How do we keep the exciting promotion from reducing the quality of the site?

Question

So the big promotion is getting far more participation than I ever expected was possible. This is a big win, and CHAOS pulled off an amazing thing here. Kudos for all your planning work! Less than a week into the contest, there are already as many people at level 2 as I expected would be there by the end of the competition.

But along with a giant spike in readership, quastion-asking, and question-answering, I worry that we're getting an increase in some less-desirable participation also. Lower quality questions, and attempts to game the system seem slightly less rare than they have been in the past. The addition of stakes higher than badges and reputation points (themselves a powerful motivator) has the potential to bring out an uglier side of our competitive natures. Here be dragons.

While the promotion is bringing in wonderful new participants and dramatically increasing the activity of our current participants, what do we need to do as a community to help ensure that increased quantity doesn't degrade quality or community?

Answer 1

Thanks for posting this, it definitely warrants a discussion.

Whenever we run a promotion, the goal (in addition to rewarding you for all the hard work you do) is to increase activity on the site by attracting new users or getting current users more engaged. Anything that increases activity carries some risk that the site will see an increase in lower-quality content. Because of that, we always try to run promotions past the moderators to get feedback, and we only run large promotions like this on sites that have a community strong enough to moderate a large influx of content.

In this case, we designed the promotion a little differently than past promotions because we thought it would be more interesting, less competitive, and have better results. Our past promotions were mostly tag contests, where we'd pick a tag and a timeframe and give prizes to the top or most viewed posts. The drawback with that kind of promotion is there can only be 1 or 2 winners, so it causes more competition. Although there are limits to the number of people that can win iPads and iPods in this contest, it's pretty high. That doesn't mean it eliminates all competition, but it does mean that you can cheer people on and vote for their posts without dramatically decreasing your own chances of winning. Because of that, our hope was that people would still be willing to vote other posts up or down when appropriate, as per usual, without concern for whether it would affect their own status in the contest.

The real-time updates were meant to make the promotion more interesting to watch and increase the likelihood that people would participate. It just makes it easier to keep track of things that you can mostly find anyway. Since it’s easy to see where you are, we figured it would make people more likely to come back and try to reach that next level. I see stuffe’s point that it might cause people to flood their communication channels - we tried to reduce this by allowing you to share posts anywhere you want instead of just your own Facebook and Twitter.

This is not a perfect promotion, so I appreciate the discussion and feedback. To some extent, there’s no way to avoid a little bit of competition when prizes are involved. My hope is that most people are on the site because they love Apple and find the Q&A helpful and interesting. The bar for winning this contest is set pretty high, which I think keeps out most people who are just here to win an iPad. Those people will always be around, but we're doing our best to reduce the damage they might do.

Answer 2

I want to expand on my comment to Lauren.

Some of my remarks in the chat room of late might be interpreted to mean that I'm against the this contest. This is not true. I am very appreciative of the efforts that Lauren (and whomever else is deserving) put into creating and promoting the contest. I am thankful to Stack Exchange for committing the resources necessary to provide killer prizes like iPads and iPod Touchs. I am hopeful that the exposure that the contest brings will bring new growth and excitement to Ask Different.

I am especially hopeful that a few members of our community who have put forth extraordinary efforts, particularly in the area of providing high quality answers, will be rewarded for their efforts. I know it wasn't the lure of an iPad that motivated them to begin with, but that doesn't mean they can't be rewarded after the fact. And should I be fortunate enough to receive a prize, I won't turn it down. There is a lot of good in this contest and it has brought a lot of life to Ask Different.

What I am against is the effect that the contest is having on our community. New faces are popping up and posting without taking the time to learn what Ask Different is all about. Previously dormant users are rising up and posting multiple questions for which answers are a simple RTFM (or Google search) away. Even some of our long time members are posting questions about which I harbor doubts. Is it a real question? Will an answer actually make a difference? Does the OP really care or is this just to put some points on the scoreboard? The quality that we fight hard to maintain is being eroded and there is something distasteful about the edge that competition brings to our site.

I know that we have the tools do deal with all of these challenges but at times I feel like Nick Fury battling HYDRA; Cut off an arm and another grows to take its place.

The bottom line is that hanging out on Ask Different has been less pleasant of late. Ironic since my numbers would seemingly indicate I'm spending more time here. Perhaps that's true, but it's just not as enjoyable as it was last week.

As I said to Lauren, this will pass. We'll survive and by discussing our grievances and, most importantly by virtue of business as usual—asking, answering, commenting and voting—we'll emerge a stronger community in the end. Despite the sense of impending doom I may express in my darker moments.

A special shout-out and thanks to Bmike and Kyle Cronin, who have both talked me down off the ledge this week and to Stuffe and Daniel L for keeping it real for me. You are the best amongst the best. We may disagree on specifics from time to time, but in the end I know we share a common goal.

Answer 3

I think the promotion is a good thing, and I am calling it a promotion rather than a competition because the competetive element is potentially a bad thing here. However I have a few misgivings about the format:

"Realtime Updates" - This is some really great code, but it's intended purpose has a dark side, and knowing you are X shares away from getting Level 2 etc will just prompt some people to flood their communications channels with links because they know they are close. Check chat for the reduced number of onebox links and the increase in the personalised links. I've done this myself, and am not proud. The fact you can check on the other guys too means you can get too involved trying to beat the others, rather than put out genuine content that would be welcome even without a promotion. If the idea is to promote being a good citizen, then the winning citizen by definition would have done it anyway, and shouldn't be doing it merely because there's a prize. Removing the update would potentially stop borderline genuine users from caning the site for 3 weeks then leaving because they won't have a clue if they are caning it in vain or not, whereas a "better" user will carry on regardless.

"New Users" - Obviously you had a brief when setting this competition up, and you will know what your expected outcomes are and what you wanted to achieve. No way to know exactly what these things were, but what we seem to have is an influx of people (some new, some previously dormant) throwing up a lot of content. The "acheivements" for want of a better word for getting X answers voted on etc reward quantity without necessarily have any controls of the quality. If I ruled the world (in my lovely hat) I would like to see the promotion be more tailored to getting people to use the site properly, rather than to use it lots; this should foster the same end result inadvertently anyway. I know this is difficult, but helpful flags and other badges may be a good way of getting the existing audience to use the site better, which should in turn equate to more at the same time.

"Tactical voting" - I'm not saying this has happened, but I am saying when the time comes close to the end, it likely will happen. Someone with the required 4x6 upvotes will find one get's mysteriously downvoted to 5 votes and rules them out of Level 3. I'm raising this now so you are prepared for it, I know votes are supposed to be invisible, but you will for sure get a query I feel. That's not me saying I will do it of course, but it's human nature when a big prize is at stake. And there's the other side too, people not voting for someone who is close to moving up a level in the competition etc as a way to maintain a gap in the scoring.

"Shares" - I've tried to share and point out exactly why I am doing it. Each time I have put shares out on Twitter I have book ended it with non share tweets about AD and the promotion. I don't want to link bait my friends, but I do want to turn some of them into users here. As I mentioned previously, I'm sure a lot of the shares are actually being shared in chat where discussion is pretty busy at the moment. These shares defeat the intended purpose, as they never leave the site, and are only ever seen by existing users. Perhaps a better mechanism is to allow a referal scheme in a future competition. Having a challenge to introduce 2 new users who get say 250 Rep and 5 selected badges within a month of joining would actually be a way to ensure higher quality inrease in visits that results in tangible benefits.

As I have other thoughts, I may edit them in here. Overall I am 70% positive on the promotion. I hope it acheives it's aims, and of course I hope I win an iPad, and further still if I do, I hope I deserved to win on the basis of maintaining very active but more importantly very focussed input to the sites real aims which still stand apart as worthwhile content. I'll still be here after, as I was here before. This is not an "old guard" (I've only been here what 7 months?) piece of disgruntlement against new users though, I hope that the conversion or "sticky" ratio of the more recently active users is increased post promotion. I would hate to see someone with enough free time simple hammer the site for 3 weeks and then hit the sunset with iPad in hand.

Answer 4

Thank you for posting this. This is almost exactly how I feel, and part of my convo in the chat last night.

I thought about it a little more, and think in some cases, we can even use the current tools and voting to almost regulate it. As in, if a user is just doing 'jeopardy' questions, we should vote accordingly.

Answer 5

Given that the contest has increased the activity of the site considerably, and there are many competitors, it seems that the "highest combined score" part of the rules is going to come into play (unless more iPads and iPods magically appear as prizes).

Given Lauren's post here:

How, exactly, would winners be determined if more than five users complete Level 3?

The strategy of making your 20 or 35 contest entries have very high scores seems to indicate that people will be pushed to improve their existing contest entries, and carefully choose what questions to post and what questions to answer - only those that are likely to be highly voted will get much attention going forward.

Other strategies, such as bumping and promoting your own answers on twitter and elsewhere, will come into play since simply providing more posts isn't going to help those who have already reached the level they intend to compete at.

Answer 6

Here's a great example of tactical voting: https://apple.stackexchange.com/a/46171/3117

Downvote, no comment, although my answer is 100% correct and was posted before any other answer was visible.

I have a very good idea who is responsible and of course it is another person close to reaching level 3. I don't expect any action taken except to note that this is happening and I expect it will only become more common in the next week or so.