Monday, February 9, 2015
Is crowdsourcing a myth?
"On November 24, an email from an anonymous Hushmail address landed in the team’s inbox. It taunted USCD about the team’s security lapses, claimed that the sender had recruited his own horde of hackers from the notorious 4chan bulletin board, and revealed exactly how he had used proxy servers and virtual private networks (VPNs) to launch his attacks.
'I too am working on the puzzle and feel that crowdsourcing is basically cheating,' read the email. 'For what should be a programming challenge about computer vision algorithms, crowdsourcing really just seems like a brute force and ugly plan of attack, even if it is effective (which I guess remains to be seen).' He signed off with the phrase 'All Your Shreds are Belong to U.S.'”
* * *
"By Stefanovitch’s reckoning, just two individuals had accounted for almost all the destruction, eviscerating the completed puzzle in about one percent of the moves and two percent of the time it had taken a crowd of thousands to assemble it. Yet the attacker had left one more clue, a blunder that pointed right back to his door. During the first attack, he had logged in with an email address from his very own domain."
* * *
"Luckily for platforms like Wikipedia or Amazon’s Mechanical Turk, the prospect for longer-term crowdsourcing projects are not so bleak. Game theorists have found that systems where individuals can build up a good reputation, are (probably) not as prone to devastating attacks from within.
But wily humans are good at finding their way around even the most secure digital systems. In a paper last year, researchers at the University of California, Santa Barbara, used AI software to detect spammers in China’s Weibo social network with an accuracy of up to 99%. Despite that, the authors concluded that 'adversarial attacks are effective against all machine learning algorithms, and coordinated attacks are particularly effective.'”
Ok, so designing a crowdsourcing problem is critical. And some problems cannot be handled by crowdsourcing because trolls and saboteurs can get into the system and wreck things faster than you can solve them. Got it. But this doesn't destroy crowdsourcing. The Internet itself is flawed because the original design did not take robust security into account. But we still use the Internet.
at 12:04 PM