Crowdsourcing and cloudsourcing

Is crowdsourced prediction highly accurate, or is its claimed accuracy due to nothing more than circular reasoning?

1/30/2012

Why IT and business just can't get along

Crowdsourced prediction is the Next Big Thing.

It's a shoe-in, because it combines last year's Next Big Thing -- the Cloud -- with ridiculing experts, a perennial crowd-pleaser.

Crowdsourced prediction is based on a simple premise -- that crowds are wiser than experts. Take InTrade, which lets people bet on such matters as which Republican presidential candidate will become the nominee (yes, it's really just an on-line bookie). Those who place their faith in markets insist that on-line betting on these outcomes delivers more accurate results than the experts. As in many domains, experts predict more poorly than random chance, this is likely. For example:

2009 counted as a good year for actively managed mutual funds. According to "'Active' Did Better in '09," (Annelena Lobb, 1/6/2010, The Wall Street Journal), their performance improved markedly that year -- almost half outperformed simple index funds.

Pure random chance would have resulted in exactly half underperforming -- the experts do worse relying on their expertise than they'd do by relying on a Ouija Board.

These aren't stupid people, and they do know their subject. How could they do such a bad job?

Hey! The ManagementSpeak database is starting to run a bit low.

Keep your ears open, and let me know of any great phrases you hear. Thanks! - Bob

My best guess: Business success entails luck as well as skill. Because investment experts can't predict luck, they ignore it, substituting patterns they perceive that aren't actually there.

These substituted patterns are convenient narratives, not empirically tested theories, which means they're more likely to be wrong than right.

The issue goes well beyond stock-pickers. Many other sorts of experts also rely on unsubstantiated narratives to support their predictions -- among them political commentators and, here in the field of information technology, market analysts.

In my expert opinion, of course.

Which is why Crowdsourcing is the new savior of the predictions business. And yet, if the Crowd makes a prediction that's awesomely accurate today, how can it change tomorrow? InTrade's predictions, for example, seem to change on a daily basis.

Might the purported accuracy of crowdsourcing be nothing more than circular logic --accurate because we define "accurate" as "what the crowd is saying"?

The study I've never found, which would answer this question quite well is of horse racing.

If crowdsourcing works as advertised, were we to tabulate the results of all horseraces we'd find that exactly one-third of all horses that ran with 2:1 odds won. Otherwise, the so-called wisdom of crowds is just another in a long line of appealing narratives that have nothing at all to support them beyond their natural appeal.

As crowdsourcing depends on Cloudsourcing, let's move on, to a prediction: "Why aren't we in the Cloud?" will supersede "Why aren't our factories in China?" as the most-often asked rhetorical question in business.

It's time, because those who have been involved in making Cloud-based computing work have started to figure out that its economics are just as situation-dependent as those of offshoring.

<Digression> Whether they offshored manufacturing or programming, business decision-makers focused on raw price more than the whole picture of total cost, plus risk, plus the increased complexity of managing operations halfway around the world, with all the attendant differences in language, culture, public policy, and simple clock time. It was all about cheap labor.

That's the case even though, when it comes to manufacturing, it appears that direct labor contributes astonishingly little to the cost of manufactured items (in the case of automobiles, roughly 10%). As for software development, cost is rarely as important as such factors as reliable on-time delivery, code quality, and fit to function.

Which is why offshoring ended up disappointing its clients far more often than you likely read in the cheerleading articles that dominate the business press.</Digression>

Tally up the Cloud's direct costs and the decision to go there is far from no-brainer territory, especially for companies big enough to need such niceties as identity management (Active Directory or an alternative) and server-managed print queues.

The Cloud shines brightest (there's a visual!) when processing loads are unpredictable and highly variable. That's when its ability to add and shed capacity more or less on demand is hugely advantageous.

For small and mid-sized companies, add the economies of scale that come from making technology management Someone Else's Problem. For new, growth-oriented companies, also add cost avoidance, from not having to build a data center.

So here's some advice you can use (finally!): Be prepared. When asked "Why aren't we in the Cloud?" answer, "We are, wherever it makes financial and strategic sense."

Or, if it doesn't, answer, "We're on the lookout for opportunities. So far, much to our surprise, we'd have to spend more to go there."

Visit our e-store to buy books by Bob Lewis

New!

Second Edition of the "IT leadership bible," with twice the content of the original

Special Offer

Permalink: http://www.weblog.keepthejointrunning.com/?p=4528

Maybe it isn't horseracing, but do you know of any truly objective tests that validate crowdsourcing as an accurate source of predictions?

Do you know anyone or any discussion groups that should be talking about this?

Bob Lewis is president of IT Catalysts, Inc. ( www.itcatalysts.com ) an independent consultancy specializing in helping businesses improve their ability to achieve designed, planned business change … which can only happen with effective IT organizations that are strategically integrated into the enterprise. Contact him at rdlewis@itcatalysts.com.

Don't leave me sitting here in a vacuum! If you think I'm full of beans, let me know. Leave a Comment (just click on the Comments button at the bottom of either the homepage or any archived column) to share your thoughts with the whole KJR community.

Or, drop me a line -- the address is Letters@ISSurvivor.com.I sometimes use Comments or reader letters in my columns. The rules:

Comments are public, because they're ... well, they're right at the bottom of every column in the archives.
If you send me a letter, let me know if and how I can use it (as is, sanitized, or don't be ridiculous - you'll be found out and run out of town). Once you give me permission to use it, the permission is eternal.
Also let me know if you'd prefer to remain completely anonymous, or whether I may give you credit by name.

If you like this article, why not let a friend enjoy it, too? It's fine with me, and in fact I'd be flattered. All I ask is that you send the whole thing, including this notice. But don't be shy ... if you think they'd like it, don't you think they should see it? But only those people - you wouldn't want me to get a reputation as a spammer, would you?