I’ve increasingly noted that some people take it literally, equating “big data” with “lots of data.” That’s in direct contradiction to how the industry defines it, which is generally the combination of “structured data” and “unstructured data.”
I’ve especially noted this recently, with this month’s dual cultural lollapaloozas of both the Super Bowl and the Academy Awards. The attempts to predict the outcomes of these events beg the question: what is big data analysis really supposed to achieve? Can it really predict discrete outcomes? Or is it really designed to identify patterns and potential outcomes?
Not being a data scientist, I’m truly intrigued by this dichotomy. One could argue, especially after reading this article in the Los Angeles Times about the Oscars, that predicting the race involves taking the snapshot of overall trends – that is, identifying a specific point along a wave. And Oscar prognostication really is an amalgamation of structured and unstructured data.
The structured data might include the analysis that Meryl Streep has been nominated more than her fair share of times, and thus less likely to win yet again. The unstructured data might include the fact that the other nominees in her category are four actresses from Eastern Europe that no one’s ever heard of. Draw your own conclusion of the inevitable result.
Then there’s the Super Bowl, another event with a discrete conclusion. Forbes contributor Roger Groves, a professor at Florida Coastal School of Law, went out on a limb before the game and predicted that the Seattle Seahawks would win the Super Bowl thanks to its superior defensive analytics capabilities. Hence my conundrum: big data analytics can predict everything except for somebody like Malcolm Butler stepping in front of Ricardo Lockette for a goal-line interception. At that moment, it was the Patriots with the superior defense capabilities, but anyone who’d seen the NFC Championship Game the week before might have assumed a pattern of Seattle never saying die and coming back at the last minute.