Does Big Data Live up to its Hype?
The last decade has been characterized by extraordinary developments in technology. As computer power has increased exponentially, so too has the volume of data. This has resulted in countless opportunities in industries as varied as market research, healthcare and retail. One particular manifestation of this has been in the form of Big Data.
Within the retail segment, most major retailers across the globe today utilize sophisticated analytical tools in order to better understand their customers and thus provide them with more personalized shopping experiences. In an age where customer loyalty is an increasingly rare commodity, such practices are designed to increase fidelity and ultimately improve their bottom line.
Whilst the widespread uptake of Big Data from retailers as varied as Carrefour and Chanel would point to significant advantages, this article will look at some of its limitations. Although the focus will be on the retail segment, many of these limitations are applicable beyond the sector.
Compared to twenty or thirty years ago, the modern retail experience is virtually unrecognizable. As consumers interact with brands across different channels and touch points, retailers find themselves faced with an unprecedented volume of collected data. (90% of all data has been created only in the last two years). Considering the volume, velocity and variety of data being collected, determining which data sets are most relevant for identifying future consumer buying behavior is no easy feat. What’s more, in trying to separate noise from valuable information, a number of potentially important details become lost in the process. As such, data processing tools still have considerable room for development to avoid excluding key variables.
- What, Not Why
Behavioral economics has taught us that consumers respond to many more elements than just simply price when looking to make a purchase decision. In its current state of development, Big Data does not presently allow for the possibility to understand the role different elements play in encouraging or discouraging a purchase. This could include aspects such as discounts, product displays, calls-to-action, artwork, weather, competition activities, etc. Human beings are not binary entities and it is worth remembering that not everything can be quantified. As beneficial as spreadsheets may be, they do little to convey elements such as context or human emotion. In other words, Big Data can prove useful in demonstrating the “what”, but not the “why”.
By its very nature, Big Data is backward looking. As such, its application is contingent on the fact that an entity has some historic data to analyze (therefore limiting its use in the case of new ventures). Additionally, regardless of the scale or volume, utilizing data based on historic transactions in order to determine future strategies inherently limits insights to those points that can be culled from the given pool of data. Simply put, Big Data can tell you what worked (or did not work) in the past, but not necessarily what will work in the future. Not only is it inaccurate to assume that the future will be a rational and linear projection of the past, but maintaining the status quo also has the possibility to limit innovation.
- General Limitations
Though Big Data can typically process large quantities of information more quickly than traditional research tools, it is worth noting that the speed of this is not as instantaneous as commonly perceived. Additionally, not all correlations supposedly put forward by the data sets are meaningful. For instance, between 2000 and 2009, both the number of divorces and the consumption per capita of margarine decreased in the US state of Maine. Whilst a correlation technically exists, it is meaningless. Finally, Big Data tends to be based in in-house data. As such, consolidating information on competitors through Big Data is typically not possible unless such information is open to the public.
Over the coming years, the field of Big Data is likely to continue to see considerable development much to the benefit of the retail segment as well as other industries. However, even if data processing tools become more sophisticated, Big Data’s inability to gain insight into unquantifiable topics such as human emotion or its inability to incorporate the impact of context, are likely to remain some of its key shortcomings. As such, the development of Big Data should also be complemented with the development of qualitative insight, in order to further improve both human understanding along with technological advancement.