Thoughts on Big Data: are you prepared for the revolution?

As I was reading the article titled “Big Data: Go Big or Go Home?” by Lora Cecere (Supply Chain Insights, LLC), I was thinking how far we have come from the days when megabytes of data storage was a luxury. Today, terabyte hard drives are commonplace on home computers.

The amount of data we store has also grown exponentially as more and more cheap storage became available. As individuals, we are storing thousands of pictures, videos, songs and documents that we may never retrieve or review again. It is easy to lose sight of content and context and simply become a data hoarder – guilty as charged. Are companies any better equipped than us to not just store but benefit from large amounts of data, let alone Big Data which is mostly a nebulous concept for non-IT? Capturing, moving and storing data has become relatively easy. And various sensors that follow interactions throughout the supply chain and wireless communications that move data rapidly give access to data that so difficult to get to previously. On the other hand, analyzing and understanding even transactional supply chain data can be a daunting task for a lot of organizations. I think there is still an opportunity for many companies to review how and why they store what they collect and be aware of the content, context and last but not least integrity of the data. After all, to paraphrase Lora, dirty data is the number one issue for supply chain teams. When you add social data from various sources to the mix, it becomes even more important for the various parties that handle data between the source and the consumer of the data to work together and understand the common goal. Silo mentality will not work with Big Data. Big Data is coming and we should welcome Big Data as with the right tools, techniques and approaches, we can answer many more questions that we don’t even think of asking today. As we prepare for the Big Data revolution, companies need to consider putting in necessary controls and tools in place in order to ensure integrity of existing data as well as the deluge of new data that is created both by individuals who may or may not be customers as well as sensors that are making their way into all aspects of supply chains. The goal should be to extract information and answers from Big Data and not end up a hoarder of even more data.