Big data is not about data!

…or: For Hard Results, You Need to Get Soft!

The Neglected Side of Big Data

“Big data is not about data! It is not even about technology!” – Quite a bold statement, especially if you follow the public debate on this topic. Most articles and discussions are centred on technical developments like Hadoop, cloud computing or self learning algorithms. This leads to the impression that big data projects are successfully accomplished by just finding the right technical tool. But this point of view falls utterly short of a fundamental characteristic of big data. You might call it the soft side of big data. This recklessly neglected side more often than not is the decisive factor that determines whether a big data project or programme succeeds, or fails.

So, what is this mysterious soft side of big data? In the style of the 3 V’s (Volume, Velocity and Variety) often used to define big data, you may summarize it as the 3 C’s: Creativity, Collaboration and Culture.

Creativity

At the beginning of each big data project stand creative ideas on how to take advantage of data. These ideas typically are formulated in questions that start with “What if we would/could…..?”. Think for example of a logistics company that could ask “What if we would combine our own data with open data, such as weather data or traffic data? Could we improve our daily delivery predictions and routing? And if so, by how much?”. But also during the data analysis, the core of each big data project, creativity is essential. With some creative ideas and programming, the analysis code can be optimized in performance and new and often unexpected insides can be revealed, like e.g. that an inefficient use of the operation theatre in a hospital is not caused by emergencies but by the long scheduled surgeries.

Working with big data involves a lot of experimentation, since most of the times you are exploring uncharted territory. You need to be creative to overcome and solve the challenges you are facing during a big data project and analysis, and creativity is an essential ingredient to succeed. Without creativity every big data project is doomed to fail already from the beginning.

Collaboration

To take full advantage of the possibilities that come with big data one needs the collaboration of multiple departments of a company. The focus of the analytics part of a big data project is provided by the business, commonly by sales, marketing or operations. The technical elements themselves are under the responsibility of IT or an autonomous data department. Essential for a fruitful collaboration is that the interdisciplinary members develop a common language. This is not trivial, seeing that most collaboration members typically come from very different backgrounds. A proven approach to swiftly support the development of a common language is through visualizations that act as a common basis of understanding. As part of our day-to-day business we often support the creation of a common language for business and IT using our proven operating model canvas. The same technique can be used to enhance the communication in big data projects.

Next to this internal collaboration, the collaboration with external partners such as research institutes, universities or consultants can help to deliver the desired value of a big data project.

Culture

Successful big data projects and programmes require a change in culture for many organizations. The related projects and programmes are very dynamic and the necessary approach is an agile one. A lot of experimentation is involved with iterative cycles. This requires a dynamic, creative and inspiring environment where people dare to try something new and unconventional, an environment often found at start-ups. This is why various large companies like ING approach the big data challenges by first founding, and later on integrating start-ups. However, also the creation of company internal, interdisciplinary teams for big data projects can be very successful.

It pays off to pay close attention to the 3 C’s of big data. It will not only support the project and help to reach the defined goals faster and more efficiently but it is also likely to create higher quality results with a higher impact. For hard results, you need to get soft.

Do your big data ambitions need more creativity, collaboration or a more inspiring cultural environment? Do not hesitate to contact Anderson MacGyver.