It has been 10 years since McKinsey published its report on the future significance of analysing large data sets, birthing the term big data in the process. In those 10 years, many businesses across many industries have used data to compete more effectively.

Yet, familiar issues continue to highlight why many have failed to get to grips with data-driven business transformation. And now that the waves of intelligent automation (IA) are up to our elbows, it's worth looking at what these issues are to remain competitive.

A combination of years of creating large data sets and the advent of RPA and IA means we're entering a technological season of harvesting, that, if done right, will ensure business success.

Enabling automation

If big data is defined as a "compilation of unstructured data", then maximum value can be gleaned by structuring it. Large data sets are only valuable if they're coherent, as well as uniformly applicable to the right people at the right time.

One of the sticking points for a failed automation project is the yoke of having big data no one can make sense of. It's disorganised, or split across multiple disconnected silos. Yet, big data is a key component of a successful automation strategy, so this has to change.

RPA can help to structure large amounts of unstructured data, so that the harvesting of big data is more efficient. But this can only be achieved by wholesale (or near-wholesale) acceptance by your workforce of the role automation can play. This is where communication comes to the fore, underpinned by great leadership. When everyone is rooting for automation, you can begin to apply it to disparate data sets and get decent, workable analysis from it. And for automation to truly be incorporated into your transformation framework, you would be wise to enable people to do it themselves with simple user interfaces and low-code environments.

Leadership and the big picture

You can only see useful patterns in big data if it's structured, accessible and easy to analyse. From there, RPA can create views of analysed data that helps you more effectively evaluate how your business performs. The ultimate outcome is better value for your business and better value for your customer (or whoever's waiting at the next link of the chain). And I'm referring to internal as well as external performance.

Machine learning and intelligent automation are at their best when they see the big picture. If the big picture is hard to make sense of, your automation ambitions may fall at the first hurdle based on the complexity or costs associated with trying to untangle the mess.

But if you have nurtured a foundation of adaptability to change, and have the means and the willing to take on your huge mountains of data, intelligent automation is your game-changer.

As Randy Bean says in a 2020 Forbes article: The [2011 McKinsey] report contained one important caveat, however, noting that these advances were all predicated “as long as the right policies and enablers are in place". So, the big question is: are you a big enough leader to handle your big data?