Big data privacy challenges.

Australian organisations are struggling to keep up with big data privacy challenges, according to analyst firm Telsyte, with four in 10 companies unsure they are not ‘being creepy’ using customer data.

The Telsyte Big Data and Analytics Study 2018 found around 80% of organisations with more than 20 employees have a big data strategy, although many are in the embryonic stages with pilot programs and exploratory projects.

Despite this growing wave of investment in big data, only 61% of businesses were conscious of privacy and regulatory concerns, despite the Notifiable Data Breaches (NDB) scheme that came into effect in February 2018, and the European Union’s (EU) General Data Protection Regulation (GDPR) coming into play in May.

According to the study, many organisations are unsure if they are being “careful not to be creepy”. Concerns include handling of sensitive data which customers might have shared without understanding what they had consented to, such as during the installation of mobile apps or signing up to free web-based services, e.g. social media sites and email services.

“There is a huge temptation to highly target and customise offers to individuals, but also predict behaviours which generate profits,” Telsyte managing director, Foad Fadaghi said. “However, many have had to draw the line at how sensitive customer data, such as location movements, or passively scraped data is used to target customers.”

Despite self-imposed regulation, nearly two thirds of organisations claim they protect data like other assets, such as physical or IP assets, with around half willing to share data if there is tangible benefit to their organisation.

Overall, 84% of business and technology leaders saw an improvement to their organisation by using big data analytics. Productivity and better decision making were rated highly, as was the ability to improve customer intelligence and control operating costs.

CEOs and chief data leaders critical for big data success

The CEO and CDO (chief data officer) are becoming the main sponsors of big data strategies in Australian companies, as the decision-making role of the CEO becomes less about ‘gut feel’, but  about data-driven and an informed process that can be reviewed by the board.

Telsyte research shows that companies with CDOs or CEOs spearheading efforts are more likely to have an organisation-wide digital strategy, or the ability to break down silos of data that might sit across various business units.

This addresses the fastest growing challenge relating to the adoption of, and use of, big data, a challenge cited by twice as many companies than it was in 2016.

Other challenges include the cost of big data software or services and lack of in-house skills.  The main drivers for big data analytics adoption include digital transformation, customer behaviour measurement and e-commerce applications.

Security and fraud detection is growing in popularity amongst organisations with a big data strategy now used by over 40% of organisations.

Service providers in demand

With a lack of skills and the high cost of entry holding some organisations back, Telsyte research shows service providers are the big winners when it comes to delivering big data analytics capability.

Around one-third of companies with a big data strategy are using a service provider, or outsourcers, compared to around one-quarter a year ago.  To plug the capability gaps, over one-third of businesses are investing in big data training for staff, which has seen a mini-boom during the last two years.