Lack of data confidence leads to AI failures

More than a third of AI and analytics projects in the cloud are failing due to poor quality data according to a new survey.

The study from data specialist Trifacta surveyed 646 data professionals across different industries and titles to examine how organizations are handling the accelerating transition of data to the cloud, the obstacles of data cleaning for analytics and the time constraints they face when preparing data.

The findings show that data preparation is an ongoing issue, with 46 percent of respondents spending over 10 hours properly preparing data for an analytics and AI/ML initiative while others spend upwards of 40 hours on data preparation processes alone on a weekly basis.

But although data preparation is a time-consuming, inefficient process, it's vital to the success of every analytics project. Some of the leading implications of data inaccuracy result from miscalculating demand (59 percent) and targeting the wrong prospects (26 percent). Decisions made from data would improve if organizations were able to incorporate a broader set of data into their analysis, such as unstructured third-party data from customers, semi-structured data or data from relational databases.

These problems are being recognized at a senior level. Among C-suite respondents 38 percent say poor data quality has caused analytics and AI/ML projects to take longer, while 36 percent say they cost more or fail to achieve the anticipated results (33 percent). With 71 percent of organizations relying on data analysis to drive future business decisions, these inefficiencies are draining resources and inhibiting the ability to glean insights that are crucial to overall business growth.

Cloud is clearly important to new projects too, with 66 percent of respondents stating that all or most of their analytics and AI/ML initiatives are running in the cloud, 69 percent reporting their organization's use of cloud infrastructure for data management, and 68 percent of IT pros using the cloud to store more or all of their data -- a trend that's only likely to grow. In two years from now, 88 percent of IT professionals estimate that all or most of their data will be stored in the cloud.

"The growth of cloud computing is fundamental to the future of AI, analytics and machine learning initiatives," says Trifacta CEO Adam Wilson. "Unfortunately, the pace and scale at which this growth is happening underscores the need for coordinated data preparation, as data quality remains one of the largest obstacles in every organization's quest to modernize their analytics processes in the cloud."

The full report is available from the Trifacta site.

Photo credit: pathdoc / Shutterstock