Improving data analysis with AI [Q&A]

Generative AI is making its presence felt in more and more areas, but there are well-founded concerns about the accuracy of information that it provides.

Is it possible to provide the convenience of a large language model AI system, with the logic and accuracy of advanced analytics? Arina Curtis, CEO and co-founder of DataGPT, thinks so. We spoke to her to find out more.

BN: Why are existing large language models ill-suited to data analysis applications?

AC: Large language models (LLMs) are limited in their ability to perform data analysis, and struggle with computationally complex questions. While they can generate human-like text and answer general knowledge questions pretty well, they cannot interpret data in a meaningful way without the addition of a dedicated data interface. Also, LLMs can generate ambiguous or inaccurate information, presenting a major challenge to the precision and accuracy needed for effective and actionable data analysis.

BN: What are the main challenges involved in using LLMs effectively?

AC: Popular solutions fall into two main categories: LLMs with a simple data interface (e.g. LLM + Databricks) or BI solutions integrating generative AI. The first category handles limited data volumes and source integrations. They also lack depth of analysis and awareness of the business context for the data. The second category leverages generative AI to modestly accelerate the traditional BI workflow to create the same kind of narrow reports and dashboard outputs.

BN: How is DataGPT different?

AC: DataGPT delivers a completely new data experience, empowering any users to access comprehensive analysis by simply asking questions in everyday language.

The LLM is the right brain of the system. It's really good at contextual comprehension. But you also need the left brain, the DataAPI, an algorithm to perform critical logic functions and conclusions. Many platforms falter when it comes to combining the logical, 'left-brained' tasks of deep data analysis with the interpretation functions of the 'right brained' LLM.

DataGPT's self-hosted LLM combined with core analytical technology makes sophisticated analysis accessible, empowering faster data-driven decision making and freeing analysts from the burden of ad hoc and follow-up requests.

BN: Can you tell us a bit about the technologies that underpin this?

AC: DataGPT's AI Analyst is driven by three key pillars:

• Lightning cache: our proprietary database is 90 times faster than traditional databases, enabling 15 times cheaper analysis and running queries 600 times faster than standard business intelligence tools.
• Data analytics engine: our algorithm conducts a comprehensive, logical analysis of customer data. Ask DataGPT a question, and it will execute millions of queries and calculations to determine the most relevant and impactful insights. Our sophisticated analysis can answer complex questions, such as, 'why did this happen?'
• Data analyst agent: Smart enough to handle the ambiguities of human speech and accurately interpret which data elements are of interest to the user, while managing the holistic/contextual interpretation and completely avoiding the inaccuracies (e.g., 'hallucinations') of generic foundation models. Uses natural language query (NLQ) functionality for searching and providing insights from corporate data sources, and creates narratives around those generated insights used to make more informed business decisions.

BN: Is this just for data analysts or can anyone use it?

AC: DataGPT serves product, UX, marketing, ad sales and customer service teams across a wide range of B2B and B2C industries, including fintech, e-commerce, gaming, media, entertainment, and travel.

While data teams make up the bulk of our users, our mission is to empower every person, in every role, in any industry, to use data to make decisions, big and small. Non-technical business users and department managers can ask questions using a familiar chat window with everyday language in order to seize revenue and growth opportunities.

Image credit: BiancoBlue/Dreamstime.com