Key developments and challenges in LLMs [Q&A]

Large language models (LLMs) have undergone rapid evolution in recent years, but can often be viewed as something of a 'black-box' as a lack of transparency makes it difficult to identify how decisions are made, trace errors, or understand biases within the model.

We spoke to Pramod Beligere, vice president -- generative AI practice head at Hexaware, to discuss this along with the tools that are being developed, such as explainable AI and interpretable models, to make AI systems more understandable, trustworthy and accountable.

BN: How are LLMs evolving?

PB: Three primary branches are emerging on their evolutionary tree: Encoder-only, Encoder-Decoder, and Decoder-only groups of models.

Initially, encoder-only models like BERT (released under an open source license in 2018) introduced bidirectional training, improving comprehension tasks. The decoder-only model GPT-2 (2019) demonstrated impressive text generation capabilities. The release of GPT-3 (another decoder-only model released in 2020) was a big leap, with 175 billion parameters, enabling better understanding of context and text generation. OpenAI's Codex (2021) focused on showcasing code generation capabilities. More recently, models like GPT-4o have exhibited even more advanced abilities, including multimodal understanding.

BN: How has the scalability of deep learning impacted the capabilities of LLMs?

PB: Deep learning has been a key approach underlying the development of LLMs. By leveraging neural networks, especially transformer architectures, this enables LLMs to process and generate human-like text. Techniques such as attention mechanisms allow models to focus on relevant sections of the input data, enabling better understanding of the context. The scalability of deep learning has enabled training on vast datasets, enhancing language comprehension and text generation. Innovations like transfer learning and fine-tuning have further refined the capabilities of LLMs, enabling them to perform a wide variety of tasks from translation to summarisation with high accuracy.

BN: What are the key challenges for LLMs when it comes to transparency and interpretability?

PB: The primary challenges facing LLMs include:

• Lack of details on the data size as well as the content and origin of the data being used for training (which could have copyright-related legal ramifications, among other issues).
• Their complex architecture and vast parameter count make it difficult to understand how they arrive at specific outputs.
• The 'black-box' nature of LLMs obscures the decision-making process, raising concerns about biases embedded in the training data.

There are ongoing efforts to enhance transparency and interpretability but they are yet to fully address these challenges.

BN: Why is transparency a concern with the current multi-layered architecture of LLMs?

PB: The black-box nature of large-scale LLMs (based on the transformer deep learning architecture) results in their internal workings not being interpretable. The models consist of millions/ billions of parameters, making it difficult to understand how specific inputs lead to specific outputs. Each layer in their complex multi-layered architecture transforms the data in ways that are not straightforward to trace or explain. It is also challenging to understand what these models have learnt and how they are making decisions, since the features learned by these models are abstract, with their decision pathway not being traceable. The complexity and opacity is a clear concern especially when regulations demand transparency.

BN: How has the lack of transparency in LLMs led to significant issues related to privacy, consent and bias?

PB: OpenAI's model has been criticised for generating biased/ discriminatory content and enabling the creation of misinformation. They have also been sued by multiple newspaper companies for using their content without permission or payment. More recently, the actress Scarlett Johansson has complained that a synthetic voice for ChatGPT called 'Sky' is too similar to her own, and was created without her permission.

Similarly, Google's Gemini faced controversy for producing biased and offensive image outputs based on race and gender. This raised ethical concerns and highlighted the opaque nature of its decision-making process, complicating efforts to identify and mitigate biases. These examples underscore the need for greater transparency and accountability in LLMs.

BN: Can you explain what impacts the interpretability of LLMs?

PB: The complexities of deep learning architecture has a significant impact. These models often involve millions/ billions of parameters, organised in intricate layers of neural networks. Such complexity makes it challenging to trace how specific inputs lead to particular outputs. As a result, understanding the decision-making process becomes difficult, hindering efforts to identify and mitigate biases or errors. This can reduce trust in the model's outputs and complicate debugging, auditing, and improving the system. Consequently, it raises ethical and practical concerns about the deployment and use of LLMs in critical applications.

BN: What are the primary ethical concerns associated with the training data used for LLMs?

PB: These primarily involve the data used for training. Key concerns include:

• Bias and fairness: Training data often contains biases reflecting societal prejudices, which can be perpetuated and amplified by LLMs, leading to unfair or discriminatory outputs.
• Privacy: Training on large datasets may inadvertently include sensitive or personal information, raising privacy concerns.
• Consent: Data used for training is often scraped from the internet without explicit consent from the content creators, raising ethical issues about data ownership and usage rights.
• Transparency: Lack of transparency about the data sources can hinder accountability and trust in the model's outputs.

These considerations necessitate careful data curation and ethical guidelines to ensure responsible AI development.

BN: What approaches are being used to improve the transparency of deep learning models?

PB: Several methodologies and tools are being developed:

• Explainable AI: Techniques like LIME (Local Interpretable Model-agnostic Explanations) and SHAP (SHapley Additive exPlanations) help interpret model predictions by highlighting important features.
• Model Auditing: Tools such as IBM's AI Fairness 360 and Google's What-If Tool enable auditing for bias and fairness.
• Visualisation: Techniques like attention maps and saliency maps provide visual insights into model decision-making processes.
• Interpretable Models: Developing inherently interpretable models, such as decision trees or rule-based systems, alongside deep learning models.
• Transparency reports: Detailed documentation of model architecture, training data, and evaluation metrics to improve accountability.

These efforts aim to make AI systems more understandable, trustworthy, and accountable.

BN: What's the role of regulatory frameworks in addressing transparency issues of LLMs, and how may this evolve in the future?

PB: Current regulatory frameworks are still evolving. Regulations like GDPR emphasise data protection and the right to explanation, requiring organizations to provide understandable information about automated decision-making processes. The EU AI Act aims to set stricter transparency and accountability standards for high-risk AI systems. In the future, we might see more comprehensive regulations mandating detailed documentation, bias audits, and explainability requirements for AI models. These changes could drive the development of more transparent, fair, and accountable AI systems, fostering greater trust and ethical use of AI technologies.

BN: How can businesses balance the risks and benefits of using more transparent LLMs?

PB: Benefits of using more transparent LLMs:

• Trust and accountability: Enhanced transparency builds user trust and facilitates accountability.
• Bias detection: Easier identification and mitigation of biases, leading to fairer outcomes.
• Regulatory compliance: Simplifies adherence to legal and ethical standards.
• Improved debugging: Facilitates troubleshooting and model improvement.

Risks:

• Complexity: Transparency tools can add complexity and computational overhead.
• Intellectual property: Revealing model internals may expose proprietary information.
• Security: Increased transparency might reveal vulnerabilities that could be exploited.

Businesses can balance these by adopting a layered transparency approach -- providing sufficient detail to stakeholders without compromising proprietary information or security. Implementing robust governance frameworks and regularly auditing models can also help manage risks while reaping the benefits.

BN: How might LLMs develop in the future?

PB: Future developments will likely focus on enhancing interpretability and reducing biases to foster greater trust and ethical use (as well as ensuring regulatory compliance). We can expect advancements in model efficiency, enabling more powerful LLMs to run on less computationally intensive hardware, including on edge devices. Multimodal LLMs that can process diverse inputs with minimal latency will become pervasive. Specialised LLMs tailored for industry domains will see increased adoption.

We are also seeing the rising number of small and powerful open source models, giving better choices for organizations. Smaller models would also make sense when implementing agentic workflows, as each model would focus on specific tasks that would not require cost intensive large general purpose LLMs.

Image Credit: Sascha Winter/Dreamstime.com