How businesses need to address the security risks of LLMs [Q&A]

Businesses are increasingly adopting AI and large language models in search of greater efficiency and savings. But these tools also present risks when it comes to cybersecurity.

We spoke to Aqsa Taylor, director of product management at Gutsy, to learn more about these risks and what organizations can do to address them.

BN: What are some of the unique challenges that large language models (LLMs) pose to traditional cybersecurity practices?

AT: LLMs introduce a set of unique challenges to traditional cybersecurity practices because of their advanced capabilities and the nature of what they produce. Unlike standard cybersecurity tools, LLMs are trained on massive datasets, which inadvertently boosts the intelligence of both security teams and potential threats.

LLMs often generate outputs that aren't easily predictable or explainable using conventional methods, which can be tricky for security analysts who are used to more straightforward tools. The possibility that LLMs might create new attack methods or exploit vulnerabilities that traditional tools overlook adds complexity to identifying and addressing threats.

Also, the sheer scale and complexity of LLMs requires new approaches for validating security and assessing risks. Traditional methods may not fully grasp the nuanced risks associated with LLMs, such as their potential to amplify biases or misinterpret inputs in unexpected ways.

BN: Could you explain the concept of adversarial training in the context of LLMs, and how it helps mitigate potential risks?

AT: Adversarial training means exposing LLMs to inputs deliberately designed to test their boundaries and induce them to behave maliciously or break established rules. This is critical during the training or tuning stage of LLM development, before full implementation, to pre-emptively identify vulnerabilities and improve resilience against potential attacks.

For LLMs, adversarial training can include techniques like injecting noise into input data, crafting misleading prompts, or simulating known attack patterns. By subjecting LLMs to these adversarial examples, cybersecurity teams can evaluate if they can stand up to real-world threats.

Continuing adversarial attacks post-deployment is also a good practice to maintain ongoing compliance and identify any evolving risks or failures. This helps mitigate the risks posed by LLMs by enhancing their ability to withstand adversarial inputs and reducing the likelihood of malicious exploitation in operational environments.

BN: Why is 'explainability' crucial for LLMs in cybersecurity, and what steps can organizations take to enhance it?

AT: Explainability in LLMs refers to their ability to justify why specific outputs or decisions are made, which is critical for cybersecurity operations to understand and trust their behavior. However, achieving explainability in LLMs is challenging due to the complex nature of deep neural networks and the inherent opacity of their decision-making processes.

Organizations can enhance explainability by integrating additional layers or tools specifically designed to provide insights into LLM outputs. Techniques such as model interpretability frameworks, attention mechanisms, or layer-wise relevance propagation can offer partial explanations of LLM decisions.

Despite these efforts, full explainability remains elusive for many security-focused LLMs, often due to the trade-offs between model complexity, performance, and interpretability. That's why organizations should adopt a pragmatic approach that combines technical advancements with rigorous testing and validation to increase confidence in LLM outputs while acknowledging and mitigating the inherent risks of opacity.

BN: How does continuous monitoring play a role in mitigating the risks associated with LLMs in real-world cybersecurity environments?

AT: Continuous monitoring of LLMs can help detect anomalous or unexpected outputs that may indicate potential threats or vulnerabilities. Unlike traditional security controls, which rely on predefined rules or signatures, LLMs can produce outputs that evolve over time and in response to dynamic inputs.

Implementing robust systems for continuous monitoring allows organizations to track LLM behavior in real-time, identifying deviations from expected norms or patterns. This enables timely intervention and remediation of potential security incidents before they escalate.

Major AI providers are deploying specialized LLMs designed specifically for monitoring other LLMs, akin to a 'spy catching a spy.' This enhances oversight and accountability within AI-driven cybersecurity frameworks, reinforcing the importance of continuous monitoring as a cornerstone of risk management.

BN: What is the importance of human oversight when integrating LLMs into cybersecurity operations, and what strategies can organizations adopt to maintain a balanced approach between automation and human decision-making?

AT: LLMs require human oversight to mitigate their inherent risks. While LLMs offer significant automation benefits, LLMs don’t have the human intuition to make complex decisions.

One of the biggest mistakes is over-rely on LLM outputs, even if they appear authoritative or conclusive. This phenomenon can lead to complacency or errors in judgment, particularly in dynamic and unpredictable cybersecurity environments.

A few recommendations cybersecurity teams can put in place to ensure LLM safety include:

• Initial 'Suggestion Only' Mode: Introduce LLMs in a mode where they provide advice or recommendations but do not autonomously enact changes or interact with systems without explicit human permission.
• Continuous Training and Awareness: Educate cybersecurity teams on the capabilities and limitations of LLMs, promoting a culture of continuous learning and adaptation to new technologies.
• Escalation Protocols: Establish clear protocols for escalating critical decisions to human operators, particularly in scenarios involving high-stakes security incidents or ambiguous LLM outputs.
• Ethical Guidelines and Governance: Develop ethical guidelines and governance frameworks that govern the use of LLMs in cybersecurity, ensuring alignment with organizational values and compliance requirements.

Establishing processes like the above can enable organizations to leverage LLMs for their operations while exercising caution mitigating risks associated with automated decision-making.

Image Credit: Sascha Winter/Dreamstime.com