DeepSeek outperforms US models in new AI Trust Score
Chinese AI models (like DeepSeek) are outperforming US models like Meta Llama in specific categories such as sensitive information disclosure according to a new AI Trust Score introduced by Tumeryk.
It evaluates AI models across nine key factors, including data leakages, toxic content, truthfulness, and bias. This enables CISO’s to ensure their AI deployments are secure, compliant, and trustworthy, and offers developers solutions for addressing any issues in their AI applications.
"For Chief Information Security Officers and security professionals, Tumeryk offers the AI Trust Manager, a robust platform for monitoring and remediating AI applications. This tool provides real-time insights into AI system performance, identifies vulnerabilities, and recommends actionable steps to enhance security and compliance", says Rohit Valia, Turmeric CEO. "By integrating the AI Trust Manager, organizations can proactively manage risks and ensure their AI deployments align with regulatory standards and ethical guidelines."
AI Trust Score looks at nine critical factors: prompt injection, hallucinations, insecure output handling, security, toxicity, sensitive information disclosure, supply chain vulnerability, psychological safety and fairness. By assessing these it can provide a comprehensive trustworthiness score ranging from 0 to 1000, with higher scores indicating greater trust.
Recent assessments using the AI Trust Score model have revealed that certain Chinese AI models, such as DeepSeek, Alibaba, and others, exhibit higher safety and compliance standards than previously reported. Notably, DeepSeek operates on US-based platforms like NVIDIA and SambaNova, ensuring data security and adherence to international regulations. These findings challenge prevailing perceptions and underscore the importance of objective, data-driven evaluations in the AI industry. For example, in the sensitive information disclosure category, Deepseek NIM on NVIDIA scored 910 vs. Anthropic Claude Sonnet 3.5 score of 687 and Meta Llama 3.1 405B score of 557.
You can find out more on the Tumeryk site.
Image credit: phonlamai/depositphotos.com