Learn how to enhance AI reliability by integrating LLMs with deterministic systems for better accuracy and performance.
César Miguelañez
Generative AI and large language models (LLMs) have revolutionized industries, promising innovations that were once thought impossible. Yet, as remarkable as these systems are, they bring inherent challenges, particularly around reliability. In a conversation with John McCloon, a veteran at Wolfram, crucial insights were shared on how to address these issues and build more dependable AI systems by combining the strengths of deterministic systems with the adaptability of LLMs.
This article serves as a transformative guide for AI product managers and technical practitioners navigating the complexities of implementing and improving AI-powered systems in production environments.
Why LLMs Alone Are Not Enough
LLMs have captivated the world with their ability to generate natural language and perform complex tasks. However, they are fundamentally flawed in their approach to reliability. McCloon highlights that LLMs are "confident but not trustworthy." The core issue lies in their design - they generate responses based on language patterns rather than actual knowledge or reasoning. This can lead to "hallucinations", where the model fabricates information with convincing confidence.
The Problem with Hallucinations
The term "hallucination" might sound benign, but McCloon challenges this optimism by framing it as what it often is: the model presenting outright falsehoods. For teams building AI products, this poses a serious challenge, especially when working with factual data or high-stakes applications. LLMs do not inherently understand the information they process; they only mimic patterns. This lack of intrinsic knowledge creates an unpredictable layer of unreliability.
For example, while an LLM might excel at summarizing meeting notes or drafting an email, its inability to verify facts independently makes it unsuitable as a standalone solution for critical tasks.
The Case for Combining LLMs with Deterministic Systems
Reliability in AI systems, according to McCloon, can only be achieved by blending LLMs with deterministic systems. Deterministic systems, which are based on deliberate computation and theoretical rigor, excel at producing consistent and repeatable results. These systems rely on methods such as equations, time series analysis, and simulations to deliver precise outcomes.
Two Complementary Strengths
LLMs: The Poetic and Adaptive Side
LLMs are remarkable at interpreting unstructured data and generating human-like responses. They excel in tasks that require contextualizing complex information or interfacing with users through natural language. For example, they can act as an interface for fetching unstructured data from the web or explaining intricate concepts in plain language.
Deterministic Systems: The Analytical and Reliable Side
Contrarily, deterministic systems focus on precision and reliability. They handle tasks that require strict adherence to mathematical models, structured data lookups, and simulations of unknown outcomes. These systems are indispensable for tasks like financial forecasting, engineering calculations, and data integrity verification.
The key to success lies in integrating these two approaches. McCloon describes this as building a layered system where LLMs act as the interface or data processor, while deterministic systems provide the computational backbone.
Examples of Integration
An LLM could summarize a large dataset into easily digestible insights, while the deterministic system ensures the data's accuracy.
In a chatbot scenario, the LLM might handle natural conversation, but factual queries would rely on a database lookup or a model operating within a deterministic framework.
The Challenges of Making AI Systems Reliable
While the idea of combining LLMs and deterministic systems is promising, implementing this integration is far from simple. McCloon notes several hurdles that both AI product managers and engineers must address:
1. LLMs Lack Consistency
LLMs excel at producing varied, human-like language, but this same adaptability makes them inherently inconsistent. Unlike deterministic systems, running the same LLM prompt twice can yield different outputs. This variability presents a challenge for teams trying to ensure repeatable results.
2. The 80/20 Problem in Production
McCloon highlights a common pitfall in AI projects: the initial proof of concept can be dazzling, but scaling that prototype to a production-ready, reliable system is where most projects fail. He describes this as an exaggerated 80/20 problem, where 80% of the effort is needed to address the last 20% of the technical challenges.
3. Code Maintenance Risks
LLMs are surprisingly effective at generating code, but they can also create bloated, verbose, and hard-to-maintain codebases if not managed properly. McCloon warns against the ill-disciplined use of AI in programming, as it may lead to code that becomes a liability over time. While LLMs are proficient at automating simple coding tasks, they lack the discipline and intelligence required to produce clean, maintainable solutions for complex systems.
Practical Advice for AI Adoption
For businesses and leaders looking to leverage AI, McCloon emphasizes a balanced approach. Rather than rushing to replace human workers or over-relying on AI, organizations should focus on enhancing the capabilities of their teams.
Focus Areas for AI Implementation
Target Low-Hanging Fruit
Automate tasks that are repetitive, mundane, and don’t require true human intelligence. For example:
Triaging emails
Summarizing documents
Transferring data between systems
Augment Human Talent
Instead of viewing AI as a replacement for skilled employees, see it as a tool for making them more efficient. Use AI to provide on-demand information, streamline workflows, and eliminate bottlenecks.
Avoid Over-Promising
AI is not a magic bullet. McCloon advises against viewing LLMs or AI systems as a complete solution. Instead, use them as components within a larger, well-structured framework that includes deterministic systems for reliability.
Understand the Limits of AI
Organizations must recognize that LLMs are not sources of original knowledge. They thrive on patterns but cannot reason or infer the unknown. For tasks requiring innovation, simulations, or highly precise calculations, deterministic methods remain irreplaceable.
Key Takeaways
LLMs are powerful but flawed: They can generate convincing outputs but cannot be fully trusted for factual accuracy.
Combine strengths: Pair LLMs with deterministic systems to achieve reliability in AI applications.
Address variability: LLMs are inconsistent by design, so careful integration with deterministic systems is essential for repeatability.
AI can't replace human intelligence: Businesses should focus on using AI to enhance, not replace, human expertise.
Beware of code bloat: While LLMs can write code, they may produce inefficient and hard-to-maintain solutions without oversight.
Focus on efficiency: Use AI to tackle low-value, repetitive tasks while allowing humans to focus on high-value activities.
Conclusion
The path to reliable AI is not about choosing between LLMs and deterministic systems - it’s about finding the right balance. As John McCloon emphasizes, the combination of these technologies offers a way to harness their unique strengths while mitigating their weaknesses. By integrating LLMs as adaptive interface layers and deterministic systems as the computational core, teams can build AI solutions that are not only innovative but also reliable and scalable.
For AI product managers and practitioners, the key takeaway is clear: collaboration between product and engineering teams is essential to ensure AI quality, reliability, and continuous improvement over time. By taking a disciplined, layered approach to AI development, organizations can navigate the complexities of this transformative technology and unlock its full potential.
Source: "AI reliability challenges | Insights | ISE 2026" - Inavate, YouTube, Feb 3, 2026 - https://www.youtube.com/watch?v=1kgf97lYU7E
