Why AI Hallucinations Still Plague Us: Jensen Huang Discusses the Next Decade of AI

In an insightful conversation at the Hong Kong University of Science and Technology, Jensen Huang, CEO of Nvidia, discussed some of the most pressing challenges and developments in artificial intelligence (AI), as well as the role his company plays in shaping the future of this rapidly evolving field. Huang’s thoughts on the ongoing AI hallucination problem, the massive computational needs of AI, and Nvidia’s significant contributions to the tech industry offer a fascinating glimpse into the future of machine learning and AI technologies.

The Hallucination Problem: Years Away from a Solution

One of the most intriguing aspects of the discussion was Huang’s candid take on the AI hallucination issue. AI hallucination refers to a phenomenon where AI systems create false information or “hallucinations” to fill in gaps in their knowledge. It’s a significant problem, particularly when it comes to applications where accuracy is critical, such as medical diagnostics, legal advice, or financial analysis.

Huang explained that solving the AI hallucination problem is still “several years away.” While current AI models, including Nvidia’s offerings, are making progress in understanding and generating information, they are far from perfect. “Today, the answers that we have are the best that we can,” Huang acknowledged, but he stressed that the goal is to develop AI that can deliver more trustworthy, reliable answers. He added, “We need to get to a point where the answer you get is not the best we can provide, but one that you can largely trust.” This level of trust is crucial for AI systems to gain widespread adoption in industries where human life and significant decisions are at stake.

However, Huang noted that the path to this goal is not simple. It involves overcoming not just the limitations of the current generation of AI but also the inherent computational challenges. AI needs to process vast amounts of data, and as Huang pointed out, this requires significant computing power, which Nvidia is constantly working to improve.

The Growth of AI Computing Power: A Million Times Over

Huang also addressed the exponential growth in AI’s computing requirements. In the past decade, the demand for computational power has grown at an astounding rate. The host of the interview pointed out that AI computing needs have increased fourfold every year, and over the span of ten years, that results in a mind-boggling increase—about a million times more computing power.

While Huang did not predict that this trend would continue at the same pace over the next decade, he did explain that Nvidia’s role has been crucial in managing this surge in demand. “What Nvidia has contributed is that we’ve taken the marginal cost of computing and reduced it by a million times,” Huang said, noting that this cost reduction has made AI research more accessible and widespread.

This is perhaps Nvidia’s single greatest contribution to AI. Huang pointed out that by drastically reducing the cost of AI training and machine learning, Nvidia has enabled researchers and developers to use these powerful tools without the same financial barriers that existed in the past. This dramatic shift has made it possible for AI to move from a niche technology to one that is central to industries worldwide.

However, as AI continues to evolve, Huang emphasized the importance of increasing computational capabilities. He stated that “in the meantime, we have to keep increasing our computation” to keep up with the expanding scope of AI. The evolution of AI is deeply tied to the ability to process more data faster and more accurately, and Nvidia’s ongoing advancements in GPU technology are at the forefront of this battle.

Nvidia’s GPUs: From Expensive to ‘Practically Free’

Despite Nvidia’s contributions to making AI more accessible, Huang was also asked about the high cost of Nvidia’s AI GPUs, which are still seen as expensive by many. Huang responded by offering a compelling perspective: without Nvidia, the cost of AI GPUs would be “a million times more expensive.” He explained that while the current price may seem high, it is a significant discount compared to what it would have been if Nvidia hadn’t made such breakthroughs in the last decade. He quipped, “It’s practically free!”

This cost reduction has fundamentally changed how AI researchers approach their work. What once would have been considered prohibitively expensive is now within reach for many institutions and organizations, significantly accelerating the pace of innovation in AI.

AI’s Development Process: Pre-training, Post-training, and Test Time Scaling

Beyond the hardware and computational challenges, Huang also provided insight into the development process of AI. He outlined three key stages in the training of AI systems: pre-training, post-training, and test time scaling.

1. Pre-training: Huang likened this phase to going to college, where AI systems consume vast amounts of data to learn and develop foundational knowledge. Pre-training is a crucial step, as it enables the AI to understand the world and its basic principles.
2. Post-training: In this phase, the AI focuses on refining specific skills, often through reinforcement learning. This process involves both human feedback and AI-generated feedback, which helps the system improve its performance in specific tasks.
3. Test Time Scaling: Huang described this final stage as “thinking.” In this phase, AI breaks down complex problems step by step, simulating various outcomes to arrive at a solution. This stage is far more intricate and can involve several iterations and adjustments to reach the optimal answer. Huang noted that the longer an AI spends “thinking,” the more accurate and reliable its responses become.

Huang’s Personal Story: A CEO by 30

The interview also delved into Huang’s personal journey, particularly his early days as an entrepreneur. Huang shared a humorous anecdote about how he wooed his wife, Lori, by promising that he would be a CEO by the age of 30. “That was my pickup line; I didn’t mean it,” Huang laughed, recounting how he had made the promise at the age of 20 without any real plan for how to achieve it. Despite his lack of a concrete strategy at the time, Huang did indeed become the CEO of Nvidia before turning 30, turning a playful remark into a defining milestone in his life and career.

He also shared his advice for aspiring entrepreneurs, saying that sometimes the best course of action is simply to take bold steps, even if they seem uncertain at the time. Huang’s story of determination and self-confidence serves as a reminder of how ambition and perseverance can lead to extraordinary achievements.

The Future of AI: More Work to Be Done

Huang’s remarks paint a picture of a future where AI continues to evolve, but also one where challenges such as hallucinations and computing demands must be addressed. His candid acknowledgment that AI systems are not yet fully reliable highlights the ongoing work that needs to be done. However, Nvidia’s role in reducing the cost of computing, advancing GPU technology, and supporting AI’s rapid development cannot be overstated. As AI becomes increasingly integrated into various industries, the need for powerful computing systems and reliable AI responses will only grow, and Nvidia’s contributions will remain central to this evolution.

The next decade will likely bring further breakthroughs in AI, as researchers and developers continue to push the boundaries of what’s possible. For now, Huang’s insights provide a clear understanding of where the industry stands and the work still to be done before AI can fully reach its potential.