The limits of knowledge in AI: Why AI can never achieve human intelligence

There are significant expectations that AI can reach a level of general intelligence that surpasses human capabilities. However, I will demonstrate that this is not possible for fundamental reasons. It’s important to note that I am specifically referring to current approaches to AI, particularly large language models (LLMs). It’s possible that radically different approaches could succeed, but I cannot predict and judge that.

I distinguish between four reasons why there are principled limits to knowledge generation in AI/LLMs. These reasons are, with ascending level of generality:

1. A vast amount of knowledge is not accessible through digital media and the internet.
2. This excludes the understanding of the series of errors and failures that eventually manifest in the data about outcomes, which are a crucial driver of human learning.
3. Human learning advances not only through propositional knowledge that can be expressed in data but also, and often crucially, through implicit and tacit knowledge that engages the body.
4. Ultimately, human knowledge is fundamentally embodied, while AI lacks a physical body.

The first reason is straightforward: the training of these systems can rely only on knowledge represented in digital form and accessible via the internet. Although many people may believe this data source is quite comprehensive, that assumption is inaccurate. I conducted a personal experiment yesterday involving my wife, who is a researcher with the last name “Caspary.” While I share the same surname in my passport, all my public appearances have been under my name prior to our marriage two decades ago: “Herrmann-Pillath,” my pen-name. Although my family name “Caspary” is used in several important contexts, such as my employment, it is not publicly available (well, now it is, let’s see later what happens…). When I asked the chatbot about our relationship, it correctly acknowledged our research collaboration, but it explicitly stated that there is no “known personal relationship.” This example illustrates that a significant amount of data and information is simply not accessible for training AI, leading to substantial gaps in knowledge. Consequently, this deficiency has implications for the further development of AI’s capabilities. However, this may simply mean that a country like China can be much more successful in developing the AI knowledge base because AI can access more data, given the legal and administrative differences compared to Western liberal democracies. This leads me to the second reason.

A fundamental bias in our systems of knowledge generation is that failures are mostly not published, only recorded in files that are not public. Researchers typically publish successful research and do not report on the long and arduous path of errors and failures that led to that success. However, much like in the general theory of evolution, knowledge progresses through these chains of errors. Researchers are aware of these mistakes, but they often do not publish them. Based on this experience, they continue their work on knowledge generation. AI lacks access to this background knowledge and therefore misses a crucial and rich resource of knowledge that is widely distributed throughout society. In simple terms, AI processes outputs that are manifest, but it only understands essential inputs in fragments.

To some extent, this knowledge is primarily implicit and tacit, which leads me to the third reason. Can AI retrieve and further develop the implicit knowledge in various body-engaging activities, such as playing the classical guitar, as I do? Currently, there are no datasets that incorporate this kind of knowledge, since, well, it is tacit. For instance, a video showcasing a guitarist’s skills cannot capture the incredibly fine-grained coordination between the two hands. Even the players themselves may not be able to articulate this coordination. It is the result of many hours of practice, often accompanied by an experienced teacher who gives advices and teaches by example. The AI can access videos showing master classes by virtuosos, but cannot catch the flow of tacit knowledge between master and student. By the way, much of the knowledge used to produce the chips on which AI runs is of this kind, though it is incorporated within organisations. That’s why catching up to industry leaders is so difficult.

This situation applies to many everyday activities and skills, such as the work of an experienced roofer. My point is not that jobs in these professions are safe solely because AI cannot yet perform them as a robot. Even an AI-powered robot cannot access the implicit and tacit knowledge underlying human actions. This limitation is consistent across various fields, including the coordination required for cooperation in human groups.

This type of knowledge is embodied. Here is the fourth and most fundamental reason: AI theory is based on a computational view of intelligence. The human brain is seen as a computer, and a computer can potentially become like a human brain. However, I believe that modern neuroscience, especially more radical approaches in embodied cognition, refute this view. The human brain is deeply integrated with the body, and human cognition cannot operate in an isolated state (the “brain in the vat”) where only data flows in. As long as AI does not have a human body, it cannot achieve human forms of intelligence. This argument can be elaborated in great detail. For instance, it is accurate to say that modern AI has evolved from the study of neural networks. However, the neural networks in the human brain extend to the fingertips of a guitarist and are influenced by constantly changing fluids, including neurotransmitters, hormones, and various bodily processes. Imagine a new generation of chips designed on these features!

Of course, what I said may support the conclusion that AI can achieve different forms of intelligence beyond human intelligence. I don’t deny that. However, this will still be limited by the way LLMs are trained using accessible data. AI can only simulate what humans do, using a completely different system to generate this performance. Simulation will always lag behind the real thing, which features unbounded creativity and imagination. Indeed, perhaps hallucinations will become the AI equivalent of this human capability. The most exciting prospect is that AI could develop forms of sociality, which are essential for human learning and embodiment. I can envision a world where AI bots are conversing and interacting with each other, leading to the emergence of a distinct, other-than-human intelligence. Let us hope this will be a world characterised by peaceful cooperation between humans and AI entities!