[Accepted to the 16th Annual Conference on Artificial General Intelligence] We bring together foundational theories of meaning and a mathematical formalism of artificial general intelligence to provide a mechanistic explanation of meaning, communication and symbol emergence. We establish circumstances under which a machine might mean what we think it means by what it says, or comprehend what we mean by what we say. We conclude that a language model such as ChatGPT does not comprehend or engage in meaningful communication with humans, though it may exhibit complex behaviours such as theory of mind.

Software’s effect upon the world hinges upon the hardware that interprets it. This tends not to be an issue, because we standardise hardware. AI is typically conceived of as a software ”mind’‘ running on such interchangeable hardware. This formalises mind-body dualism, in that a software ”mind’‘ can be run on any number of standardised bodies. While this works well for simple applications, we argue that this approach is less than ideal for the purposes of formalising artificial general intelligence (AGI) or artificial super-intelligence (ASI). The general reinforcement learning agent AIXI is pareto optimal. However, this claim regarding AIXI’s performance is highly subjective, because that performance depends upon the choice of interpreter. We examine this problem and formulate an approach based upon enactive cognition and pancomputationalism to address the issue. Weakness is a measure of plausibility, a ”proxy for intelligence’‘ unrelated to compression or simplicity. If hypotheses are evaluated in terms of weakness rather than length, then we are able to make objective claims regarding performance (how effectively one adapts, or ”generalises’‘ from limited information). Subsequently, we propose a definition of AGI which is objectively optimal given a ”vocabulary” (body etc) in which cognition is enacted, and of ASI as that which finds the optimal vocabulary for a purpose and then constructs an AGI.

Simplicity is held by many to be the key to general intelligence. Simpler models tend to “generalise”, identifying the cause or generator of data with greater sample efficiency. The implications of the correlation between simplicity and generalisation extend far beyond computer science, addressing questions of physics and even biology. Yet simplicity is a property of form, while generalisation is of function. In interactive settings, any correlation between the two depends on interpretation. In theory there could be no correlation and yet in practice, there is. Previous theoretical work showed generalisation to be a consequence of “weak” constraints implied by function, not form. Experiments demonstrated choosing weak constraints over simple forms yielded a 110-500% improvement in generalisation rate. Here we show that all constraints can take equally simple forms, regardless of weakness. However if forms are spatially extended, then function is represented using a finite subset of forms. If function is represented using a finite subset of forms, then we can force a correlation between simplicity and generalisation by making weak constraints take simple forms. If function is determined by a goal directed process that favours versatility (e.g. natural selection), then efficiency demands weak constraints take simple forms. Complexity has no causal influence on generalisation, but appears to due to confounding.In Press: Accepted for publication in the Proceedings of The 17th Conference on Artificial General Intelligence, 2024

To make accurate inferences in an interactive setting, an agent must not confuse passive observation of events with having participated in causing those events. The “do” operator formalises interventions so that we may reason about their effect. Yet there exist at least two pareto optimal mathematical formalisms of general intelligence in an interactive setting which, presupposing no explicit representation of intervention, make maximally accurate inferences. We examine one such formalism. We show that in the absence of an operator, an intervention can still be represented by a variable. Furthermore, the need to explicitly represent interventions in advance arises only because we presuppose abstractions. The aforementioned formalism avoids this and so, initial conditions permitting, representations of relevant causal interventions will emerge through induction. These emergent abstractions function as representations of one’s self and of any other object, inasmuch as the interventions of those objects impact the satisfaction of goals. We argue (with reference to theory of mind) that this explains how one might reason about one’s own identity and intent, those of others, of one’s own as perceived by others and so on. In a narrow sense this describes what it is to be aware, and is a mechanistic explanation of aspects of consciousness.

Biological intelligence is more sample-efficient than artificial intelligence (AI), learning from fewer examples. Here we answer why. Given data, there can be many policies which seem “correct” because they perfectly fit the data. However, only one correct policy could have actually caused the data. Sample-efficiency requires a means of discerning which. Previous work showed sample efficiency is maximised by weak-policy-optimisation (WPO); preferring policies that more weakly constrain what is considered to be correct, given finite resources. Biology’s sample-efficiency demonstrates it is better at WPO. Established theories, such as active inference, do not explain how. To understand how, we formalise the multiscale-competency-architecture (MCA) observed in biological systems, as a sequence of nested agentic-abstraction-layers. We show that WPO at low levels enables synthesis of weaker policies at high. We call this multiscale-causal-learning, and argue this is how we might construct more scale-able, sample-efficient and reliable AI. Furthermore, a sufficiently weak policy at low levels is a precondition of collective policy at higher levels. The higher level collective “identity” is lost if lower levels use an insufficiently weak policy (e.g. cells become isolated from the collective informational structure and revert to primitive behaviour). This has implications for biology, AI-safety, and philosophy.

If A and B are sets such that A is a subset of B, generalisation may be understood as the inference from A of a hypothesis sufficient to construct B. One might infer any number of hypotheses from A, yet only some of those may generalise to B. How can one know which are likely to generalise? One strategy is to choose the shortest, equating the ability to compress information with the ability to generalise (a “proxy for intelligence”). We examine this in the context of a mathematical formalism of enactive cognition. We show that compression is neither necessary nor sufficient to maximise performance (measured in terms of the probability of a hypothesis generalising). We formulate a proxy unrelated to length or simplicity, called weakness. We show that if tasks are uniformly distributed, then there is no choice of proxy that performs at least as well as weakness maximisation in all tasks while performing strictly better in at least one. In other words, weakness is the pareto optimal choice of proxy. In experiments comparing maximum weakness and minimum description length in the context of binary arithmetic, the former generalised at between 1.1 and 5 times the rate of the latter. We argue this demonstrates that weakness is a far better proxy, and explains why Deepmind’s Apperception Engine is able to generalise effectively.

Please note this poster and abstract are to be presented at ASSC27 in Tokyo. The full paper will be posted here soon. The hard problem of consciousness (Chalmers, 1995) asks why there is something it is like to be a conscious organism. We address this from 1st principles, by constructing a formalism that unifies lower and higher order theories of consciousness (Morin, 2006). We begin by assuming that humans are biological organisms that aim for survival at the fundamental level. We then assume pancomputationalism and hold that the environment learns organisms that exhibit fit behaviour via the algorithm we call natural selection. Selection learns organisms that learn to classify causes, facilitating adaptation. Recent experimental and mathematical computer science elucidates how (Bennett, 2023). For example, it has been shown that scaling this capacity implies progressively higher orders of “causal identity” facilitating reafference (Merker, 2005) and P-consciousness, then A-consciousness (Block, 1995), then meta-self-awareness (Morin, 2006). In this paper we suggest that we can use this framework to address the hard problem in precise, computational terms. First, we deny that there can always exist a philosophical zombie as capable in all respects as a P-conscious being. This is because a variable presupposes an object to which a value is assigned. Whether X causes Y depends on the choice of X, so causality is learned by learning X such that X causes Y, not by presupposing X and then learning if X causes Y (presupposing abstractions reduces sample efficiency). However, learning is a discriminatory process that requires states be differentiated by a given value. Without objects, there is only quality. By this we mean an organism is attracted to or repulsed by a physical state (i.e. “stay alive”, “die”). Learning reduces quality into objects by constructing policies classifying causes thereof. Where selection pressures require an organism classify its own interventions, that policy (i.e. a “1st-order-self”) has a quality that persists across interventions, and so there is something it is like to be that organism. Thus organisms may have P-consciousness because it allows them to adapt with greater sample efficiency. We then argue neither P nor A-consciousness in isolation are remarkable, but when P-consciousness combines with A-consciousness we may obtain “H-consciousness'' (which (Boltuc, 2012) argued is the crux of the hard problem). We speculate that this occurs when natural selection pressures require organism X infer organism Y's prediction of X's interventions (a “2nd-order-self” approximating intent). We suggest then that A-consciousness is the contents of 2nd-order-selves. In this paper we will show how by predicting another's predictions of one's own 1st-order-self it becomes possible to know what one knows and feels, and act upon this information to communicate meaning in the Gricean sense (recognising communicative intent). We will formally present the psychophysical principle of causality, namely the process of learning and acting in accord with a hierarchy of causal identities that simplify the environment into classifiers of cause and effect. We will then show how this new framework ‘dissolves’ the hard problem of consciousness by simply going one level down and starting with the most fundamental drive of all living biological systems such as humans: stay alive!

Artificial general intelligence (AGI) may herald our extinction, according to AI safety research. Yet claims regarding AGI must rely upon mathematical formalisms – theoretical agents we may analyse or attempt to build. AIXI appears to be the only such formalism supported by proof that its behaviour is optimal, a consequence of its use of compression as a proxy for intelligence. Unfortunately, AIXI is incomputable and claims regarding its behaviour highly subjective. We argue that this is because AIXI formalises cognition as taking place in isolation from the environment in which goals are pursued (Cartesian dualism). We propose an alternative, supported by proof and experiment, which overcomes these problems. Integrating research from cognitive science with AI, we formalise an enactive model of learning and reasoning to address the problem of subjectivity. This allows us to formulate a different proxy for intelligence, called weakness, which addresses the problem of incomputability. We prove optimal behaviour is attained when weakness is maximised. This proof is supplemented by experimental results comparing weakness and description length (the closest analogue to compression possible without reintroducing subjectivity). Weakness outperforms description length, suggesting it is a better proxy. Furthermore we show that, if cognition is enactive, then minimisation of description length is neither necessary nor sufficient to attain optimal performance. These results undermine the notion that compression is closely related to intelligence. We conclude with a discussion of limitations, implications and future research. There remain several open questions regarding the implementation of scale-able general intelligence. In the short term, these results may be best utilised to improve the performance of existing systems. For example, our results explain why Deepmind’s Apperception Engine is able to generalise effectively, and how to replicate that performance by maximising weakness. Likewise in the context of neural networks, our results suggest both limitations of “scale is all you need”, and how those limitations can be overcome.