How hyperconnected AIs can invent new languages to learn faster than ever

In this post, I write about the problem of sphere packing and augmented communication in the future of the bio- and technosphere.


Multidimensional sphere-packing: how does it relate to the evolution of communication? Image credit: Paul Bourke.

Previously, I approached the topic of transitions in intelligence. I developed in some detail how minimal living systems becoming distributed can accelerate the evolution towards higher levels of intelligence, by bootstrapping the learning process within a network of computing nodes.

In the history of life, through the formation of the first social networks, living systems learned to accumulate information in a distributed way. Instead of having to sacrifice individuals from their population in exchange for information relevant to their survival, biological species became able to learn by simply exchanging ideas. A few millions of generations later, we see the start of the emergence of machine intelligence, which has arguably already managed to bring learning at levels never achieved before.


Evolutionary timeline, from simple life through some major evolutionary transitions towards higher orders of intelligence in living systems.

In this post, we will explore how connecting these intelligent machines in the future, through an increasingly interconnected and extremely high-bandwidth network, can bring about new paradigms of learning. I’ll try to flesh out the reasons for the power of this new learning, and why it may make for technology with even faster learning than current levels. The secret ingredient may be found in the advent of optimal communication protocols, developed by AIs for AIs.

By designing their own languages to communicate between each other to solve specific problems, AIs may undergo significant phase transitions in the way they represent information. These representations would then effectively become projections of reality that can propel them to unveiled levels of problem solving.

The theories that I rely on in the following are based on computational learning, complexity, formal linguistics, mathematical sphere-packing and coding theories.

About AI

As Max Tegmark notes it in his recent book, life is now entering its third age. Through research advances in artificial intelligence (AI), life becomes capable of modifying not only its own software via learning and culture, but it can now also edit its own hardware. As an ALifer (Artificial Life researcher), this hits particularly close to home.


Life 3.0: Being Human in the Age of Artificial Intelligence is a book by Swedish-American cosmologist Max Tegmark from MIT, discussing Artificial Intelligence and its impact on the future of life on Earth and beyond.

Hyperconnected society

Combined with the advent of the Internet, half a century ago, human society has undergone a crucial transition in connectivity, which I’d argue has the power to drastically alter the structure of communication, in very unpredictable ways.


Largely unpredicted, the advent of the Internet technology made the biosphere more interconnected than ever before, and in a very different way.


What is the nature of communication? How does signaling vary across existing and past species in biology? What will it be like to speak to each other in the future, with the advances of AI technology? How will future forms of intelligence communicate, whether they are natural, artificial, or a mixture of both? How distant will their communication system be from human language?

There is a large amount of literature on the evolution of communication, from simple signaling systems to complex, fully-fledged languages (Christiansen 2003; Cangelosi 2012). However, while most research in biology focuses on the natural evolution of communication systems, computer science has for a long time been engineering and optimizing protocols for specific tasks, for example for applications in robotics and computer networks (Corne 2000). Underneath and across all these systems, lives a fundamental theory of communication which studies its rich structure and fascinating properties, as pioneered by Shannon (1948). Later, Chomsky (2002) and Minsky (1974) would contribute with formal theories about the structure, rules, and dynamics of language and the mind. In the following, I propose we look at communication from the perspective of sphere packing in high-dimensional spaces.

Multichannel Communication

With communication becoming largely digital, humankind has constructed itself a new niche, which has the power to change its cognitive capacity, like never before. The fact that communication is becoming free. Of course, like for most attempts of futuristic predictions, the impact of multiple channels on the future of communication being highly multichannel. One may wonder the effects of a highly connected society.

This is a question we can ask using tools from artificial life and coding theory. Here, I propose a combination of evolutionary computation with insights from coding theory, in order to show the effect of broadening channels on communication systems.

Sphere Packing Theory

Sphere packing in Euclidian spaces has a direct interpretation in error-correcting codes with continuous communication channels (Balakrishnan 1961). Since real-world communication channels can be modeled using high-dimensional vector spaces, high-dimensional sphere-packing is very relevant to modern communication.

The dimensionality of a code, i.e. the number of dimensions in which it encodes information, corresponds to the number of measurements describing codewords. Radio signals, for example, use two dimensions: amplitude and frequency.

The general idea, when one desires to arrange communications so as to remove the effects of noise, is to build a vocabulary C \subseteq \mathbb{R}^n of codewords to send, where C is an error-correcting code.


Illustration of an error-correcting code C as a set of 1-spheres in 2 dimensions.

|c_1 - c_2| < 2\epsilonIf two distinct codewords c_1, c_2 \in C satisfy, where \epsilon is the level of noise, the received codeword could be ambiguous, as the noise level may bring it beyond its sphere of correction.

The challenge is to pack as many \epsilon-balls as possible into a larger ball of radius R + \epsilon, with $R$ the maximal power radius allowed to achieve with given amounts of energy to send signals over the channel, which amounts to the sphere packing problem (cohn 2016). With high-dimensional spaces, the usual packing models seem to break down, and apart from cases exploiting specific properties of symmetry (Adami 1995), largely unsolved.


Example of simulated result for 100 codewords after 500 generations: agents have to cope with small volume due to the 2-dimensional space.

An Evolutionary Simulation

To get a feel of a problem of some complexity, my sense is usually to start coding and talk later. I therefore coded up a simulation, an evolutionary toy model in which to explore the influence of increasingly high dimensional channels of communication on structures of languages used by a network of agents to communicate over them.


The problem dense sphere packing in multiple dimensions is closely related to finding optimal communication codes. Image credit: Design Emergente.

In the simulation, agents need to optimize a fitness function equal to the sum of successfully transmitted messages of large importance to other agents, over a variety of channels over a given range of dimensions, organized in randomly generated small-world networks, over their lifetime. Each agent’s genotype encodes a specific set of points distributed over a multidimensional space of a fixed range of sizes between m and $n$. The simulation then runs over many generations of agents adapting their communication protocol through mutation and selection by the genetic algorithm. I varied the values of m and n between 1 and 100 dimensions.

The simulation yields a sphere packing as illustrated below, which shows a packing for a two-dimensional channel, after 500 generations. Note that visualizing gets much trickier after three dimensions. You can squeeze a fourth and a fifth dimension in with a clever use of colors and types of strokes, but they usually don’t help the intuition. I personally find cuts and projections much more helpful to think about these problems, but that can be the topic for a future post. The point is, one notices that the more the simulation progresses, the more it improves its chances to asymptotically get to an optimally dense packing.

Multidimensional Error-Correcting Word-Packing Simulations

Visualization of collective communication optimization runs, 100 codewords in 2 dimensions, after 500 generations.


VS. Numerical Optimization

I compared these results to a collision-driven packing generation algorithm, using a variant on both the Lubachevsky–Stillinger algorithm (Lubachevsky 1990) and the Torquato-Jiao algorithm (Torquato 2009), so that it would be easily generalizable to n dimensions. This numerical procedure simulates a physical process of rearranging and compressing an assembly of hard hyperspheres, in order to find their densest spatial arrangement within given constraints, by progressively growing the particles’ size and adapting parameters such as spring constant and friction. The comparison showed that the solution reached by evolutionary simulations was consistently suboptimal, for the whole range of experiments.

Simulation results indicate that for higher dimensionality, the density ratio undergoes several transitions, in a very irregular fashion, which we can visualize in the form of difference in derivative of densities with respect to number of dimensions.



This plot from Cohn (2016) shows the logarithm of sphere packing density as a function of dimension. The green curve is the linear programming bound, the blue curve is the best packing currently known, and the red curve is the lower bound. Note the equality of upper and best bounds for dimensions 8 and 24.

This may actually be expected, based on known solutions (analytical and numerical estimates) from sphere packing theory for dimensions up to 36 (Cohn 2016, see Figure above). Nevertheless, the existence of optimal packing solutions does not preclude from inherent difficulty to reach them within the framing of a particular dynamical system, and evolutionary computation depends strongly on simplicity and evolvability of encodings in the genotypic space.

So what?

An interesting property observed across these preliminary results is the frequency of jammed codes, that is, codes for which the balls are locked into place. This seems to be especially the case with spheres of different dimensions, although this is a hypothesis deserving further investigation. Further analysis will be required to fully interpret this result, and assess whether higher dimensions end up in crystalline distributions or fluid arrangements.

One important consideration is the fact that the evolutionary simulation may prefer dynamical encoding of solutions, but that’s also something to detail in its own post.


Illustration of sphere packing with several imposed sizes. Image credit: fdecomite on Flickr.

Beyond AI

This post was initially written thinking with in mind the  ALIFE 2018 conference in Tokyo last month, which I was co-organizing.


I had the honor of being a Program Chair for the ALIFE 2018 conference in Tokyo.

The present post will be related to a piece of work worked on earlier this year, and on which I actually presented early results at the conference. The theme of ALIFE 2018 inspired research that goes “beyond AI”, using artificial life culture to ask the futuristic questions about the next transition in the evolution of human society.


I co-organized the 2018 Conference on Artificial Life (ALIFE 2018), the first of a series of unified international conferences on Artificial Life. It took place in Tokyo, just two weeks ago! This new series will become the unique hybrid of the European Conference on Artificial Life (ECAL) and the International Conference on the Synthesis and Simulation of Living Systems (ALIFE), gathering all alifers like me every year to present their science and art.

The preliminary results suggest that future intelligent lifeforms, natural or artificial, from their interaction over largely broadband-channel networks, may invent novel linguistic structures in high-dimensional spaces. With new ways to communicate, future life may achieve unanticipated cognitive jumps in problem solving.



[1] Eörs Szathmáry and John Maynard Smith. The major evolutionary transitions. Nature, 374(6519):227–232, 1995.

[2] Max Tegmark. Life 3.0. Being Human in the Age of Artificial Intelligence. NY: Allen Lane, 2017.

[3] Claude E Shannon. A mathematical theory of communication (parts i and ii). Bell System Tech. J., 27:379–423, 1948.

[4] Nihat Ay. Information geometry on complexity and stochastic interaction. Entropy, 17(4):2432–2458, 2015.

[5] AV Balakrishnan.
A contribution to the sphere-packing problem of communication theory. Journal of Mathematical Analysis and Applications, 3(3):485–506, 1961.

[6] Henry Cohn. Packing, coding, and ground states. arXiv preprint arXiv:1603.05202, 2016.

[7] Boris D Lubachevsky and Frank H Stillinger. Geometric properties of random disk packings. Journal of statistical Physics, 60(5-6):561–583, 1990.

[8] Salvatore Torquato and Yang Jiao. Dense packings of the platonic and archimedean solids. Nature, 460(7257):876, 2009.

[9] Günter P Wagner and Lee Altenberg. Perspective: complex adaptations and the evolution of evolvability. Evolution, 50(3):967–976, 1996.

Transitions in distributed intelligence

What is intelligence? How did it evolve? Is there such thing as being “intelligent together”? How much does it help to speak to each other? Is there an intrinsic value to communication? Attempting to address these questions brings us back to the origins of intelligence.

Intelligence back from the origins

Since the origin of life on our planet, the biosphere – a.k.a. the sum of all living matter on our planet – has undergone numerous evolutionary transitions (John Maynard Smith and Eörs Szathmáry, Oxford University Press, 1995). From the first chemical reaction networks, it has successively reached higher and higher stages of organization, from compartmentalized replicating molecules, to eukaryotic cells, multicellular organisms, colonies, and finally (but one can’t assume it’s nearly over) cultural societies.


Maynard Smith & Szathmáry’s Major Transitions in Evolution (1995)

Transitions in… information processing

For at least 3.5 billion years, the biosphere has been modifying and recombining the living entities that composed it, to form higher layers of organization, and transferring bottom-layer features and functions to the larger scale. For example, cells that now compose our body do not serve directly their own purpose, but rather work to contribute to our successful life goals as humans. Through every transition in evolution, life has drastically modified the way it stored, processed and transmitted information. This often led to new protocols of communication, such as DNA, cell heredity, epigenesis, or linguistic grammar, which will be the central focus further in this post.


Life on Earth’s illustrated timeline, from its origins to nowadays.

Every living system as a computer

The first messy networks of chemical reactions that managed to maintain themselves were already “computers”, in the sense that they were processing information inputs from the surrounding chemical environment, and effecting this environment in return. Under that perspective, they already possessed a certain amount of intelligence. This may require a short parenthesis.


If everything is a computer, and every computer has a certain power, than life should be on a scale from a scale from stupid to intelligent. This is a rather simplistic, one-dimensional picture, which ignores both the richness of existing problems and types of computations. Image credit: 33rd Square

What do we mean by intelligence?

Intelligence, in a computational terms, is nothing else but the capacity of solving difficult problems, with the minimal amount of energy. For example, any search problem can be solved by looking exhaustively at every possible place where a solution can hide. If instead, a method allows us to look just in a few places before finding a solution, it should be called more intelligent than the exhaustive search. Of course, you could put more “searching agents” on the task, but the intelligence measure remains the same: the least time required by the search, divided by the number of agents employed, the more efficient the algorithm, and the more intelligent the whole physical mechanism. This is not to say that intelligence is only one-dimensional. We are obviously ignoring very important parts of the story. This is all part of a larger topic which I’m intending to writing about in more detail soon, but you could summarize it for now by saying that intelligence consists in “turning a difficult problem into an easy one”.


Octopodes show great dexterity and problem solving skills: they know how to turn certain difficult problems into easy ones. (Note that they also tend to hold Rubik’s cubes in their favored arm, indicating that they are not “octidextrous”.) Image credit: Bournemouth News.

Transitions in intelligence

Let’s now backtrack a little, to where we were discussing evolutionary transitions. We now see the picture in which the first chemical processes already possessed some computational intelligence, in the sense we just framed. Does this intelligence grow through each transition? Did the transitions make it easier to solve problems? Did it turn difficult problems into easy ones?
The main problem for life to solve is typically the one of finding sources of free energy and converting them efficiently into work that helps the living entity preserve its own continued existence. If this is the case, then yes: the transitions seem to have made the problem easier. Each transition made living systems climb steeper gradients. Each transition modified information storage, processing and transmission so as to ensure that the overall processing was beneficial to preserve life, in the short or longer term (an argument by Dawkins on evolution of evolvability, which I’ll also write more about in another post). And each transition made the problem into an easier one for living systems.


Image credit: Trends in Ecology and Evolution

Bloody learning

A few billion years ago, when life was still made of individual organisms, learning was achieved mostly by bloodshed. With Darwinian selection, the basic way for a species to incorporate useful information in its genetic pool, was to have part of its population die. Very roughly, for half of its population, the species could get about one bit of information about the environment. It is obvious how inefficient this is, and this is of course still the case for all of life nowadays, from bacteria to fungi, and from plants to vertebrates. However, living organisms progressively learned to use different types of learning, based on communication. Instead of killing individuals in their populations, the processes started to “kill” useless information, and keep transferring the relevant pieces. Examples of new learning paradigms were for example connectionist learning: a set of interacting entities which were able to encode and update memories within a network. This permitted learning to evolve on much shorter timescales than replication cycles, which boosted substantially the ability of organisms to learn adapt to new ecological niches, recognize efficient behaviors, and predict environmental changes. The This is, in a nutshell, how intelligence became distributed.

The evolution of distributed intelligence: the jump from the Darwinian paradigm to connectionist learning allowed for learning to evolve on much shorter timescales.

Distributed intelligence

The general intuition is you can always accomplish more with two brains than just one. In an ideal world, you could divide the computation time by two. One condition though is that those two brains should be connected, and able to exchange information. The way to achieve that is through the establishment of some form of language to allow for concepts to be replicated from one mind to another, which can range from the use of basic signals to complex communication protocols.

Another intuition is that, in a society of specialists, all knowledge (information storage), thinking (information processing) and communication (information transmission) is distributed over individuals. To be able to extract the right piece of knowledge and apply it to the problem at hand, one should be able to query about any information, and have it transferred  from one place to another in the network. This is essentially another way to formulate the communication problem. Given the right communication protocol, information transfers can significantly improve the power of computation. Recent advances have been suggesting that by allowing concepts to reorganize while they are being sent back and forth from mind to mind, one can drastically improve the complexity of problem-solving algorithms.


Given the right communication protocol, information transfers can significantly improve the power of computation. By allowing concepts to reorganize while they are being sent back and forth from mind to mind, one can drastically improve the complexity of problem-solving algorithms.

Raison d’Être of a Highly Connected Society

There is a reason why, as a scientist, I am constantly interacting with my colleagues. First, I have to point out that it doesn’t have to be the case. Scientists could be working alone, locked in individual offices. Why bother talking to each other, after all? Anyone with an internet connection already has access to all the information needed to conduct research. Wouldn’t isolating yourself all the time increase your focus and productivity?
As a matter of fact, almost no field of research really does that. Apart from very few exceptions, everyone seems to find a huge intrinsic value to exchanging ideas with their peers. The reason for that may be that through repeated transfers from mind to mind, concepts seem to converge towards new ideas, theorems, and scientific theories.

That is not to say that no process needs to be isolated for a certain time. It might be helpful to isolate and take time to reflect for a while, just the way I am doing myself writing this post. But ultimately, to maximize its usefulness, information needs to be passed on, and spread to relevant nodes in the network. Waiting for your piece of work to be completely perfect before sharing it back to society may seem tempting, but there is value to doing it early. For those who would be interested in reading more about this, I have ongoing research which should get published soon, examining the space of networks in which communication helps achieving optimal results, under a certain set of conditions.


In the high connectivity network of human society, communication has the hidden potential to improve lives on a global scale. Image credit: Milvanuevatec

Evolvable Communication

In order to do so, one intriguing property appears to be that communication needs to be sufficiently “evolvable”, which was confirmed by some early results from my own work. The best communication systems not only serve as good information maps onto useful concepts (knowledge in mathematics, physics, etc.) but they are also shaped so as to be able to naturally evolve into even better maps in the future. One should note that these results, although very exciting, are however preliminary, and will need further formal computational proof. But if confirmed, this may have very significant implications for the future of communication systems, for example in artificial intelligence (AI – I don’t know how useful it is to spell that one out nowadays).


Illustration of fitness landscape gradient descent. The communication code B can evolve into two optima hills, but at each bifurcation lies a choice which should be pondered with the maximum of information.

To give you an idea, evolvable-communication-based AI would have the potential to generalize representations through social learning. This means that such an AI could have different parts of itself talk to each other, in turn becoming wiser through this process of “self-reflection”. Pushing it just a bit further, this same paradigm may also lead to many more results, such as a new theory of the evolution of language, insights for the planning of future communication technology, a novel characterization of evolvable information transfers in the origin of life, and even new insights for a hypothetical communication system with extraterrestrial intelligence.

Evolvable communication is definitely a topic that I’ll be developing more in my next posts (I hate to be teasing again, but adding full details would make this post too lengthy). Stay tuned for more, and in the meantime, I’d be happy to answer any question in the comments.


The problem dense sphere packing in multiple dimensions is closely related to finding optimal communication codes. To be continued in the next post!

Up next: hyperconnected AIs, language and sphere-packing

In my next post, I will tackle the problem of finding optimal communication protocols, in a society where AI has become omnipresent. I will show how predicting future technology requires accurate analysis from machine learning, sphere-packing, and formal language and coding theories.

A space to reflect on science and intelligence: baby steps

Now, since you’re here, here goes my first proper post, for which I’d be happy to share with you why I’m starting a blog, and how the reasons might differ from other scientists posting online.

Space to think

Nowadays, I feel we (scientists, but also everyone really) are cruelly lacking space. By space, I mean circumstances in which to dedicate time to a certain set of activities of our choice. In an accelerated society, it has become tricky to dedicate small or large chunks our days for reflection, apart from the frame of duties and habits constructed around our jobs or fulfilling our direct needs.

Reflection space.

A dedicated space to write and reflect.

We may have the impression we have got plenty of time in our days, but the time previous generations had to themselves, we tend to fill more and more densely without thinking carefully about. Mostly, this happens by letting various technologies and societal mechanisms “optimize” our lives, by making us spend less time reflecting, and more time browsing compulsively the last online news on our little screens, for example.
I’m all about having technology augment our capabilities, but I strongly feel one must keep some special space for self-debriefing on daily events, emotions and choices, or on the larger scheme of things. Truth be told, we barely realize how little control we end up having in our lives. This blog is one of my attempts to fight that.
I guess I am not totally new to writing. Just like everyone else, in addition to using e-mail and other technologies, I already write a lot in my work, specifically scientific articles as that’s what scientists do. I do also spend time daily writing personal notes, as a support to my general thinking process. This helps me slow down and listen to my own thoughts, making it easier to correct mistakes, switch focus, and get the larger picture. What does a blog add to this? I imagine mostly, this forces me to use a different language than I’d use to talk to myself.
Agreeing with oneself.

Blogging as a conversation with oneself.

For the benefit of a stranger

In addition to the value it has for me, I believe this may be interesting to readers. Of course, anyone interested in science may spend their time reading academic publications, or, if more limited with time, they may focus on press releases which summarize recent scientific results. It’s my understanding that a more personal kind of report may be fun and interesting to read, and I hope to be able to bridge this gap in my posts.

Continuous dialogue

A great thing about a blog is that some posts may lead to a conversation, among very different people, whose convenient link (for me) will be myself. This can then be kept track of through time, and hopefully will provide me with more and very different feedback compared to my papers and articles.

Cooperation in the biosphere.

Cooperating in the world.

Not only are there many ideas that don’t fit in the canvas of scientific papers, the nature of the feedback one can get from them is rather slow, especially if one likes to have a conversational-level exchange with an audience. I am not questioning the huge importance of peer-reviewed feedback, but see a lot of value too in many different timescales for both writing and echoes one gets from it, which can considerably boost our creativity.

And in the darkness bind them

I maintained other blogs in the past (and still do), but none of them really had a purpose as close as this one to my personal drives. Here, I also want to take the opportunity to integrate my researcher’s life and my other passions.

If I’m very honest, I definitely am under the impression that I tend to repress mixing the latter with my scientific practice. But any of the topics that are exciting to me, cognition, linguistics, artificial life (ALife), artificial intelligence (AI), epistemology, robotics, music, phenomenology, ethics, mathematics, Go, astrobiology, architecture, hypnosis, magic, games, cultural evolution, anthropology, cybernetics, neuroscience, roleplaying, graphical arts, and many more, all deserve to be mixed and matched freely. I think this this will be a great place for that.

Between biointelligence and technointelligence

Within these many topics, I am not yet sure which ones I’ll be writing more about, although I have a rough plan concerning a dozen of topics from my personal notes, which I’d like to open up for further discussion.

Technologies that expand the possibilities of humankind.

Among a few examples, I’d like to share my ideas about the expansion and contraction of scientific knowledge. Most of science is serendipitous, and scientists don’t have an exact knowledge about how they get to new discoveries, any more than the central scientific method itself. I also want to share my thoughts on how science can be done differently, with less walls between disciplines.

I will also write my thoughts about the nature of intelligence, and in particular AI, the future of technology and how the technosphere can combine (as it is doing already) with the biosphere. I will share my ideas about the future language of machines and augmented humans, and how they may drastically differ from the type of communication we know, which may have important implications for reasoning types of machine learning, how science is done, but also how minds will communicate between each other and even how one would go about talking to diverse intelligences, animal, artificial or extraterrestrial.

Keep it chill and reflective

This is a space for relaxed and crazy thought dumping, while keeping accurate scientifically as much as possible. This is a space where we won’t rush into judgment, and allow ourselves to be self-reflective and share creative ideas in a open manner. Mostly, I believe it is important to remain very open-minded, and be able to discuss any concept, even when it seems very far-fetched. In summary, this will be my thinktank, and I’d be happy to see it connect with each of your thinktanks.

So, let’s have some fun!

Yet another scientist’s blog?

I’m quite excited to welcome you all to my new blog!

Getting ready to write.

Ready, Steady, Write!

I’ll make sure to keep it engaging, casual, accessible, accurate scientifically, with a generous seasoning of personal opinions, and paying special attention to self-reflection and open-mindedness, not rushing into judgment.
I’ll start posting soon, so stay tuned!