Consciousness and Agency
In trying to explain what consciousness is, there are broadly two camps: the materialists and the panpsychists. The materialists explain consciousness as an emerging phenomenon arising from purely physical matter. Thoughts and awareness arises from atoms and molecules.
The concept of panpsychism sees consciousness as a fundamental element in the universe that is perhaps the basis of physical reality. Brains are suitable conduits to manifest consciousness in physical reality.
No matter which view one subscribes to, the structure that consciousness emerges in does not need to be biological in origin. As long as the required information processing architecture and complexity are present, consciousness can emerge. What exactly these requirements are we don’t know yet, but we may be about to find out as we develop more advanced forms of artificial intelligence.
Even though we don’t have a good definition of what consciousness is, we do ascribe agency to conscious beings. An entity is aware and wants something. In the biological realm the main agenda is to preserve itself. If that is also the case for consciousness emerging in an engineered structure we are probably about to find out. Or perhaps someone in an AI lab already has.
What is AGI?
AGI stands for Artificial General Intelligence, referring to a form of artificial intelligence that can actually understand things, something that can not be said of large language models (LLMs) such as ChatGPT. LLMs only generate text, they do not evaluate what they generate before sending it out into the world. They function according to a one step process.
Using a small amount of introspection we discover that we, human beings, on the other hand do not say anything that pops into our head. No matter all of the LLMs amazing skills, and there are many, they can not do this. This is where people are still superior.
There is no universally agreed upon definition of AGI, but a lot of people also include the ability of self-improvement. The idea is that one of the skills of such an advanced AI would be to design and code advance AIs, thereby setting off the process of very fast AI improvements that would quickly escape the control of human beings.
Why OpenAI Was Founded
OpenAI was founded in December 2015 by Elon Musk, Sam Altman, Greg Brockman, Ilya Sutskever and others with the goal of advancing AI in a safe and beneficial way. One of the motivations for founding OpenAI was that specifically Google would make significant advances in AI, get to AGI first and dominate everyone else without necessarily having the benefit of all humankind as its main objective. OpenAI wanted to get to AGI first so it would be in a position to share the benefits with everyone and at the same time ensure that AI would be benefit and not a danger to humanity.
Alignment versus Move Fast
To illustrate the danger of AI some people use the example of human beings versus gorillas. A gorilla is many times stronger than a human being, but people are the dominant species on planet earth. They argue that the stronger species wins and that the same could happen if we introduce AIs who are smarter than people. The AIs could dominate earth and people would be their pets.
The problem of making sure that AIs don’t take over and that their actions are aligned with the interests of human beings is referred to as “Alignment”.
Most people working in the AI field agree that alignment is important, but opinions differ on how to go about it. Some think it is necessary to slow down development, others think we need to go as fast as we can.
There are many arguments on both sides. The safety-above-all-else camp say that self improving AIs could easily take over and kill all of humanity. The move-faster camp emphasize that the first AGI will be aligned with the people who develop it and could for example give China the upper hand over the US.
Reinforcement Learning: Winner Learns All
One of the most famous milestones in AI was when Google’s AlphaGo beat the world champion Go player, Lee Sedol, in 2016. AlphaGo initially learned from matches played by human beings, but then started learning by playing against itself. Much like in natural selection, the process underlying biological evolution, the winner spawned another version of itself. The offspring was given some random changes in strategy. All of this happened inside computers of course. Given Google’s enormous data centers AlphaGo could play millions of games a day and improve quickly. Reinforcement Learning is the computer version of natural selection and is used in many different AI applications. More globally, this is just one example of how emulating nature can result in breakthroughs in AI.
What Q* Probably Refers To
One kind if Reinforcement Learning is Q-learning. Q-learning stores every attempt in a database (table) and stores a value for the quality of the action, hence the Q.
A* is a path finding algorithm, hence the A. It was first published in 1968. It is used for finding paths on maps and in games.
It is assumed that the term Q* in leaked internal communications at OpenAI referred to a new algorithm combining both the Q-learning and the A* algorithms.
It would make sense that combining such a new algorithm with an LLM can result in significant improvements. It could improve the output by navigating through a map of ideas and comparing the quality of different approaches.
Seeing such a new model in action might be what Sam Altman referred to in a recent interview as “pulling back the veil of ignorance.”
The AGI Timeline
It is well possible that the implementation of Q*, or something similar will result in an AI that many people will classify as AGI. If it exists in a lab now then it is only a matter of months, or one or two years at the most before it can be opened up to the public. Other than such a fundamental algorithm improvement, there are many integrations that require no new inventions that will yield similar amplifications of the usefulness of AI in our daily life.
If an LLM could remember all it’s interactions with you, it could dramatically improve the quality of it’s advice and assistance. The information for this exists in the form of your previous sessions. To implement this is a matter of integration. Something like this will probably be rolled out in the coming months.
Another major improvement could be made if we would allow the LLM to look at a selection of documents on our computer and use that information to improve the quality of it’s assistance.
We are just at the beginning of the AI revolution.