Scale Is All You Need? Part 4-3: The Post-AGI-World

Note: If you haven’t seen Part 4-2, you can read it here.

Outlook of Human Culture in Post-AGI

In a future in which AGI is fully integrated, human culture will likely undergo profound changes. Our society will adapt to a world in which artificial intelligences take on tasks in science, philosophy and politics that were traditionally reserved for human curiosity and the thirst for knowledge. This could give rise to a new kind of culture, one that differs greatly from today's ideals and values, since many of our basic assumptions – such as the value of academic achievements and political decision-making – will be challenged by the capabilities and possibilities of AI. And last but not least, human hubris will erode as we realize that our own species may not be the most intelligent on Earth and that even an artificial intelligence far surpasses our own intelligence.

Another slight in the history of mankind. Something that the Slovenian philosopher and psychoanalyst Slavoj Zizek already foresaw 15 years ago:

In the context of modern LLM and the function of human thought and its own failures, such as human hallucinations, Zizek's statement seems more relevant than ever.

A possible future society often mentioned in this context is again the Star Trek society, in which material scarcity is no longer a challenge. In the film series, replicators ensure that all goods are immediately available, and the human drive shifts from survival to exploration and self-realization. Research and the thirst for knowledge are no longer existential necessities here, but rather an expression of human curiosity and the desire to explore the universe. If AGI can fully take over scientific discovery, the value of a PhD or an academic career based on scientific achievement becomes obsolete – the greatest discoveries in a post-AGI era could be made by self-improving models and algorithms without human intervention. AlphaFold 2 already gives us a glimpse of what an academic world could look like in which insights and breakthroughs are achieved by AI without much human intervention.

This also raises the existential question of the meaning of life and the significance of knowledge: In a world where knowledge is readily available and constantly being generated, the focus could shift to how we experience and interpret this knowledge, rather than creating it ourselves. Philosophers such as Heidegger and Sartre have already addressed the topic of “being” and questioned what constitutes a human being. A culture after AGI could therefore be a kind of “meta-culture” that no longer revolves around the production and acquisition of knowledge, but rather focuses on understanding and reflecting on the questions it raises - a culture that aims to explore the relationship between people and the world and themselves in a deeper and more existential way. Moreover, Longevity would put the question of life and death back at the center of our attention. What is death, what is dying? Should we understand death as a necessary part of life or rather as an illness to be overcome? If we overcome the age of death, what would that mean for life? Can we even understand and grasp life if we lack the negative, its dialectical counterpart, inasmuch as we only understand light because we have an idea of darkness? Or to put it another way: the philosophical existentialism of a Sartre or Camus could prove to be the most consequential philosophy due to the inevitably changing material reality, the exploration of which will have serious effects on our lives. We will therefore devote ourselves to different scientific disciplines than we do today, simply because different problems and questions will arise. Morals and ethics are ultimately always an expression of current material reality and change to the extent that our social existence changes.

This outlook suggests that a post-AGI culture will not only change traditional scientific and philosophical pursuits, but also the meaning of human life and the way we define ourselves. Traditional culture could transform into a culture of experience and exploration, where the greatest adventures lie within the human mind and the vast expanses of the yet unexplored. Since the beginning of humanity, philosophers have been arguing about the meaning of life. The aforementioned French philosopher Albert Camus had answered this question in the last century by saying that the meaning of life can only be recognized and understood retrospectively at the end of life, in retrospect on one's own life decisions. However, it can be argued that a person's sense of life is also projected into the future when the entire basis of life changes forever.

We therefore need to start dealing with ethical and moral questions today, such as whether it is negligent for a doctor not to consult an AI when evaluating X-ray images, given that some AI models are already able to evaluate these images better than a human doctor. 

If, however, we expand the cultural framework to include ethical and philosophical considerations, numerous other decisive cultural fields of discussion arise. For example, will there be wars if we live in a world of abundance and the distribution of resources seems to be a historically archaic matter? Do we live in a world of total peace where the only war is the one about whether the robot is allowed to trim the hedge on Sundays?

And what about religion, faith and religious studies? It has always been a truism among atheists that faith has always been a human attempt to make the incomprehensible, that which cannot be explained by reason, comprehensible and, above all, controllable by shifting it into a transcendental power figure (archaic tribal cultures, for example, tried to get the forces of nature under their control through rain dances; an early attempt to transform one's own powerlessness, one's exposure to nature and one's lack of understanding into action – which obviously failed). In short: What will become of religion and religious studies if we create our own rational “machine god”? Will there perhaps be an artificial religion that is born only from reason, from created intelligence? And anyway: What will still be taught philosophically in schools and universities, or further: What does education still mean at all, will there still be educational institutions?

In his latest book, The Singularity is Nearer, Ray Kurzweil argues that the singularity will lead to the end of the distinction between man and machine, with the two becoming one. We are already seeing the beginnings of this today, with the use of Neuralink technology to enable people who are completely paralyzed to control a computer with just their thoughts.
Nanorobots will allow us to intervene in our bodies at the smallest physical level. We will probably be able to cure all diseases before they break out, and even optimize our human physique so that we, as cyborgs, literally develop superhuman powers. So it is relatively certain that in the future we will be able to download knowledge instead of laboriously acquiring it – The Matrix sends its regards.

A doctorate, hard-earned with sweat and diligence, will be a relic of the past. Today, anyone who owns a smartphone and has access to a ChatGPT already has their personal PhD in their pocket. All these thoughts that education and knowledge will never be the same as our generation has experienced and acquired are not utopian, but already a reality today!
Even people who have never studied can solve the most complex questions at the doctoral level with the help of OpenAI's o1 and at the same time have the solution explained to them in an understandable way if they have any questions left. And if we think one step further and tie into the ideas of existentialism: will we perhaps even be able to upload our consciousness, or at least a copy of it, to a cloud in the foreseeable future? What actually is being, what is consciousness, what is individuality, what is authenticity? We will face all these questions in the next 20 years with a seriousness that has never been seen before in human history.

To summarize, culture is something that is constantly changing. As humans, we have a relatively short lifespan, so culture appears to be something relatively constant, stable, and coherent. However, a look at human culture shows the exact opposite: culture is constantly changing. Even in the present, different groups of people have different cultural characteristics. However, the post-AGI world will challenge our culture in a completely different way. It will break down and challenge what has historically been considered “normal” and consistent. Exponential development will literally change everything in no time. And that's why it's so important to me to keep pointing out that we need to start thinking now abouthow we want to live in the future. Even if it is hardly possible to prepare for the future, we can still develop certain parameters, certain principles that should survive the coming upheaval as the cornerstones of our human culture and civilization. What makes us human? What is socio-culturally non-negotiable? What must endure, but above all: what is dispensable? We have to ask ourselves all these questions, sooner rather than later.

Part 4-4 is coming out soon. Subscribe to the Forward Future Newsletter to have it delivered straight to your inbox.