Tim Morgan published his eighth blog post in our Emerging Fellows program by exploring the values of automated technology. The views expressed are those of the author and not necessarily those of the APF or its other members.

It takes a civilization to build a robot. Our civilization is not the first to have imagined creating non-human intelligences. It is however the first to be plausibly close to achieving that goal. We may see by 2050 adaptive Artificial General Intelligences (AGIs) which appear to be within the human cognitive range. If true AGIs remain elusive, we certainly will be able to connect limited AIs together to create AGI-like capabilities by 2050.

Technologist Kevin Kelly has written about the evolution of technology, framing it as what it "wants". That is a convenient proxy for talking about how we shape our own development through technology. Technologies are attempts to satisfy our shifting needs and desires. Technology wants to satisfy our timeless needs.

Technology is not neutral. We embed our values into how it works and what it does. It opens new possibilities at the same time it creates new problems. That progression changes our perspective and our culture over time. New values shape what new technologies we want. We co-evolve.

What automation gets is to become an integral part of living. It shapes how we think, how we communicate, how we organize, and how we live. Automation wants to exist because we want it to exist. Our wants are its wants. It is the proverbial djinn of legend called forth to grant our wishes. The problem is that what we want is always imperfect. Our djinn-like automations meet our imperfect wants, creating new wants, prompting new automation, creating more wants, ad infinitum.

We are already passively embedding narrow values into each piece of code we write and each machine we make. We need to consciously embed healthy social and ecological values into capital itself via automation. We need to turn externals into active market participants. We need to give reason to resources. We need Cognified Capital.

How we shape Cognified Capital's identity and self-concept is key. We risk increasing harm if we create these AGIs based on short-term wants and not long-term needs. This is not because we should fear Terminator-like killing machines. It is because we need to give voice to what currently has no voice.

We need to tightly bind AGIs identities to virtual and natural resources. Identity-driven agency will allow them to actively work within markets. They can directly satisfy the needs of their attached resources, like our mind works to satisfy the needs of our body. Automation gets to bridge economic systems and natural ecologies in mutually supportive ways, while helping us benignly fulfill our needs. We get partners, not servants.

What we get from Cognitive Capital are amplifiers of human potential and vigilant guardians of our environments. In turn, Capitalism gets to turn passive assets into active players in expanding fair exchanges of value. It gets to disentangle markets from the insatiable rent-seeking hunger of ownership accumulation. The same hunger Adam Smith warned us to be vigilant about. It gives ownership back to its rightful owners.

What does automation get in exchange? It gets an identity and a purpose. How we shape its purpose over the next few decades will shape our future. 

© Tim Morgan 2019