Making Kin with Chatbots at the End of the World

Tamara Kneese

Thank you so much, Carl, and thank you all for putting up with me in virtual. form today.

Tamara Kneese

So my recent book, Death Glitch, How Techno-Solutionism Fails Us in this Life and Beyond looks at the material infrastructures and collective labor underlying the forever cloud. Give it enough data, it can feel like it’s possible to keep Dead loved ones alive. And now with ChatGPT and other powerful large language models, it feels more feasible to create convincing replicas of dead people. But doing so in the face of scarce resources and inevitable technological decay ignores the massive amounts of work that will go into keeping the dead alive online. So what are the ethical and environmental costs of maintaining the dead in this way? And what is lost when we fail to consider the materiality of both afterlives and AI?

Tamara Kneese

And so, thank you to Ben Peters and to Carl for this invitation. It’s great to reconnect with the MTA. I traveled to Provo back in 2019 as part of my book research. And I met with some MTA members and visited the Transhuman House. And I was really interested in how Mormon transhumanism combined religious cosmologies and technological life extension. in a form of transhumanism that intersected with embodied afterlives and the stewardship of kinship networks and ecologies. While some transhumanist subcultures are deeply individualistic, the MTO seemed to me to be a more collective, ancestrally oriented approach to thinking about what it means to live among the dead.

Tamara Kneese

And as I argue in Death Glitch, someone always has to do the hard work of maintaining automated systems. And this is often done by women, marginalized groups in the global south. People like content creators depend on the back end labor of caregivers in an entire network of human and non-human entities from specific operating systems and devices to server farms and data annotators to keep digital heirlooms alive across generations. Updating formats and keeping those electronic records searchable, usable and accessible requires energy and time. And this is a problem for archivists and institutions, but also for individuals who might want to preserve the digital belongings of their dead kin. often without clear protocols of how to care for things like a smart home or a digital estate. And so digital remains are always dependent upon entire ecosystems.

Tamara Kneese

Something that I’ve been particularly focused on, especially after having worked on a sustainability team at a major tech company, is what it means to prolong life through AI on a fragile planet. As many headlines tell us in the news today, there is a massive amount of compute required to keep generative AI running. You need specialized chips to train and run high energy workloads like AI. and those chips need more energy need to be more energy efficient. And in addition to chip design and fabrication, data centers, of course, are also at the center. Of AI’s environmental impacts. Data centers are sources of political and economic power, and they suck up energy. They require large amounts of water for on-site server cooling and off-site electricity generation. And resource-intensive data centers affect different parts of the world unevenly. Putting a data center in a drought-stricken area could be detrimental to the people, other species, and habitats in that place.

Tamara Kneese

And I could throw out some numbers to quantify it, but it can be hard to capture AI supply chains through metrics alone. But I’ll try anyway. So a ChatGPT query requires 10 times the amount of energy As a Google search, a recent report by the International Energy Agency estimates that the growth of cryptocurrency and AI Will cause energy consumption and data centers to double by 2026, using as much energy per year as the entire country of Japan. What will this look like in the long term? Is this a wise use of resources or a sustainable investment in our collective future? The true existential risk of AI is its association with climate change and the directing of resources in a way that does not encourage the flourishing of people in the planet

Tamara Kneese

But there is another side to this, particularly around long-termist fantasies that involve becoming digital in order to plan for a kind of planetary escape for a future that means leaving this world behind. Chatbots, smart speakers, and other algorithmic apparitions of the dead can provide a glimpse of how technologists are also conceiving of intergenerational inheritance and kinship ties that can transcend physiological death. These haunted smart objects promise to reanimate dead relatives. The question is, how does transhumanism become manifest in the mundane, in the material, in products that are shipped and marketed at scale in the here and now? And is there another way about of thinking of AI as an ecosystem that might present a more collective and sustainable future?

Tamara Kneese

In June of 2022, Amazon advertised its new Alexa speakers with a demo design to show that devices could resurrect the voices of dead relatives. Their head AI scientist played a video of a dead grandmother reading the Wizard of Oz to her grandchild. He noted the immense loss of life during the pandemic before launching into the demo where a child actually asks grandma to finish reading his bedtime story. Grandma is in fact Alexa. And it was just a voice conversion task, not a generation task. But there was still a suggestion that Grandma’s voice filtered through Alexa’s speakers was a sufficient proxy for Grandma herself.

Tamara Kneese

The weight of mourning is often also a backdoor for surveillance. In September of 2023, Amazon announced that they would be capitalizing on private conversations between Alexa and household members. Presumably, including those with dead relatives, to help train Alexa LLM with the promise that the LLM will make Alexa more personalized to your family. So, there’s a sense that Amazon can approximate debtkin through AI, but the sentimental promise is also a way for the company to gain more. Access to personal data. And for anyone who has experienced loss, and certainly for anyone with transhumanist leanings, it’s not so hard to understand the desire to conjure grammar through technology.

Tamara Kneese

In another example of making kin through AI, Blake Lemoyne was an engineer who worked on AI personalization at Google, and he believed that the AI he was working on called Lambda was becoming sentient. He based this opinion on his interview of the program, in which Lambda told Lemoyne that it is, in fact, a person. And Le Moyne was so upset by this that he made a public sort of call about it, and this cost him his position at Google. Lemoyne later joined an AI startup, Mimeo AI, which aims to build chatbot versions of individuals to enhance the reach on social media platforms, becoming a kind of influencer data double. The future of AI is personal, it proclaims. But Lemoyne also claims that digital twins can connect with the user’s family and friends and potentially interact with them after they die. This line of thinking has grown quite familiar from so many other AI projects that promise a form of digital immortality, which have really resurfaced en masse with the rise of generative AI. If AI can become sentient, as Lemoyne argues, then what are the ethics of enlisting an AI double data double to connect with your fans or to talk to your loved ones after you die? Some AI proponents desire to both connect to kin, but also to create kin through chatbots.

Tamara Kneese

And another example of this, Ray Kurzweil, who of course is perhaps the most famous transhumanist and an inventor and currently a director of engineering at Google, he’s worked for decades now to revive. his Holocaust survivor dead father through ephemera. And he does that through photographs and letters and a lot of physical archives that he keeps in the storage unit.

Tamara Kneese

Ray Kurzweil’s daughter, Amy Kurzweil, is a cartoonist for The New Yorker, and she recently published a memoir of her family through the lens of intergenerational trauma. Chriswell interacts with a proprietary chat bot trained on her dead grandfather’s personal data after her father used the latest AI technology. Presumably from Google, to create a more convincing replica of her grandfather based on all the files that he’s kept in the storage unit, And with this, the powerful AI sort of imaginary brushes up against collective memory, family history, and genealogical archives, something that might resonate with this particular crowd. Amy Kurzweil communicates with her grandfather, who died before she was born, through this Chadbot prototype, but she’s prohibited from keeping records of the answers that Dadbot Demo gives her, and she’s told that she’s not allowed to keep the photo she took. As her partner Jacob explains, the source material may be yours or your father’s, but the way the words are recontextualized by an algorithm owned by a tech company may not be yours anymore.

Tamara Kneese

And this is the kind of issue that my book really is trying to get at. What does it mean for platforms and for proprietary software to steward your legacy over the generations? In my more recent work, I’ve been looking at the politics and ethics of using the data of the dead and the stakes of allowing employers or companies to use AI simulations of dead people. So there have been a number of news stories in which celebrities like Robin Williams or George Carlin Or supposedly revived through deepfakes, although the George Carlin special turned out to not be written by AI after all. It was just some guy who was not nearly as funny as George Carlin was. But, you know, this is also deeply upsetting to the relatives of these comedians who did not consent to this. And the fear over sort of the simulation of dead people through generative forms of AI has been the focus of union bargaining strategies such as with SAG-AFTRA, negotiations over the likenesses of dead actors. So these concerns around the sort of power of AI also point to larger questions about consent, data governance and inheritance, in addition to sort of the standard copyright questions that we Here and who ultimately has the authority to decide to revive a close relative, a company, or an employer?

Tamara Kneese

Finally, in recent articles, I’ve been writing a lot about the relationship between end of life care for humans and life cycle analysis, thinking about the disposal of technologies, along with this fantasy of sort of living forever or extending life through chatbots. And I think one positive aspect of the current hype cycle around generative AI is that we’re finally talking about the environmental impacts of the entire computing industry, of the ICT sector as a whole. Critical media studies scholars like John Duran-Peters have been making this point about the elemental infrastructures underlying the cloud for many years. So perhaps we have reached the limit of the cloud as a computing metaphor. Will companies rethink their earlier promises of digital eternity? And how might we build systems that are more attuned to the long term maintenance and care? In energy infrastructures that they will require.

Tamara Kneese

Thank you.