Safeguarding the Future - Spring 2023 Syllabus

Example project: engineering/solutions (reducing infectious disease transmission in buildings)

Example project: research/analysis (assessing the contribution of feeder rodents to animal suffering)

(We'll also share some example projects from last year, one of which impacted this year's course)

Syllabus (dates and assignments subject to change with guest speaker availability)

Feb 7 - The present has no future

At our current rate of economic growth, we’d be using every atom in the galaxy in less than 10,000 years… which is a problem because the Milky Way is more than 50,000 light-years across. In less that 150 years, we’ve gone from fountain pens to tweets, cavalry to ICBMs, and snake oil to CRISPR-mediated T cell therapy … yet somehow we imagine that the future will consist of people like us who are essentially still making all the decisions. If the present world cannot endure without catastrophe or radical transformation, then the future will be quite different from what most people expect. It might offer delight and fulfillment literally beyond our current capacity to imagine, a hellish dystopia, or an empty universe. If we prefer the first option, we’d best figure out how to nudge our trajectory in a positive direction  How do we go about even deciding how to do that without creating unintended consequences that might be worse than we could imagine?

Feb 9 - Storytelling and history

We’re creatures of the story. There is no human history without somebody capable of presenting a compelling narrative. And we can shape the future by deciding how to tell our story. How, then, can we describe the arc of human history in ways that will improve the future? What can we learn from it? How many people lived in each epoch, and under what conditions? When and what were the turning points that changed history? Were they inventions, ideas, or wars? Which stories have changed the world? …for the better and for the worse? 

Due:

• “This can’t go on” (H. Karnofsky)
• “All possible views of humanity’s future are wild” (H. Karnofsky)
• “1960: The year the singularity was cancelled” (S. Alexander)

Feb 14 - A Dearth of Energy                    Guest: Anne White

Some claim that recent technological advances were constrained by energy limitations. Computation, which is unquestionably the most transformative area of technology of the last fifty years, is noteworthy for requiring minimal energy. If we had access to unlimited clean energy, almost every material problem could, in principle, be solved. How likely is that? And what do we need to do to make it happen?

Due: 

• Review of Energy and Civilization (B. Gates)
• “Why nuclear power has been a flop” (J. Crawford)
• “The limits to growth in organic economies” (E.A. Wrigley)

Recommended: 

• Energy and Civilization (V. Smil)
• LongNow Video: Saul Griffith, Climate Change Recalculated

Feb 16 - The planetary thermostat

Few people continue to doubt that climate change is real and that human activity is the major cause.  Many assume that warming of 2C - a figure that only twenty years ago was considered catastrophic - is now unavoidable. But we have implemented only minor efforts to lessen our dependence on fossil fuels. That will dramatically increase water and food shortages in the poorest parts of the world and gradually submerge land occupied by a significant fraction of humanity. Should we intervene to speed up our efforts to correct our mistakes?  What are the possible consequences, and who should decide?

 The long-term impacts, let alone the tail risks, will almost certainly make everything else more difficult. What can we do?

Due: 

• “The Climate Fixers: Is there a technological solution to Global Warming” (M. Specter)
• “Economic impacts of tipping points in the climate system” (Dietz et al)
• “Evaluating climate geoengineering proposals in the context of the Paris Climate Agreement temperature goals” (Lawrence et al)
• “Marine cloud brightening” (Latham et al)
• “Halving warming with idealized solar geoengineering moderates key climate hazards” (Irvine et al)
• “How green sand could capture billions of tons of carbon dioxide” (Temple)

Feb 21 - Existential risk CLASS CANCELLED DUE TO MIT HOLIDAY

That the world is getting hotter is often described as an existential risk. Is it? What does “existential” mean? Does it include global catastrophic risks? What defines those? Local or worldwide civilizational collapse? Stagnation? Human extinction? Becoming locked into a social or technological trap from which there is no escape? How important is each of these... and how can we be sure we're not just millenarians fixed on the end times?

Feb 23 - Right idea, wrong PR

Some view technology as deterministic. But even if people will eventually adopt whatever is useful, multiple technologies may fill a need, in which case outside factors can decide which of them will be developed. To what extent does the success of technology depend on its inventor? On a popularizer? A funder? What happens when a true and important idea is devised by the wrong person or the wrong place? Or the opposite?

Due:

• “The doctor who championed hand-washing and briefly saved lives” (R. Davis)
• “The vulnerable world hypothesis” (N. Bostrom)
• "Fritz Haber and Carl Bosch" (C. Flavell-While)

Feb 28 - Nuclear game theory            Guest: Andrew C. Weber  

Leo Szilard, the inventor of the chain reaction, warned that dropping the bomb on Japan in 1945 would accelerate an arms race that would make American cities vulnerable to foreign militaries for the first time in half a century. He was right. Nuclear weapons cannot be blocked and can only be deterred through mutually assured destruction. Whether they kept the Cold War cold is debatable, but it was always touch and go. We came close to losing around half of humanity in 1983. What can we learn from our first experience with a technology capable of threatening so many of us? And what does the current war between Russia and Ukraine mean for the wider nuclear threat?

Due:

• Would US and Russian nuclear forces survive a first strike? (L. Rodriguez)
• How Many People Would Be Killed by a Direct US-Russia Nuclear Exchange?
• How Bad Would a Nuclear Winter Caused by a US-Russia Nuclear Exchange Be?
• How Likely is a Nuclear Exchange Between the US and Russia?

Recommended:

• The Dead Hand (D. Hoffman)
• NUKEMAP (A. Wellerstein)

Mar 2 - Biological catastrophe

COVID-19 was caused by a single, comparatively mild pandemic virus that wasn’t even engineered to cause harm.  Around thirty thousand people today could single-handedly assemble most viruses from synthetic DNA using cheap equipment. That number gets significantly larger every year as the price of computing power drops and the tools of genetic engineering grow. For years, scientists have tried to find viruses in nature that can harm us. Should they be doing that?  What would happen if they were to post a list of credible pandemic viruses found in nature? Or learn how to engineer more destructive ones? How can we best immunize civilization against catastrophic biology… and delay disaster until we have succeeded?

Due:

• “The Deadliest Virus: Did a scientist put millions of lives at risk, and was he right to do it?” (M. Specter)
• “Dangerous biological research - is it worth it?” (K. Esvelt and J. Galef)

Mar 7 - Mistakes and Madmen            Guest: Sophie Rose

Do most catastrophic risks arise from accidents or deliberate actions? History suggests that our record of accidents is mixed and could certainly be improved. But the real uncertainty concerns the number of people who would push a doomsday button if given the opportunity. To date, only great powers have had that chance, and their actions are largely tempered by game theory. Should many more individuals gain that power, we may expect a different outcome.

Due:

• Parsons, Keith M.; Zaballa, Robert A. (2017). Bombing the Marshall Islands: A Cold War Tragedy. Cambridge University Press. Chapter 3: Runaway Bomb. ISBN 978-1108508742. (accessible through MIT libraries as PDF download)
• The fear of setting the planet on fire (Y. Yiu)
• “The Unilateralist's Curse” (N. Bostrom et al)
• “Who would destroy the world? Omnicidal agents and related phenomena” (P. Torres)

Mar 9 - Project blizzard

Discussions are all very well and good, but we also hope to achieve something concrete this semester. Think of 3 project ideas you consider intriguing that are relevant to safeguarding the future. They can be on any subject. Maybe you're worried about technological stagnation, or a totalitarian autocracy trap, or contacting aliens. If you've a creative concept, bring it and share it.

Due:

• Come ready to present 3 project ideas

Mar 14 - What is transformative AI?                Guest: Ajeya Cotra

We have already created systems with superhuman capacities. Soon AIs will be able to accelerate the pace of science and technology development reliably. That will likely return the world to a cycle of exponential growth and innovation that halted when we stopped having as many babies as we could feed, interrupting the sequential conversion of inventions into calories into inventors. The resumption of that cycle will mark the end of our current world. When will it happen, and how can we begin to prepare? Open AI’s recent release or Chat GPT - while not in any way challenging the dominance of humanity - seemed to wake many people up to the fact that AI will have abilities we have not yet imagined or planned for.

Due:

• “Timelines to transformative AI” (A. Cotra)
• “Provably beneficial artificial intelligence” (S. Russell)
• Work with ChatGPT to produce a treatise of interest

Recommended:

• Superintelligence (N. Bostrom)

Mar 16 - Paths to AI alignment               Guest: Paul Christiano

What happens when we create agents more capable than we are? After all, chimpanzees are nearly extinct and mostly reside in zoos. Ideally, we want anything with power to be not only aligned with our interests as the average human but much more so. Even given perfect alignment with human interests, machines with significantly superhuman competence will still upend the strategic gameboard, potentially creating incentives for people to act dangerously. To give just one example, superhuman competence in biology might create catastrophic threats that could reliably overcome our human-designed defenses. How can we both align transformative AI systems and prevent their superior capabilities from causing harm?

Due:

• [NEW] "This Changes Everything" (E. Klein)
• “What failure looks like” (P. Christiano)
• “Overview of the AI alignment landscape” (N. Nanda)
• "Yudkowsky Contra Christiano on AI Takeoff Speeds" (S. Alexander)

Recommended:

• Human Compatible (S. Russell)
• Living with Artificial Intelligence (S. Russell)

Mar 21 -  A world without biorisk

The world is demonstrably vulnerable to the introduction of a single
pandemic virus with a comparatively low case fatality rate. The deliberate
and simultaneous release of many pandemic viruses across travel hubs
could threaten the stability of civilization. Current trends suggest that within
a decade, tens of thousands of skilled individuals will be able to access the
information required for them to single-handedly cause new pandemics.
Safeguarding civilization from the catastrophic misuse of biotechnology
requires delaying the development and misuse of pandemic-class agents
while building systems capable of reliably detecting threats and preventing
nearly all infections. This appears surprisingly feasible.

Due:

• "Delay, Detect, Defend" (Esvelt)

Mar 23 - Project development

Apr 4 - Group's Choice: Persuasion of the Powerful

Due:

• Op-Ed #2

Suggested readings

• pg 19-29, What Can We Learn from Asilomar? of "The Asilomar Conference: A Case Study in Risk Mitigation" (Grace)
• What an actually pessimistic containment strategy looks like 

Apr 6 - On reflection

Let’s try and focus on realistic measures we might take to address the impact AI will have over the next twenty to thirty years. It would be good if everyone could come to class with some concrete ideas for how we  can use AI - because obviously we will use it - in a measured and valuable way.    Also,  we may well have to reorganze society if AI eliminates most of our jobs. But rather than just sort of say; hmmm, we will need a revolution or some acceptable approach to socialism, maybe everyone can think a bit about how we actually WOULD go about reorganizing. What do you want AI to do for you and what to you want it not to do?  None of us have been that concrete about these issues because there are no magic or concrete solutions. But let’s pretend human’s will still run the world for a while and that we continue to have the power to deploy technology the ways we want it used. What will that mean?

Due:

• OpenAi's "Planning for AGI and Beyond" (Astral Codex Ten)

Apr 11 - Coordinating preparedness                 Guest: Rick Bright

COVID-19 gave us a reasonable snapshot of how well human civilization can be expected to respond in a global crisis. What can we learn from the last two years? Which institutions can we rely on? In which countries? Are there any patterns? How can we compensate for weaknesses and uncertainties?

Due:

• “10 lessons I’ve learned from COVID-19” (H. Branswell)
• “The American Pandemic Preparedness Plan” (White House)

Apr 13 - Resilient logistics: feeding everyone

There is an inherent tradeoff between efficiency and robustness. By definition, a maximally efficient system has no redundancy. Just-in-time manufacturing minimizes warehouse storage time and wasted materials but is fragile to supply chain disruptions. How can we render our supply chains resilient when market forces work to render them fragile? Which are particularly at risk, and what can we do?

Due:

• "The fragility of interdependency" (A. Vespignani)
• “Supply chains and the coronavirus” (L. O’Leary)
• “Feeding everyone if the sun is obscured and industry is disabled” (D. Denkenberger)

Recommended:

• 1177 BC: The Year Civilization Collapsed (E. Cline)
• The Collapse of Complex Societies (J. Tainter)

Apr 18 - Raising the well-being waterline                 Guest: Rosalind Picard

Nobody wants a future filled with misery. But suffering is incredibly widespread. At any given time, an estimated 20% of people experience some symptoms of mental illness. Many more are miserable. Suffering and mental illness make it hard to cooperate, so helping people lets them help more. To what degree are these problems biological, and how many are environmental? Do people care more about subjective happiness or fulfillment? What are the most effective treatments today? What can we invent to improve well-being?

Due:

• “Burden of Depressive Disorders by Country, Sex, Age, and Year” (Ferrari et al)
• “Development, Freedom, and Rising Happiness: A Global Perspective (1981–2007)” (Inglehart et al)
• “Finland is the happiest country in the world, and the Finns aren’t happy about it” (F. Martela)

Apr 20 - Unknown unknowns

Due:

• Op-Ed #3

Apr 25 - Open discussion: Neurotech

Due

• "Physical principles for scalable neural recording" (K. Kording et al)
• "Book Review: Age of Em" (S. Alexander)
• "The Adventure: a new Utopia story" (S. Armstrong)
• "Neurotech at Work" (N. Farahany)

Apr 27 - Rebuilding civilization

Given the non-trivial probability of civilization collapsing, it’s remarkable how little has been done to help the survivors rebuild. What do we most want to pass on to our successors? To their descendants? Can we learn anything from past civilizational collapses and recoveries? To what extent do the answers depend on the nature of the threat that ends the rest of us?

Due: 

• “The Library of Utility” (K. Kelly)
• “Manual for civilization” (A. Rose)

Recommended:

• The Fall of Rome, and the End of Civilization (B. Ward-Perkins)

May 2 - TBD

May 4 - Projects

Either talk by Rosalind Picard or students, or time for project development

May 9 - Project Development Time

May 11 - Project Presentation

Due:

• Physical submission of Final Project on Canvas
• Project presentation at class

May 16 - No class