Course Syllabus

Safeguarding the Future - Spring 2025 Syllabus

Feb 5    Feb 12      Feb 19      Feb 26     Mar 5     Mar 12   Mar 19    Mar 26    Apr 2    Apr 9      Apr 16   Apr 23 Apr 30 May 7      May 14

The present cannot endure (Esvelt, Specter) The history and future of human agency (Crawford) Coordinating to solve problems (Gellin) Project Day Nuclear game theory (Weber) Rewriting life (Esvelt) Transformative AI (Cotra) <Spring break> Project Day / Communicating technical subjects (Specter) The energy bottleneck and climate change (Brown) Avoiding the tyranny trap (TBC) Raising the wellness waterline (Picard) Class choice or surprise (influential result of 2023's class)  Project Day Final project presentations

 

Syllabus (dates and assignments subject to change with guest speaker availability)

 

February 5

The present cannot endure                                             Kevin Esvelt and Michael Specter

At our current rate of economic growth, we’d be using every atom in the galaxy in less than 10,000 years… which is a problem because the Milky Way is more than 50,000 light-years across. In under 150 years, we’ve gone from fountain pens to tweets, cavalry to ICBMs, and snake oil to CRISPR-mediated T-cell therapy … yet somehow, we imagine that the future will consist of people like us, living much the way we do today. If the present world cannot endure without catastrophe or radical transformation, then the future will be quite different from what most people expect. It might offer delight and fulfillment literally beyond our current capacity to imagine, a hellish dystopia, or an empty universe. If we prefer the first option, we’d best figure out how to nudge our trajectory in a positive direction. Are we moving towards a radically different future? What are the crucial drivers? And how can we communicate problems and solutions to best safeguard the future?

 

February 12

The history and future of human agency             Guest: Jason Crawford, Roots of Progress

Humanity has gained increasing control over the essentials of life and the economy: our food supply, our materials and manufacturing processes, our use of energy, and our transportation networks. A prerequisite of safeguarding the future is the continuation of this historical process. Commentators from Samuel Butler to Vernor Vinge to the present have expressed concern that humanity will lose control. We will approach this topic first historically, arguing the apparently unpopular position that human agency has been increasing, not decreasing, over time. We will cover present evidence for this view, and suggest key drivers of the trend. Then we will discuss whether we can expect this trend to continue or whether it might reverse, and why.

Required reading:

• “The short history of global living conditions” (M. Roser)
• “We need a new science of progress” (P. Collison, T. Cowen)
• "Darwin Among the Machines" (S. Butler)
• "The Future as a Way of Life (NEW LINK 11 Feb)" (A. Toffler)
• "The Coming Technological Singularity" (V. Vinge)
• "Do we get better or worse at adapting to change?" (J. Crawford)
• “1960: The year the singularity was cancelled” (S. Alexander)
• “This can’t go on” (H. Karnofsky)

Recommended Reading:

• Invention and Innovation: A Brief History of Hype and Failure, 2023  (V. Smil)

 

February 19

Coordinating to solve problems                                                                        Guest: Bruce Gellin

Our world is governed by a constellation of bodies: governments, industry, academia, to name a few. They need to work together and they rarely do. The COVID pandemic was the most recent and horrifying example of government bodies competing with each other (even just within the United States). Between the ability of government, the desires of industry, and the often theoretical goals of academic researchers, we need to shape how we develop, adopt, and govern use of technologies. How do we corral these forces together? What are historic examples of large coordination efforts, and what lessons from them can we apply today?

Required reading:

• "The Asilomar Conference: A Case Study in Risk Mitigation" (K. Grace)
• “The Early Days of Pugwash” (J. Rotblat)
•  Trinity: “The most significant hazard of the entire Manhattan Project” (By Kathleen M. Tucker, Robert Alvarez)
• Reconstructing the Cost of the One Giant Leap, Casey Drier, The Planetary Society

 

February 26

Project Day

To safeguard the future, we can't simply learn from the past and discuss the challenges of the present. We need to act. That may involve creating compelling videos or conducting class exercises that are turned into manuscripts, covered in Science, and help catalyze the new field of AI/biosafety research and the Executive Order on AI. Accordingly, today will focus on project planning.

Due: Writing #1

 

March 5

Nuclear game theory                         Guest: Andrew C. Weber, Council on Strategic Risks

Leo Szilard, the inventor of the chain reaction, warned that dropping the bomb on Japan in 1945 would accelerate an arms race that would make American cities vulnerable to foreign militaries for the first time in half a century. He was right. Nuclear weapons cannot be blocked and can only be deterred through mutually assured destruction. Whether they kept the Cold War cold is debatable, but it was always touch and go. We arguably came close to losing half of humanity in 1983. What can we learn from our first experience with a technology capable of threatening so many of us? And what do the war between Russia and Ukraine and rising tensions between the United States and a China that is modernizing its arsenal imply for global nuclear risk?

Required reading: 

• “Nuclear weapons: A beginner’s guide to the threat” (P. Webber)

• “The Return of Nuclear Escalation: How America’s Adversaries Have Hijacked Its Old Deterrence Strategy” (K. Lieber and D. Press)

• “How Likely is a Nuclear Exchange Between the US and Russia?” (L. Rodriguez)

• “Doomsday Clock Stands at 90 Seconds to Midnight, Closer Than Ever to Global Ruin” (UN General Assembly)

• NUKEMAP (A. Wellerstein) [instructions: try out some parameters]

• “Bombing the Marshall Islands: A Cold War Tragedy. Cambridge University Press. Chapter 3: Runaway Bomb. (K. Parsons and R. Zaballa) (accessible through MIT libraries as PDF download)

• “The fear of setting the planet on fire” (Y. Yiu)

• “The Unilateralist's Curse” (N. Bostrom et al)

Recommended reading:

• The Dead Hand (D. Hoffman)
• Nuclear Ethics Revisited (Joseph S. Nye Jr.) 

Due: Rewrite of Writing #1

 

March 12

Rewriting Life                                                                                                    Kevin Esvelt

COVID-19 was caused by a single, comparatively mild pandemic virus that wasn’t even engineered to cause harm. Around thirty thousand people today could single-handedly assemble pandemic influenza viruses from synthetic DNA using cheap equipment. For years, scientists have tried to find viruses in nature that can harm us. Should they be doing that? What would happen if they were to post a list of credible pandemic viruses found in nature or describe how to engineer more destructive ones? Can we avoid this while still ending disease and aging? How can we best immunize civilization against catastrophic biology… and delay disaster until we have succeeded?

Required reading: 

• Publish and Perish: The Hazards of Sharing Biological Information.  Chapter 5, Higher Animals, audiobook M. Specter Pushkin Press (Class will be sent mp3s)

• “Delay, Detect, Defend” (K. Esvelt)
• “Guarding against catastrophic biological risks” (J. Yassif, S. Korol, A. Kane)
• “Securing civilisation against catastrophic pandemics” (Gopal et al)
• "Confronting Risks of Mirror Life" (Adamala et al)

Recommended reading:

• Who would destroy the world?” (P. Torres)
• Technical Report on Mirror Bacteria: Feasibility and Risks(various)

EDIT: Op-Ed 1 Re-write due 

 

March 19

Aligning transformative AI                              Guest: Ajeya Cotra, Open Philanthropy Project

We have already created systems with narrowly superhuman capabilities. At some point, AI will noticeably accelerate the pace of technology development. That will likely return the world to a cycle of exponential growth and innovation that stalled when we stopped having as many babies as we could feed, interrupting the sequential conversion of inventions into calories into inventors. The resumption of that cycle will mark the end of our current world. When will it happen, and how can we begin to prepare? ChatGPT – while not in any way challenging the dominance of humanity – seemed to wake up many people to the fact that AI will have abilities we have not yet imagined or planned for.

What will happen when we create agents more capable than we are? It's concerning that chimpanzees are nearly extinct and mostly reside in zoos. Anything with tremendous power must be much more aligned with our interests than the average human. And even given perfect alignment, machines with significantly superhuman competence will still upend the strategic gameboard, potentially creating incentives for people to act dangerously or even catastrophically. How might we align transformative AI systems and prevent their capabilities from causing harm? 

Required reading: 

• “A Race to Extinction: How Great Power Competition Is Making Artificial Intelligence Existentially Dangerous” (S. Meacham)
• "Machines of Loving Grace" (D. Amodei)
• "Situational Awareness" (L. Aschenbrenner)
• “A Playbook for AI Risk Reduction” (H. Karnofsky)
• “What failure looks like” (P. Christiano)
• "Yudkowsky Contra Christiano on AI Takeoff Speeds” (S. Alexander)

Recommended reading:

• "Sleeper Agents" (Hubinger et al)
• "Alignment faking in large language models" (Greenblatt et al)
• "Emergent Misalignment" (Betley et al)
• Executive Order 14110 (U.S. government)

• “Timelines to transformative AI” (A. Cotra)

• Human Compatible (S. Russell)

• Living with Artificial Intelligence (S. Russell)
• Deep Utopia (N. Bostrom)

Due: Writing #2

 

March 26

Spring Break

 

April 2

Project Day + Communicating technical subjects with minimal controversy             Michael Specter

The technical challenges we will have to adopt - or choose to reject -will make it essential that technologists express themselves plainly, directly and in ways that non-scientific people do not find threatening. If you can’t express a philosophical point of view in a way that is widely understood how can you hope to set an agenda for a technological future?

Required reading:

•  “My Techno-Optimism” (V. Buterin)
• “ An Ecomodernist Manifesto” (J. Asafu-Adjaye)                                                                 

EDIT: Op-Ed 2 due 

Due by the end of class: Final project proposal.

 

April 9

Energy bottlenecks and the global thermostat                      Guest: Benjamin Brown, CU Boulder

Some claim that recent technological advances were constrained by energy limitations. Computation, which is unquestionably the most transformative area of technology of the last fifty years, is noteworthy for requiring minimal energy. If we had access to unlimited clean energy, almost every material problem could, in principle, be solved. What might we do to make it happen?

Many now assume that warming of 2oC – a figure that only twenty years ago was considered catastrophic – is unavoidable. That will dramatically increase water and food shortages in the poorest parts of the world and gradually submerge land occupied by a significant fraction of humanity. Which technologies could plausibly change this? Should we engage in geoengineering to correct our mistakes? What are the possible consequences, and who should decide? 

Required reading: 

• Review of Energy and Civilization (B. Gates)
• “Why nuclear power has been a flop” (J. Crawford)
• “The limits to growth in organic economies” (E.A. Wrigley)
• “The Climate Fixers: Is there a technological solution to Global Warming” (M. Specter)
• “Evaluating climate geoengineering proposals in the context of the Paris Climate Agreement temperature goals” (Lawrence et al)
• “Marine cloud brightening” (Latham et al)
• “Halving warming with idealized solar geoengineering moderates key climate hazards” (Irvine et al)
• “How green sand could capture billions of tons of carbon dioxide” (Temple)

Recommended reading: 

• Energy and Civilization (V. Smil)
• LongNow Video: Saul Griffith, Climate Change Recalculated

Due: Writing #3

 

April 16

Avoiding the Tyranny Trap                                               Guest: TBC

We can see more than ever – and others can see us. Ubiquitous cameras and smartphones make it possible to keep track of everyone, including where they go and whom they meet. Some nations have already instituted broad systems of surveillance. When does public safety yield to autocratic control? Are there ways of enhancing security without compromising freedoms? How will increasingly powerful artificial intelligence shape the centralization of power in society?

Required reading: 

• “The grim fate that could be worse than extinction” (D. Minardi)
• “The totalitarian threat” (B. Caplan) (free PDF)

• “Why technology favors tyranny” (Y. Harari) (free PDF)

• “Geopolitical implications of AI and digital surveillance” (D. Peterson)

• “Designing alternatives to China’s surveillance state” (D. Peterson)

Recommended reading:

• “The malicious use of artificial intelligence” (Brundage et al)

 

April 23

Raising the wellness waterline                                      Guest: Rosalind Picard, MIT

Nobody wants a future filled with misery. But suffering is incredibly widespread. At any given time, an estimated 20% of people experience some symptoms of mental illness. Many more are miserable. Suffering and mental illness make it hard to cooperate, so helping people lets them help more. To what degree are these problems biological, and how many are environmental? Do people care more about subjective happiness or fulfillment? What are the most effective treatments today? What can we invent to scalably improve well-being?

Required reading: 

• “Development, Freedom, and Rising Happiness: A Global Perspective (1981–2007)” (Inglehart et al)
• "Bowling Alone: America's Declining Social Capital" (R. Putnam)
• “Burden of Depressive Disorders by Country, Sex, Age, and Year” (Ferrari et al)
• “Finland is the happiest country in the world, and the Finns aren’t happy about it” (F. Martela)

Recommended reading:

• "Brave New World" (A. Huxley)

EDIT: Op-Ed 3 due 

April 30

TBD depending on class interest and technological advances in spring 2025   Kevin Esvelt and Michael Specter

Required reading:

• “The Expanding Circle: Ethics, Evolution, and Moral Progress” (pp. 111-124) (P. Singer)
• “The Extremist: The woman behind PETA, the most successful radical group in America” (M. Specter)
• “Evolution and Ethics” (pp. 79-86) (T.H. Huxley)
• "The Lifecycle of Software Objects" (T. Chiang)  (recommended: the entire book)

 

May 7

Project Day

Last chance to advance class projects.

 

May 14

Final project presentations

Due: Final projects