Course Syllabus

Safeguarding the Future - Spring 2024 Syllabus

Feb 7   Feb 14      Feb 21     Feb 28    Mar 6     Mar 13   Mar 20    Mar 27    Apr 3    Apr 10      Apr 17   Apr 24 May 1      May 8

The present cannot endure (Esvelt, Specter) The history and future of human agency (Crawford) Avoiding the tyranny trap (Peterson) Rewriting life (Esvelt) The energy bottleneck and climate change (White) Transformative AI (Cotra) Nuclear game theory (Weber) <Spring break> Communicating technical subjects (Specter) Coordinating to solve problems (tbc) Raising the wellness waterline (Picard) Moral circle expansion (Esvelt and Specter) Class choice or surprise (influential result of 2023's class)  Final project presentations

 

Syllabus (dates and assignments subject to change with guest speaker availability)

 

February 7

The present cannot endure                                             Kevin Esvelt and Michael Specter

At our current rate of economic growth, we’d be using every atom in the galaxy in less than 10,000 years… which is a problem because the Milky Way is more than 50,000 light-years across. In under 150 years, we’ve gone from fountain pens to tweets, cavalry to ICBMs, and snake oil to CRISPR-mediated T-cell therapy … yet somehow, we imagine that the future will consist of people like us, living much the way we do today. If the present world cannot endure without catastrophe or radical transformation, then the future will be quite different from what most people expect. It might offer delight and fulfillment literally beyond our current capacity to imagine, a hellish dystopia, or an empty universe. If we prefer the first option, we’d best figure out how to nudge our trajectory in a positive direction. Are we moving towards a radically different future? What are the crucial drivers? And how can we communicate problems and solutions to best safeguard the future?

 

 

February 14

The history and future of human agency             Guest: Jason Crawford, Roots of Progress

Humanity has gained increasing control over the essentials of life and the economy: our food supply, our materials and manufacturing processes, our use of energy, and our transportation networks. A prerequisite of safeguarding the future is the continuation of this historical process. Commentators from Samuel Butler to Vernor Vinge to the present have expressed concern that humanity will lose control. We will approach this topic first historically, arguing the apparently unpopular position that human agency has been increasing, not decreasing, over time. We will cover present evidence for this view, and suggest key drivers of the trend. Then we will discuss whether we can expect this trend to continue or whether it might reverse, and why.

 

Required reading:

• “The short history of global living conditions” (M. Roser)
• “We need a new science of progress” (P. Collison, T. Cowen)
• "Darwin Among the Machines" (S. Butler)
• "The Future as a Way of Life" (A. Toffler)
• "The Coming Technological Singularity" (V. Vinge)
• "Do we get better or worse at adapting to change?" (J. Crawford)
• “1960: The year the singularity was cancelled” (S. Alexander)
• “This can’t go on” (H. Karnofsky)

Due: Sketch of a future much-improved universe

Recommended Reading:

• Invention and Innovation: A Brief History of Hype and Failure, 2023  (V. Smil)

 

 

February 21

Avoiding the Tyranny Trap                                               Guest: Dahlia Peterson, CSET

We can see more than ever – and others can see us. Ubiquitous cameras and smartphones make it possible to keep track of everyone, including where they go and whom they meet. Some nations have already instituted broad systems of surveillance. When does public safety yield to autocratic control? Are there ways of enhancing security without compromising freedoms? How will increasingly powerful artificial intelligence shape the centralization of power in society?

Required reading: 

• “The grim fate that could be worse than extinction” (D. Minardi)
• “The totalitarian threat” (B. Caplan) (free PDF)

• “Why technology favors tyranny” (Y. Harari) (free PDF)

• “Geopolitical implications of AI and digital surveillance” (D. Peterson)

• “Designing alternatives to China’s surveillance state” (D. Peterson)

Recommended reading:

• “The malicious use of artificial intelligence” (Brundage et al)

 

 

February 28

Rewriting Life                                                                                                    Kevin Esvelt

COVID-19 was caused by a single, comparatively mild pandemic virus that wasn’t even engineered to cause harm. Around thirty thousand people today could single-handedly assemble pandemic influenza viruses from synthetic DNA using cheap equipment. For years, scientists have tried to find viruses in nature that can harm us. Should they be doing that? What would happen if they were to post a list of credible pandemic viruses found in nature or describe how to engineer more destructive ones? Can we avoid this while still ending disease and aging? How can we best immunize civilization against catastrophic biology… and delay disaster until we have succeeded?

Required reading: 

• Publish and Perish: The Hazards of Sharing Biological Information.  Chapter 5, Higher Animals, audiobook M. Specter Pushkin Press (Class will be sent mp3s)

• “Delay, Detect, Defend” (K. Esvelt)

• “Guarding against catastrophic biological risks” (J. Yassif, S. Korol, A. Kane)

• “Who would destroy the world?” (P. Torres)
• “Securing civilisation against catastrophic pandemics” (Gopal et al)

Due: Writing #1

 

 

March 6

Energy bottlenecks and the global thermostat                      Guest: Anne White, MIT

Some claim that recent technological advances were constrained by energy limitations. Computation, which is unquestionably the most transformative area of technology of the last fifty years, is noteworthy for requiring minimal energy. If we had access to unlimited clean energy, almost every material problem could, in principle, be solved. What might we do to make it happen?

Many now assume that warming of 2oC – a figure that only twenty years ago was considered catastrophic – is unavoidable. That will dramatically increase water and food shortages in the poorest parts of the world and gradually submerge land occupied by a significant fraction of humanity. Should we engage in geoengineering to correct our mistakes? What are the possible consequences, and who should decide? 

Required reading: 

• Review of Energy and Civilization (B. Gates)
• “Why nuclear power has been a flop” (J. Crawford)
• “The limits to growth in organic economies” (E.A. Wrigley)
• “The Climate Fixers: Is there a technological solution to Global Warming” (M. Specter)
• “Evaluating climate geoengineering proposals in the context of the Paris Climate Agreement temperature goals” (Lawrence et al)
• “Marine cloud brightening” (Latham et al)
• “Halving warming with idealized solar geoengineering moderates key climate hazards” (Irvine et al)
• “How green sand could capture billions of tons of carbon dioxide” (Temple)

Recommended reading: 

• Energy and Civilization (V. Smil)
• LongNow Video: Saul Griffith, Climate Change Recalculated

 

 

March 13

Aligning transformative AI                              Guest: Ajeya Cotra, Open Philanthropy Project

We have already created systems with narrowly superhuman capabilities. At some point, AI will noticeably accelerate the pace of technology development. That will likely return the world to a cycle of exponential growth and innovation that stalled when we stopped having as many babies as we could feed, interrupting the sequential conversion of inventions into calories into inventors. The resumption of that cycle will mark the end of our current world. When will it happen, and how can we begin to prepare? ChatGPT – while not in any way challenging the dominance of humanity – seemed to wake up many people to the fact that AI will have abilities we have not yet imagined or planned for.

What will happen when we create agents more capable than we are? It's concerning that chimpanzees are nearly extinct and mostly reside in zoos. Anything with tremendous power must be much more aligned with our interests than the average human. And even given perfect alignment, machines with significantly superhuman competence will still upend the strategic gameboard, potentially creating incentives for people to act dangerously or even catastrophically. How might we align transformative AI systems and prevent their capabilities from causing harm? 

Required reading: 

• “A Playbook for AI Risk Reduction” (H. Karnofsky)

• “A Race to Extinction: How Great Power Competition Is Making Artificial Intelligence Existentially Dangerous” (S. Meacham)

• "This Changes Everything" (E. Klein)
• “What failure looks like” (P. Christiano)
• “Overview of the AI alignment landscape” (N. Nanda)
• "Yudkowsky Contra Christiano on AI Takeoff Speeds” (S. Alexander)

Recommended reading:

• Executive Order 14110 (U.S. government)

• “Timelines to transformative AI” (A. Cotra)

• Human Compatible (S. Russell)

• Living with Artificial Intelligence (S. Russell)

Due: Rewrite of Writing #1

 

 

March 20

Nuclear game theory                         Guest: Andrew C. Weber, Council on Strategic Risks

Leo Szilard, the inventor of the chain reaction, warned that dropping the bomb on Japan in 1945 would accelerate an arms race that would make American cities vulnerable to foreign militaries for the first time in half a century. He was right. Nuclear weapons cannot be blocked and can only be deterred through mutually assured destruction. Whether they kept the Cold War cold is debatable, but it was always touch and go. We arguably came close to losing half of humanity in 1983. What can we learn from our first experience with a technology capable of threatening so many of us? And what do the war between Russia and Ukraine and rising tensions between the United States and a China that is modernizing its arsenal imply for global nuclear risk?

Required reading: 

• “Nuclear weapons: A beginner’s guide to the threat” (P. Webber)

• “The Return of Nuclear Escalation: How America’s Adversaries Have Hijacked Its Old Deterrence Strategy” (K. Lieber and D. Press)

• “How Likely is a Nuclear Exchange Between the US and Russia?” (L. Rodriguez)

• “Doomsday Clock Stands at 90 Seconds to Midnight, Closer Than Ever to Global Ruin” (UN General Assembly)

• NUKEMAP (A. Wellerstein) [instructions: try out some parameters]

• “Bombing the Marshall Islands: A Cold War Tragedy. Cambridge University Press. Chapter 3: Runaway Bomb. (K. Parsons and R. Zaballa) (accessible through MIT libraries as PDF download)

• “The fear of setting the planet on fire” (Y. Yiu)

• “The Unilateralist's Curse” (N. Bostrom et al)

Recommended reading:

• The Dead Hand (D. Hoffman)

• Nuclear Ethics Revisited (Joseph S. Nye Jr.) 

 

 

April 3

Communicating technical subjects with minimal controversy             Michael Specter

The technical challenges we will have to adopt - or choose to reject -will make it essential that technologists express themselves plainly, directly and in ways that non-scientific people do not find threatening. If you can’t express a philosophical point of view in a way that is widely understood how can you hope to set an agenda for a technological future?

Required reading:

•  “My Techno-Optimism” (V. Buterin)
• “ An Ecomodernist Manifesto” (J. Asafu-Adjaye)                                                                 

Due: Writing #2

Due: Final project proposal

 

 

April 10

Coordinating to solve problems                                                                        Guest: tbc

Our world is governed by a constellation of bodies: governments, industry, academia, to name a few. They need to work together and they rarely do. The COVID pandemic was the most recent and horrifying example of government bodies competing with each other (even just within the United States). Between the ability of government, the desires of industry, and the often theoretical goals of academic researchers, we need to shape how we develop, adopt, and govern use of technologies. How do we corral these forces together? What are historic examples of large coordination efforts, and what lessons from them can we apply today?

Required reading:

• "The Asilomar Conference: A Case Study in Risk Mitigation" (K. Grace)
• “The Early Days of Pugwash” (J. Rotblat)
•  Trinity: “The most significant hazard of the entire Manhattan Project” (By Kathleen M. Tucker, Robert Alvarez)
• Reconstructing the Cost of the One Giant Leap, Casey Drier, The Planetary Society

 

 

April 17

Raising the wellness waterline                                      Guest: Rosalind Picard, MIT

Nobody wants a future filled with misery. But suffering is incredibly widespread. At any given time, an estimated 20% of people experience some symptoms of mental illness. Many more are miserable. Suffering and mental illness make it hard to cooperate, so helping people lets them help more. To what degree are these problems biological, and how many are environmental? Do people care more about subjective happiness or fulfillment? What are the most effective treatments today? What can we invent to scalably improve well-being?

Required reading: 

• “Development, Freedom, and Rising Happiness: A Global Perspective (1981–2007)” (Inglehart et al)
• "Bowling Alone: America's Declining Social Capital" (R. Putnam)
• “Burden of Depressive Disorders by Country, Sex, Age, and Year” (Ferrari et al)
• “Finland is the happiest country in the world, and the Finns aren’t happy about it” (F. Martela)

Due: Writing #3

Recommended reading:

• "Brave New World" (A. Huxley)

April 24

Moral circle expansion                                                       Kevin Esvelt and Michael Specter

Over the past twenty five years there has been a significant shift in our view of other species and the role they play on this planet. How can we continue to justify the way we treat other species - from a moral or an environmental perspective.  The idea, propagated from the early verses of Genesis until now suggests humans have “dominion” over other animals and the planet.  What rights should animals have, and how, for that matter, should humans treat each other - particularly those who define their own humanity in more complicated terms than: male or female.

Required reading:

• “The Expanding Circle: Ethics, Evolution, and Moral Progress” (pp. 111-124) (P. Singer)
• “The Extremist: The woman behind PETA, the most successful radical group in America” (M. Specter)
• “Evolution and Ethics” (pp. 79-86) (T.H. Huxley)
• "The Lifecycle of Software Objects" (T. Chiang)  (recommended: the entire book)

 

May 1

TBD based on interest and tech advances

 

 

May 8

Final project presentations

Due: Final projects