Course Syllabus

Safeguarding the Future - Spring 2023 Syllabus

Semester schedule

Feb 7  

Feb 14     

Feb 21    

Feb 28   

Mar 7    

Mar 14  

Mar 21   

Mar 28  

Apr 4   

Apr 11     

Apr 18  

Apr 25

May 2    

May 9  

May 16

The present has no future Links to an external site.

A dearth of energy Links to an external site. (White)

Existential risk CLASS CANCELLED

Nuclear game theory Links to an external site. (Weber Links to an external site.)    

Mistakes and madmen (Rose Links to an external site.)

Transformative AI (Cotra Links to an external site.)

* A world without biorisk

<Spring Break, no class>

Persuastion of the Powerful Links to an external site. Links to an external site. (op-ed review) Links to an external site.

Coordinating preparedness (Bright Links to an external site.)

Raising the wellness waterline Links to an external site. (Picard)  (op-ed review) Links to an external site.

Open discussion: Neurotech 

Red-team LLMs

Final project development Links to an external site.

No class

Feb 9       

Feb 16      

Feb 23   

Mar 2

Mar 9

Mar 16

Mar 23

Mar 30  

Apr 6 

Apr 13    

Apr 20

Apr 27

May 4   

May 11 

 

Storytelling and history Links to an external site.

The planetary thermostat and Geoengineering Links to an external site.

* Right idea, wrong PR (op-ed review)

Biological catastrophe

Project blizzard (bring 3 ideas)

Paths to AI Alignment (Christiano Links to an external site.)

Project development (finalize teams)

<Spring Break, no class>

* On reflection (op-ed review)

Resilient logistics: feeding everyone

Unknown unknowns Links to an external site.

Rebuilding civilization

Final project development Links to an external site.

Final project presentations (Project due) Links to an external site.

 

Example project: engineering/solutions Links to an external site. (reducing infectious disease transmission in buildings)

Example project: research/analysis Links to an external site. (assessing the contribution of feeder rodents to animal suffering)

(We'll also share some example projects from last year, one of which impacted this year's course)

 

Syllabus (dates and assignments subject to change with guest speaker availability)

 

Feb 7 - The present has no future

At our current rate of economic growth, we’d be using every atom in the galaxy in less than 10,000 years… which is a problem because the Milky Way is more than 50,000 light-years across. In less that 150 years, we’ve gone from fountain pens to tweets, cavalry to ICBMs, and snake oil to CRISPR-mediated T cell therapy … yet somehow we imagine that the future will consist of people like us who are essentially still making all the decisions. If the present world cannot endure without catastrophe or radical transformation, then the future will be quite different from what most people expect. It might offer delight and fulfillment literally beyond our current capacity to imagine, a hellish dystopia, or an empty universe. If we prefer the first option, we’d best figure out how to nudge our trajectory in a positive direction  How do we go about even deciding how to do that without creating unintended consequences that might be worse than we could imagine?

 

 

Feb 9 - Storytelling and history

We’re creatures of the story. There is no human history without somebody capable of presenting a compelling narrative. And we can shape the future by deciding how to tell our story. How, then, can we describe the arc of human history in ways that will improve the future? What can we learn from it? How many people lived in each epoch, and under what conditions? When and what were the turning points that changed history? Were they inventions, ideas, or wars? Which stories have changed the world? …for the better and for the worse? 

Due:

 

 

Feb 14 - A Dearth of Energy                    Guest: Anne White

Some claim that recent technological advances were constrained by energy limitations. Computation, which is unquestionably the most transformative area of technology of the last fifty years, is noteworthy for requiring minimal energy. If we had access to unlimited clean energy, almost every material problem could, in principle, be solved. How likely is that? And what do we need to do to make it happen?

Due: 

Recommended: 

 

 

Feb 16 - The planetary thermostat

Few people continue to doubt that climate change is real and that human activity is the major cause.  Many assume that warming of 2C - a figure that only twenty years ago was considered catastrophic - is now unavoidable. But we have implemented only minor efforts to lessen our dependence on fossil fuels. That will dramatically increase water and food shortages in the poorest parts of the world and gradually submerge land occupied by a significant fraction of humanity. Should we intervene to speed up our efforts to correct our mistakes?  What are the possible consequences, and who should decide?

 The long-term impacts, let alone the tail risks, will almost certainly make everything else more difficult. What can we do?

Due: 

 

 

Feb 21 - Existential risk CLASS CANCELLED DUE TO MIT HOLIDAY

That the world is getting hotter is often described as an existential risk. Is it? What does “existential” mean? Does it include global catastrophic risks? What defines those? Local or worldwide civilizational collapse? Stagnation? Human extinction? Becoming locked into a social or technological trap from which there is no escape? How important is each of these... and how can we be sure we're not just millenarians fixed on the end times?

 

 

Feb 23 - Right idea, wrong PR

Some view technology as deterministic. But even if people will eventually adopt whatever is useful, multiple technologies may fill a need, in which case outside factors can decide which of them will be developed. To what extent does the success of technology depend on its inventor? On a popularizer? A funder? What happens when a true and important idea is devised by the wrong person or the wrong place? Or the opposite?

Due:

 

 

Feb 28 - Nuclear game theory            Guest: Andrew C. Weber Links to an external site.  

Leo Szilard, the inventor of the chain reaction, warned that dropping the bomb on Japan in 1945 would accelerate an arms race that would make American cities vulnerable to foreign militaries for the first time in half a century. He was right. Nuclear weapons cannot be blocked and can only be deterred through mutually assured destruction. Whether they kept the Cold War cold is debatable, but it was always touch and go. We came close to losing around half of humanity in 1983. What can we learn from our first experience with a technology capable of threatening so many of us? And what does the current war between Russia and Ukraine mean for the wider nuclear threat?

Due:

Recommended:

 

 

Mar 2 - Biological catastrophe

COVID-19 was caused by a single, comparatively mild pandemic virus that wasn’t even engineered to cause harm.  Around thirty thousand people today could single-handedly assemble most viruses from synthetic DNA using cheap equipment. That number gets significantly larger every year as the price of computing power drops and the tools of genetic engineering grow. For years, scientists have tried to find viruses in nature that can harm us. Should they be doing that?  What would happen if they were to post a list of credible pandemic viruses found in nature? Or learn how to engineer more destructive ones? How can we best immunize civilization against catastrophic biology… and delay disaster until we have succeeded?

Due:

 

 

Mar 7 - Mistakes and Madmen            Guest: Sophie Rose Links to an external site.

Do most catastrophic risks arise from accidents or deliberate actions? History suggests that our record of accidents is mixed and could certainly be improved. But the real uncertainty concerns the number of people who would push a doomsday button if given the opportunity. To date, only great powers have had that chance, and their actions are largely tempered by game theory. Should many more individuals gain that power, we may expect a different outcome.

Due:

 

 

Mar 9 - Project blizzard

Discussions are all very well and good, but we also hope to achieve something concrete this semester. Think of 3 project ideas you consider intriguing that are relevant to safeguarding the future. They can be on any subject. Maybe you're worried about technological stagnation, or a totalitarian autocracy trap, or contacting aliens. If you've a creative concept, bring it and share it.

Due:

  • Come ready to present 3 project ideas

 

 

Mar 14 - What is transformative AI?                Guest: Ajeya Cotra Links to an external site.

We have already created systems with superhuman capacities. Soon AIs will be able to accelerate the pace of science and technology development reliably. That will likely return the world to a cycle of exponential growth and innovation that halted when we stopped having as many babies as we could feed, interrupting the sequential conversion of inventions into calories into inventors. The resumption of that cycle will mark the end of our current world. When will it happen, and how can we begin to prepare? Open AI’s recent release or Chat GPT - while not in any way challenging the dominance of humanity - seemed to wake many people up to the fact that AI will have abilities we have not yet imagined or planned for.

Due:

Recommended:

 

 

Mar 16 - Paths to AI alignment               Guest: Paul Christiano Links to an external site. Links to an external site.

What happens when we create agents more capable than we are? After all, chimpanzees are nearly extinct and mostly reside in zoos. Ideally, we want anything with power to be not only aligned with our interests as the average human but much more so. Even given perfect alignment with human interests, machines with significantly superhuman competence will still upend the strategic gameboard, potentially creating incentives for people to act dangerously. To give just one example, superhuman competence in biology might create catastrophic threats that could reliably overcome our human-designed defenses. How can we both align transformative AI systems and prevent their superior capabilities from causing harm?

Due:

Recommended:

 

 

Mar 21 -  A world without biorisk

The world is demonstrably vulnerable to the introduction of a single
pandemic virus with a comparatively low case fatality rate. The deliberate
and simultaneous release of many pandemic viruses across travel hubs
could threaten the stability of civilization. Current trends suggest that within
a decade, tens of thousands of skilled individuals will be able to access the
information required for them to single-handedly cause new pandemics.
Safeguarding civilization from the catastrophic misuse of biotechnology
requires delaying the development and misuse of pandemic-class agents
while building systems capable of reliably detecting threats and preventing
nearly all infections. This appears surprisingly feasible.

Due:

 

 

Mar 23 - Project development

 

 

Apr 4 - Group's Choice: Persuasion of the Powerful

Due:

  • Op-Ed #2

Suggested readings

 

Apr 6 - On reflection

Let’s try and focus on realistic measures we might take to address the impact AI will have over the next twenty to thirty years. It would be good if everyone could come to class with some concrete ideas for how we  can use AI - because obviously we will use it - in a measured and valuable way.    Also,  we may well have to reorganze society if AI eliminates most of our jobs. But rather than just sort of say; hmmm, we will need a revolution or some acceptable approach to socialism, maybe everyone can think a bit about how we actually WOULD go about reorganizing. What do you want AI to do for you and what to you want it not to do?  None of us have been that concrete about these issues because there are no magic or concrete solutions. But let’s pretend human’s will still run the world for a while and that we continue to have the power to deploy technology the ways we want it used. What will that mean?

Due:

 

Apr 11 - Coordinating preparedness                 Guest: Rick Bright Links to an external site.

COVID-19 gave us a reasonable snapshot of how well human civilization can be expected to respond in a global crisis. What can we learn from the last two years? Which institutions can we rely on? In which countries? Are there any patterns? How can we compensate for weaknesses and uncertainties?

Due:

 

 

Apr 13 - Resilient logistics: feeding everyone

There is an inherent tradeoff between efficiency and robustness. By definition, a maximally efficient system has no redundancy. Just-in-time manufacturing minimizes warehouse storage time and wasted materials but is fragile to supply chain disruptions. How can we render our supply chains resilient when market forces work to render them fragile? Which are particularly at risk, and what can we do?

Due:

Recommended:

 

 

Apr 18 - Raising the well-being waterline                 Guest: Rosalind Picard

Nobody wants a future filled with misery. But suffering is incredibly widespread. At any given time, an estimated 20% of people experience some symptoms of mental illness. Many more are miserable. Suffering and mental illness make it hard to cooperate, so helping people lets them help more. To what degree are these problems biological, and how many are environmental? Do people care more about subjective happiness or fulfillment? What are the most effective treatments today? What can we invent to improve well-being?

Due:

 

 

Apr 20 - Unknown unknowns

Due:

  • Op-Ed #3

 

 

 

Apr 25 - Open discussion: Neurotech

We've spent a lot of time this semester talking about AI, but what about the future of the human brain? Let's explore the possibilities of neurotech. For a technical overview of the possibilities, we'll read "Physical principles for scalable neural recording." For a logical-extreme, we'll read a review of Age of Em by Robin Hanson (notably, a pioneer in prediction markets). For a guardian angel story, that also double as a "what if everything goes right" story, we'll go with "The Adventure." And as a view on the business side, we'll read "Neuratech at Work."

Due

 

Apr 27 - Rebuilding civilization Links to an external site.

Given the non-trivial probability of civilization collapsing, it’s remarkable how little has been done to help the survivors rebuild. What do we most want to pass on to our successors? To their descendants? Can we learn anything from past civilizational collapses and recoveries? To what extent do the answers depend on the nature of the threat that ends the rest of us?

Due: 

Recommended:

 

 

May 2 - TBD

 

 

May 4 - Projects

Either talk by Rosalind Picard or students, or time for project development

 

 

May 9 - Project Development Time

 

 

May 11 - Project Presentation

Due:

  • Physical submission of Final Project on Canvas
  • Project presentation at class

 

May 16 - No class