Extended audio/transcript from my conversation with Dwarkesh Patel

Extra content on my Otherness series, AI takeover, p(God), and more.

I recently had the pleasure of recording a podcast with Dwarkesh Patel. The final episode was ~2.5 hours, but we actually spoke for about six hours – four about my “Otherness and control in the age of AGI” series, and two about the basic AI takeover story. I thought that the extra content might be worth making public in some form, in case it might of interest to anyone. So with Dwarkesh’s permission, I'm posting the full conversation on my audio feed here (audio files also embedded in this post).¹ There are also transcripts available on my website.

I’ve split the conversation into two parts: “Otherness and control” (audio, transcript) and “AI takeover” (audio, transcript).

(Part 1 audio)

Extra content in part 1 includes:

• More on why we might regret doing alignment;

• Predictable updating about AI takeover scenarios;

• Marxism and the population bomb as pessimistic analogies for the discourse about AGI;

• More on deep atheism and how much we should trust competition;

• More on whether alignment is too parochial, given humanity’s evolutionary history;

• Whether concerns about AI moral patienthood actually change the bottom line;

• How to deal with uncertainty about factors that could totally change your world picture, including out-there factors like weird decision theory;

• Intersections between meditation and AI alignment;

• More on why consciousness might not be necessary for moral patienthood;

• p(God). 

(Part 2 audio)

Extra content in part 2 includes:

• Whether misaligned AIs would form a natural coalition against humans;

• Whether AI takeover would be analogous to the US choosing to invade Costa Rica;

• Whether stories about AI scheming assume the wrong picture of the nature of intelligence;

• What a concrete scenario that leads to AI takeover might look like;

• Whether to expect flaws in the training signal to lead to the bad type of misalignment;

• Whether to expect *long-horizon* forms of goal mis-generalization and/or reward-seeking in particular;

• More on what a good outcome might look like;

• What we can do now to promote a good outcome;

• Whether AI risk is *suspiciously* interesting;

• Tribalism and mistake theory in the AI risk discourse.

Enjoy!

1. Dwarkesh's team made some edits to the conversation's original audio -- for example, to smooth up various transitions. I'm not sure exactly what got cut here, but I think the cuts were pretty minimal. I also made one edit myself, to cut a section where I misspoke.