Discussing chapters 1-10 of Mary Shelley’s 1818 genre mash-up Frankenstein.
On Mary Shelley’s stacked genetics, the ‘scenius’ with Lord Byron and Percy Shelley, questions over authorship including a suspiciously accurate depiction of post-nut clarity.
Forbidden knowledge: are infohazards real, taking accountability for new technology, guilt and the disgust instinct, strong parallels with AGI, arguments for and against creating new species. Can we defend a parochial concern for our own family/friends/species?
Is the monster innately evil? Or a product of his environment?
We love this book. hyped to hear the monster’s side of the story in part 2.
pop culture Frankenstein and namespace collision
Rich: She’s a bit sturdy, right? By her own admission she’s like maybe a bit on the spectrum, or autistic.
Cam: You know, now after reading that autistic booker.
Rich: So I can diagnose them. I just have to listen to anyone speak for 30 seconds and I can offer them a diagnosis for free. I’ll do you guys right now if you like.
Benny: Had you guys read this in school? It’s not something you have to read in school.
Rich: This is pretty sophisticated for high school. But I definitely read it when I was probably 19 or 20 or something like that. I’ve forgotten everything. I remember a couple of little bits here and there, but I can’t remember the plot, so don’t spoil me.
Cam: So you’ve been saying Frankenstein’s monster for like 10 years.
Rich: That was so funny in the foreword, where they were like, people started making that mistake from like the 1840s or something, like 20 years after it came out. They were already calling the monster Frankenstein.
Cam: To me, the monster is Frankenstein. It could be useful, just when we’re talking about it, to refer to him as Victor Frankenstein.
Rich: Or just Victor. I feel like we’re on first name terms.
Cam: Victor’s useful. But it reminds me of — I think it’s kind of like, was it a gif or jif? And the guy that created it even won a Webby Award, which is online internet, where you have to say five words in your acceptance speech. His was, “it’s pronounced gif” or something like that. But no one pronounces it jif, so I’m kind of like — it’s the same as regular vocabulary, it’s how people use it, which descriptively is what makes it. It’s gif, regardless of what this guy says. So I’m kind of on that side.
Rich: But then what are you going to call the doctor?
Cam: Victor. Or Dr. Frankenstein. Or Victor Frankenstein. And sometimes you can still call it Frankenstein’s monster to be —
Rich: To be fair, if it’s his son, then we could call it like George Frankenstein, or I don’t know. It would inherit his surname if he’d had it by more natural methods. So yeah, maybe it’s fair to call it Frankenstein.
Rich: Do you think there’s any Jews out there who are still called Frankenstein? I don’t know, I just sort of assume so.
Benny: I’m not sure if it was a very popular name. I was reading some of the etymology and I think she pulled the name — there was like a Frankstein sort of name that maybe she got some inspiration from. But I’m not sure if Frankenstein itself was an actual common name at that point. It is a good name. It’s a fucking cool name. Especially if you were to be a professor. Like if you had to report to Professor Frankenstein, that’s an awesome name.
Rich: You’re at uni and you do undergrad, you get your masters, and then they’re like, do you want to do your PhD? I’ve got to become Dr. Frankenstein.
Benny: I’m a terrible student but I don’t care, that’s why I’m going for it. I just want to be Dr. Chug, that’s the only reason.
Rich: This has migrated so much from the source material, which I had also forgotten. When I think of Frankenstein — like big green zombie guy, bolt in his neck, flashes of lightning and stuff. None of that’s in this book at all. It’s a totally different book to that. It’s not like a monster-of-the-week type thing at all. But the pop version of it is so much dumber and less interesting. It would be cool to do a faithful remake of this.
Cam: Well, I haven’t watched — the most famous movie was, I think, the 1930s Frankenstein movie, played by Boris Karloff. That image of Frankenstein is — I don’t think it’s black and white so it’s not green, but with the bolts and the big forehead, that’s almost the image of people’s Frankenstein. He was so good in that. But now, if you’re going to throw Frankenstein in The Simpsons or something, you’re making him green and hanging out with Dracula and shit.
synopsis
Rich: So we start out with a frame story from the point of view of an Arctic explorer called Captain Walton, who is heading for the North Pole. He rescues a man who’s sledding across the ice in pursuit of some unknown figure, and that man turns out to be our boy, Victor Frankenstein. He’s a Swiss scientist. He is befriended by Walton, and then Walton coaxes his story out of him as to how he ends up in the Arctic. So he tells us about his early life, how he was obsessed with alchemy, and then that sort of transitioned into natural philosophy and chemistry. He studies hard and figures out a way to breathe life into some kind of creation that he’s made up of purloined body parts.
Rich: And then as soon as he does it and the monster awakens, he immediately regrets his decision and feels intense guilt and disgust. The monster runs away. He has a nervous breakdown for several weeks. And then his friend comes and nurses him back to good health. And just as he’s feeling good about himself again and ready to go out into the world, they get a letter from Geneva saying that his little brother William has been murdered. And so everyone is extremely upset. They flee back to Geneva as quickly as possible.
Rich: So Victor is pretty certain — well, he is completely certain — that the monster killed William. But his sort of foster sister, Justine, gets framed for the murder instead. And then it goes to trial and she’s found guilty and executed, which plunges everyone into even darker despair, especially Victor’s cousin Elizabeth, who’s really close with Justine. And Victor himself, but he’s feeling terrible for slightly different reasons, because he thinks that this is all his fault. So he goes hiking really deep in the Alps on a hiking expedition and encounters the monster again, who sort of begs him to listen to his story and hear him out. And so that becomes like the third nested story within the frame story, where the monster is telling his story to Victor, who is telling his story to Captain Walton, who also is actually telling his story to his sister, I think, back home. He’s writing letters about all of this. And that’s where we stop reading, which is a couple of chapters into part two, where it transitions from the monster’s point of view as to what he’s been up to in the years since he fled from Victor’s table.
Cam: Damn, we should have said at the start, you know, with the first part chapters one to seven and first two chapters of part two — anyway, no spoilers after that.
Initial reactions
Rich: I’m so hyped. I was a little bit bored to begin with because I think it takes a while to warm up with the frame stories — the letters, and I don’t really care about Captain Walton. And then even Victor’s childhood, where maybe she spends a little bit too long telling about his childhood and education. I wasn’t gripped, and I thought we might be in for a rough one. But then as soon as it got to William’s death and then Justine’s trial, I was completely hooked. And now I’m fascinated, and it’s like a page-turner, and I’m really excited to hear the monster’s journey.
Benny: I’m really enjoying the writing, which is surprising a bit, because it’s written in 18-something, which is amazing. But it’s flowery in some sense, but in the best way, and especially the prose coming out of the characters — like the monster’s prose, I guess we’re not quite gonna get there, but is amazingly eloquent in just the choice of words. I just love the way it’s written, it’s amazing. And you couldn’t get away with writing a book like this now because it would be totally excessive and unwarranted. But it’s right in a sweet spot where it’s not Shakespearean old, so the language hasn’t changed so much that you can’t — it’s easy to understand what’s written on the page, but it’s written differently enough that it’s different from reading anything modern.
Rich: Yeah, it’s completely accessible, right? There’s a couple of words that are used differently, but you can always figure it out from context, and it’s charming. Or misspellings? I think I’ve seen either typos or just like different ways of spelling words at the time.
Benny: Yeah, different ways of spelling a word with an end and stuff like that. Some old hyphenated words that are no longer hyphenated.
Cam: Just looks quite nice, but you know, a bit much. I just wanted to quickly say, I totally agree with Rich — I thought we were potentially in for a rough one, and then it was like, damn, this is classic, this is great. Although I reread the opening letters after I read volume one, and then actually I think it deserves a spot there as well. Maybe I shouldn’t start with it, I don’t know, but you can see the parallels between Walton’s journey, and it’s just nice to see the monster and Victor from another perspective as well. Actually, funnily enough, I think she wrote this a decade later, perhaps — well, maybe not a decade, but it wasn’t in there, and then she put it in there.
Rich: Which bit? The Walton frame?
Cam: I think the letters, and it probably wasn’t the second version. Maybe it was just during the process, after her drafts went to Percy Shelley and Byron and stuff, and then she threw it in there. There are open questions of how much did Percy Shelley contribute. I think the consensus is —
Rich: Yeah, a woman couldn’t have written this by herself.
Cam: I know, so it’s kind of flavored with that. I think the consensus is he probably edited it at most.
Suspiciously accurate depiction of post-nut clarity
Rich: I would have never guessed that this was written by a woman if I were reading it. And especially not at the time. I think it would have been even more shocking with the command of the language of chemistry and science and the philosophers and all of these facets of education which were barred from women at the time. And then the passage that completely sealed it for me, where I would never have guessed, is that when the monster wakes up and he suddenly goes from being manic and super into it, and then he switches straight to revulsion, and it reads to me like a perfect depiction of post-nut clarity. Where he’s like — read that passage, it’s really funny. It’s like, “oh, I had desired it with every throbbing fibre of my heart, I was feverishly excited.” And then suddenly my stomach dropped and I felt revolted and disgusted at what I had done. It’s like, “breathless horror and disgust filled my heart.” And then he runs out of the room.
Mary Shelley’s elite genetics
Cam: Yeah, I can imagine the fact it’s written by a woman at the time — just not just disbelief, I imagine everyone’s looking shocked, like more shocked than bloody Victor with this monster, like, how — they’re a woman? I imagine the book would be shocking itself as well, because it’s touching on these like creation of life and relinquishing the natural. But I suppose the thing with Mary Shelley is, damn, this chick’s a genius. How precocious she was — she was like 20 or something when she wrote this. It’s insane. But like, as you know, I’m pretty big on genetics, to my detriment. When you look at her family — William Godwin, as we know through David Deutsch, ahead of his time, potentially a genius.
Rich: And it’s crazy that he invented Godwin’s Law before Hitler as well.
Cam: That always gets mixed up with Goodhart’s. Which one’s Godwin?
Rich: That’s the one about how any internet argument will inevitably devolve into mentions of Hitler.
Benny: Even more impressive that he came up with it when he did. It’s amazing. Both the internet and Hitler. Way before its time.
Cam: And then Mary Wollstonecraft, her mother, who was like maybe the first feminist. I think maybe not — there’s some proto-feminists earlier. I mean, you can argue about Plato. There’s parts of him.
Rich: Wait, was there a feminist before Taylor Swift? Damn.
Cam: No, it started in the third wave.
Rich: Yeah, her heritage is insane. Also, Percy Shelley is an absolute freak as well. Because he’s very young too. He’s like 20.
Rich: Another thing which jumps out at me whenever I read stuff from this time is how the standard of education for the cultural elite is just absolutely astonishing. Like, Victor I think is saying how he’s somewhat self-educated and he’s embarrassed that a 15-year-old has more education than him, and meanwhile he speaks Latin and Greek and German and English. Or maybe he teaches himself Latin and Greek because he feels inadequate that the typical 15-year-old is better educated than him. Like, what the fuck?
Cam: Like a median university student back then is just so erudite and well-read. And you fast forward to now, and the median doesn’t even read. Like, you’re not even reading — you get readings, you just don’t do the reads.
Rich: Yeah, so erudition, that’s the key difference, right? That’s what’s so impressive. And these beautiful letters — I mean, I know these letters are fake, but the way that people write and communicate is just so much more sophisticated.
Cam: Yeah, you’re pretty boss to be a man of letters back in the day. And like gorgeous prose that’s good enough to publish. I suppose we can sort of do that with emails.
Benny: Yeah, no one’s publishing a pamphlet of my emails, I’ll tell you that much. Not much in there.
Rich: It’d be so humiliating if they did too, right? It’d just be so banal.
Cam: So quickly, just on the starting letters from Walton to his sister or his wife —
Rich: I think it’s his sister.
Cam: Which, yeah, I didn’t like it first, but I went back and quite liked it. There’s parallels between him and the monster a bit — I mean, we’ll find out more of what the monster’s going through, but Walton talks around how he’s really lonely. He’s on the ship with all these people and he’s just a total INTP, trying to learn, and he doesn’t really connect with anyone. And then he finally meets the stranded Victor, and Victor tells him, “I too once sought knowledge to my detriment, I see that in you, and so I’ll tell you the story.” And then there’s also this element of forbidden knowledge. It’s like one of the things —
Rich: Yeah, isn’t that an info hazard? Like, shouldn’t he just say nothing at all?
Forbidden knowledge and infohazards
Cam: Yeah, you can. But one thing I read is this idea of forbidden knowledge.
Rich: I noticed that he skips over the exact method by which he animates the corpse, right? Which is very convenient for Mary Shelley’s science fiction, as well, to not have to actually give a plausible mechanism. See what happened there?
Cam: I’m quite pulled by this forbidden knowledge. I don’t like reading about — there’s stuff that’s more like creating info hazards, but it seems related. I was just trying to — why do I like forbidden knowledge? I think one is, it’s likely to update your views on things. That’s why it’s forbidden — that’s the reason why it’s forbidden. And then the impact of why it’s forbidden is like, it’s hard to find, and so there’s that cool element to it as well. Well, there is a little bit of like creating new knowledge rather than finding existing stuff. But Victor himself read all these old — so I looked a little bit into the old people he was reading. Like, I think Agrippa, and I’m forgetting the others now.
Rich: Paracelsus.
Cam: Paracelsus, ibuprofen, Albertus Magnus. But they were all kind of like alchemists and other things. But I think even those guys were known to be at a time controversial for forbidden knowledge then. And just this kind of like shark with blood — Victor himself is so intrigued by it. I think the post-nut clarity is quite good.
Benny: Yeah, that’s actually a nice parallel between Walton being lonely and the monster being lonely, I hadn’t actually thought of that. I think also worth noting, I think there were a couple people while Mary Shelley was writing this, or at least around the same time, that were interested in trying to reanimate corpses. There were like some Italian dude doing experiments on dead corpses with electricity, trying to re-shock them into life and stuff. And so, I don’t — explicitly refers to any of them. But certainly these sort of ideas were, to some extent, circulating around scientific circles at the time. Which makes sense, right? You get a certain level of sophistication in your science and you start wondering about this thing called death and wondering if you can cure it. And the initial attempts to do that are —
Rich: If you shoot electricity, does it make like your fingers contract or something like that? Presumably it could, right? It could cause the nerves to fire.
Benny: I would imagine, yeah. At least if you’re fresh enough.
Cam: So question — is this the first science fiction?
Rich: That’s what they say, yeah. It’s an intersection of a lot of genres. I think it’s a really fucking clever book, because there’s these big questions. Should we get to some of the big questions? So like, building on what you’re saying about forbidden knowledge, Cam — it’s basically the exact same debate that we’re having today. How much progress should we have? Are there certain things that we should just leave well enough alone and beyond our dominion as humans, and that sort of amount to playing god? And so the book’s subtitle, I think in some editions, is “A Modern Prometheus.” So there’s a very obvious parallel there. It’s not clear to me yet whether Shelley is sympathetic to Frankenstein’s attempts, or it’s meant to be a tale of hubris. I guess it’s probably meant to be a tale of hubris, which would be sort of in line with all the classical teachings on this thing.
Cam: Maybe I’m influenced by knowing Shelley is Godwin’s daughter, and Godwin was so ahead of his time, not only with the internet phenomenon, but he’s so optimistic and ahead of his time in terms of allowing children to learn everything, allowing women to learn everything. And I know there’s this interesting paradox with why the fuck Mary Shelley fell out with Godwin. It’s kind of both, I feel. I think we’ll know more once we read volume two. If Frankenstein is causing catastrophe, what’s driving that? Okay, so one thing I was thinking — why it’s so on thing for us around this AI debate, is just, as David Deutsch says, you create AI, it will be like a person, and we need to not enslave it. And if you enslave it, that’s going to cause it to revolt. But with Frankenstein, you create this life, and if you choose to enslave it, yes, it’s going to kill everyone. Now, we’re going to find out exactly how Frankenstein was treated. Because he’s arguing, well, he wasn’t really enslaved — like maybe he’s deserted. It’s like, come on. He’s been deserted, you know —
Rich: He’s got major daddy issues.
Cam: Yeah, so I probably wouldn’t go kill everyone.
Rich: Imagine the first time you meet your dad and he’s like — and just runs out of the room.
Cam: It’s only like the ugliest people in the world. I don’t remember when you’re a baby, you just have weird reactions, like an ugly baby or something.
Benny: No one ever knows they have an ugly baby. You don’t have to be euphemistic about it.
Rich: So, yeah, we will find out more later. It’s interesting though that Frankenstein himself, contrary to I guess the pop culture depictions, is not mad. He’s not really a mad scientist. And he’s not obviously hubristic, and he’s not obviously portrayed as being a bad guy. In fact, not at all. He’s not evil and he’s not insane, he’s just obsessive and —
Benny: He just seems extremely curious, right?
Rich: Extremely curious. So the question is, is this a parable about curiosity kills a cat, or something on the obsession theme? At least there’s some direct text on that, where Frankenstein’s father is saying, while you’re away studying you need to be writing home at least as often as you are working on your research, because it’s important to be well balanced in all things. And then Victor himself agrees with that and is like, yeah, you should maintain your tranquility at all times, and you shouldn’t be swept up in passion for anything, and that includes the pursuit of knowledge. Which is a weird kind of — it reminds me of like a Roman moderation virtue or something. Like, not being inflamed by the passions. But like, come on, surely you’re getting out for knowledge —
Benny: Yeah, sure. If you’re going to be inflamed by anything, it’s something as pure as the pursuit of knowledge.
Rich: But that’s the framing, is that obsession is bad, and that it can perhaps make you out of touch with the world or with what is important, and can lead you down strange paths. But I think that’s directly contrary to the history of science and technology, in which obsessive freaks 100 percent have driven the way forward. Like, hugely responsible for making important big advances, have massively boosted human well-being, has not been through gentle moderation. Usually it’s often been through people who are completely obsessed with solving a certain problem.
Cam: Yeah, it’s almost like you don’t want to be one of them — like it might be destructive for yourself, but we need some of those.
Benny: Like, you wouldn’t want to be Elon Musk, but you’re sort of glad he exists.
Rich: Well not only that — you wouldn’t want to be Musk’s wife or Musk’s 17 children or whatever.
Cam: I also have a thing about that. Like, I wouldn’t want to be David Foster Wallace, right? That’s so tormented, but produces this great art. At times you kind of like, yeah, I wish I was David Foster Wallace. No, you wouldn’t, you wouldn’t take that.
Rich: I keep expecting you to wear a bandana on the call. Start wearing a bandana next time.
Benny: It’s not time to panic, but if you start wearing a bandana — you okay, Cam? You okay, buddy?
Victor as deadbeat dad
Cam: I wonder if Victor regrets just making this new person, like, you know, nine foot tall.
Benny: Scaling mountains, like, just on his own. Crazy. One place to go is, so what do you think of his reaction to the monster waking up? Because, I mean, in some sense, this is an extremely weird reaction to have. He just closes his eyes basically to what he perceives as this horror that he’s potentially unleashed on the world. He leaves his house, he just stays somewhere else for a while. When he comes back the monster’s gone, and he basically just crosses his fingers and hopes for the best.
Rich: That’s what I did when my daughter was born, but she was still there when I got back.
Cam: Yeah, it captures the male human condition very well. You hear about a fucking flight to a different country.
Benny: Distract yourself with other stuff.
Rich: That’s a good point though, Benny. It’s not explained why he has such a strong reaction, because he’s not portrayed as a religious man. Although perhaps that’s just meant to be the water in which everyone swims at this time, so perhaps it’s not even meant to be worth mentioning.
Benny: Yeah, I don’t know. I mean, I think the best way to analyze it is just as an analogy with having created something that you’re ashamed of. The particulars of the story are perhaps somewhat hard to believe — I mean, presumably he kind of knew what this thing looked like, and then the fact that it opens its eyes and all of a sudden he’s revolted, those specifics are perhaps hard to believe. But it’s not hard to believe that you’re working hard on something and when it’s finally finished you realize that this thing ended up not being what you thought, or you now believe it’s gonna have some perverse side effects when you unleash it into the world or whatever. Like, you can imagine people working on bombs, people working on DNA sequencing for various — virology, et cetera. There’s lots of these quote-unquote info hazards.
Cam: Yeah, there’s like Oppenheimer. But it is just weird, like, in the story it’s like, how does he know already that this is a terrible mistake? You probably wouldn’t have that reaction. And it’s almost like he spiritually knows. He has some kind of spiritual feeling. He already knows how dangerous this thing could be. And, you know, ugly duckling, maybe, it’s this ugly thing.
Rich: Actually, it’s not inconceivable to me when you consider that, as far as we know, he didn’t do a whole bunch of prototypes and try to reanimate little animals or whatever. So imagine that all he’s been ever working with is slabs of dead tissue, and then suddenly one opens its eye. I think that alone could be enough to give you that —
Cam: I’d be pretty scared. It’s a fucking zombie, man.
Cam: It was kind of like that moment where, like, Hagrid tells his fucking first years how to get the Philosopher’s Stone, and he’s like, oh, shouldn’t I say that? Should not?
Rich: I honestly think the post-nut clarity thing gives me the exact empathy for that. Maybe you’re jerking off and you’ve gone on some weird porn rabbit hole or something, and you’re like, yeah, this is awesome. And then you do like a total 180 within 30 seconds, and you’re like, what the fuck? Like, how did I get here? Like, why did I just one dwarf — you’re just like — the flip from fervor and passion to sudden self-disgust and recrimination.
Cam: It may not be this disgust, but just this odd feeling once you’ve done it.
Rich: The Oppenheimer stuff is a really good comparison, right, where they were so happy when they cracked the problem, and then — yeah, totally. It seems believable.
Cam: Almost in this nerdy, autistic thrive just to scientifically solve a problem.
Benny: You just enjoy the process of all the incremental problem solving and you’re enjoying figuring it out, and then the final product, you take a step back and realize, like, wait a second, what have I just created? Yes, I think that’s right. And so I’m trying to decide if Shelley is blaming Victor Frankenstein at all for the lack of care that he has, or his sort of wishful thinking after he brings his monster into existence that the problem will just go away. Because you can imagine her being pro-science, pro-progress, but taking the position that you need to be extremely careful over what sort of intellectual products, if you will, that you’re bringing into the world. And that you can’t just invent something and then put it into the — make it everyone else’s problem. Assuming it’s going to be sorted out by everyone else. You need to take ownership and responsibility over whatever you’re creating.
Cam: Yeah, I think the warning for the info hazard, the dangers of knowledge, is not just creating knowledge, but it’s our actions around that as well. And if you then neglect it or enslave it or whatever, that there is perhaps the bigger risk than just discovering something new.
AGI comparison: how do we feel about creating a new species?
Rich: Yeah, so I feel like I can predict extremely well how you guys feel about the general question of progress and gaining new scientific knowledge and so on, but I don’t think I’ve actually talked to you about the idea of creating a new species itself. Like, if and when we get AGI, we create another type of being that has personhood, that has consciousness and has moral rights in the same way that humans do — do you think we should do that or not?
Cam: Well, even before AGI, there’s this huge taboo around just cloning something, or bringing back like a dinosaur or something. Which, you know, would be cool. But I remember in school, you hear about the sheep, and you just think it’s going to take over, and then just like nothing happened much for 20 years. And there’s this artificial life — you even see with artificial meat, just non-natural, people don’t like it. And I share a bit of that — I have a bit of a yuck instinct of, you know, clones knocking about and shit like that. It probably doesn’t answer your question exactly.
Rich: Yeah, I guess I’m less interested in aesthetic considerations and more just, like, should we — I mean, for one thing —
Cam: Well, I think it’s a bit of why Frankenstein is like so serious, because he wasn’t born — like, we’re defying God’s method and being God ourselves. And yeah, that’s what people can sit around with AGI for sure.
Rich: Yeah, a lot of times our instincts serve us well. The disgust reaction is super useful. But this is something beyond the realms of our instincts, I think. And I think most people don’t even realize this, because they think that AI will be a big pocket calculator. And to be fair, the current ones almost certainly are big pocket calculators, and we don’t have to think about this stuff. But we’re plausibly on track that we should at least be considering it. And it might very well be within our lifetimes. So there’s the moral considerations of enslaving sentient beings, which I think we don’t have a good track record of, given how we treat animals.
Cam: But then there’s also the moral consideration of, well, if you don’t do that, is it going to destroy us? Very smart people are very concerned around this existential risk of destroying us and doing something bad — enslaving it might be worth it.
Rich: Well, that would be another reason not to bring it into existence at all, right? If you had the option. Because then you don’t have to worry about whether or not it —
Cam: But then you get the question of, someone’s going to.
Rich: Yeah. And I think that’s the actual answer to these Promethean-type concerns. And it’s the same answer for the nuclear bomb. It’s the same answer for AI — that unless you have some kind of totalitarian world government, there’s no way of stopping it from happening anyway.
Cam: Well, I suppose you can say, no, it’s the wrong thing to do. You don’t drop a nuclear bomb on people because it’s the wrong thing to do. Well, yeah, but what if the journalists find it first? I mean, we’re all a bit utilitarian bent, mixed with virtue.
Rich: Yeah, I’m pretty convinced by — I watched Oppenheimer, I really enjoyed it, I thought it was really good. And then I listened to the Dwarkesh episode again with the guy who wrote the —
Benny: Richard Rhodes.
Rich: Yeah, Richard Rhodes. And I am quite convinced that needed to happen in some sense. You could debate about individual details about whether or not they should have done Nagasaki or whatever, but I think more or less people made the right difficult decisions there. And I think certainly other countries would have developed the bomb and things could have gone much differently. So yeah, it does seem inevitable absent some kind of totalizing power that lets you stamp out innovation worldwide. As long as you have a multipolar world —
Cam: Frankenstein’s monster before the Chinese or something.
Rich: Yeah, yeah. Without saying which faction is the right one to do it or who’s better than who, just looking at it more abstractly, you can’t just say don’t do it. I mean, unless you’re a naive Kantian or whatever, it doesn’t really make sense to say, oh, let’s just not do it.
Benny: Well, I was going to say, another way to thread the needle is just to take the obvious problems that came about him creating the Frankenstein’s monster alone in the book — the issues that raises. Like, I think part of the issue is that he did this totally — it was a total solitary effort. He didn’t tell anyone about it, no one knew this was happening. So he had no input from the outside, he didn’t do this in any institutional framework that could help guide his actions, or ask various questions like, if you bring one of these into existence, is it now normatively required of us to bring others? Because it’s not going to be quite like a human, and do we expect it to integrate perfectly into human societies? If we bring one in, are you obliged to now bring a group of them in? He wasn’t wrestling with any of these questions. But if he had been doing this out in the open, I think it’s much more likely people would have started to raise these kind of questions. And these are the kind of things — the details that actually become extremely important once you actually create this sort of thing. I would take issue with, like, we should start thinking about this with respect to current AI systems, because I think we’re nowhere close to anything sentient. But admittedly at one point we will be closer to something like that.
Cam: Well, the secrecy is interesting because part of it — you don’t want to be scooped. You’re like Andrew Wiles, it’s been five years trying to correct the theorem. And some people don’t work like that, but some people do. I don’t think I could work like that. But also the secrecy after the monster gets loose, and then Vic just has to say, well, he probably committed murder, and he’s not really telling anyone.
Rich: Did you notice how no one probes further on that, where he’s like, I know who the murderer is. And they’re like, oh, great. He says he knows who the murderer is. But they don’t ask, like, who? Or how do you know?
Cam: Different legal system. But yeah, surely you say, mate, look, I can have some inside knowledge. But yeah, you kind of think he’s wracked with all this remorse and guilt, but he’s still not letting people in.
The burden of guilt (the bumblebee incident)
Benny: Which is understandable, after he’s done it. Like, I don’t know if you guys have ever been in a situation like that, where you just feel so bad or guilty about something that it just wrecks any thoughts you have about anything else. Like, you’re just carrying around this burden of guilt with you.
Cam: Yeah, but I remember in primary school I had a bumblebee on my arm and we were all sitting down outside, and I just put my elbow next to this kid next to me, and it just crawled across. And the next minute he’s got a couple of fucking stings, and it’s this big deal, and I just kept that shit close to my chest, and I felt so guilty to this day, 20 years later. That’s my Frankenstein moment, fucking bumblebees. I remember the way young Kirk —
Benny: That’s like the most wholesome fucking story I’ve ever heard. That kid grew up to be a fucking serial killer.
Rich: You got re-traumatised in that David Foster Wallace story where he talks about the flapping of the bumblebee’s wings.
Cam: Yeah. You don’t tell anyone, but you still want to do stuff about it. You have this remorse and you feel responsible, but you can’t bring yourself to actually be honest.
Rich: He does want to sort of take care of the problem, in the sense that he decides that he’s going to kill the monster if he sees it again, and that’s his idea of taking responsibility.
Cam: Which is a kind of understandable reaction, right? Like, your belief that this monster is evil and it’s killed people, right?
Rich: Although he has zero evidence for that.
Cam: Well, that’s another interesting thing. It’s kind of like, he just has this felt sense. But I think, didn’t he see Frankenstein kind of lurking, right?
Benny: As he was driving into his family’s town in Geneva, he saw the monster, which is when he first thought that it was actually the monster that killed —
Cam: Well, yes, it’s an open question — if he did do it, was it self-defense? Was it something? We don’t know if it’s guilt at this stage.
Benny: I’m astonished I can’t remember the end of the story, having read it before. It’s unbelievable to me that I’m reading it as if for the first time, wondering, oh my god, yeah, did the monster do it? I can’t believe I can’t remember. I should’ve been inking things for the past 10 years.
Cam: This guy’s been strangled, and he’s bringing his eyes back to roll hands. Maybe someone else.
Rich: I mean, I assume he probably did it, but I hope he didn’t, because I’m now feeling really sympathetic towards the monster. Especially because, you know, after reading the first few pages of him talking, he’s so eloquent and he makes such good points.
Cam: Well, that’s what’s interesting around how it does — like, doesn’t he even mention Paradise Lost or some shit, when he’s talking on the mountain?
Benny: Yeah, Milton’s Paradise Lost.
Cam: Maybe I read something that mentioned it.
Rich: He does seem like a representation of Milton’s Satan, right? Maybe he does mention it. Maybe that’s why, because I was thinking about that.
Benny: I think it’s on the wiki or something, maybe.
Nature vs nurture and rebelling against god
Rich: No, we should talk about that though, because I think Milton was super influential on Shelley and Byron.
Cam: The Romantics.
Rich: Yeah, and I think influential in ways that he wouldn’t have intended. So basically, in Paradise Lost, the character of Satan is really weirdly — or weirdly by the standards of the time — sympathetic, in that you see how God is a tyrant essentially and has not allowed the angels any autonomy. And Lucifer’s rebellion seems kind of fair. And then when he gets tossed down to earth and so on, you sort of find yourself rooting for him. And apparently Milton didn’t actually intend that portrayal to be sympathetic, but it comes across as sympathetic. The idea of wanting to have individual rights and freedoms — and, you know, better to reign in hell than serve in heaven kind of thing. This does feel like a Lucifer figure, because his father has abandoned him and he is rebelling perhaps with just cause. He’s been made bad because of the neglect of his father figure, who to him is literally God, because he’s the creator.
Cam: I suppose there is this nature-nurture argument as well. If he is partly evil, is it because of neglect? Are you for an ugly guy? Or is it partly inherent in him and brought out worse because of how people treat him?
Rich: Maybe it’s further than what we’ve read, but I think it’s definitely on the side of nurture. Or she is arguing on the side of nature, based on —
Cam: Well, I know William Godwin was huge — like, just nurture, everything. There’s a few quotes from him just saying, it’s all obviously nurture. And he feels like he solved the problem back then, which, you know, as you guys know, I think contradicts behavioral genetics. But David Deutsch is on William Godwin’s side.
Rich: Yeah, and the Deutschian frame here is that Frankenstein is a person who is just another kind of person, and we know that people are not inherently good or evil, and it’s pretty silly to even think about them that way. They just respond to the incentives that they are faced with, and animalistic impulses can be reigned in by certain types of social technology and blah blah blah, layers of culture and so on.
Cam: Well, the most sort of spiritual or religious framing would be like, evil isn’t inherent to people, evil and good act on people. It’s like this kind of spiritual force that acts on people and potentially acted on Victor. Depending on your view of this obsession-fighting-God thing, that’s evil acting on him, and perhaps evil acts on Frankenstein’s monster as well.
Rich: Yeah, I guess I find it hard to think about that. But in my language it’d be like, certain actions have good or bad consequences, and we can try and come up with rules, laws, social mores that lead to more of the good consequences and fewer of the bad consequences. But there’s no platonic forces outside of us that are acting upon us.
Cam: Yeah, I’m not supernatural. But I suppose what I’m saying is, maybe it’s more metaphorical. We do good sometimes even if we don’t get benefit from it. Maybe you could argue that we always do because we look like good people. But there’s something that’s causing people to do good and causing people to do bad. If you want to be metaphorical or supernatural about that, maybe not. Anyway, this may be a little bit too mystical.
Back to the question of AGI and creating new species
Rich: Can I try and drag you guys back to my original question. I’m really interested to know — not as an analogy, but just concretely — do you think that we should create a new type of sentient species that might be even better than us or smarter than us? Do you find that attractive or scary? Have you thought about it at all?
Benny: Yeah, I haven’t thought about it much, to be honest. I do think — my earlier answer wasn’t totally unrelated, in that I think you’d want to think really hard about not just the first one that you’re going to create, but whatever consequences are going to arise as a result of bringing the first one into the world. First and foremost is probably, are you going to do this multiple times, or are you just going to do this once? Because if you do know that you’re going to bring a new sentient being into the world, it does seem kind of cruel to let it be sort of the only one of its type. Even if you buy the sort of universal explainer sort of stuff — because in the movie the biology is going to be different, it’s going to think at different speeds if it’s, for instance, made out of silicon, etc. And so I think you want to think really hard about the world you’re giving it. From its perspective, is this going to be an appealing place? Like, humans are super social.
Rich: Because it wasn’t adapted to its world through natural selection, right?
Benny: I mean, depends on your views of humans, I guess, but we’re obviously very social creatures. And even our whole epistemology, our way of working with the world, of figuring things out, our scientific method, is all still extremely social. And so it’d be quite cruel to bring a being that was wired up as a human, in some sense, into a world where there’s no other humans. Like, imagine yourself waking up in a world where you’re surrounded by aliens, and there’s no other humans, only aliens.
Cam: But if you’re a universal explainer, aren’t they fundamentally the same as us? Like, Frankenstein’s a person, and all this other stuff is superficial — which would be hard to do. You know, I’m sort of imagining the school classroom, that’s another man, the guy’s twice the size of everyone with green skin, it feels a little bit like the odd one out. But in terms of, like, why can’t he connect with people like anyone else?
Benny: I mean, I think you can. It’s just there’s going to be barriers, right? Like, to give one really concrete one — suppose you bring something into the world that is sort of human-like but thinks at a computer speed. So its brain is basically silicon. Okay. So even if it has all the same social tendencies as a human being, being able to think — I don’t even know how much faster, you know, a thousand times as fast as humans — everyone else would seem like a fucking moron, right? And it would be infuriating to try and have a conversation with anyone.
Cam: I know what you mean.
Benny: I can’t wait. I do this every Sunday night. My God. And so that’s just one very concrete one, where even if you just buy universality — all the ideas are accessible between these two people, it can explain whatever it wants — it’s going to be super lonely, right? Assuming it feels, it has the same sort of emotional landscape as a human does. And then you start asking, well, what if it doesn’t have the same sort of emotional landscape as a human does? What if it’s wired up so much differently? And that doesn’t mean it’s not a universal explainer, but different species, et cetera, are going to have different proclivities to socialization and how much they’re going to want to —
Rich: They might have no socialization drive at all, right?
Benny: Yeah, they might be totally happy being alone, in which case it’s less of an issue. But how are you going to know that beforehand? It’s very hard. And presumably there’s some sort of incrementalism here where we’re learning how to build AGIs — or maybe the process is slow and you’re actually, it’s more of a process of almost like creating baby-type things, and then they learn over time, et cetera. Who knows what the actual process looks like? But yeah, I think you want to take those sorts of questions very seriously. Sort of like taking theory of mind of what you’re creating seriously, which I think Victor totally failed to do here. I think it’s one of those things where it’s so hard for me to imagine — it’s genuinely hard for me to have an emotional reaction to it. I can think about it intellectually and sort of be excited about the prospect of bringing new things that have the possibility of advancing science, and being like other humans who can be happy and flourish in the world. Like, that’s cool. But if I’m being honest, just even being able to conceive of creating something like a human in a lab —
Cam: That’s easier to imagine. I feel like a Frankenstein-type thing, or just a clone or something — my reaction to there is, yeah, we sort of — I mean, maybe we don’t do too much of it because it’s not okay, but if not then yeah, it’s fine, and we should treat people equal. When you start against AGI — the normal take with AGI is that it probably will be, if not qualitatively different, very quantitatively different, so much that it might as well be qualitatively different. Then, yeah, I find — to be honest, I don’t think about it too much. To answer your question directly, Rich —
Benny: I mean, even cloning though — so you brought up cloning, that’s an interesting example. If you thought about taking cloning seriously, suppose you wake up tomorrow and everyone has the ability themselves.
Cam: Cloning myself? I think that’s a different question — how would I feel about cloning myself versus just a clone of someone? I meet the clone of Benny, I’ve never met you — I think he’d be similar to Benny. But it kind of depends on me.
Benny: Yeah, like, how do you feel about a world where you have the same genetics as that?
Rich: But what is the argument against cloning? Assuming you recognise them as people.
Cam: I think the main argument, and I’m not sure if it’s a good argument, is that it’s unnatural. That’s why people don’t want to do it. There may be other arguments.
Benny: Yeah, I’m assuming putting — so you’re assuming you can do this in a very healthy way. You’re not doing damage to anyone. So, given that, I don’t know what the argument against would be.
Rich: There’s a biological, very high level argument that you need to do sexual recombination in order to make sure human fitness remains.
Cam: Yeah, the Red Queen.
Benny: Is it still true though? Even modern environments?
Rich: But maybe not. We adapt our environment to us instead of the other way around.
Cam: Well, I think one argument there — there’s kind of this open question in evolution, why does sex exist? Why do we not just asexually produce clones? And I think the main theory is around parasites co-evolving, and it’s kind of defense against that. So that feels like it’s inherently always gonna be the case, but I’m not sure that’s the reason for sex. I mean, I’d feel funny with my clone knocking about, right? Very similar to me.
Rich: But a clone is essentially the same as an identical twin, right? It’s the same thing. So these things sort of exist in the world and it’s basically fine. But I guess it’s to do with the intent of why is someone wanting to make clones.
Cam: I suppose when you scale that shit up — one identical twin, alright. Octomom, whatever her name is. Hey kids.
Rich: Like 10 Cams, I don’t know. And of course, like, every rich person will clone themselves and raise themselves as a child, right? I wouldn’t want to clone myself. I would not want to raise myself as a child. I way prefer having the random recombination version. There’s like more of a slot machine effect. Or maybe one of me as a control, I guess.
Cam: There’s probably a Goldilocks optimal number of Cams.
Cam: So I suppose with them — what about just creating a child asexually or non-sexually in a lab, if we could do that, so it’s not your clone?
Rich: We can do that, but what do you mean by that?
Cam: So can we do that without the sperm and the egg?
Rich: IVF.
Cam: I mean, just like, from fucking — like, with electricity or some shit. I mean, a sexual process is underpinning IVF, isn’t it? They just do it artificially.
Rich: So you mean cloning?
Benny: I think he means sort of designing it from scratch — like designer babies sort of deal. Like, you’re like, okay, I’m going to have a — you know, I’m going to just design someone who’s six foot and has this skin color and has this eye color and this hair color.
Rich: Oh, I see. So you have a huge DNA bank and you choose any allele for any position or something. Like the character sliders on video games or whatever.
Benny: Yeah, exactly.
Rich: Yeah, I don’t know. I think that’s a whole other can of worms, I guess.
Cam: What do you think to your original question — are you hyped?
Rich: I haven’t thought about it much either. It doesn’t feel real to me. I think partly because I don’t think that we’re currently on the track for AGI, so it doesn’t feel imminent enough. But then, we quite likely will get there, so I should be thinking about it more. And my gut reaction is definitely worry, and wanting to be extremely cautious about it, because of concerns for the moral rights of those beings, and because I am parochially attached to humanity and don’t want us to become obsolete, basically.
Cam: Yeah, I appreciate that, appropriate concerns.
Rich: Yeah, but I don’t even know if that’s right. I think the accelerationists maybe have some kind of a point — I don’t actually know what the arguments really are, most of them sound dumb. But if you were to argue that you could create a new type of being that is better than you in some way — depending on how you regard better, but let’s say it experiences more utility and is better at pushing the frontiers of knowledge and is, I don’t know, thinks faster or whatever, or is less entropic — then it would feel a bit parochial to think, oh, just cause I’m a human I should protect my own lot. If these AIs have lots of other great attributes and just don’t happen to be human, then it would be parochial to favor humans.
Parochialism and expanding moral circles
Cam: Yeah, I mean, that’s the whole argument with Singer’s moral expanding circle and stuff — originally it’s your family, and then it’s your nation, citizens, and then it’s actually others. I mean, what’s funny with that argument is that people still show bias towards their family, and it doesn’t feel necessarily wrong, right? The fact that you bias your daughter, Rich, over other kids — there seems to be something good about that. And it might just be this parochial hangover that humans have, or you might be able to justify it on consequentialist grounds — that’s just the most efficient way to allocate resources, that the parents themselves with this natural feeling of affinity towards their kid can do it, and maybe we shouldn’t raise everyone up the books or something.
Benny: Singer’s moral circle is an argument that you should treat everyone equal in that sense, just — it’s not saying you shouldn’t have a bias towards your family. He might feel like that because of other arguments, but his moral circle thing is simply about, like, who should you ascribe moral —
Rich: Who’s a moral patient?
Benny: Patienthood to. Yeah, exactly. Just a slightly potential point.
Cam: Well, but it just feels like a natural causal arrow to saying, well, okay, so the African kid shouldn’t be treated different to the American kid. And then it’s like, well, what about my kid that I’m gonna pay to university if I can, compared to the American kid let alone the African kid? And consequentialist argument aside of, that’s an efficient way to allocate stuff, there’s still something in me that feels like, yeah, it’s good and fine to bias your family and friends. That might just be appropriate. But like — okay, what if there was a utilitarian argument —
Rich: No, I think you can make a utilitarian argument for all of that stuff very easily, so it doesn’t feel like —
Cam: Again, so actually it’s better everyone’s in a kibbutz. Well, it’s really, we do it to an excess, right? The average person spends a lot of resources on their family probably, and maybe the argument is at the margin we just need to decrease that.
Rich: Yeah, yeah, I’d agree with that.
Benny: Yeah, me too.
Cam: I struggle with — you keep going down, you keep going that margin down, like, it gets to a point which isn’t that normal. You don’t give your kids much because your resource could be doing better elsewhere.
Rich: No, I just think that doesn’t lead to the best set of worlds. Like, it seems very obvious to me that humans are social animals, humans naturally feel massive affection for their children and their relatives and their friends. And so the type of world which is best is the one in which people meet their natural desires by allocating resources to people not only who they have the most information about and can therefore make way better resource allocation decisions for, but to receive the pleasure from doing so, which we don’t receive except in a very abstract way. But, like, the EA people are right that on the margin you should move that dial quite further down. But it never gets even close to being like your kids are on the bread line or whatever. Like, that’s obviously not a good way to live, to me. It’s like, is it universalizable, right? Like, is it scalable? And that doesn’t scale — if everyone’s doing that, then everyone’s fucking miserable and resentful.
Cam: Yeah, it’s kind of this way that we’re wired, so we just deal with it. Even if the results are not perfectly fair, that’s the best way to do it. Just imagine rationalizing to the young Frankenstein adopted kid, well, you know, I just don’t have the same natural feelings towards you.
Benny: I do find families that mix adopted kids with biological kids to be pretty fascinating. My mom’s family is one such family, actually. She has a sister who’s adopted.
Cam: The Jolie-Pitts is another famous one — Brangelina. They’ve got Shiloh.
Benny: I think those are all — aren’t those all adopted? No, you’re right.
Cam: They’ve got one from everywhere around the world and then they’ve got three biological kids. And I do wonder if there is this kind of — I would imagine, maybe not from Ang, I sort of imagine myself, if I was a parent, you kind of see a bit of yourself when your kids — and I imagine there would be a little bit of natural bias, natural tendency biologically that you feel guilty about.
Benny: The three extremely good looking kids — they’re totally average looking.
Cam: Not one-percentile bangers.
Cam: Anyway, let’s diverge a little bit to your pre-podcast question. I was actually a little bit confused by something. I’m interested in your thoughts just around the role of romances in this novel, the flaws of it.
Benny: Yeah, me too.
Rich: Yeah, I wonder if we even just talk about that next time. Might be something to save for another discussion. Because I want to think about it more. I don’t know anything about it either. And I’ve read like vanishingly little from the Romantic era. This wouldn’t be quite the first, but, you know, I don’t know.
Cam: Yeah, I mean, I personally read that famous Ozymandias poem, which is a banger, around the statue.
Rich: That’s the one Shelley poem that I’ve read. I assume that’s the one that most people have.
Benny: I was like, was it Ozymandias?
Cam: It’s funny now just thinking about them, like, you know, you got Mary Shelley chilling with Lord Byron. You know, when you find out famous names — like Aristotle and Plato — they all kind of knew each other. You read these famous names from the early 1800s and you kind of get surprised, like, all these people hanging out. But then it kind of makes sense. That’s kind of why, one, talent attracts talent. Two, it also contributes to their fame, the fact that they’re all kind of mates and referencing each other potentially. But man — well, actually, on that, it kind of made me think — it’s not this new thing in San Francisco when you go to these like 140 IQ angles, where everyone’s like fucking each other and running stories. I think they’re all in a polycule back then, just fucking, being creative.
Rich: Yeah, they’re being creative in a different direction though. I think it is cool though. I agree with you. I mean, they’re all exploring similar themes apparently too. You know, I don’t know anything about this, but Percy Shelley also has a work about Prometheus, and I think Byron wrote about it as well. They’re clearly forming a scenius-type thing. And we should say for people who don’t know as well — the genesis of this, I don’t think we said explicitly, is that the Shelleys and Byron are sitting around during bad weather in their holiday home or whatever in Switzerland. And to pass the time they had a competition with each other — who can come up with the coolest ghost story basically. And this is the outcome of that. Like one of the greatest stories ever written. It’s just like a funny writing prompt.
Cam: And actually, I think the essence of Dracula — the embryonic version was also a product of, maybe not this weekend, but that kind of same group just trying to write ghost stories.
Rich: I wonder if Mary Shelley has way more cultural influence over the long course of history than Percy Shelley or Lord Byron does. Or if that’s just because I’m an idiot and I don’t know anything about the romantic poets.
Cam: Assuming Percy didn’t write this. Well, I don’t know too much about Byron. I know there’s this trope of like the Byronic hero, which is the kind of aloof — I don’t know, imagine even that fucking vampire from — what’s it? The one that all the girls read?
Benny: Pretend you don’t know the name.
Cam: No, I knew that.
Rich: Your literature references on this call so far have covered Harry Potter, Twilight —
Cam: Dante?
Cam: I’m revealing myself as a fucking Harry Potter Dante bro. That was my internet stream, out of curiosity. Not doing anything wrong with that.
Cultural legacy of this book
Benny: I mean, I do think it’s probably undoubted that Mary Shelley’s gonna have more long-term influence than either of those, just because I think she’s hit on one of the most timeless themes possible — which is the possible destructive nature of things you’re creating, and responsibility for your creations.
Cam: And also just sci-fi — just the idea of science and literature combining.
Benny: But yeah, exactly — like the perils of progress. Anytime you have a society that’s pushing the boundaries of what’s known, creating new things, science, technology, engineering, this theme is going to continually emerge.
Cam: It’s going to be timeless, right?
Benny: And honestly, going back and reading this now, I’m kind of surprised — and maybe it just comes back to the fact that everyone thinks they know the story of Frankenstein and it’s sort of just cultural cash at this point. But it does feel weird that it’s not more front and center in a lot of conversations, because it does feel like, going back and reading it, it’s so relevant to so many of the cultural battles we’re having just around open sourcing AI software and all this stuff. And I mean, perhaps there’s not that many lessons because it’s from so long ago, but —
Cam: Maybe it’s one of those things that’s become a bit clichéd, right? I mean, everyone does talk about this — like, man creates something that destroys man. It feels a bit tired referencing it, and it feels like you kind of know it. But I suppose clichéd is a cliché for a reason, including that phrase.
Benny: Yeah, I think that’s a big part of it. Everyone feels like they know it.
Rich: We probably just have more salient recent examples too. Like, I think the atom bomb is an extremely good modern example that has way more salience in most people’s minds, because it actually happened, and we can actually observe the downstream consequences and think about, you know, what was a good decision, what wasn’t. And it analogizes quite well into AI as well in various ways.
Cam: Yeah, it does feel like the one example — and AGI might be the next one — that it’s hard to kind of shake off. Because you always go back, you know, people were worried about textile machines and people were worried about bullets and stuff, and then probably it’s good that we got them in the end and you can have safety around them. But like, nuclear weapons is kind of like, yeah, people are worried that they exist, right? It’s like the one thing people still want to turn back if they could.
Rich: Yeah, I always had this naive empiricism about this. You know, like Matt Ridley type arguments or Pinker-esque arguments — there’s a long line of doomsayers and Malthusians and Luddites that have always proclaimed that this is going to be the end of the world, and they’ve always been wrong. Or, to the extent they’ve been right, human ingenuity has always been able to provide the solutions to whatever new problems arise. And I used to think that line of argument was really compelling. And now, because I’m thinking more from a Nassim Taleb, Popperian point of view, I’m like, well, that doesn’t mean anything. That’s just what happened in the past. We can absolutely come up with new technology that is the end of us.
Cam: Yeah, but I think it’s still an important argument to consider, right? It’s like, this is the same bias that everyone gets, and this might be another instance of that. However, as you point out —
Rich: But this is like Bulverism, right? It’s trying to say why people are biased for the arguments they make, instead of talking about the actual specific arguments that they make. And the arguments might be right or wrong, but it has nothing to do with historical precedent.
Cam: I suppose I think you’ve got to do both. You’ve got to say, like, why is this case different to all the others? And if you can kind of point that out, then it’s like, yep.
Rich: Yeah, that’s fair enough, I suppose. To take it as a starting point, I don’t know, I still don’t know about that. But yeah, to say as a starting point, we’ll figure out how to solve these problems —
Cam: Well, it’s kind of like AGI — just even not destroying us. AGI just taking large jobs, right? Like, everyone’s been worried about that for time immemorial. You need progress, and that doesn’t happen. But as we just said, maybe it would take away the jobs.
Rich: Well, this is the one thing where it literally will take all our jobs.
Cam: Well, people argue against that. I don’t want to necessarily look at that.
Rich: The pocket calculator version won’t, to be fair, but —
Cam: Well, Deutschians argue that the AGI version won’t either. But without getting in — I’m not sure exactly what the arguments are. But this certainly feels like, you know, the one area that the long historic base rates would hold potentially.
should Zuckerberg and friends try to model consequences of AI?
Rich: Yeah, I’m surprised just how unwilling people are to get into the specifics of this kind of thing, and actually try and project what will happen, even in a speculative way. Like, on that — I don’t know if you guys listen to the Mark Zuckerberg interview on Dwarkesh, and he hits all the talking points and stuff, but he just doesn’t seem to think to actually try and create a vision of, if this leads to this, then that leads to that, then how will that cash out kind of thing? Like, what will humans be good at? What will be our competitive advantage? Presumably the human touch — things that other humans will find valuable, or something. Or, you know, there’ll be a huge rush for literal physical touch-based jobs, like masseuse or home care, or something that maybe the robots are not as good at. And then that would absolutely tank the labor market — I mean, it would tank the price of labor for those kinds of professions. So we would have, for the first time, an inversion where humans are doing menial work for less money instead of, you know, more knowledge work for more money while the manual work gets outsourced.
Benny: Yeah, I mean, to be fair to Zuck a bit there — people are so wrong typically about their forecasts for even a decade into the future in terms of automation and what the economy is going to look like, what the labor market is going to look like, which jobs are going to be most in demand, etc. I can sort of forgive him for not giving his five-point plan for how to run the economy in 2033 or whatever.
Rich: Yeah, but surely — if you’re proposing, if you believe that you’re creating this thing which is better than humans, or equal to or better than humans in every way, surely you are thinking about what that implies. Like, how could you not be? It seems crazy to me.
Benny: I don’t think he thinks like that about AI, though. I think he seems much more like Sam Altman or any of the sort of realistic people who are running these AI companies, who don’t have their head in the clouds with respect to their capabilities. I think they just view it as sort of a continual improvement in these capabilities that slowly takes away some of the jobs that are more rote and manual and stuff we don’t want to be doing anyway. And like, yeah, that’ll shake up the economy, but there’s no reason to think that overnight we’re going to go from humans are doing 95% of the labor to humans are only required for 3% of the labor.
Rich: Yeah, but it’s not the rate of change that I’m worried about. It’s the end point, right? Like, once you get to the end of your process — I thought that Altman and co all think that AGI will be better than humans. I mean, maybe Zuckerberg doesn’t think that, but if Altman does, then I would sort of level the same question of him.
Benny: The rate of change is extremely important though, right? Because it tells you how fast you’re going to adapt as a society. But there is a huge difference between going from humans required to do 95% to tomorrow we’re only required to do 3%. If that happens in the span of 24 hours versus the span of five years, that’s a huge difference.
Rich: Yeah, but isn’t the implication there that there’s something that humans are migrating towards? And what I’m trying to say is, isn’t the end case that there’s nothing that humans can do, and that AI is capture all value and humans are completely obsolete?
Benny: But at that point you’re so far into the future. Like, I don’t want my tech executives thinking about that. That’s just like pure sci-fi scenario. Like, we figured out how to automate every code.
Rich: Well, isn’t that just AGI scenario? Like, you find out the formula for personhood and you create AGIs and they’re instantly better than humans.
Benny: Yeah, so, I mean, yes, exactly. So that is sci-fi. I thought we should go — figured out the formula for personhood. But then also the additional assumptions that they will be better than us in every way and want to do all these jobs. And you’re also baking in not only that we’re going to figure out a job, but tons of assumptions about how they’re going to behave in the world, which I think is — you know, you’re already aligning a bunch of — and I’m not saying I agree with the way you’re talking about a job, but you’re aligning a bunch of like Deutschian points here, where you would just view it as like, yeah, you’re birthing a bunch of more creative people in the world. They’re going to have propensities and want to do different sorts of —
Rich: Yeah. Even though some of those people can think like a thousand times faster than other of those people. I think it’s crazy to think that there’ll be a level playing field even with universality. But anyway, this will get way too far off topic.
Benny: Yeah, maybe. I agree with that. I’m just saying you’re baking in like an extreme number of assumptions about what the post-AGI — you’re one, assuming we’re gonna get AGI pretty soon, and two, assuming a lot about what the post-AGI world looks like. And I’m totally fine for the tech executive, the head of Meta, to not have a plan for what that world’s gonna look like.
Rich: Yeah, yeah, okay, that’s fair enough. But I think, like, obviously you don’t find it crazy that A, we can get AGI, and whether or not we do, it is conceivable, right? Obviously. And B, that AGIs — it’s plausible that AGIs will be able to do everything better than humans. And then your economist brain would have to think, oh, what would that — I think it’s an interesting thing to think about. Even if Zuckerberg is not thinking about it in great detail, it seems important to think about. Maybe we’ve got plenty of time to think about it.
Benny: Yeah, maybe. I don’t know if I’d say it’s important to think about. Like, yeah, maybe.
Rich: Yeah, just insofar as that AI probably won’t be incremental, right? It’ll actually be a leap forward. It’ll be a zero-to-one thing.
Benny: Well, it’s just — right, it’ll look much more like raising kids. So it’ll be very influential in that sense. Like, you’ll have the formula quote-unquote, but it’s not like you get super intelligent sort of overnight. You basically have things that you still need to socialize, you still need to be in conversation with — all these sorts of things. So I think you envisioned it more like kids. I guess, kids that can think at light speed.
Rich: Well, kids take like decades to train on a bunch of sensory data that happens after their core wiring, right? The reason that it’s disanalogous is that you train an AGI in a month or whatever, but not in 20 years. And then it just exists and is like an adult or whatever.
Benny: Well, now you’re assuming the training process for — when you say training process, you’re conflating whatever the training process, quote-unquote, for AGI would be, which I think will be highly disanalogous to the training process that we use for LLMs. But you’re saying the timelines that we use to train LLMs — now you’re importing that into the conversation about AGI. We have no reason to think those training processes will look the same at all. And if Deutsch is right that AGI is basically just humans with better hardware, then they’re still going to have all these mental hangups. There’s still going to be all this social epistemology required. No matter how smart one person is, they’re still going to have to converse with other people to have their ideas checked and their ideas criticized and stuff like that.
Rich: So you think it’ll happen on our timescale, not on theirs, because we’ll be the ones instructing them in some sense?
Benny: I don’t know. But I just — I wouldn’t say that because we can train a large language model in X number of months, therefore AGI will go from baby-like to adult-like in X number of months. I think that analogy is just — we just have no idea what training an AGI will look like. So I’m just uncomfortable with basing the timelines on a totally disanalogous automated system.
Rich: Yeah. Cool. Anyone got anything else? I’ll show you later.
Cam: I’d have to leave it there.
Rich: Oh, yeah. We went over time a bit.
Benny: Do we want to try and finish for next time? Or how are you guys feeling?
Cam: I think so. Yeah, alright.
Rich: Yeah, let’s do it.
Benny: Just over 100 pages.
Rich: Easy peasy.
Benny: And yeah, it only gets more exciting as the monster starts talking.
Cam: Yeah, the genius comes out.
Rich: Sweet. See you next time.
