Every year at Christmas I treat myself to a viewing of a YouTube video by a family of musicians called The Five Strings. The video is titled “O Come All Ye Faithful – Epic Flash Mob Carol,” and that title pretty much summarizes it—but hardly communicates the warmth and effect of the piece. Here’s a link: you might want to watch the 4:45 video before going on to my spoilers.
I ration my viewing because I don’t want to let the effect grow stale. By the end of the song there are tears in my eyes, my chest is swelling with joy, and I feel rejuvenated with the grace of the season.
To start at the least important end, it’s awfully cute. The young performers are adorable, and the idea of a set of siblings setting out to bring cheer to the world is charming. Second point: they’re good. The voices are wonderful, and the instrumental performances are great. And of course the surprise and joy of the people listening are lovely to see—not to mention the visible enthusiasm of the performers.
The music itself is brilliant. We all know the song, of course. Here, though, the lyrics are slightly different from the familiar ones—just different enough to catch our attention. And the musical arrangement raises the well-known tune to a new level.
The flash mob method means the song starts off with a single lonely voice, which is gradually joined by more and more voices and instruments. Variety grows as the song proceeds. There’s a quick violin interlude at 1:40 that’s almost like an Irish jig, and it seems to shift time signatures as it segues into the second verse.
The low notes of the cello that enters about 2:20 add depth to the sound. And at 2:34 the phrase is unexpectedly drawn out, leading into a crescendo to the third verse and the sight of the whole company facing the viewer in song, with a lighted star behind them. At 3:13 the harmonization of the refrain shifts, along with the lyrics, lifting the song to a new level of glory. And the very end brings all this grandeur to a hushed conclusion.
And the Singers
The very idea of a flash mob has an effect here. A group of people coming together in a seemingly random fashion, out of careful planning, just to entertain or amuse, is pretty nifty in itself. A 2010 video of a flash mob in a food court doing the Hallelujah Chorus is fun partly just because of the absurdity of imagining that something as complex as the Hallelujah Chorus could come together out of nowhere. It’s reminiscent of the movie musicals in which everyday people seem to be able to break out spontaneously into song-and-dance numbers at will (see TV Tropes’ “Musical World Hypotheses” for an entertaining discussion). One of my favorite examples is “That’s How You Know” in Enchanted, which slyly pokes fun at itself (as does the whole movie). Apparently Giselle came into the mundane world with the ability to cast the spell ‘Summon Production Number’.
But in the Five Strings video, the gradual accretion of a mob of musicians has a greater effect. It’s as if the faithful really are coming together at the summons of the song. There’s no attempt to make it seem spontaneous—it’s obvious that the effort has been planned in advance—but the stepwise addition of more and more strands to the performance is like opening a series of Christmas presents. Except that here, the whole adds up to more than the sum of the parts.
It occurs to me that the emotional effect lies near to what Samantha Edmonds calls “the rally” (or “the Riders of Rohan phenomenon”). She puts it this way:
This isn’t just a “sudden happy turn” in a narrative: as I’ve argued, what sets these scenes apart from other kinds of eucatastrophes is that the RoR phenomenon is specifically a choice made by a community. This is the moment your team shows up. Another friend agreed, explaining, “What makes these scenes so important is that often it’s chosen family who is arriving in these moments of great need, when things are darkest and most unwinnable.” . . . .
. . . . .I argue that the greatest “Consolation” of the fantasy genre isn’t the happy ending: it’s Companionship.
Granted, we don’t think of Christmastime this way when we wax sentimental about it. And there’s nothing wrong with that sentiment. But the Christmas story itself is precisely about sudden hope arriving in the moment of great need. And when one enthusiastic group of companions after another arrives to join in the song, that is rather like seeing “your team” show up. Even if they’re strangers to the people listening from the front doors, they belong to a common cause.
Maybe I’m placing too much weight on just a few minutes of celebratory song. But do take a look at the video. I think you’ll enjoy it, and if you too find that it moves you, maybe someday we’ll figure out why.
Recently I picked up Katherine Paterson’s Bridge to Terabithia (1977), which I kept hearing about but had never actually read. I had had the impression it was a portal fantasy about a couple of children, but that isn’t correct: it’s about a couple of children with vivid imaginations who play at fantasy.
The edition I read had an introduction that tipped me off that something terrible was going to happen in the story. (Since this book is almost fifty years old, I’m going to forgo a spoiler alert.) So I was prepared when a tragedy did occur. But I found the ending uplifting nonetheless, because of how the author describes the change this tragedy works in the main character, Jess. Which may be the point of tragedy, on the whole.
The denouement did get me thinking about something I’d never quite been able to articulate, having to do with the Narnia stories. Narnia keeps coming up in Bridge to Terabithia; it largely inspires the characters’ imagined world. So the connection was out in the open. It isn’t mentioned specifically in the Terabithia quote that sparked my reflection:
It was Leslie who had taken him from the cow pasture into Terabithia and turned him into a king. He had thought that was it. Wasn’t king the best you could be? Now it occurred to him that perhaps Terabithia was like a castle where you came to be knighted. After you stayed for a while and grew strong you had to move on. For hadn’t Leslie, even in Terabithia, tried to push back the walls of his mind and make him see beyond to the shining world—huge and terrible and beautiful and very fragile? (Handle with care—everything—even the predators.)
Now it was time for him to move out. She wasn’t there, so he must go for both of them. It was up to him to pay back to the world in beauty and caring what Leslie had loaned him in vision and strength. (ch. 13, p. 161)
The aphorism that kept coming to mind was “Once a king or queen in Narnia, always a king or queen in Narnia” (The Lion, the Witch and the Wardrobe, ch. 17). But that’s not quite it. That line is about continuing to be a king or queen. But what I kept thinking about was becoming a king or queen in the first place.
See, the Pevensie children, who become kings and queens in Narnia, don’t start out as heroes of mythic stature. They’re a pretty normal bunch of kids. There’s no suggestion that they are of extraordinary strength, courage, intelligence, or virtue. They certainly rise to the occasion: we see them becoming heroes as the story develops. But their only qualification for royalty, in the first book, seems to be simply the fact that they are “sons of Adam and daughters of Eve”—that is, human beings. (ch. 2 and 8)
The notion that a human being as such has a royal destiny isn’t far from Jess’s thought in the Terabithia quote above. The other world is where you’re knighted, where your true self is recognized; but you’ve already got the potential. The other world simply calls you to realize it and carry it out (“pay back to the world in beauty and caring”).
The underlying thought that my reading of Terabithia teased out is that simply being a human being (or, more broadly, a person), a child of God, is enough to qualify one for the kind of dignity and respect that we associate with royalty. And that’s something worth remembering.
Here’s the appropriate song—unrelated to either book, but matching very closely in theme: “Kings and Queens” by Audio Adrenaline (lyrics).
Can one have too many books? I want to say no. (See the illustration, which hangs on our front hall next to the library.) On the other hand, it’s become difficult to fit my accumulated collection comfortably into any reasonable home—and doubly so when you add in my wife’s lifelong collection as well—even without the eight or ten boxes of books I sold off to the fabulous Wonder Books in Frederick in an attempt to thin out the collection. And nonfiction is another story. (Well, the stories are in the fiction section; but you see what I mean.)
This is so even though I’ve tried to exercise restraint by buying only those books I’ll want to read over again. I prefer to try out a new book by getting it from the library, and, after reading, to decide whether this is a “keeper” that I always want to have available. The rare exceptions are new books that I already know I’ll want to own, generally in a series. While Lois McMaster Bujold’s Vorkosigan saga was still coming out, for instance, I’d grab those off the bookstore shelves at once, sight unseen. (Though I admit I do have only an electronic copy of Gentleman Jole and the Red Queen—but that’s another discussion.)
Aside from considerations of space, it occurred to me some years ago that the “read again” rationale necessarily diminishes over time. Given my likely lifespan, how many times am I likely to reread any given volume? That number is constantly decreasing. This makes me a little cautious about buying new books—as opposed to using the library. One of the great perks of my current job is that there’s a county library branch right on the first floor of our building. I can put a book on hold and nip down while at the office to pick it up. Same thing to return it. A wonderful arrangement . . . at least until I retire . . .
The other motivation I’ve had for accumulating books has been a vague notion that they serve as a kind of reference library. In writing posts like these, it’s convenient to be able to grab a favorite book to check on some point or to make a citation precise. For years, I kept around an old paperback copy of Neil Jones’s The Planet of the Double Sun (1931, paperback edition 1967), merely as an example of a really crummy old-time SF yarn. It finally occurred to me that there was no particular purpose in having it—rereading it would be too boring to be worth any use I might make of the mention—and I ditched, er, released my copy to find a new home with some historian of SF. Haven’t missed it since.
The Great Reread
In the process of going over my packed-and-unpacked-and-reshelved books (two moves in the last two and a half years!), I’ve sometimes found it hard to recall what some of the stories were actually like. Or I feel as if I’d like to read a book once more before I consign it to the outermost depths. So I find myself rereading a lot of old material. Some of it holds up pretty well. Others, I wonder what I saw in the tale in the first place. The rereads end up being a combination of final trial—do I want to keep this book?—and fond farewell.
Of course I continue to read new stories too: new good books are coming out all the time. But it’s rather enjoyable to revisit some old friends as part of a balanced bookish diet.
And sometimes the old stuff may be worthy of comment. So I’m going to include the occasional post here from the Great Reread. The inspiration comes from Jo Walton’s brilliant series of essays, “What Makes This Book So Great,” originally hosted, I believe, at Tor and published in a 2014 collection. (There seems to be a list of links here; Walton’s own comments on rereading are here.) I’m not spotlighting what are necessarily the greatest books—but there may be interesting things to say even about the non-great or non-classic tomes I find cluttering my shelves.
Since these books tend to be fairly well aged, I don’t think it’s necessary to issue spoiler alerts in each case. The comments all have spoilers.
Brian Daley’s Peripatetic Heroes
I first ran across Brian Daley (1947-1996) in his 1977 novel The Doomfarers of Coramonde, which has the irresistible premise that a sorcerer in a fantasy land summons magical assistance from another world—and it turns out to be an armored personnel carrier, with crew, from the war in Vietnam. The fish-out-of-water contrast was delightful and the action first-rate. I was primed to look for more good fun a few years later when Daley published Requiem for a Ruler of Worlds (1985), the first in a trilogy of adventures featuring the mismatched buddies Hobart Floyt and Alacrity Fitzhugh.
They’re still good fun. The heroes are both likable and relatable—dashing and mercurial Alacrity, a knockabout itinerant spacer with “friends in low places” across the starlanes, and staid Hobart, a minor bureaucrat from a congested backwater Earth. Alacrity has some personal issues to get over, mostly in the third volume, but Hobart develops most, rising to the occasion as he travels into space to obtain a bequest inexplicably left to him by an interstellar magnate. The worldbuilding is brilliant: planet after exotic planet, oddball character after still odder character. The sheer variety and colorfulness of Daley’s cosmos is pleasing.
The Floyt-Fitzhugh saga falls into the category of the “picaresque” tale. Originally I misunderstood that term to mean the same thing as simply “picturesque,” and the story certainly is that. When I actually looked up the word, many years ago, I realized it had a more specific meaning: “Characteristic of a genre of Spanish satiric novel dealing with the adventures of a roguish hero,” as Wiktionary puts it. That fits our heroes pretty well—Alacrity from the start, Hobart learning fast. They’re lovable rogues, often on the wrong side of the law (or at least local customs), with hearts of about 12-carat gold. Even in the last scene of the series, they’re off again to who knows where, one jump ahead of pursuit.
And thereby hang my possibly idiosyncratic reservations about the story. The plot of the first book is very good; there’s a clear goal and climax. In the second and third books, not quite so much. Alacrity and Hobart achieve some interesting results, but they themselves don’t actually stop to absorb or enjoy them. Our Heroes are always on to the next thing. Daley gives each of them a pretty good romance subplot, but these never quite resolve, since the MCs are always leaving their girlfriends behind—even at the end of the third book, Fall of the White Ship Avatar. I was left wondering, do these two ever get to settle down and enjoy the fruits of their labors?
This is a standard issue with wandering, picaresque heroes, particularly those who star in open-ended series. Daley even lampshades this aspect by including a whimsical journalist in the story who features Floyt and Fitzhugh in a series of utterly melodramatic “penny dreadfuls,” to Our Heroes’ perpetual embarrassment and annoyance. Daley seems to want to place them in the company of series heroes whose adventures never really end. James Bond is never going to stop secret-agenting; he just gets re-cast with increasingly younger actors. At the end of Indiana Jones and the Kingdom of the Crystal Skull, just when we might expect Indy to retire and be replaced by his son, Indy snatches back his rolling fedora at the last moment. The ending of The Incredibles or Spider-Man 2 show the characters resolving their personal character arcs, but nonetheless leaping into the next adventure.
One wonders if these perpetual adventurers ever retire, even as they inevitably age. We’ve pondered this previously in considering the “professional hero.” There has been the occasional hero who explicitly retires and passes on the mantle to a successor. There’s a neat little story in Kurt Busiek’s Astro City showing how superhero Jack-in-the-Box is training his successor. And Captain America eventually retires in the MCU, though the ‘line of succession’ is rather fraught.
Personally, I like to see the main characters settle down at some point, if only to enjoy a well-earned respite from perpetual danger and damage. They can continue to have interesting adventures later on; “happily ever after” need not be a static condition. Consider, for example, the somewhat atypical honeymoon of Miles and Ekaterin Vorkosigan in Diplomatic Immunity. But it’s satisfying to see beloved characters enjoying some degree of stability.
On the other hand, there’s also a certain appeal to the hero who’s eternally “out there,” a legendary figure, always at their peak and unabatedly themselves. In a way I don’t want to see Nero Wolfe or James Bond retire. So I do rather appreciate the perpetual motion of Floyt and Fitzhugh, despite the lack of resolution. To resolve, or not to resolve? That is the question—but either answer may be acceptable. I’ll be keeping the Daley books on my overstuffed shelves.
I finally got around to watching the 2024 film The Lord of the Rings: The War of the Rohirrim. Missed it in the theatres, but I caught it on streaming video while my wife (who is not quite so headlong a Tolkien fan as I am) was off on a storytelling adventure.
The movie is very good. Grim, but good. Of course, this came out recently enough that I should issue a
Canon and Its Discontents
I refrained from reading about the plot until I had seen the film, so as not to spoil anything for myself. When I went back to the source material—the tale of Helm Hammerhand, King of Rohan, about 200 years before the War of the Ring—I was surprised to see how closely the screenwriters had actually followed what Tolkien provided in Appendix A to The Lord of the Rings (section II, The House of Eorl). This isn’t essential—but it’s helpful.
In other words, I’m not going to be scandalized just because a Middle-Earth tale isn’t entirely consistent with the canon. Firing salvos back and forth about canonicity isn’t very productive. What matters is what we do with the canon (or without it).
At the same time, if a work is worth adapting in the first place—book into movie, for example—then the adapters would do well to understand what made the source appealing. We can use a source simply as a convenient hook on which to hang a different story, but we’d be wasting what’s worthwhile in the original. Merely arbitrary or casual departures from the canon often degrade the result, rather than enhancing it. We have to look at the changes case by case. (The Peter Jackson Lord of the Rings movies themselves are full of examples both good and bad—but that’s a subject for a much longer dissection, one of these days.)
Thus—to take an SF example—when Gregory Benford wrote one of the sequels to Isaac Asimov’s Foundation series, Foundation’s Fear, Benford threw in a new means of interstellar travel—via wormholes—in addition to the “hyperspatial jump” that was standard in Asimov’s works. This addition made the story seem a little more up-to-date, since wormholes had come into fashion in SF after Asimov wrote the original stories. But inserting these wormholes into the background created unnecessary tensions in the worldbuilding, since events in the original series might have played out quite differently if these wormholes had been around as Benford assumes. And it added nothing much to the actual interest of the novel. It wasn’t a big deal, but it was an annoyance.
Thus, we can evaluate War of the Rohirrim fairly by looking at how well it used the original Tolkien material, and how effectively it went beyond.
Adding and Expanding
I thought the screenwriters here did a good job. The Riders of Rohan as pictured in the movie are recognizably the same civilization we saw in the book and the LotR movies. Helm Hammerhand is not at all the same character as Theoden, but he comes from the same culture. Rohan’s relationship with Gondor is consistent with what we see in the books, allowing for two hundred years’ difference.
When the movie brings in LotR’s “oliphaunts,” or mûmakil (appropriately, only the latter name is used, since the former is more of a hobbitish coinage), that may seem an anticipation of the LotR storyline, but it’s plausible enough, drawing on Tolkien’s worldbuilding to add a danger and a challenge to the bare-bones tale of Helm in the books’ appendix. The Wikipedia article on the movie makes some good observations here, and I’m inclined to agree.
The writers’ key decision, I think, was to use Helm’s daughter as the primary viewpoint character for this story. She’s mentioned in the book, but her name is never given. The screenwriters’ choice of “Héra” rather unfortunately invites confusion with the Greek goddess of the same name (less an accent), but it does fit her appropriately into a family containing Helm, Haleth, and Hama.
Telling the story as Héra’s tale allows the movie to create a sense of continuity that would be hard to achieve otherwise, since Helm and his two sons all die in the course of this war, one by one. None of them survive to carry the viewer through to the end. It is plausible, though, for Héra to come out alive. And that also enables the screenwriters to give us something more like a happy ending than would have been possible if the focus had been on, for example, Helm himself. Helm’s nephew Fréaláf takes the throne, and that’s fine. Héra, however, can continue forward unconstrained and free. The writers have wisely left her tale open-ended, rather than resolving it with a romance or other specific commitment. That open prospect—“the Road goes ever on”—seems an appropriate conclusion for a story that is wound almost too tightly around one set of royal family conflicts and geographic environments.
I’ve found Amazon’s The Rings of Power somewhat disappointing; I’m cheered by this modest, but enjoyable, alternative venture into Middle-Earth. Who knows: someday (if the intellectual property rights align) we might even see the stories of the First Age, the Silmarillion, come to the big screen—and wouldn’t that be a sight to see!
I recently read an excellent cautionary tale (and with a romance to boot), David Walton’s Three Laws Lethal (2019). The subject of “artificial intelligence” or AI (it isn’t really intelligence, but that’s another story) is hot. To take only one rather specialized example, the Federal Communications Commission’s Consumer Advisory Committee last year carried out a brief survey of the roles of AI, both harmful and helpful, in dealing with robocalls and robotexts. So it seems like an appropriate moment to take a look at Walton’s insights.
Frankenstein and the Three Laws
It’s well known that the early history of SF—starting with what’s considered by some to be the first modern SF story, Mary Shelley’s Frankenstein (1818)—is replete with tales of constructed creatures that turn on their creators and destroy them. Almost as well known is how Isaac Asimov, as he explains in the introduction to his anthology The Rest of the Robots (1964), “quickly grew tired of this dull hundred-times-told tale.” Like other tools, Asimov suggested, robots would be made with built-in safeguards. Knives have hilts, stairs have banisters, electric wiring is insulated. The safeguards Asimov devised, around 1942, gradually and through conversations with SF editor John W. Campbell, were his celebrated Three Laws of Robotics:
1—A robot may not injure a human being, or, through inaction, allow a human being to come to harm.
2—A robot must obey the orders given it by human beings except where such orders would conflict with the First Law.
3—A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
The “positronic” or “Asenion” robots in many of Asimov’s stories were thus unable to attack their human creators (First Law). They could not run wild and disobey orders (Second Law). Asimov’s robot could not become yet another Frankenstein’s monster.
That still left plenty of room for fascinating stories—many of them logic puzzles built around the Three Laws themselves. A number of the loopholes and ramifications of the Three Laws are described in the Wikipedia article linked above. It even turned out to be possible to use robots to commit murder, albeit unwittingly, as in Asimov’s novel The Naked Sun (1956). When he eventually integrated his robot stories with his Foundation series, he expanded the Three Laws construct considerably—but that’s beyond our scope here.
Autonomous Vehicles
To discuss Three Laws Lethal, I must of course issue a
Walton’s book does cite Asimov’s laws just before Chapter One, but his characters don’t start out by trying to create Asenion-type humanoid robots. They’re just trying to start a company to design and sell self-driving cars.
The book starts with a vignette in which a family riding in a “fully automated Mercedes” runs into an accident. To save the passengers from a falling tree, the car swerves out of the way, in the process hitting a motorcyclist in the next lane. The motorcyclist is killed. The resulting lawsuit by the cyclist’s wife turns up at various points in the story that follows.
Tyler Daniels and Brandon Kincannon are friends, contemporary Silicon Valley types trying to get funding for a startup. Computer genius Naomi Sumner comes up with a unique way to make their automated taxi service a success: she sets up a machine learning process by creating a virtual world with independent AIs that compete for resources to “live” on (ch. 2 and 5). She names them “Mikes” after a famous science-fictional self-aware computer. The Mikes win resources by driving real-world cars successfully. In a kind of natural selection, the Mikes that succeed in driving get more resources and live longer: the desired behavior is “reinforced.”
Things start to go wrong almost at once, though. The learning or reinforcement methods programmed into the AIs don’t include anything like the Three Laws. A human being who’s been testing the first set of autonomous (Mike-guided) vehicles by trying to crash into them is killed by the cars themselves—since they perceive that human being as a threat. Two competing fleets of self-guided vehicles see each other as adversaries and can be turned against their controllers’ enemies. The story is both convincing—the AI development method sounds quite plausible and up-to-the-minute (at least to this layman)—and unnerving.
But the hypothetical AI system in the novel, it seems to me, casts some light on an aspect of AI that is not highlighted in Three Laws-type stories.
Having a Goal
The Mikes in Three Laws Lethal are implicitly given a purpose by being set up to fight for survival. That purpose is survival itself. We recall that a robot’s survival is also included as the third of the Three Laws—but in that context survival is subordinated to protecting humans and obeying orders. Asimov’s robots are conceived as basically passive. They would resist being destroyed (unless given orders to the contrary), but they don’t take positive action to seek the preservation or extension of their own existence. The Mikes, however, like living beings, are motivated to affirmatively seek and maintain themselves.
If an AI population is given a goal of survival or expansion, then we’re all set up for Frankensteinian violations of the First Law. That’s what the book depicts, although in a far more sophisticated and thoughtful way than the old-style SF potboilers Asimov so disliked.
At one point in Walton’s story, Naomi decides to “change the objective. She didn’t want them to learn to drive anymore. She wanted them to learn to speak” (ch. 23, p. 248)—in order to show they are sapient. Changing the goal would change the behavior. As another character puts it later on, “[i]t’s not a matter of preventing them from doing that they want” (as if a Law of Robotics were constraining them from pursuing a purpose, like a commandment telling humans what not to do). Rather, “[w]e teach them what to want in the first place.” (ch. 27, p. 288)
Goals and Ethics
Immanuel Kant
The Three Laws approach assumes that the robot or AI has been given a purpose—in Asimov’s conception, by being given orders—and the Laws set limits to the actions it can take in pursuing that purpose. If the Laws can be considered a set of ethical principles, then they correspond to what’s called “deontological” ethics, a set of rules that constrain how a being is allowed to act. What defines right action is based on these rules, rather than on consequences or outcomes. In the terms used by philosopher Immanuel Kant, the categorical imperative, the basic moral law, determines whether we can lawfully act in accordance with our inclinations. The inclinations, which are impulses directing us toward some goal or desired end, are taken for granted; restraining them is the job of ethics.
Some other forms of ethics focus primarily on the end to be achieved, rather than on the guardrails to be observed in getting there. The classic formulation is that of Aristotle: “Every art and every investigation, and similarly every action and pursuit, is considered to aim at some good.” (Nicomachean Ethics, I.i, 1094a1) Some forms of good-based or axiological ethics focus mostly on the results, as in utilitarianism; others focus more on the actions of the good or virtuous person. When Naomi, in Walton’s story, talks about changing the objective of the AI(s), she’s implicitly dealing with an axiological or good-based ethic.
As we’ve seen above, Asimov’s robots are essentially servants; they don’t have purposes of their own. There is a possible exception: the proviso in the First Law that a robot may not through inaction allow harm to come to humans does suggest an implicit purpose of protecting humans. In the original Three Laws stories, however, that proviso did not tend to stimulate the robots to affirmative action to protect or promote humans. Later on, Asimov did use something like this pro-human interest to expand the robot storyline and connect it with the Foundation stories. So my description of Three Laws robots as non-purposive is not absolutely precise. But it does, I think, capture something significant about the Asenion conception of AI.
Selecting a Purpose
There has been some discussion, factual and fictional, about an AI’s possible purposes. I see, for example, that there’s a Wikipedia page on “instrumental convergence,” which talks about the kinds of goals that might be given to an AI—and how an oversimplified goal might go wrong. A classic example is that of the “paperclip maximizer.” An AI whose only goal was to make as many paper clips as possible might end by turning the entire universe into paper clips, consistent with its sole purpose. In the process, it might decide, as the Wikipedia article notes, “that it would be much better if there were no humans because humans might decide to switch it off,” which would diminish the number of paper clips. (Apparently there’s actually a game built on this thought-experiment. Available at office-supply stores near you, no doubt . . .)
A widget-producing machine like the paperclip maximizer has a simple and concrete purpose. But the purpose need not be so mundane. Three Laws Lethal has one character instilling the goal of learning to speak, as noted above. A recent article by Lydia Denworth describes a real-life robot named Torso that’s being programmed to “pursue curiosity.” (Scientific American, Dec. 2024, at 64, 68)
It should be possible in principle to program multiple purposes into an AI. A robot might have the goal of producing paper clips, but also the goal of protecting human life, say. But it would then also be necessary to include in the program some way of balancing or prioritizing the goals, since they would often conflict or compete with each other. There’s precedent for this, too, in ethical theory, such as the traditional “principle of double effect” to evaluate actions that have both good and bad results.
Note that we’ve been speaking of goals give to or programmed into the AI by a human designer. Could an AI choose its own goals? The question that immediately arises is, how or by what criteria would the AI make that choice? That methodological or procedural question arises even before the more interesting and disturbing question of whether an AI might choose bad goals or good ones. There’s an analogy here to the uncertainty faced by parents in raising children: how does one (try to) ensure that the child will embrace the right ethics or value system? I seem to recall that David Brin has suggested somewhere that the best way to develop beneficent AIs is actually to give them a childhood of a sort, though I can’t recall the reference at the moment.
Conclusions, Highly Tentative
The above ruminations suggest that if we want AIs that behave ethically, it may be necessary to give them both purposes and rules. We want an autonomous vehicle that gets us to our destination speedily, but we want it to respect Asimov’s First Law about protecting humans in the process. The more we consider the problem, the more it seems that what we want for our AI offspring is something like a full-blown ethical system, more complex and nuanced than the Three Laws, more qualified and guarded than Naomi’s survival-seeking Mikes.
This is one of those cases where contemporary science is actually beginning to implement something science fiction has long discussed. (Just the other day, I read an article by Geoffrey Fowler (11/30/2024) about how Waymo robotaxis don’t always stop for a human at a crosswalk.) Clearly, it’s time to get serious about how we grapple with the problem Walton so admirably sets up in his book.
Rick Ellrod's Observatory 