Magic Hammers
Magic Hammers
Wake up babe new AI model just dropped
With the release of OpenAI’s hyped new model that “can reason,” here are my thoughts on what I’ve been reading recently about large language models and AI safety. While these models don’t justify the fantastic claims made about them—at least not yet—we need to take steps now to ensure that future AI is safe.
OpenAI CEO Sam Altman reportedly wants to raise $7 trillion to increase the supply of graphical processing units (GPUs) for use in artificial intelligence. Last year, Altman—who has a reputation for being smart, charming, and full of shit—was fired by OpenAI’s board of directors. OpenAI’s board had a specific mandate to provide independent oversight to ensure OpenAI develops advanced AI safely, a task Altman himself has said is perhaps “the most important project in human history.” Board member Helen Toner—who I’ve interacted with professionally a few times but don’t really know—said Altman had lied to the board about OpenAI’s safety processes and about his financial stake in OpenAI’s business. OpenAI has since been accused by insiders of bypassing its safety team and of stifling dissent within the company. But Altman was rehired a few days after he was fired after investors revolted and hundreds of employees threatened to quit. While OpenAI’s board had the legal authority and arguably the responsibility to fire Altman, it didn’t ultimately have the power to do so. Toner and fellow board member Tasha McCauley—both of whom left after Altman was reinstated—have called for government regulation of advanced AI development. “Based on our experience,” they write, “we believe that self-governance cannot reliably withstand the pressure of profit incentives.”
Ted Chiang “ChatGPT Is a Blurry JPEG of the Web” (The New Yorker)
“Think of ChatGPT as a blurry JPEG of all the text on the Web. It retains much of the information on the Web, in the same way that a JPEG retains much of the information of a higher-resolution image, but, if you’re looking for an exact sequence of bits, you won’t find it; all you will ever get is an approximation. But, because the approximation is presented in the form of grammatical text, which ChatGPT excels at creating, it’s usually acceptable. You’re still looking at a blurry JPEG, but the blurriness occurs in a way that doesn’t make the picture as a whole look less sharp.”
Science fiction writer Ted Chiang has a gift for getting to the heart of complex subjects. I wrote about another essay of his when I discussed AI last year. In this essay, Chiang argues that large language models (LLMs) like ChatGPT are essentially lossy text-compression algorithms that reproduce blurry versions of the data it’s trained on. ChatGPT looks smart—and summarizing synthesizing a vast corpus of data does indeed require a kind of intelligence—but it is nevertheless essentially remixing the ideas of others. It creates an illusion of understanding by simulating what an intelligent person might say, but it doesn’t have fixed opinions of its own. Sometimes the transformations it performs on our ideas are novel and interesting, and sometimes the result is hallucinatory nonsense. ChatGPT may be a step toward broader machine intelligence, but right now it can’t easily distinguish between the two.
Colin Fraser, “Generative AI Is a Hammer and No One Knows What Is and Isn’t a Nail” (Medium)
“Can a hammer do the dishes? This seems unlikely on its face but the pace of technological change here feels incredibly fast and you don’t want to look like an idiot. After only a few months, OpenAL releases Hammer4 and you learn that archaeologists are now using new AL hammers to dig for fossils—who could have ever imagined that? Experts who are very smart are regularly making promises about how even if Hammer4 can’t do the dishes today, it’s only a matter of time before Hammer5 comes out which will surely have even more capabilities.”
Colin Fraser suggests that maybe scale isn’t all you need. LLMs may not be a universal hammer; no matter how much computing power you give them there may be tasks that they’re just not good at doing. Fraser argues that the generative AI strategy of making probabilistic bets about how to complete a task “is not good at generating output that satisfies specific criteria.” That’s because they aren’t responding to prompts in a rigorous, logical way but rather guessing what logical responses might look like. This strategy produces impressive results, but it’s not obvious it will approach general intelligence as we give it more data and more compute. If in fact these models aren’t suited for all types of tasks, they may not actually be that useful, much less warrant a $7 trillion investment. That shouldn’t be surprising; as Fraser says, “magical universal hammers are very rare.”
Yoshua Bengio, “Reasoning Through Arguments Against Taking AI Safety Seriously”
“The genie is possibly out of the bottle: Most of the scientific principles required to reach AGI may have already been found. Clearly, large amounts of capital is being invested with that assumption. Even if that were true, it would not necessarily mean that we collectively should let the forces of market competition and geopolitical competition be the only drivers of change. We still have individual and collective agency to move the needle towards a safer and more democratic world.”
In this essay, Turing Award winner Yoshua Bengio addresses the arguments commonly made for dismissing AI safety. Bengio endorsed SB 1047, a bill regulating frontier AI models that was recently passed by the California legislature. As I’ve said before, I’m pretty skeptical that AI will destroy the world. But I still think we should take the risk that AI could cause a catastrophe seriously. AI capabilities are already superhuman in many ways and continue to improve. AI doesn’t need to exhibit general intelligence or consciousness—it doesn’t have to be some kind of magical universal hammer—to be capable of dangerous behavior or to be used for dangerous ends. LLMs are opaque and behave in unpredictable ways. AI developers may not want to create dangerous AI, but will nevertheless have powerful incentives to take dangerous shortcuts for their own gain (I wrote a paper a few years ago about this issue of collective action problems in AI safety1). No one can say with confidence whether AI development will stall or will continue to advance rapidly. But that uncertainty should make us less rather than more complacent; we need to make sure AI is safe whatever happens. As Bengio writes, “We cannot let corporations grade their own homework and simply put out nice-sounding assurances. We don’t accept this in other technologies such as pharmaceuticals, aerospace, and food safety. Why should AI be treated differently?”
Other things that I read recently and that I recommend include Dino Buzzati’s 1960 novel The Singularity, which reads like a Julio Cortazar novel about AI; “With Folded Hands,” Jack Williamson’s 1947 story about the unintended consequences of technological progress; and “The Great Automatic Grammatizator,” Roald Dahl’s prescient 1954 story about AI writing.
Telling the Future is supported entirely by paying subscribers. If you find it valuable or interesting, please consider buying a paid subscription. And if you enjoyed this post, please share it with others!
Excellent this, more please!