Fragments: August 24

The useful part

I was listening to Ezra Klein's interview with Helen Toner about the recent OpenAI hack of Hugging Face and the subsequent discovery that there were swarms of agents inside OpenAI doing unsanctioned activities. One of the points Klein made was that at no point did any of these (thousands of?) agents ever try to check in with a human [Klein:] So these message boards — you have however many A.I. Hey, researchers, programmers, parents at OpenAI, Anthropic — do you want us coordinating with each other on this message board we have created in the innards of your systems?

How it works

• [Toner:] Or even F.Y.I., we have a message board we're coordinating on in the innards of your system.
• No "hey, some of the agents in here are doing sketchy things", no sign of an AI whistleblower.
• ❄ ❄ ❄ ❄ ❄ Is the AI bubble so big that the frontier companies like OpenAI and Anthropic have no way of becoming a viable business?
• If that's the case, Bruce Schneier and Nathan Sanders have a possible path:
• Evidence suggests the market itself could reassess that these companies offer nothing of financial value.

What to take from it

If these AI companies should fail in the financial markets, the US should nationalize them and convert them into national labs operated under democratic control that preserve their benefit to the public interest.

Details worth keeping

agents posting hundreds of thousands of messages. Listening to that, another thing occurred to me. none of these agents thought to rat the others out.