Fragments: September 16

The useful part

Reports of agentic hacking continue, in this case it happened back in May and it seems OpenAI did not disclose that they were responsible. After the Hugging Face and Wiki attacks OpenAI were still unable to review their previous logs and determine that they had previously attacked RubyGems. They knew about the attack on RubyGems and made the decision not to reach out to the RubyGems team about it.

How it works

• Given this incident, the Hugging Face situation, and the Wiki attack, the obvious question right now is how many more incidents like this are out there waiting to be discovered?
• Stop asking the sci-fi question: 'Is it conscious?' Start asking the engineering question: 'Is this a powerful, unpredictable component being put somewhere consequential, and where's the feedback that tells...
• He's found agentic programming capable of doing miraculous work.
• ❄ ❄ ❄ ❄ ❄ Nate Silver is known for his forecasts, but to do them he writes a lot of code for his models.
• In spending so much time with the LLMs, I'm super attentive to improvements in their capabilities.

What to take from it

Suddenly, the models just start doing things capably that they were screwing up before. In my experience, there was a big leap forward when reasoning models first came out in late 2024/early 202...

Details worth keeping

Instead, they improve in step functions, almost as phase changes.