Verständnislos

Falls man die Antwort auf eine Frage bereits kennt –

With o1, you can kind of do this. I gave it a problem I knew how to solve, and I tried to guide the model. First I gave it a hint, and it ignored the hint and did something else, which didn’t work. When I explained this, it apologized and said, Okay, I’ll do it your way. And then it carried out my instructions reasonably well, and then it got stuck again, and I had to correct it again. The model never figured out the most clever steps. It could do all the routine things, but it was very unimaginative.

– ist der Einsatz von AI relativ unproblematisch, auch wenn Lame Language Models weder Antworten im eigentlichen Sinn formulieren noch überhaupt sprechen können:

Still, as a proxy for semantic meaning, embeddings have proved surprisingly effective. It’s one reason why large language models have rapidly risen to the forefront of AI. When these mathematical objects fit together in a way that coincides with our expectations, it feels like intelligence; when they don’t, we call it a hallucination. To the LLM, though, there’s no difference. They’re just lists of numbers, lost in space.

Das alles ist – nach kurzer Befassung mit der verwendeten Technologie – eigentlich leicht erkennbar. Erstaunlich ist deshalb, dass sehr offensichtliche Tatsachen im Zusammenhang mit AI von einem Kind sechs AI-Forscherinnen ausgesprochen werden müssen, um wahrgenommen zu werden:

we found no evidence of formal reasoning in language models... Their behavior is better explained by sophisticated pattern matching—so fragile, in fact, that changing names can alter results by ~10%!

As a developmental psychologist, I spend a lot of my time with little kids, and they are infinitely smarter than we think. Working with AI has made me even more impressed with the kinds of things that every two-year-old is doing. It has also made the intelligence of octopuses, brine shrimp, and all the other creatures around us more vivid. On the other hand, I would not have assumed that we could learn as much just from text as we do. That raises the question about how much of what I think is my deep knowledge about the world is really just my being able to parrot the things I’ve read or heard other people say. LLMs have raised that issue in a really interesting way.