By using this site, you agree to our Privacy Policy and our Terms of Use. Close
sc94597 5 hours ago
Edit Thread

I think Ben Goertzel describes the abilities of LLM's (and this applies to diffusion models too) pretty well at 9:05 - 19:56 in this video.


There is emergent "reasoning" and "creativity" but it isn't of the same kind or extent as a human, yet.

For example he says at 12:36

"I've often given in the domain of music modeling is if you train an LLM on music up to the year 1900 only, it's not going to invent grindcore, neoclassical metal or progressive jazz. I mean, it may do cool, creative things. It'll make up new songs that you never heard before if you ask it to put West African drumming together with Western classical music. It may even manage to, like, set Mozart to a polyrhythm or something, right? But to get to jazz and metal and world music and all, this is just a number of creative leaps that I really don't think anything like a current LLM is going to be able to take.

And certainly experimentation with Suno or any music in any existing music model. You can see the practical limitations here, right? It's pretty on the initially it's really cool. Like you can pick an artist to make an arbitrary number of songs in the style of that artist. It gets their voice right. You can make a competent metal guitar solo. On the other hand, it's all banal music in the end. Like you're not getting anything really awesome out of it. Even within the defined genres that it knows, let alone like inventing some new genre of music or some profoundly new new style, right?

So there clearly are limitations which are more severe than the limitations we have, but it's not quite clear how to formalize or quantify those limitations right now.

Yeah, that's a great example. I saw a vision generator model. Oh that's okay. I saw a vision generator model and they prompted it to generate for 1956, 1957, 1958. And you could just see the morph of the styles for all of the different years. And of course, when it went past 2024, it just ran out of distribution because there was no training data. So it kind of mode collapsed. But interestingly, if you went forward enough to about 2070, you started seeing Star Trek uniforms and so on. But but you know, my intuition though is if these models are learning very abstract representations of the world, then why wouldn't they? I don't think they are.

Oh. That's interesting. Why not?

I don't think they're learning very abstract representations of the world just from looking looking inside of what they're doing. I don't I don't see how they could be. And when you, when you try to use them to derive mathematical proof, which I've played with quite a lot because that's my original background of a PhD in math, when you try to use them to derive mathematical proof, they mix things up in very silly, basic ways that lead you to think if they're building an abstract representation, it's not the right one, right? Like it doesn't represent the actual mathematical structures. And then in the case of math, there sort of is a correct abstract representation and they're not getting it right. Like you, you can in many cases give a proof sketch and it will fill in the details of your sketch, which is which is interesting. You can even give it you can give a verbal outline of a theorem, and it will turn that into like a formal logic theorem statement. So it can do quite a lot of things, but then it will it will mix things up in very, very silly ways, which like no, no graduate student would, would ever do. Right. And so it seems from that example, the abstractions it's learning are really not the ones that a human mathematician would would use.

And that's probably connected with the fact that, I mean, in the automated theorem proving world, which we had represented here by Joseph Urban from Czech Technical Institute. I mean, using llms to do theorem proving. I mean, that's not what they're doing, right? I mean, that's not what Google did with alpha geometry either, right? I mean, they use the LLM to translate Math Olympiad problems and such into formal logic and then use different sort of AI system to do the actual math, right?

So I think music is a domain where it's clear that creativity is limited, and it feels like the internal representation is not quite the right one to be profoundly creative, but math is a bit more rigorous. So when you see the sorts of errors it makes, it's really quite clear that it's not getting the abstraction right, even when it can spit out what the definition is. And this is the frustrating thing we've all seen. Like, it will it will spit out the precise definition of, say, a non-well-founded set based on Axel's Anti-foundation axiom. But then when you ask it to derive a consequence of that, it'll 70% of the time be good, 30% of the time come up with other gibberish where like if you if you understood that definition that you just cited, you could never come up with that, with that, with that gibberish."