I have already learned a lot from the human knowledge LLM was trained on (and yes i know about halus and of course I fact check everything) but learning coding using a LLM teacher fucking rocks
Thanks to copilot, I “understand” linux kernel modules and what is needed to backport, for example.
Of course, the training data contains all that information, and the LLM is able to explain it in a thousand different ways until anyone can understand it.
But flip that around.
You could never explain a brand new concept to an LLM which isn’t already contained somewhere in its training data. You can’t just give it a book about a new thing, or have a conversation about it, and then have it understand it.
A single book isn’t enough. It needs terabytes of redundant examples and centuries of cpu-time to model the relevant concepts.
Where a human can read a single physics book, and then write part 2 that re-explains and perhaps explores new extrapolated phenomenon, an LLM cannot.
Write a completely new OS that works in a completely new way, and there is no way you could ever get an LLM to understand it by just talking to it. To train it, you’d need to produce those several terabytes of training data about it, first.
And once you do, how do you know it isn’t just pseudo-plagiarizing the contents of that training data?
I have already learned a lot from the human knowledge LLM was trained on (and yes i know about halus and of course I fact check everything) but learning coding using a LLM teacher fucking rocks
Thanks to copilot, I “understand” linux kernel modules and what is needed to backport, for example.
Of course, the training data contains all that information, and the LLM is able to explain it in a thousand different ways until anyone can understand it.
But flip that around.
You could never explain a brand new concept to an LLM which isn’t already contained somewhere in its training data. You can’t just give it a book about a new thing, or have a conversation about it, and then have it understand it.
A single book isn’t enough. It needs terabytes of redundant examples and centuries of cpu-time to model the relevant concepts.
Where a human can read a single physics book, and then write part 2 that re-explains and perhaps explores new extrapolated phenomenon, an LLM cannot.
Write a completely new OS that works in a completely new way, and there is no way you could ever get an LLM to understand it by just talking to it. To train it, you’d need to produce those several terabytes of training data about it, first.
And once you do, how do you know it isn’t just pseudo-plagiarizing the contents of that training data?