Beyond words: What AI is really learning -- and what it knows that we never taught it
Imagine you had to finish every sentence in every book ever written -- with just your best guess of the next word. That’s how large language models (LLMs) like GPT-4 start learning.
LLMs use self-supervised learning, meaning they don’t need someone to label or explain the data to them. Instead, they learn by reading vast amounts of text from books, code, academic papers, Wikipedia (and its 57 million+ articles), Reddit forums, and news articles, in addition to billions of others, and then predicting what word comes next in a sentence -- over and over again.