Talking with AI, Part 2: How Can Predicting the Next Token Lead to Coding?

Part 2 of 11 in Talking with AI: From LLM Training to AGI

A conversation between Dr. Suttisak Denduangchai and ChatGPT-5.6 Reasoning Instant, translated from the original Thai chat. ChatGPT’s explanations and examples are preserved, with headings, tables and mathematical notation adjusted for reading on the web.

Read the complete series

Me: Tell me more.

ChatGPT-5.6 Reasoning Instant:

Absolutely. This is the most interesting part of LLMs 🧠, because the central question is:

If it is trained only to predict the next token, why can it ultimately write programs, translate languages, summarize articles, or solve problems it has never seen?

The short answer is that to predict text well enough, the model is forced to build representations of what lies behind that text.

From next-token prediction to “understanding”

Ask the model to complete this sentence:

John put the ice cream on the table and left it there for five hours. When he returned, the ice cream was ____

To accurately answer melted, the model has to capture several relationships at once:

ice cream
   ↓
temperature
   ↓
room conditions
   ↓
time = 5 hours
   ↓
solid → liquid

It was not directly taught:

RULE #492:
ice cream + room temperature + 5 hours = melted

But if it does not learn a pattern like this, it will predict huge amounts of training text poorly.

The more vast and varied the text it must predict, the more useful internal representations it needs to create.

What does a Transformer do?

This is the main engine.

Suppose I write:

The animal didn't cross the street because it was too tired.

What does it refer to?

A Transformer uses an important mechanism called Attention.

In a very simple form:

                    ┌── animal
                    │
The animal ... because it was too tired
                       │
                       └──── Attention

When processing it, the model can give different weights to earlier tokens:

animal     ██████████
street     ██
cross      █
because    █

Each attention head can learn different patterns.

One head might attend to:

Pronoun → noun

Another might capture:

Subject → verb

Another:

Opening bracket → closing bracket

Another might learn dependencies in code:

var customer = GetCustomer();

Console.WriteLine(customer.Name);
                  ↑
             customer

Transformers also have many layers.

So processing proceeds roughly as follows:

Tokens
  ↓
Layer 1
  ↓
Layer 2
  ↓
Layer 3
  ↓
...
  ↓
Layer N
  ↓
Next-token probabilities

At every layer, the representation keeps changing.

A token is not necessarily a “word”

This is very important.

The text:

The cat is sleeping

is first converted by the Tokenizer into token IDs, conceptually like this:

"The"       → 1234
" cat"      → 5678
" is"       → 318
" sleeping" → 9012

Each token is then converted into a vector.

For example, this:

cat

becomes roughly:

[0.21, -0.81, 0.14, 0.53, ...]

In reality, it has very high dimensionality.

This vector space lets the model form representations of many relationships.

We can imagine, very roughly, something like:

             animal
                ↑
       cat •    |    • dog
                |
                |
     car •      |           • horse
                |
────────────────────────────→ living thing

But in reality it is not a 2D graph like this; it is a far more complex high-dimensional space.

How does it “learn a concept”?

Suppose the training data contains countless statements about cats:

Cats have four legs.

My cat is sleeping.

Cats are mammals.

The kitten chased a mouse.

Unlike dogs, cats...

Nobody tells the neural network:

Create a concept called CAT

But creating a representation that gathers the properties of “cat” helps reduce its Loss.

The concept therefore emerges from the pressure of optimization.

Broadly, this is called representation learning.

Where do abilities that look like Reasoning come from?

Consider this problem:

Alice has 5 apples. She gives Bob 2. How many remain?

The answer is 3.

But the training data contains huge amounts of problems, explanatory examples, mathematics, code proofs, and text with logical structure.

The model can therefore learn computational patterns such as:

initial quantity
      ↓
operation
      ↓
new quantity

or:

5
↓
subtract 2
↓
3

As models become larger, data increases, and training improves, some abilities can emerge more clearly, even though we did not create a module named:

reasoning.dll

inside the model 😆

Code makes this even clearer

Suppose the training data contains vast amounts of code:

if (user == null)
{
    return;
}

and:

if user is None:
    return

and:

if (!user) {
    return;
}

To predict this code well, memorizing syntax alone is not enough.

It benefits from capturing the concept:

Check whether value exists
        ↓
If invalid
        ↓
Exit early

Some representations can therefore sit above any one programming language.

This produces something interesting:

C#
 │
 │
 ├──────→ underlying concept ←──── Python
 │
 │
JavaScript

So you can ask:

Convert this C# algorithm to Python

even if that Python output may never have appeared directly in the training set.

This leads to Generalization

This is bigger than memorization.

Suppose a model has seen:

2 + 3 = 5
4 + 7 = 11
10 + 5 = 15

If it learned only a lookup table:

2+3 → 5
4+7 → 11

it would fail when it encountered something new.

But if it learned the pattern of addition to some degree, it can handle new input:

13 + 8

This is generalization.

Of course, LLMs do not generalize perfectly, and model arithmetic has limitations, especially when problems are complex or long.

Where does Scaling fit in?

Researchers have found that increasing factors such as:

Model parameters ↑
Training data ↑
Compute ↑
Training quality ↑

generally reduces loss and increases capability.

A small model might only manage:

autocomplete

A larger one:

write a paragraph

Larger still:

summarize

translate

code

solve problems

follow complicated instructions

Today, though, we should not think “more parameters = always smarter.”

Architecture, data quality, post-training, inference-time compute, tools, and the training recipe also matter enormously.

The most puzzling part 🤯

During training, no programmer writes:

model.learn("economics")
model.learn("C#")
model.learn("Thai")
model.learn("medicine")
model.learn("sarcasm")

Programmers specify the architecture and training objective.

At its simplest, roughly:

Read text
      ↓
Predict next token
      ↓
How wrong was it?
      ↓
Adjust parameters
      ↓
Do it again

Repeated at an enormous scale.

Then much of the internal structure emerges through learning.

But we need to be careful with the word “understanding”

There is a major debate about whether LLMs really understand.

One view says:

They are statistical models that are extremely good at producing language; they do not understand the world as humans do.

Another says:

If they construct representations of objects, relationships, cause and effect, and can use these to generalize, it is too easy to dismiss all of that as not being understanding.

I think the safer explanation is:

LLMs have internal representations that support some behaviours we call reasoning and understanding, but we should not assume their internal processes or experiences are like humans’.

We still have an incomplete understanding of the inner mechanisms of large models.

Now there is something even better

After training finishes, the weights barely change while we talk to it.

TRAINING
Huge amounts of data
     ↓
Change weights
     ↓
████████████
     ↓
Finished model

INFERENCE
You ask a question
     ↓
Same weights
     ↓
Compute
     ↓
Answer

So the question is...

If the weights do not change, why can an LLM read something we told it 10 pages ago and appear to “learn” from it immediately?

That takes us to Context Window, In-Context Learning, KV Cache, and Attention, which are entirely separate from Training and are essential to understanding how ChatGPT works while we are talking to it 🧠⚡

← Previous: What Does Training an LLM Mean?

Next: Context, Attention and Memory During a Conversation →

Read the series

Back to Notes