Talking with AI, Part 4: Why More Thinking Can Improve an Answer
How inference-time compute, reasoning, multiple attempts and verification can improve results without changing the model’s weights.
Part 4 of 11 in Talking with AI: From LLM Training to AGI
A conversation between Dr. Suttisak Denduangchai and ChatGPT-5.6 Reasoning Instant, translated from the original Thai chat. ChatGPT’s explanations and examples are preserved, with headings, tables and mathematical notation adjusted for reading on the web.
Me: Tell me more.
ChatGPT-5.6 Reasoning Instant:
Now we reach the point where modern LLMs begin to look unlike autocomplete 🧠⚙️
The question is:
If the weights are the same, why does letting a model “think more” let it answer better?
Because the intelligence we see in use does not depend only on what is stored in the weights. It also depends on how much computation we allow the model to use to find the answer at inference time.
This is called inference-time compute or test-time compute.
Start with an ordinary LLM
Suppose we ask:
A shop sells pens for 17 baht and pencils for 8 baht. If you buy 7 pens and 13 pencils, how much do you pay?
The direct method is:
17 × 7 = 119
8 × 13 = 104
119 + 104 = 223But the model does not have to produce all these steps. It might try to jump straight to:
Question
↓
Transformer
↓
"223 baht"There is no problem for an easy question.
But if the task is:
Analyze this architecture
Find a race condition
Examine security implications
Compare 4 approaches
Then propose a migration planjumping directly from the question to a final answer becomes dangerous.
What can extra computation do?
Instead of:
Problem
↓
Answerthe idea can be roughly:
Problem
↓
Analyze
↓
Decompose
↓
Check assumptions
↓
Try solution
↓
Verify
↓
Revise
↓
AnswerThe important point is that the weights may be identical.
But the computational path is longer.
Comparing it to a person helps illustrate it:
47 × 83 = ?
You may not get a “new brain” after picking up paper.
But instead of:
Think for 1 secondyou can do:
47 × 80 = 3760
47 × 3 = 141
3760 + 141 = 3901
Check againThe probability of a correct answer rises because you used more computation.
A reasoning model is not simply “a bigger LLM”
This is an important distinction.
There are two axes:
Training compute
↑
Used when creating the model
Inference compute
↑
Used when answering a questionConceptually:
Inference Compute
↑
│
Reasoning │ ●
│
│
│
Normal assistant │ ●
│
└────────────→
Training
capabilityWe can invest more on both sides.
What is it “thinking” about?
We need to be careful with that word.
We should not imagine a tiny human sitting in a neural network saying:
“Hmm... how shall we solve this equation today?”
😂
What actually happens is computation in the neural network and an inference system designed to use more steps/resources to reach a better output.
Each company’s reasoning system differs, and many details are not disclosed.
But conceptually we can think of:
Input
↓
intermediate computation
↓
evaluate possibilities
↓
refine
↓
OutputProgramming provides a very clear example
Suppose you tell a coding agent:
TidySpace scans the NTFS MFT slowly. Please optimize it.
Using this approach:
Prompt
↓
LLM
↓
Edit FileA.cs
↓
Donehas a high chance of breaking something 💣
It does not even know where the bottleneck is yet.
A better agent should:
Read repository
↓
Find MFT scanner
↓
Trace call path
↓
Inspect allocations
↓
Inspect I/O pattern
↓
Form hypothesis
↓
Change implementation
↓
Build
↓
Benchmark/Test
↓
FAIL?
↙ ↘
Yes No
↓ ↓
Investigate Review diff
↓ ↓
Fix Final answerNotice that the LLM itself does not need to change.
What increases is the number of interactions with the environment and the amount of computation.
This is an Agentic Loop
An ordinary LLM:
User → LLM → AnswerAn agent:
┌──────────────┐
│ ↓
User → LLM → Action → Result
↑ │
└──────────────┘
repeatFor example:
LLM:
I need to find the implementation first
→ search code
Result:
MftScanner.cs
LLM:
Open the file
→ read file
Result:
...
LLM:
This allocates byte[] for every record.
I need to check whether it is really the hot path.
→ search usages
...
→ edit
→ build
Build failed
→ inspect compiler error
→ edit again
→ build
PASS
→ tests
PASSThis is the enormous difference between:
LLM
and:
AI Agent
An Agent therefore has an “outside world”
An LLM alone is confined to text:
┌─────────┐
Text → │ LLM │ → Text
└─────────┘An agent gains hands and eyes:
┌───────────┐
│ LLM │
└─────┬─────┘
│
┌────────────┼─────────────┐
↓ ↓ ↓
Files Browser Shell
↓ ↓ ↓
Code Web Program
│ │ │
└────────────┼─────────────┘
↓
Results
│
└────→ LLMTools can include:
Git
GitHub
Compiler
Tests
Database
Email
Calendar
AWS
Browser
APIs
Python
...The capability of the whole system is therefore not equal to the capability of the LLM alone.
This produces an equation I want you to remember
It is not an academic equation, but it is useful for thinking:
AI System Capability
≈
Model
× Context
× Tools
× Compute
× FeedbackA very capable model without information:
Great Model × Bad Context
↓
🤷A capable model using old information:
Great Model × Stale Data
↓
Confident nonsenseA capable model + repository + compiler:
Model
+ Code Search
+ Compiler
+ Tests
+ Git
+ Iteration
↓
Coding AgentThis is why comparing AI through benchmarks of the model alone is becoming insufficient for agentic work.
Verification is extremely important
LLMs have one basic problem:
They can create an answer that sounds convincing even when it is wrong.
But when there is an environment that can check it, we can change the task.
From:
LLM:
"I think this code compiles"to:
LLM
↓
write code
↓
dotnet build
↓
Compiler:
CS1503 ...Now we do not have to “believe” the LLM.
The compiler is the judge 🧑⚖️
Fix it again:
LLM
↓
Fix
↓
dotnet build
↓
PASS
↓
dotnet test
↓
137 passed
↓
Review diffThis is more powerful than simply increasing the model’s intelligence.
Reinforcement Learning returns here
Remember our discussion of training in the first episode:
Model
↓
Output
↓
Reward
↓
Adjust weightsIf we can create environments with verifiable answers, such as:
Math
Coding
Games
Formal proofswe can give fairly clear rewards.
For coding:
Generated patch
↓
Tests
↓
150/150 PASS
↓
Reward ↑or:
Generated patch
↓
Tests
↓
73 failures
↓
Reward ↓This gives better feedback than simply asking a human:
“Does this answer look good?”
Why are Math and Code important for training reasoning?
Because they have verifiable rewards.
The problem:
Solve:
x² - 5x + 6 = 0can be checked as:
x = 2, 3Code:
Implement function Xcan be checked with:
unit testsChess:
win / loseFormal theorem:
proof valid / invalidThis produces a very sharp training signal.
The real-world task:
“Design the best architecture”
is much harder to verify, because there is no:
ArchitectureTests.exe
PASS: This is objectively the best architecture.Unfortunately 😆
What is the “Thinking level” you see in ChatGPT?
At the product-concept level, choosing to use more reasoning means accepting:
Compute ↑
Latency ↑
Cost ↑to increase the chance of:
Analysis quality ↑
Planning ↑
Verification ↑
Complex problem solving ↑So the highest setting should not be used for everything.
Ask:
What is HTTP 404?
There is no need to send the AI into a cave to meditate for 40 minutes 🧘
But:
Analyze the entire HealthTAG architecture and create a migration plan with no downtime
Here, a larger reasoning budget is useful.
This leads to an idea bigger than “Model”
We used to think:
Better AI
=
Bigger modelNow the picture is becoming:
Better AI System
=
Good Model
+
Reasoning Compute
+
Context Management
+
Memory
+
Retrieval
+
Tools
+
Environment
+
Verification
+
Agent LoopAnd you can see that many of these things do not live in the neural network at all.
This is very important.
Let’s combine everything we have learned
The full picture now is:
TRAINING
│
┌────────────▼────────────┐
│ Training Data │
└────────────┬────────────┘
↓
Token prediction
↓
Backprop
↓
Update weights
↓
┌────────────────┐
│ LLM │
│ Transformer │
│ Weights │
└───────┬────────┘
│
══════════════════════╪══════════════════════
│
INFERENCE
│
┌────────▼────────┐
│ Context │
│ prompt / files │
│ retrieved info │
└────────┬────────┘
↓
┌────────────────┐
│ LLM │
└───────┬────────┘
↓
Reasoning
↓
Need a tool?
↙ ↘
Yes No
↓ ↓
┌────────────────┐ │
│ Web / Code / │ │
│ Git / DB / API │ │
└───────┬────────┘ │
↓ │
Result │
↓ │
└─────┬─────┘
↓
Verify
↓
Good enough?
↙ ↘
No Yes
↓ ↓
Loop AnswerThis is much closer to the mental model of a modern AI system than the word “chatbot.”
And one biggest puzzle piece remains 🧩:
When we train for real, what do we do with tens of thousands of GPUs? Where do hundreds of billions of parameters live? Why is so much VRAM required? What are FP32, BF16, and FP8? What are Tensor Parallel, Data Parallel, and MoE?
If we continue into that topic, you will begin to see the hardware/infrastructure level: how an LLM built from the equation predict the next token becomes a data center that consumes enormous amounts of electricity ⚡🏭
← Previous: Context, Attention and Memory During a Conversation
Next: GPUs, VRAM and the Infrastructure Behind LLM Training →
Read the series
- Part 1: What Does Training an LLM Mean?
- Part 2: How Can Predicting the Next Token Lead to Coding?
- Part 3: Context, Attention and Memory During a Conversation
- Part 4: Why More Thinking Can Improve an Answer (this episode)
- Part 5: GPUs, VRAM and the Infrastructure Behind LLM Training
- Part 6: From a Base Model to an Assistant
- Part 7: Does an LLM Memorize, Learn or Guess?
- Part 8: Following a Token Through a Transformer
- Part 9: From an LLM to an AI System and Agent
- Part 10: More Compute for Training or for Answering?
- Part 11: From More Capable LLMs to the Question of AGI