Why AI Can't "Think" Like Humans: A Deep Dive into the Engineering Gap Between System 1 and System 2

In current AI discussions, we frequently hear the term "Reasoning." Whether it’s OpenAI’s o1 series or various Chain-of-Thought (CoT) techniques, the goal is to make models "think" before answering. However, from the perspectives of cognitive science and engineering implementation, current Large Language Models (LLMs) are still undergoing a difficult transition from "intuitive reaction" to "logical thinking."

To understand this, we need to introduce the two cognitive systems proposed by Nobel laureate Daniel Kahneman in Thinking, Fast and Slow: System 1 (Fast Thinking) and System 2 (Slow Thinking).

System 1: Probabilistic "Intuitive Snapshots"

Current standard LLMs (such as GPT-4 and Claude 3.5) are essentially extremely powerful System 1 engines.

When you input a question, the model performs large-scale probabilistic prediction. It does not construct a logical model in its "mind"; instead, it rapidly predicts the next token based on patterns found in its training data. This process is similar to human "intuition": when you see a red, round object, you instantly recognize it as an "apple," without needing to go through the logical deduction of "color is red $\rightarrow$ shape is round $\rightarrow$ plant classification $\rightarrow$ it is an apple."

Characteristics of System 1:
- Extremely Fast: Results are generated almost instantaneously.
- Heuristic: Based on pattern matching rather than logical proof.
- Prone to Error: When facing complex mathematics, deep logic, or unseen edge cases, it produces so-called "hallucinations."

System 2: Costly "Logical Deduction"

System 2 represents the state humans enter when handling complex tasks: when calculating $17 \times 24$ or planning an international trip, you enter a focused, slow, and energy-consuming state. You operate step-by-step, check intermediate results, and backtrack when errors are detected.

In AI engineering, the core of implementing System 2 lies in shifting "computational load" from the training phase to the inference phase (Inference-time Compute).

From Token Generation to Path Search

Traditional LLMs produce output in a unidirectional, streaming manner. True System 2 requires:
1. Planning: Breaking down large goals into subtasks.
2. Verification: Self-checking generated intermediate steps (Self-Correction).
3. Search: Finding the optimal solution among multiple possible paths (e.g., using Monte Carlo Tree Search, MCTS).

This is why models like o1 have introduced reinforcement learning-driven Chain-of-Thought (CoT). They no longer simply predict the next word; instead, they perform multiple attempts, failures, and corrections internally until they find an answer that passes verification, only then outputting it to the user.

Engineering Trade-offs: Latency vs. Accuracy

Pushing AI from System 1 to System 2 faces significant engineering challenges:

• Explosion in Inference Costs: If a problem requires the model to internally "think" through $1000$ tokens to output an answer of only $10$ tokens, the inference cost increases by $100$ times.
• Unacceptable Latency: Users are accustomed to instant responses. If a simple greeting requires the model to "ponder" for three seconds, the product experience will degrade significantly.
• The Verification Challenge: To enable self-correction, there must be a reliable reward function or verifier. While this is easy for math problems (is the result correct?), how do we define "correctness" for literary creation or strategic analysis?

Future Directions: Dynamic Routing and Hybrid Architectures

Future AI systems will not rely on a single mode but will function as dynamic routing systems:
- Simple Requests $\rightarrow$ System 1 $\rightarrow$ Low-latency, low-cost responses.
- Complex Requests $\rightarrow$ System 2 $\rightarrow$ High-latency, high-precision deduction.

When we talk about AGI (Artificial General Intelligence), we are essentially talking about an agent capable of autonomously deciding when to invoke System 2. It knows when it can answer based on intuition and when it must stop and "think seriously," just like a human.