AI大问题小问题 杂问答 集中贴

aidj

请写出AI研究员过去3年依旧无法大语言模型的10大痛点或天花板。

Here are the 10 core pain points or ceilings that AI researchers still find insurmountable in large language models, despite three years of rapid progress:

---

**1. Hallucination: From "Nonsense" to "Plausible Falsehood"**
This has been a core pain point since the GPT-3 era. Today's model hallucinations are more subtle and dangerous — the model generates content that contradicts facts or user intent in a highly fluent, confident, and seemingly logical manner. In serious domains like medicine, law, and finance, this "well-reasoned fabrication" is far more dangerous than an obvious error, because it so easily wins human trust.

**2. Fragile and Overestimated Reasoning**
Models can solve Olympiad math problems yet stumble on simple commonsense reasoning. Their reasoning is fundamentally pattern matching, not genuine causal or logical derivation. A tiny, irrelevant distraction — such as adding an inconsequential background detail to the classic "farmer crossing the river" puzzle — can cause the model's reasoning to collapse. It memorizes the "shape" of an answer without internalizing the underlying physical or logical laws.

**3. The "Moravec's Paradox" of Value Alignment**
Making a model refrain from outputting high-level taboos like violence or pornography is easy. But making it truly grasp complex human values like "fairness," "kindness," or "honesty," and strike the right balance in dilemmas (like a white lie), remains profoundly difficult. Simple RLHF only trains a veneer of surface correctness. Once prompted adversarially, the model easily exposes internalized biases or makes rigid, mechanical moral judgments.

**4. "Lost in the Middle" of Long Contexts**
Despite context windows expanding to millions of tokens, a model's *effective attention* is far from uniform. Research shows it attends best to information at the very beginning and end, while losing focus on information in the middle. This is like a person reading a long report and remembering only the abstract and conclusion, while the critical, detailed arguments in the middle become a blur. This severely limits reliability in tasks like long-document analysis or large codebase comprehension.

**5. Catastrophic Forgetting in Continual Learning**
This is a classic, unsolved problem of connectionist models. To teach a model new knowledge (like a recent event) without erasing old knowledge, the most reliable method remains prohibitively expensive full retraining or fine-tuning. If you try to update the model with a small amount of new data, it can abruptly "forget" other crucial capabilities. A paradigm for true, gradual, human-like continuous learning has yet to emerge.

**6. Global Collapse in Planning and Multi-Step Action**
Models excel at single-step operations. But ask for a long-term, multi-step plan with dependencies (like "plan an entire developer conference"), and errors cascade and compound. It forgets the original goal, makes contradictory decisions, or falls into infinite loops. This is a fundamental flaw of its "System 1" intuitive generation, which lacks an intrinsic "System 2" mechanism for deep deliberation and global consistency checking.

**7. The Absence of Causal and Counterfactual Reasoning**
LLMs are superb correlation-finders but terrible causal reasoners. They struggle profoundly with counterfactual questions like, "If historical event X hadn't happened, how would Y be different?" They cannot construct a coherent mental model of the world and trace how altering a causal link would change the chain of events. Instead, they piece together a statistically correlated, seemingly plausible narrative.

**8. The Colossal Gap in Cost and Efficiency**
The intellectual energy density is shockingly low. For commonsense reasoning that the human brain achieves on ~20 watts, a model requires thousands of GPUs consuming megawatts of power. This "brute-force compute for emergent intelligence" model faces an almost insurmountable economic and environmental cliff on the path to ubiquity, real-time applications, and on-device deployment.

**9. The Evaluation Crisis: Benchmarks, Not Understanding**
We lack any true measure of a model's comprehensive capability. All mainstream benchmarks are rapidly saturated, and "human preference" evaluations are expensive and unstable. We don't know if a model is genuinely smarter, or just fitting the test set better. A durable test that can probe the upper limits of reasoning and reveal the depth of true "understanding" remains completely absent.

**10. The Dark Forest of Theoretical Understanding**
We know *how* to train a model, but cannot fundamentally explain *why* gradient descent enables such a massive neural network to exhibit complex emergent intelligence. Will scaling laws hold indefinitely? Where is the boundary of emergent abilities? This absence of a foundational theory turns all optimization into a form of alchemy. We have built an artifact that has surpassed our own theoretical capacity to explain it.

---

These pain points are deeply interconnected, many pointing to the same core truth: **Current LLMs are still fundamentally statistical simulators, not mental models.** Breaking through this ceiling likely requires not incremental patches on the existing path, but an entirely new paradigm and theoretical framework.

aidj

Convolutional Neural Networks (CNNs): Ideal for processing visual data, such as image and video recognition.

Recurrent Neural Networks (RNNs) / Transformers: Designed to process sequential data, making them the backbone of natural language processing and generative AI.

Generative Adversarial Networks (GANs): Used for generating new, synthetic content by pitting two networks against each other (one to generate, one to evaluate)


LLM belongs to which group in precise. No ambiguous answer.

Gemimiarge Language Models (LLMs) belong precisely to the **Transformers** group.

---

### Why this is the precise classification:

* **The Architecture:** Every modern LLM (such as the GPT series, Claude, Llama, and Gemini) is built fundamentally on the **Transformer architecture**, which was introduced in 2017.
* **The Mechanism:** They rely entirely on the **self-attention mechanism** to process and generate sequential text data, completely replacing older Recurrent Neural Network (RNN) designs in state-of-the-art applications.


Qwen: LLMs belong precisely to the **Recurrent Neural Networks (RNNs) / Transformers** group (specifically, they are built on the **Transformer** architecture).


Grok: **Recurrent Neural Networks (RNNs) / Transformers**

LLMs (Large Language Models) are **precisely** based on the **Transformer** architecture (a direct evolution/successor in the sequential data category). They are designed for processing sequential data like text, forming the backbone of natural language processing (NLP) and generative AI.

aidj

刚刚又看到 台湾人在脸书上分享 鼓吹A G I，真是令人不爽，数学上推论出 transformer 注意力机制 不可能。

连自己都骗

Engineer

AI帖文应该被禁止，特别是楼主牛头不对马嘴的帖文

aidj

Engineer 发表于 8-6-2026 07:49 AM
AI帖文应该被禁止，特别是楼主牛头不对马嘴的帖文

以Anthropic公司为例，memory 是如何使用的：runtime memory 给大语言模型、算力(token generator)，必不可少 realtime Accounting manager 。如果能拿掉后者，性能稳定又加速。最好是失败推理不计费。

**Anthropic（Claude）的 memory 使用主要围绕 LLM 推理的 KV Cache（键值缓存）、runtime 资源管理和实时计费/会计系统展开。** 你提到的 **runtime memory（给 LLM 和 token generator）** 是核心必备，而 **realtime Accounting manager** 是他们为了精确计量、计费和资源分配而实现的复杂层，如果能简化或移除部分开销，确实能提升稳定性和性能，尤其希望“失败推理不计费”。以下基于公开文档、API 行为和相关讨论的分析。

### 1. Runtime Memory：KV Cache 是核心（给 LLM 和 Token Generator 必不可少）
Anthropic 在 Claude API 和 Claude Code 等产品中 heavily 依赖 **Prompt Caching + KV Cache** 来管理上下文（context window，可达 200k~1M tokens）：

- **KV Cache 是什么**：LLM 推理时，attention 机制为每个处理过的 token 计算 Key-Value 对，存储在 GPU VRAM（runtime memory）中。后续 token 生成时直接复用，避免重复计算前缀（prefix）。这是 token generator（生成器）的性能关键——没有它，长上下文推理会慢很多倍、成本爆炸。
- **Prompt Caching**：用户可通过 `cache_control` 标记提示块。Anthropic 服务器维护 KV cache（内存中，不持久化到磁盘），支持 5 分钟或 1 小时 TTL。Cache hit 时读成本大幅降低（例如 ~10% 原价），miss 时全量计算并写入（写成本稍高）。这直接服务于 runtime 推理效率。
- **实际使用**：在多轮对话或 Claude Code（代理/工具使用场景）中，系统尽量复用 prefix KV cache。闲置超时后可能 cache miss，导致全量重新处理（成本和延迟 spikes）。Thinking（思考）模式等功能会进一步放大 cache 写入量。

**为什么必不可少**：Transformer 架构下，没有 KV cache 的 autoregressive 生成几乎无法实用，尤其大模型 + 长上下文。Anthropic 在基础设施上投入巨大来管理这个（GPU 内存分配、eviction 策略等）。

### 2. Realtime Accounting Manager（实时会计/计量系统）
Anthropic 有专门的 **Compute Accounting Manager** 等角色/系统，负责实时追踪：
- 输入/输出 tokens（包括 cached vs uncached）。
- Cache write/read、batch 处理、agent session-hours、工具调用、代码执行等**多维度计量**。
- 资源使用（compute time）、配额、billing。

这不是简单的日志，而是 **realtime** 的，因为：
- 计费是 pay-as-you-go（按 token 实时扣）。
- 需要支持 rate limits、usage alerts、subscription（Claude Max 等）与 API 配额分离。
- 防止滥用、精确分配 GPU 等稀缺资源。

这个层增加了 overhead（延迟、复杂性），因为每次请求都要经过计量、配额检查、cache 管理等。bug 发生时（如 cache 失效），会导致“全量计算却仍计费”，或失败请求也消耗 quota。

### 3. 如果“拿掉” Accounting Manager 的影响
- **性能稳定 + 加速**：是的。移除/简化 realtime 计量能减少请求路径中的检查点、锁和同步开销，让推理更直接（类似本地部署或简化 inference server）。KV cache 等 runtime memory 仍保留，但 billing/accounting 的 overhead 下降，吞吐可能提升，稳定性更好（少一些 quota-related 错误）。
- **实际案例**：Anthropic 内部 bug（如 cache write/read 失败）曾导致“最大工作量 + 全价计费”，暴露了 accounting 层与 inference 耦合的复杂性。简化它能减少这类问题。
- **挑战**：完全拿掉会丢失精确 billing、多租户隔离、滥用防护和企业级功能。开源/自托管方案（如 vLLM + 简单 metering）常采用异步/事后 accounting 来缓解。

**最佳实践建议**（你的“失败推理不计费”）：
- Anthropic 当前不总是完美实现（有用户报告失败/部分处理仍扣费，尤其 bugs 时）。API 错误（如 400 invalid_request）通常不计费，但超时或中途失败可能已部分消耗。
- 优化方式：用 explicit cache_control 最小化 miss；Batch API（非实时，成本更低，不占标准 limits）；客户端重试逻辑 + token 预估；监控 usage 对象返回。
- 自建类似系统时：把 accounting 做成异步（inference 先跑，成功后再计量），失败路径零计费 + 快速 reject。结合 prefix caching + eviction 策略，runtime memory 专注服务 LLM。

总之，Anthropic 的架构高度优化了 **runtime KV memory** 来支撑高效 token 生成，但 **realtime accounting** 是企业级 billing/控制的代价。如果你是在设计自有系统，优先解耦 inference runtime 和 metering 层，能显著提速和稳定，同时实现“失败不计费”。有具体实现细节或代码层面问题可以再细聊！