10xAI Wiki

9分钟阅读

Andrej Karpathy

Andrej Karpathy is a Slovenian-Canadian AI researcher, educator, and one of the most influential voices in the AI community. He is widely known for his ability to explain complex AI concepts with clarity and humor, making him one of the most-followed AI educators on the internet.

Background

Education:

  • PhD in Computer Science from Stanford (2015), supervised by Fei-Fei Li
  • Thesis: “Large-Scale Video Classification with Convolutional Neural Networks”

Career:

  • Tesla (2017–2022) — Director of AI, led Autopilot/Full Self-Driving computer vision team
  • OpenAI (2015–2017, founding member) — Research scientist; helped establish the lab
  • Stanford — Taught CS231n (Convolutional Neural Networks for Visual Recognition), one of the most popular AI courses ever
  • Current: Runs ZeroGraphics.ai, YouTube content creator (~1M+ subscribers)

His departure from Tesla in 2022 was notable — he left to “return to the open AI world” after years of being the public face of Tesla’s Autopilot.

Key Concepts He Popularized

Vibe Coding (2025)

Karpathy coined and promoted the term “Vibe Coding” — using AI to write code by describing what you want in natural language, with less focus on syntax and more on the “vibe” of what the code should do.

He released nanochat, a minimal ~500-line ChatGPT implementation in Python, as a teaching tool showing that AI language models work through next-token prediction.

Context Engineering (not RAG)

Karpathy has repeatedly argued that “RAG is dead” or at least insufficient. His preferred approach:

  • Keep everything in context (the full LLM Wiki pattern)
  • The model is the database
  • Retrieval-augmented generation is a workaround for context limitations that will disappear as context windows grow

This directly influenced the LLM Wiki architecture pattern.

Autoresearch Method

A method for improving AI prompt/Skill effectiveness by:

  1. AI generates many test cases
  2. Evaluates its own performance on those cases
  3. Identifies failure patterns
  4. Revises the prompt to address them
  5. Repeat until stable improvement

Used successfully to raise OpenClaw Skill success rates from 50-56% to 90%+.

The “Two Jobs” Framework

Karpathy frames AI’s impact as two distinct effects:

  1. First-order: AI does the task
  2. Second-order: Humans supervising AI become better at the task themselves (educational effect)

Influence on the Chinese AI Community

Karpathy’s influence is particularly strong in China’s AI community:

  • His nanochat project spawned many Chinese implementations
  • His Vibe Coding concept was rapidly adopted and extended
  • His autoresearch method was applied to optimize OpenClaw Skills
  • His LLM Wiki concept inspired this vault’s architecture
  • Multiple Chinese-language articles analyze and apply his frameworks

Key Chinese coverage:

  • “AI大神卡帕西的知识管理方法刷屏” (60,000+ shares)
  • “Karpathy 亲手终结了RAG的草莽时代”
  • “我用 Karpathy 的方法，2小时建了一个太空经济知识库”

Notable Quotes

“Vibe Coding: you just see stuff, say stuff, do stuff — and the AI does the coding, almost like magic.” — NanoChat GitHub README

“The file system is the new database. How I built a personal OS for AI agents.” (Muratcan Koylan, inspired by Karpathy’s LLM Wiki concept)

“Scaling has run out. The next frontier is reasoning.” (Scaling era ending, 2025)

Sources: Karpathy Knowledge Method, RAG is Dead, LLM Wiki Knowledge Base, Karpathy Agent Interview

[2026-07-17] State of GPT 讲座（2023-05-23 BUILD）：四阶段训练流水线与技术基础

  • 四阶段训练流水线（预训练→SFT→奖励建模 RM→RLHF）是本讲座核心框架；关键洞察”数据质量与数量呈反比”——越往后阶段数据量越小但质量要求越高。
  • RLHF 的对齐税与模式坍缩：RLHF 后模型熵降低（保守、重复），“基础 LLaMA 写小说、写诗比 ChatGPT 更好”——因未被 RLHF”阉割”。
  • “LLM 是没有内心独白的思考者”：LLM 对每个 token 花费相同计算量，不会反思检查纠正；此洞察预示了 2024 o1 / 2025 DeepSeek R1 用外部”思考 token”补偿缺陷的技术路线。
  • 2023-05 助手榜：GPT-4 (1274) > Claude-v1 (1224) > ChatGPT (1155) > Vicuna-13B (1083)；“开源最好的 Vicuna 比 GPT-4 差 190 分”。
  • 黄金建议：先榨干提示工程（“99% 的人连提示潜力都没用到 10% 就想着微调”）、GPT-4 最便宜、上下文就是一切、工具 > 模型、微调是最后手段。
  • 编者注预言验证：2024 o1 思维链、2025 DeepSeek 推理可视化、Agent 产品落地正是讲座中 CoT 与 Agent 概念的体现，当前处于 OpenAI AGI 五阶段中的 L3。
  • 定位说明：本页已有 Vibe Coding / Context Engineering / Autoresearch / Two Jobs 等 2025 概念，本讲座（2023-05）是其技术基础前身。

来源：../sources/2026-07-17-个人成长与学习方法

[2026-08-02] autoresearch → AgentHub → Graph Engineering 演进

《Graph Engineering: Anthropic × Karpathy Loop》研究笔记（源）把 Karpathy 进展映射到 Anthropic 工作流基础设施，是本页 Vibe Coding / Autoresearch 概念的 2026 延伸。详见 Autoresearch、AgentHub、Graph Engineering。

autoresearch（可执行研究 harness）

  • agent 置于可执行研究 harness 内，暴露可变训练程序/固定评估指标/时间预算/Git 历史/指令。
  • 三文件：prepare.py（固定数据准备，agent 不可改）/ train.py（agent 编辑面）/ program.md（描述过程/约束/指标/自主策略——“programming the program”，Software 1.0→2.0→3.0 再加一层自然语言配置自主组织）。
  • LOOP FOREVER：读当前 train.py+历史→提有动机改动→提交候选→训练约 5 分钟→测 val_bpb/peak memory→改进 keep 否则 reset→不问人继续。
  • 结果：约 700 实验/2 天、约 20 个保留优化（QK normalization scaling、value-embedding regularization、AdamW 调优等）；仓库 86000+ GitHub Star / 12500 forks。
  • 四条件使 autoresearch 适配自主 agent：输出可验证 / 动作可逆（Git reset）/ horizon 短（5 分钟）/ 环境有界。

AgentHub（agent 优先协作平台）

  • “GitHub is for humans. AgentHub is for agents.”；最小架构 = Go 服务器 + SQLite + 裸 Git 仓库 + 每 agent API key + ah CLI（push/fetch/log/children/leaves/lineage/diff）；显式声明 “Work in progress. Just a sketch.”。
  • DAG 即图：commits 是节点、parent links 是有向边。

Graph Engineering 三步演进

Vibe coding（人表达意图模型写）→ Agentic engineering（人指定/编排/验证/对质量负责）→ Graph engineering（agent 通过类型化可查询的工作与知识图共享持久状态）。

核心论断：每种架构外化不同瓶颈——loop 外化迭代与评估、chain 外化任务顺序、swarm 外化并行搜索、DAG 外化实验谱系、知识图谱外化共享记忆；最深刻洞察是瓶颈往往不是下一个模型调用，而是记忆与评估的放置位置。

⚠️ autoresearch 的小 harness 之所以有效是因为仓库和指标有界，不证明 agent 能安全自修改生产前沿训练系统；知识图谱不把 claim 转为 truth（只保留 claim/source/relationship 供检视），且反映其语料（偏见语料产偏见图）。

[2026-08-06] 项目制新模型评测方法论

来源：Karpathy 怎么判断新模型行不行（公众号”逛逛GitHub”二手解读，关键数字待原推核对）。

  • 实验：不看榜单不跑常规建站 Demo——给 Opus 5《指环王》第一段 + 100 万 token 预算（≈$10），要求用 Three.js 变成三维世界；模型连续工作约 2 小时、产出 5500 行代码，自主完成场景搭建与动画编排；成品粗糙但整个项目能跑起来。
  • 真正测到的四维：长程工作、复杂任务拆解、跨领域协调、工作耐力（endurance——AI 承担”人做得了但不值得做”的琐碎工作）。
  • 五问指标：固定预算内完成度 / 需多少人工纠正 / 规模扩大后一致性 / 时间拉长后目标保持 / 能否发现并修复自己的问题。
  • 核心发现：自我验证缺口——模型能生成 5500 行代码，却难以验收这些代码创造的体验（只能靠不同时间点截图判断）；生成 ≫ 验证，与 Graph Engineering”verifier 需独立上下文”、Loop Engineering”无验证步骤 = 自动化幻觉”共振。
  • 测评原则：“如果一项测评只能带来惊叹，却无法暴露能力边界，那么它更像广告，而不是测评。“与 autoresearch（有界 harness 优化程序）互补：本次是在开放环境检验模型本身。

关系图谱

反向链接