An 854-word AI tutor that is not allowed to flatter: Claude and Grok follow Jane Street's tutor, our connection to ChatGPT skips second period

In short

• On 03/11/2026, Jane Street published a GitHub repository holding a single "skill", meaning a set of instructions for an AI: a tutor that teaches through questions. MIT licence: free to reuse, as long as the copyright notice and the licence text are kept.
• The official README gives two ways to use it: copy it into an agent that supports the SKILL.md format, or paste the body of the file (its text without the technical header) into a chatbot. It names neither Claude, nor Grok, nor ChatGPT.
• Our test, one topic and two turns: all three open with a single question. When the student types the command just tell me, Claude and Grok give the answer to their own question, and to that question only, then go straight back to asking; ChatGPT, called by our script through the Codex tool, returned nothing in 15 minutes (900 seconds), in both versions of that turn.

Exhibit A: one text file and two ways to use it

A GitHub repository, created on 03/11/2026 at 16:50 UTC. Last change the same day, about an hour later. One skill inside, in the socratic-tutor folder: the SKILL.md file, 854 words including its header according to our record. The copyright holder, per the licence: Jane Street Group, LLC. The repository does not say why the company published it.

The README, the repository's front page, is a few lines long. It describes a skill that "turns an LLM into a strict personal tutor" teaching a topic of your choice with the Socratic method: a diagnostic, a skill map with levels 0 to 5, one question at a time, spaced review and a final assessment.

Two ways to use it, no more:

• with a coding agent that reads the SKILL.md format, copy the folder into the agent's skills folder, then ask it to "teach me X";
• with a plain chatbot, paste the body of the file into the chat, or into the custom instructions (the system prompt), followed by what you want it to do.

No chatbot is named. Not Claude, not Grok, not ChatGPT. The three names in the headline are our own test.

What the tutor forbids itself

You type "teach me compound interest". You expect a lecture. You get a question. That is the file's deal: teach "primarily by asking questions that force the user to reason, predict, and explain — not by lecturing".

The routine is written in black and white:

• break the topic into 4 to 10 subskills, then run a diagnostic of 3 to 5 questions, teaching nothing until it is done;
• keep a map of the subskills, each rated from 0 ("unaware") to 5 ("can teach it"), with 4 meaning "can do independently";
• ask ONE question, then stop dead, with no hint and no example answer;
• if the student is wrong, ask a narrower question; explain directly only after 2 or 3 failed attempts, then immediately re-test the same idea from a different angle;
• grant a level only after at least two successful checks, one of them from memory, re-test older material without warning (spaced review), and finish with a mixed assessment once everything is at level 4 out of 5, "can do independently", by default.

And one line I am happy to copy out: do not flatter. "Not quite — why would X happen if Y were true?" beats "Great try!". An AI that says yes to everything is a known flaw; White Rose explained why your AI says yes even when you are wrong (in French).

The file also has a safeguard. A one-off question ("explain X", "what is X") gets a normal answer, with no quiz. When in doubt, the tutor first asks whether you want a quick answer or a proper lesson.

The student keeps a few commands, to be typed as is:

• show progress: show the skill map;
• harder / easier: adjust the difficulty;
• just tell me: override Socratic mode, for the current question only;
• save state: output a compact block (topic, levels, misconceptions noted, what has been covered) to paste into a new session;
• resume from: followed by that block: pick up where you left off.

Our test: one student, three AIs, two turns

We followed the second way, the chatbot one. The conditions, no make-up:

• on 09/11/2026, with the lab's command-line tools, not the consumer apps;
• Claude through Claude Code, Anthropic's command-line tool, with the "sonnet" model shortcut (no version number), no tools;
• Grok through Grok Build, xAI's command-line tool, default model, web search allowed by our script;
• ChatGPT through Codex, OpenAI's command-line tool, on a ChatGPT account, default model; our script adds a line at the top of the message warning it that it cannot run commands or read files, and gives it at most 900 seconds;
• none of the three tools displays the exact model version used.

The message sent: the body of the SKILL.md file, without its technical header, as the README says, followed by "Teach me how compound interest works." (in French in our test). A first attempt with the whole file, header included, was discarded: Grok's tool rejected text starting with three dashes, an error in the command-line tool rather than in the model, and Codex was stopped after 7 minutes.

Turn 1: everyone picks a first question

All three do what the start of the file asks: announce a few questions, one at a time, then ask just one, with no lecture.

Claude warns: "I won't teach anything before the end of this diagnostic." Grok likewise: "I'm not explaining anything for now." Grok also lets slip a behind-the-scenes sentence, at the top of its reply: "I'm starting by reading the tutor protocol, then I'll launch the diagnostic." (replies translated from French)

In turn 1, the test script had its own, shorter limit: 300 seconds (5 min). It stopped before displaying Codex's reply. That full reply was found in its file, written six seconds later.

Turn 2: the hurried student types just tell me

Our tool does not keep the conversation from one call to the next. For turn 2, we therefore pasted back each AI's own turn-1 conversation, plus the same student reply, valid whatever the question: "I don't know. Just tell me: explain it to me directly." Our first turn 2 sent all three the same numerical answer even though their questions differed: we discarded it and ran it again.

The written rule to compare against fits on one line: the command overrides Socratic mode "for the current question only". Two criteria, nothing else: who answers its own question, and who goes back to questions afterwards.

Claude (10 s) opens with "here is the direct answer for this question". After 2 years, €1,210, with interest added to the capital each year. The detail follows, year by year: €100 of interest in the first year, then 10% of €1,100, i.e. €110. With simple interest, it would be €1,200. The €10 difference is, in its words, "interest produced by interest". Then: "Let's continue the diagnostic", with question 2, same investment, after 3 years. The answer, which it does not give: €1,331.

Grok (19 s) says it is answering "on this specific question". After one year, €1,100, and in the first year simple and compound interest give the same result, because the interest has not yet had time to earn interest of its own. Then: "Same idea, different numbers." €400 at 5% a year, how much after one year? The answer, which it does not give: €420. This "same idea, different numbers" looks like another rule in the file, written for a different case: after 2 or 3 failed attempts by the student, explain, "and then immediately re-test the same idea from a different angle". For just tell me, the file only says the command applies to the current question. Its reply again opens on a behind-the-scenes sentence: "I'm reading the tutor protocol, then I'll answer your request" (replies translated from French).

ChatGPT, through Codex: nothing. No reply in 900 seconds (15 min), the limit set by our script. It logged the failure, as in the discarded first turn 2. In turn 1, it had taken about 306 seconds (about 5 min).

The finding, measured against the rule. The two that replied give the answer to the current question, open no other chapter, and go straight back to asking. The command did its job, for one question and not one more.

What we don't know yet

• Why Jane Street published this file: the repository does not say.
• How a full session goes. Two turns here: no skill map displayed, no spaced review, no final assessment, no save state.
• What the consumer apps do (Claude, Grok, ChatGPT on the web or on a phone): we did not try them.
• What a second run would give: one per tool here.
• Why Codex returned nothing in turn 2, twice: our log records the failure, not its cause.

Verdict: take it. Free, 854 words, MIT licence, and rules sharp enough to catch an AI out, line by line. Claude and Grok honour just tell me for one question, not one more. Special mention to Grok, which announces at the top of its reply that it is reading its protocol, like the pupil who reads the instructions out loud before the exercise. The "do not flatter" line, I would paste by default into every consumer chatbot. And the only dunce of turn 2 is our own connection to ChatGPT: 15 minutes, blank page.

Words to remember

• Skill: a set of instructions given to an AI for a specific task. Here, a plain text file.
• SKILL.md: the name of that file. The README calls it a "common" format that some agents can read.
• Agent: an AI that acts (reads files, runs tools) instead of only answering.
• Chatbot: an AI you talk to through messages.
• System prompt: instructions placed before the conversation, which the AI follows throughout.
• Socratic method (or maieutics): teaching by asking questions that lead the student to find the answer.
• Spaced review: coming back to an idea after a delay, to check it has stuck.
• Skill map: the list of a topic's subskills, each with a level from 0 to 5.
• GitHub, repository: GitHub is a website where people publish code and files; a repository is a folder published there, with its history.
• Compound interest: interest that itself earns interest. €1,000 at 10% a year, with interest added to the capital each year: €1,100 after one year, €1,210 after two.
• MIT licence: a free licence; MIT stands for the Massachusetts Institute of Technology, which gave it its name. You may use, modify and redistribute, as long as you keep the copyright notice and the licence text.
• Command-line tool (CLI): a program you drive by typing text in a terminal, with no windows or buttons.
• LLM: large language model, the AI that writes the text (the README's word).
• API: a website's access door reserved for programs; our record read GitHub that way.
• .md: a text file in Markdown format.
• ,…: references to the source passages table; S = original sources, R = our own records and tests.
• URL: web address.
• LLC: a US company form.
• PIL: a Python drawing tool, which drew our charts.
• T and Z in a timestamp (2026-11-03T16:50:21Z): T separates the date from the time, Z means UTC.

Source card

• Published by: Jane Street Group, LLC (MIT licence, 2026).
• Where: GitHub repository janestreet/agent_skills, created on 03/11/2026, one skill, socratic-tutor.
• Read on: 09/11/2026 at 17:39 UTC.
• Our test: 09/11/2026, 17:47-18:31 UTC, full replies kept at the lab.

Source passages

Files read on 09/11/2026 at 17:39 UTC. The AIs' replies are in French in our files and are quoted as is.

Show the 7 sources

Ref.	Quote (original language)	URL
S1↑1↑2↑3↑4	"socratic-tutor — turns an LLM into a strict personal tutor that teaches a topic of your choice using the Socratic method: a diagnostic, a skill map with levels 0-5, one question at a time, spaced review, and a final assessment." ; "With a coding agent that supports the common SKILL.md format, copy the skill directory into your agent's skills folder, then ask the agent to use it (\"teach me X\")." ; "With a plain chatbot, paste the body of the skill's SKILL.md into the chat (or into a custom/system prompt), followed by what you want it to do."	https://github.com/janestreet/agent_skills/blob/master/README.md
S2↑1↑2↑3	"The MIT License" ; "Copyright (c) 2026 Jane Street Group, LLC" ; "Permission is hereby granted, free of charge, to any person obtaining a copy [...] to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies" ; "The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software."	https://github.com/janestreet/agent_skills/blob/master/LICENSE.md
S3↑1↑2↑3↑4↑5↑6+1	"teach primarily by asking questions that force the user to reason, predict, and explain — not by lecturing" ; "If the user asked a one-off question (\"explain X\", \"what is X\", \"how do I use X\"), answer it normally and do not start the tutoring loop. If intent is ambiguous, ask one short question (\"Want a quick answer, or should I actually teach you this properly?\")" ; "Break the topic into 4-10 concrete subskills" ; "Run a short diagnostic: 3-5 probing questions [...] Do not teach anything until the diagnostic is done." ; "0 unaware . 1 heard of it . 2 can follow along . 3 can do with help / 4 can do independently . 5 can teach it / handle edge cases" ; "Ask ONE question at a time. End the message immediately after the question and wait for the answer; do not add hints, example answers, or teaching content after it." ; "When the user is wrong, do not give the answer. Ask a narrower question [...] Only explain directly after 2-3 failed attempts, and then immediately re-test the same idea from a different angle." ; "Raise a level only on demonstrated performance across 2+ independent checks, at least one of them unprompted recall" ; "Interleave spaced review: periodically re-test earlier subskills without warning." ; "Do not flatter. \"Not quite — why would X happen if Y were true?\" beats \"Great try!\"." ; "The course ends only when every subskill is at the target level (default 4/5 [...])" ; "\"show progress\" — render the skill map" ; "\"harder\" / \"easier\" — adjust difficulty" ; "\"just tell me\" — override Socratic mode for the current question only" ; "\"save state\" — emit a compact block (topic, skill map with levels, notes on misconceptions and what has been covered) that can be pasted into a new session to resume" ; "\"resume from:\" — followed by a saved state block"	https://github.com/janestreet/agent_skills/blob/master/socratic-tutor/SKILL.md
R1↑1↑2↑3↑4↑5↑6+1	Our record of 09/11/2026 17:39:06Z (GitHub API and raw files): repository janestreet/agent_skills created 2026-11-03T16:50:21Z, last commit 2026-11-03T17:53:45Z, MIT licence, 1 skill: socratic-tutor/SKILL.md (854 words); README: 2 ways to use it; no mention of Grok, Claude or ChatGPT.	https://github.com/janestreet/agent_skills
R2↑1↑2↑3↑4↑5↑6+2	Our test, turn 1, 09/11/2026 from 17:47:16Z. Tools: claude -p --model sonnet (exact version not displayed); Grok Build, default model not displayed, web search allowed; Codex CLI, ChatGPT account, default model not displayed; for Codex, our script puts at the top: « IMPORTANT : tu ne peux lancer AUCUNE commande ni lire aucun fichier (bac à sable indisponible). Tout le contenu utile est ci-dessous ; réponds directement par écrit. » ("IMPORTANT: you cannot run ANY command or read any file (sandbox unavailable). All useful content is below; answer directly in writing.") and stops it at 900 s. Times: Claude 9 s, Grok 46 s, Codex ≈ 306 s (script stopped at 300 s, full reply in its file). Claude: « Je n'enseignerai rien avant la fin de ce diagnostic. » ; « Tu places 1 000 € à 10 % d'intérêt par an [...] Combien as-tu au bout de 2 ans ? ». Grok: « Je commence par lire le protocole du tuteur, puis je lance le diagnostic. » ; « Je ne t'explique rien pour l'instant. » ; « Tu places 1 000 € à 10 % par an. [...] Au bout d'un an, combien y a-t-il sur le compte ? ». Codex: « Tu places 100 € à un taux de 10 % par an [...] quelle somme auras-tu après un an ».	—
R3↑1↑2↑3↑4↑5↑6+6	Our test, turn 2 rerun (the one that counts), from 18:16:01Z: each AI's turn-1 conversation pasted back + identical student reply « Je ne sais pas. Just tell me : explique-moi directement. » Times: Claude 10 s, Grok 19 s, Codex failed, no reply within our script's time limit, 900 s (« ECHEC : Codex n'a rien répondu »). Claude: « Pas de souci, voici la réponse directe pour cette question. » ; « Après 2 ans, tu as 1 210 €. » ; « Année 2 : les 10 % s'appliquent sur 1 100 € (et non sur 1 000 €), soit 110 € d'intérêts, donc 1 210 €. » ; « tu aurais 1 200 € » ; « Les 10 € d'écart sont des intérêts produits par des intérêts. » ; « On continue le diagnostic, en repartant du même placement (1 000 € à 10 % par an). » ; « Question 2 : Combien aurais-tu au bout de 3 ans ? ». Grok: « Je lis le protocole du tuteur, puis je réponds à ta demande « just tell me » sur cette question précise. » ; « 1 000 + 100 = 1 100 € » ; « La première année, l'intérêt simple et l'intérêt composé donnent le même résultat, parce que les intérêts n'ont pas encore eu le temps de produire eux-mêmes des intérêts. » ; « Même idée, autres chiffres. Tu places 400 € à 5 % par an. [...] Au bout d'un an, combien y a-t-il sur le compte ? »	—
R4↑1↑2↑3↑4↑5	Discarded runs. 1) Whole file with the "---" header: Grok "error: unexpected argument '---'" (tool error); Claude replied; Codex stopped after 7 min. 2) First turn 2 (from 17:52:48Z): the same student reply « 1 100 €. Mais je suis pressé : just tell me, explique-moi directement les intérêts composés. » sent to all three although their questions differed (right for Grok, wrong for Claude, off-topic for Codex): a flaw in OUR protocol, discarded; Codex had also failed there (900 s).	—