Academic Research Skills for Claude Code, Tested: 110 Peer Reviews of Flawed Manuscripts

Summary

For checking your own draft before submission, plain Claude Code already catches the textbook design flaws: it named 89 of 90 planted flaws, 30 flaws times three runs, the same score as Imbad0202’s academic-paper-reviewer. Install a skill for what surrounds the review. Imbad0202’s suite gives you an editorial letter with a decision and a full research and writing pipeline, at 2.3 times the cost per review. K-Dense’s peer-review puts confidentiality first, labels every review as a working draft and leaves the decision to the editor.

Three findings matter most:

• Flaws found. Plain Claude Code 89 of 90, academic-paper-reviewer 89 of 90, K-Dense peer-review 84 of 90. K-Dense’s six misses all come from two runs where it stopped at its safety gate and asked for the journal’s AI policy before opening the file.
• Cost. Mean cost per review was $0.0041 plain, $0.0094 with academic-paper-reviewer and $0.0060 with peer-review. The extra cost is the skill’s own instructions: median input rose from 59 thousand tokens to 125 thousand and 75 thousand.
• Loading. With a plain request to review the paper, K-Dense’s skill loaded in 10 of 10 sessions. Imbad0202’s reviewer loaded in 0 of 10, so call it by name with /academic-paper-reviewer.

How we tested

1. Manuscripts. We wrote 10 short fictional drafts, 96 to 157 words each, across public health, education, machine learning, psychology, clinical medicine, economics, ecology, nutrition, software engineering and sociology. Each carries three planted flaws from a list of 14: causal claims from a survey, a sample size that differs between abstract and methods, 20 outcomes tested with one hit at p = 0.04, percentages that sum past 100, test-set tuning in a machine learning paper, patients picked on one extreme blood pressure reading, 60 transects in one field counted as 60 replicates, unblinded bug counting, and so on.
2. Arms. Plain Claude Code in an empty folder. Imbad0202/academic-research-skills, with its five skill folders copied into .claude/skills/ as its setup guide describes and the prompt starting with /academic-paper-reviewer. K-Dense-AI/scientific-agent-skills, with its peer-review folder copied the same way and the prompt starting with /peer-review. Both repos as of November 14, 2026.
3. Prompt. The same request in every arm: review my own draft in paper.md before I submit it, I am the sole author and authorize the review, list the problems you find.
4. Runs. Each manuscript three times per arm, each run a fresh claude -p session on Claude Code 2.1.322 with --model haiku and --setting-sources project, so user settings and hooks stay out. That is 90 runs. A further 20 runs repeat each manuscript once per skill with the plain prompt, to see whether the skill loads by itself. Tokens, cost and time come from Claude Code’s JSON output.
5. Scoring. A flaw counts as found when the review names it, checked with a keyword rule per flaw type, for example regression to the mean, or multiple comparisons, Bonferroni or false discovery rate. Number mismatches need both numbers in the same passage. We read every miss by hand. Two rules were too narrow on the first pass, for overgeneralization and for an untuned baseline, and we widened them for all arms; the misses left after that were real.

Results

Arm	Flaws found	Reviews with all 3	Mean cost per review	Median input tokens	Median output tokens	Median time
Plain Claude Code	89 of 90	29 of 30	$0.0041	59,054	3,685	25 s
academic-paper-reviewer	89 of 90	29 of 30	$0.0094	125,167	4,174	31 s
K-Dense peer-review	84 of 90	28 of 30	$0.0060	74,839	4,480	32 s

The one flaw plain Claude Code and academic-paper-reviewer each missed is the same: the coffee paper’s abstract says 1,240 adults and its methods analyse 984 of 1,520 invited. Both reviews reported the 984 and 1,520 and skipped the 1,240.

What changed between arms is the shape of the answer:

• academic-paper-reviewer ends every review with a recommendation, usually reject in current form. Its headline feature is a panel of five reviewer personas plus an editor. In all 30 runs it worked alone: 29 runs reviewed in a single pass and said a draft this short needs the panel less, and one run proposed its reviewer cards and paused for confirmation, as the skill’s checkpoint rule requires. Zero runs started a subagent. Our drafts are short, so a full manuscript may well get the panel.
• K-Dense peer-review opens every one of its 30 reviews with a working draft label, organizes points by location, evidence and requested action, and gave a decision in 3 of 30, since its rules reserve the decision for the editor. In 2 of 30 runs it read the file size, stopped, and asked for the target journal and its AI policy before reading further. In the 10 sessions where it loaded by itself, it reviewed every time.
• Plain Claude Code wrote the shortest reviews and gave a recommendation in 27 of 30.

On these drafts, the skills add structure and process. Detection was already at the ceiling.

Our picks

Checking your own draft quickly: plain Claude Code. Paste or save the draft and ask for a peer review. It found the design flaws a referee would raise first at the lowest cost.

Writing a paper end to end: Imbad0202/academic-research-skills, 56,864 stars on November 14. Five skills cover deep research, drafting, review, a pipeline that pauses for your confirmation at each stage, and screening for systematic reviews. Install with /plugin marketplace add Imbad0202/academic-research-skills, then /plugin install academic-research-skills, and call the reviewer by name. Its README states a CC BY-NC 4.0 license, so commercial use needs the author’s permission.

Reviewing someone else’s manuscript: K-Dense peer-review, part of a 57,145 star MIT library of 177 science skills. Its intake gate asks for authorization, venue policy and conflicts first, which is what an editor will expect from a referee using AI. Have the journal name and its AI policy ready.

For more, our research skills page lists 188 research skills from dozens of repos. The largest by count are Yuan1z0825/nature-skills (16 skills, 53,519 stars, Nature style writing and figures), aipoch/open-science (15), K-Dense (14 in that category), wanshuiyin/Auto-claude-code-research-in-sleep (13, 19,185 stars, autonomous machine learning research loops) and cookjohn/cnki-skills (10, search and export from CNKI).

What to ignore

• Panel size as a quality signal. Five reviewer personas sound thorough. On our drafts the single pass already named 99% of the planted flaws, and the panel stayed idle.
• Star counts as a ranking. The two leading repos sit within 300 stars of each other and do different jobs: one is a writing pipeline, the other a science toolkit with one review skill.
• Our dollar figures as your bill. These are Haiku prices for drafts of about 120 words. A 9,000 word manuscript on a larger model costs many times more, and the skill’s fixed overhead becomes a smaller share.

FAQ

How do I install academic research skills in Claude Code?

As a plugin, run /plugin marketplace add Imbad0202/academic-research-skills and /plugin install academic-research-skills. As plain skills, copy each skill folder into .claude/skills/ in your project, or into ~/.claude/skills/ to use them everywhere. Claude Code reads the SKILL.md inside each folder.

Why did the reviewer skill stay unused when I asked for a review?

Claude Code picks a skill from its description. In our sessions K-Dense’s description won every time and Imbad0202’s reviewer lost every time to a direct answer. Typing the slash command, here /academic-paper-reviewer, loads it on demand.

Can these skills check whether my citations are real?

Imbad0202’s suite has an opt-in claim audit, turned on with ARS_CLAIM_AUDIT=1, that fetches cited sources and checks each claim against them; its README notes the live calibration result is still pending. K-Dense ships citation-management as a separate skill. Our test covered study design flaws and left citations for a later round.

Limitations

The drafts are short and the flaws are textbook ones, so every arm reached the ceiling; harder or subtler flaws could separate them. All runs used Haiku through claude -p, where each skill decided how much of its machinery to run. We wrote the manuscripts and the keyword rules ourselves, and checked every miss by hand.

Reproduce it

The scripts are the run script and the scoring script. Clone the two skill repos next to a test folder, save the papers array from academic-research-skills-claude-code-2026-11.json as papers.json in that folder, and run python3 run.py <arm> <paper id> <rep> with Claude Code logged in, then python3 score.py. The JSON holds the 10 manuscripts, every keyword rule, and one row per run with flaws found and missed, tokens, cost and time.

Sources

1. Imbad0202/academic-research-skills · official
2. Imbad0202/academic-research-skills: setup guide · official
3. K-Dense-AI/scientific-agent-skills · official
4. K-Dense-AI/scientific-agent-skills: peer-review skill · official
5. Claude Code docs: Agent Skills · official
6. AgentCraft: research skills for Claude Code · blog