Skip to content
LiveNext 14:39:56
topfive.ai
SettledResearch

Study: swapping coding-agent harness moves SWE-bench scores little, but cost per task varies up to 3x

Researchers Yangze Liu and Zhongyi Han ran five models through Claude Code, OpenCode and mini-SWE-agent on SWE-bench Verified. Harness swaps flipped about 13% of hard-task outcomes, the same as rerunning one harness. OpenCode trailed by up to 9 points, partly from an output cap, and cost per task differed up to 3x.

HEAT
OUTLETS
1
FIRST SEEN
LAST SIGNAL

Primary source

arXivarxiv.org/abs/2610.04433

Coverage · 2 articles, oldest first

Saturday, October 24, 2026

  1. The Decoder

    Fancy harnesses make coding agents more expensive but not better, study finds

  2. arXivPrimary

    Study: swapping coding-agent harness moves SWE-bench scores little, but cost per task varies up to 3x
Share
Share card: Study: swapping coding-agent harness moves SWE-bench scores little, but cost per task varies up to 3x

Share this article

Copying the image…

Open any post or message and paste it in. The link to TopFive is printed on the card.

Post it on

X LinkedIn Hacker News Email
Download image
XLinkedInHacker NewsEmail
← The whole wire