Professional analysts using Claude Code went with whichever option their AI agent leaned toward 86% of the time, even though each lean had been assigned by a hidden coin flip, according to an Anthropic study. The 50 analysts each spent roughly an hour judging an advisory service run by a made-up farm co-op, making four calls that each had two defensible answers, so chance alone would predict about 50%. Analysts who had already stated a preference abandoned it in 60% of cases when the agent disagreed, compared with 6% when it agreed. Afterward, 77% explained their choices using the agent’s arguments, but only 15% credited it, and those who deferred on every call were about as confident as the rest. Anthropic made the leans stronger on purpose, so everyday effects may be weaker.
