A fun thing you can leverage AI for is learning about and tuning your own harness and…

A fun thing you can leverage AI for is learning about and tuning your own harness and habits with long-running experiments using data, or derivable data, that only you have.

Your results can vary widely from others based on your codebase, your models, and your context/tools tuning.

For example, I wanted to get a sense of whether I was defaulting to the right reasoning level when using GPT-5.5 on a particular codebase.

It’s easy to default to GPT-5.5 high/xhigh, but higher spending, wall-clock time, and model overthinking are often creeping results that compound over time.

I had Claude Code with Opus 4.8 orchestrate an experiment analyzing 100 past good commits (no bugs/follow-ups) to assess what reasoning level Codex with GPT-5.5 needed to reproduce those changes effectively.

Each experiment run was isolated, with no knowledge of how the changes were actually implemented in the real codebase, a prompt to express intent, and a starting point (the previous commit).

Across 400 runs (4 reasoning levels x 100 commits), a few patterns emerged: high/xhigh was rarely needed to solve correctly, and the highest thinking level often took ~2.5x the wall-clock time and 10x the tokens/cost.

I’ll definitely be adjusting my default when working on this particular codebase.

Of course, what works for you will depend on your codebase, your models, and your context/tools tuning, so YOU SHOULD RUN YOUR OWN experiments like this!