Teaching Claude my conventions through 10 PR comments

When I first started using Claude on my homelab repo, the code it produced was correct but felt off. The naming was slightly different from the rest of the file. The function signatures used Optional[X] when the rest of the file used X | None. The error messages were longer than the rest of the file. The tests used a different pattern than the existing tests. The code worked. It didn’t match.

The first fix I tried was a long CLAUDE.md that documented every convention. That worked for a session. The next session, the agent drifted again. The CLAUDE.md was 800 lines and the agent wasn’t reading it carefully.

The fix that actually worked was simpler. I let the agent write code, then I gave it specific feedback in PR comments. After about 10 comments, the agent started writing code that matched the codebase on the first try.

Here’s what those comments looked like.

Comment 1: the type hint

In this file we use X | None, not Optional[X]. The rest of the file is already in that style. Match the existing style.

Comment 2: the function signature

This function should be parse_line(line: str) -> LogEntry | None. The convention in the rest of the file is to return None for the not-found case, not to raise. Match the convention.

Comment 3: the error message

The error message is too long. The convention is “what went wrong, in 5-10 words, no apology.” See load_config() for an example. Match the length.

Comment 4: the test pattern

Use the given_when_then pattern. Look at test_parse_line_handles_empty_string in test_parser.py for the format. Match the structure.

Comment 5: the import

Don’t use from typing import .... We’re on Python 3.12. Use from collections.abc import ... instead. The rest of the file already does this.

Comment 6: the docstring

One-line docstring only for simple functions. Multi-line only for the public API. parse_line is internal. Match.

Comment 7: the variable name

Use result not ret_val. The rest of the file uses result. Match.

Comment 8: the constant

Move the magic number 0.7 to a module-level constant with a name. The convention in this file is to name the constants. Look at STABILITY_THRESHOLD_SECONDS in the same file. Match.

Comment 9: the helper

This helper is duplicated. There’s a format_timestamp() helper in utils.py. Use it. Don’t re-implement.

Comment 10: the test name

Test name should describe the behavior, not the function. test_parse_line_returns_none_for_blank_input not test_parse_line_blank. Match the existing test names.

After 10 comments like these, the agent started writing code that matched the conventions on the first try. The CLAUDE.md didn’t need to be 800 lines. The feedback loop did the work.

The pattern

The pattern is: the agent writes a draft. I review the diff. For each deviation from the codebase convention, I leave a short comment that says what the convention is and where the agent can see an example. The agent reads the comment. The next draft has fewer deviations.

The comments are not “this is wrong.” The comments are “here’s the convention and here’s where to see it.” The agent is smart enough to learn from examples. The agent is bad at inferring conventions from a 2000-line codebase without being told where to look.

The compounding effect

Each session the agent gets a little better at matching the codebase. The comments accumulate. The agent’s “model” of the codebase’s conventions improves. After 30 or 40 comments, the agent’s first drafts are usually 90% right. The remaining 10% are usually the subtle things (naming in a specific module, the order of arguments in a function call).

The compounding is the part that matters. The first session is slow. The tenth session is fast. The fiftieth session is barely any review at all. The agent has learned the codebase.

The thing this doesn’t replace

This doesn’t replace code review. The agent’s first drafts match the conventions but they don’t always match the intent. The conventions are surface. The intent is deeper. A PR comment about “use the helper” is a surface comment. A PR comment about “this helper isn’t right for this case because X” is an intent comment. The agent learns from both. The agent is better at learning from the surface comments.

For the intent comments, I still need a human reviewer. The agent can suggest the right code. The agent can’t tell whether the code matches what the system is supposed to do. That requires knowing the system. The agent knows the code. The agent doesn’t know the system.