AI agent experience logging

A useful technique is to ask Claude (or whoever) a question shortly after finishing a task: reflect on how you implemented that task - what did you learn from your experience that could help you in the future? You can use this reflection to improve existing skills or maintain notes files that the AI can consult and thereby approach related tasks in a more informed way.

Although this sounds good, in practice it comes with a few pain points:

• Remembering/bothering to ask for the reflection.
• Deciding where to store the lessons learned from reflecting.
• Getting the AI to find the recorded wisdom for the next task.

With this in mind, I'm experimenting with a workflow that uses an agent skill to make reflection into a simple slash command /reflect and stores the wisdom as longer-than-usual structured commit messages. The messages use a body section to record the AI's thrills and spills along with commit trailers for simpler, deterministic bits of knowledge (in a similar manner to the Lore commit format).

The commits on a PR get squashed when the PR is merged, which by default concatenates the existing commit messages from the PR into one long message for the squash/merge commit. However, this point is an opportunity to reflect further on the content of the richer commit messages produced by /reflect. To handle this, there is also a /finalize skill that overrides the squash message concatenation by reflecting on and summarising the individual commit reflections.

The basic workflow then is:

• Do some stuff with AI that warrants a commit.
• Type /reflect before doing the commit and ask the AI to handle the commit rather than running the git command directly yourself. Note that for many commits, the AI will choose not to add any reflections, simply because nothing interesting has been experienced.
• Repeat these steps until your PR is finished and reviewed to satisfaction.
• Type /finalize before merging and ask the AI to merge from the chat rather than doing the merge on the Github PR page.

For the time being, this is deliberately an opt-in workflow that you're free to ignore. However, if everyone loves it then we could eventually add a Github action to catch the /finalize part if you forget to do it yourself (less effective but still better than a simple message concatenation).