Prompt Analysis

Prompting AI Coding Agents is a new discipline that every developer must start mastering. Because Git AI stores every prompt and tracks the generated code through the full SDLC - it has all the data you need to analyze your prompting practices and figure out what works best for you.

Git AI ships a skill that you can use to analyze your past prompts: /git-ai:prompt-analysis.

The skill first loads all your prompts from the local prompt store, out git notes, or a cloud prompt store, then creates a one-off SQLite database for the Agent to use as a scratchpad to record analysis as it iterates through your prompts.

Example

Ask the Agent to analyze your prompts: "Go back in time. how was prompting practice / style changed over the last 3 months?"

Agent spins up sub-agents, grades each prompt, and records the results in its scratchpad SQLite:

The Agent runs some final queries, summarizes the results, and presents them:

Use with Claude Code

Each version of git-ai ships with some local skills that are versioned along side the binary.

/plugin add ~/.git-ai/skills

Depending on your settings, you may be asked to install the skill before it shows up on the Installed tab:

5 Questions to Try

/git-ai:prompt-analysis Which model has the best acceptance rate? What's the overall trend over 6 months?

/git-ai:prompt-analysis Grade my prompts according to best practices

/git-ai:prompt-analysis Categorize my prompts by work type (bug fix, feature, refactor, docs)

/git-ai:prompt-analysis Why do some of my prompts have low acceptance rates?

/git-ai:prompt-analysis Correlate my prompting techniques with acceptance rate

Compare to Your Peers

Include prompts from your whole team by mentioning "team", "everyone", or "all authors":

/prompt-analysis Compare and contrast my prompting practice to everyone on my team

/prompt-analysis Which team member ships the most AI-code with the highest acceptance rate? What are they doing differently from me?

Go Back Further in Time

By default, analysis covers the last 30 days. Mention a time range to go further.

Note: the furthest back you can go is the date you installed Git AI. Sorry, we don't have a time machine.

/prompt-analysis Grade all my prompts from the last 90 days

/prompt-analysis How has my prompting practice changed over the last 6 months?