AI Coach
AI Coach scores how effectively your team prompts AI tools like Claude, ChatGPT, and Microsoft Copilot, then turns that into coaching — a coaching signal, never a performance evaluation.
Beta — access by request. AI Coach is currently in beta and enabled per organization. If you don't see AI Coach in your left-hand navigation, contact your Herd representative to request access.
Overview
AI Coach helps security and enablement teams understand — and improve — how their people actually use AI tools like Claude, ChatGPT, and Microsoft Copilot. Once you connect your organization's AI accounts, Herd scores how effectively each conversation was prompted and turns the results into targeted, friendly coaching.
The goal is upskilling, not surveillance. AI Coach is explicitly a coaching signal, not a performance evaluation — it describes prompting habits, not the person.
To open it, click AI Coach in the left-hand sidebar. You land on the Overview, and the section is organized into three tabs:
Overview — your organization's AI posture, leaderboards, and score trend (plus the Setup/Connected flow for admins).
Conversations — the scored-conversation list and per-conversation scorecards.
Findings — AI policy violations and coaching signals, with triage actions.
How scoring works
Every scored conversation gets an overall prompting-effectiveness score from 0–100, plus a breakdown across eight prompting-craft dimensions. Each dimension is rated on an anchored 0–4 level:
0
Absent
1
Weak
2
Adequate
3
Strong
4
Excellent
A dimension can also be N/A when it doesn't apply to a given conversation (for example, tool awareness in a plain browser chat).
The dimensions
Goal clarity
States the desired outcome unambiguously.
Context sufficiency
Provides the background, data, and inputs the task needs — enough, not excessive.
Constraints & requirements
Specifies explicit requirements, output format, and limits.
Success criteria
Defines what a good or finished result looks like.
Front-loading
Supplies complete requirements up front instead of drip-feeding them across turns.
Iteration & course-correction
Gives clear feedback and corrects course across turns.
Scope & decomposition
Right-sizes the ask and breaks large tasks down.
Tool & environment awareness
Leverages the tools, files, and capabilities available to the assistant.
Herd also surfaces a model-fit signal on each conversation — whether the model the person chose was a good fit for the task.
Privacy by design
AI Coach is built so you can act on how people prompt without reading what they wrote.
Coaching, not evaluation. Scores and tips describe prompting habits and are framed as a growth signal — not a rating of the person.
Never chat content. The summaries shown in the conversation list and on each scorecard are dimension-derived one-liners. Admins — even super-admins — triage on the score and coaching, never the back-and-forth of the chat itself.
Aggregated, scores-only leaderboards. Team standings are averages, and the individual leaderboard (off by default) shows names, ranks, and scores only.
People see their own. Users can review their own scored activity and get prompt rewrites through the Herd Slack app.
What you can do with AI Coach
Setting Up AI Coach — Connect your Claude, ChatGPT, or Microsoft Copilot account and map AI activity to your people.
Reviewing AI Use — Read the Overview's posture cards, browse scored conversations, read a conversation's scorecard, and coach an individual.
Findings — Review AI policy violations and coaching signals, and triage them without ever exposing chat content.
Leaderboards & Remediation — Turn on leaderboards and self-service, and auto-assign coaching when a pattern of weak prompting emerges.
Last updated