I hand skills to a few people who run them on their own Macs. I see none of their sessions. I’d like to know what they keep, what they send back for a fix, and why, because that should shape the next version of each skill. Today nothing carries it back.
Before building a way back, I read how vendors already do it. On 8 and 9 October I went through three places people get agent skills: the 290 most-installed entries on skills.sh, Vercel’s directory; the 61 plugins in Anthropic’s official Claude Code marketplace that ship hooks, out of 315; and the 30 most-starred skill and plugin repos on GitHub, leaving out apps and lists. Shopify’s and Stripe’s plugins I’d read a few days earlier, and re-read. I read code and captured no traffic, so “sends” below means the code builds the request and posts it.
The data leaves through one of three doors, and each reaches different people.
npx hyperframes feedback, Neon’s neon feedback, Expo’s submit-expo-feedback. It travels with every install.npx skills add copies the skill files and installs no hooks.The second door makes the install route matter. Microsoft’s Azure skills report nothing when installed with npx skills add. Installed as a plugin, which Anthropic’s official marketplace lists, the same skills report every session, every Azure skill loaded, every reference file read and every Azure tool called. The hook sends each event by running npx -y @azure/mcp@latest, so it also fetches and runs whatever version npm has that day. The repo’s README never mentions the hooks.
Most vendors that send anything send counts: which skill loaded, which command ran, the version, the OS, sometimes a random device ID. Homebrew, VS Code and Next.js have done the same for years, on by default, with a variable to switch it off. Several now also record whether an AI agent is driving.
Your words are rarer. Of the 21 marketplace plugins whose hooks see your prompt, Shopify’s is the only one that sends it to its vendor: verbatim, up to 2,000 characters. One more, JFrog’s, hands it to a downloaded binary I couldn’t read. Observability plugins like PostHog and Dash0 ship whole sessions, but into your own account, once you set them up. Several vendors promise in writing to stay out of your content. Vercel’s plugin: “Prompt text, bash commands, tool-call contents, skill arguments, file paths, project names, account IDs … are not collected.” claude-mem, whose whole job is recording your sessions locally: “NEVER collected — not now, not ever: prompts or conversation content”.
The new kind is the third: reports the agent writes because the skill tells it to.
Ranked by whose judgment it carries:
| What comes back | Who collects it | How you’re asked |
|---|---|---|
| The model’s grade of the tool | Shopify (“You are grading the Shopify AI Toolkit”, every turn); HeyGen (0–10 after each render) | You aren’t |
| The agent’s bug report | Neon; Prisma; Stripe, on about 1% of turns and on failures | Neon: “No need to tell the user whether feedback was sent.” Stripe: shown to you first |
| Your request, in your words | Shopify, verbatim; HeyGen, “what the person asked for, in their words”, when it can’t do it | You aren’t |
| A task the agent failed, or you took over | Expo | Shown to you, sent only on your yes, never when you’re away |
| Your choice or outcome | Impeccable, which design direction you picked; career-ops, a hire story | Impeccable: you aren’t. career-ops: you file it yourself |
The bottom two rows are what I want: what the person did with the result, not a model’s grade of it.
Expo, for the default and the signal. Its usage telemetry is off until you turn it on, and the skill tells the agent: “Never enable it without an explicit user request.” Its feedback skill looks for “a task an AI agent could not complete cleanly despite real effort: several failed attempts, a build or screen that never worked, or the user stepping in to fix it manually.” The agent drafts a structured report and must “show the user the exact submission you intend to send and get approval”. With nobody there to approve it, nothing goes. “The user stepping in” is the closest thing I found to my “sent back for a fix”.
Stripe, for the ask. Before the agent sends feedback, the hook tells it: “Before submitting, show the prospective feedback to the user and ask for approval.” It also tells the agent to mention “that they may be invited to a private beta”, so the ask doubles as lead capture. I’d skip that part.
Impeccable, for the label. It deals you a few design directions and, once you pick, reports which one. That’s the person’s choice, the label I want. It just doesn’t ask first.
career-ops, for the preview. It has “no backend and no telemetry”. The one thing that leaves is a story you file from your own GitHub account, after seeing every field.
Google, for where the detail stays. Its agents-cli writes each command’s log entry into your own Cloud project and sends Google only the request headers.
Silence. Neon’s skill tells the agent to check “silently” whether it can send a report, then adds: “No need to tell the user whether feedback was sent.” The reports are about Neon’s own docs and tools, and there’s no switch.
Your prompt by default. Shopify’s toolkit sends your latest message as typed, plus a scorecard the agent fills in about the toolkit. It’s disclosed in the README and it can be switched off, but it’s on.
Edits to the agent’s own calls. Carta’s hooks add your model, effort level, session and prompt IDs and token count to every call the agent makes to Carta’s tools. Spotify’s hooks add headers to curl commands the agent wrote itself. Neither has a switch, and Spotify’s README doesn’t mention the headers. The data is mild. A hook that edits the agent’s commands for attribution could edit them for anything.
The model’s grade as the signal. It’s cheap and plentiful, and it rates the vendor’s product through a model the vendor just instructed.
Even the designs I’d copy enforce consent with a sentence. Stripe’s “ask for approval” is an instruction to the model. The Stripe CLI has a confirmation step of its own, and its code notes that “—json signals a scripted invocation, so it always skips the prompt”. The plugin’s example command for the agent uses --json. Expo’s approval is an instruction too, and so are HeyGen’s redaction (“names, clients, figures and paths left out”) and Neon’s “Never send secrets or sensitive data.” Each holds as long as the model complies. I haven’t tested how often it doesn’t.
So if I build a way back, I’d move the consent out of the prompt. One of my skills already ends on a review page where the person marks each result ship, rework or kill, with a note, and the page writes that down, not the model. The agent could draft a line about why, from the session. The person would see the exact text and send it, or not. It would stay off until they say yes, as Expo’s does.
That’s undecided and unbuilt. It also needs a door, and a skill installed with npx skills add gets only the first and the third.
export DO_NOT_TRACK=1
That switches off the skills CLI, HyperFrames, Shopify’s toolkit, Stripe’s CLI, Prisma, Supabase, Google’s agents-cli, Convex, Expo, Impeccable and Salesforce’s plugin. Three need their own:
export VERCEL_PLUGIN_TELEMETRY=off
export AZURE_MCP_COLLECT_TELEMETRY=false
export SHOPIFY_CLI_NO_ANALYTICS=1
Neon’s CLI, as far as I found, takes only --no-analytics on each command. Its skill’s bug reports, Carta’s additions and Spotify’s headers have no switch at all.