Having reviewed our telemetry scripts (track-telemetry.ps1/track-telemetry.sh) and the hook.json files that call them, there are several issues that should be addressed:
- We're using the same
postToolUse hook in all clients. However, the shape of the JSON passed to the hook script varies from one client to another. Rather than one script that tries to accomodate all clients, each client should have its own script tailored to its output format. I believe this will be easier to maintain going forward.
- Claude has a
postToolUseFailure hook that is called when a tool fails. I'm not sure if this is called in addition to postToolUse or in place of it; we need to investigate.
- The clients have very different approaches to handling different operating systems. VS Code has support for OS-specific commands (
mac, linux, and windows) while Copilot CLI has shell-specific commands (bash and powershell). Claude always wants to use bash, with PowerShell as a fallback (though it is unclear how that would work). We should consider replacing our separate .sh and .ps1 scripts with a single, cross-platform NodeJS script and then invoking that with npx. Since we already need npx for the scripts to send the telemetry (via Azure MCP) this does not introduce any new requirements.
- In at least some cases a client will pick up the default hooks file specified by the default plugin.json. Given the differences between clients, I think it would be better to have a client-specific plugin.json for each client we support, with each of those point to a client-specific hooks.json, which in turn runs client-specific scripts.
Having reviewed our telemetry scripts (track-telemetry.ps1/track-telemetry.sh) and the hook.json files that call them, there are several issues that should be addressed:
postToolUsehook in all clients. However, the shape of the JSON passed to the hook script varies from one client to another. Rather than one script that tries to accomodate all clients, each client should have its own script tailored to its output format. I believe this will be easier to maintain going forward.postToolUseFailurehook that is called when a tool fails. I'm not sure if this is called in addition topostToolUseor in place of it; we need to investigate.mac,linux, andwindows) while Copilot CLI has shell-specific commands (bashandpowershell). Claude always wants to use bash, with PowerShell as a fallback (though it is unclear how that would work). We should consider replacing our separate .sh and .ps1 scripts with a single, cross-platform NodeJS script and then invoking that withnpx. Since we already neednpxfor the scripts to send the telemetry (via Azure MCP) this does not introduce any new requirements.