From 52fb5d754cdad49a541adb810e5bd7cb3aa8f302 Mon Sep 17 00:00:00 2001 From: markusha77 Date: Mon, 30 Mar 2026 00:04:32 +0800 Subject: [PATCH] fix: Testing & Security skills cleanup --- .../agent-authentication/SKILL.md | 20 - .../api-security-best-practices/SKILL.md | 39 +- .../backend-security-coder/SKILL.md | 191 ++------ .../better-auth-best-practices/SKILL.md | 20 - .../broken-authentication-testing/SKILL.md | 40 +- .../checking-changes/SKILL.md | 20 - skills/Testing & Security/claims/SKILL.md | 20 - skills/Testing & Security/clerk-auth/SKILL.md | 20 - .../component-refactoring/SKILL.md | 20 - .../confidence-check/SKILL.md | 20 - .../container-security-auditor/SKILL.md | 67 +-- .../cpp-coding-standards/SKILL.md | 20 - .../Testing & Security/cpp-testing/SKILL.md | 36 +- .../developing-with-fortify/SKILL.md | 20 - .../Testing & Security/feature-flags/SKILL.md | 20 - skills/Testing & Security/flow/SKILL.md | 20 - .../frontend-code-review/SKILL.md | 37 +- .../frontend-testing/SKILL.md | 42 +- .../gdpr-data-handling/SKILL.md | 20 - .../golang-testing/SKILL.md | 20 - .../isms-audit-expert/SKILL.md | 20 - .../javascript-testing-patterns/SKILL.md | 41 +- skills/Testing & Security/libafl/SKILL.md | 36 +- .../mtls-configuration/SKILL.md | 20 - .../performing-security-audits/SKILL.md | 173 ++----- .../Testing & Security/pest-testing/SKILL.md | 20 - .../property-based-testing/SKILL.md | 450 ++---------------- .../pyrefly-type-coverage/SKILL.md | 20 - .../python-anti-patterns/SKILL.md | 20 - .../python-testing/SKILL.md | 36 +- .../realphonevalidation-automation/SKILL.md | 20 - .../receiving-code-review/SKILL.md | 20 - .../requesting-code-review/SKILL.md | 20 - .../security-audit/SKILL.md | 20 - .../security-best-practices/SKILL.md | 20 - .../Testing & Security/senior-secops/SKILL.md | 20 - .../senior-security/SKILL.md | 20 - .../setup-web-tests/SKILL.md | 20 - .../swift-protocol-di-testing/SKILL.md | 20 - .../syncable-entity-testing/SKILL.md | 20 - .../temporal-python-testing/SKILL.md | 20 - .../test-driven-development/SKILL.md | 20 - .../Testing & Security/test-writer/SKILL.md | 20 - .../tidb-test-guidelines/SKILL.md | 20 - skills/Testing & Security/ui-review/SKILL.md | 20 - .../validating-api-contracts/SKILL.md | 84 ++-- .../verification-loop/SKILL.md | 20 - skills/Testing & Security/verify/SKILL.md | 20 - skills/Testing & Security/vitest/SKILL.md | 20 - .../vulnerability-scanner/SKILL.md | 39 +- .../web-renderer-test/SKILL.md | 20 - .../write-unit-tests/SKILL.md | 47 +- .../writing-bundler-tests/SKILL.md | 42 +- .../writing-dev-server-tests/SKILL.md | 42 +- 54 files changed, 585 insertions(+), 1597 deletions(-) delete mode 100644 skills/Testing & Security/agent-authentication/SKILL.md delete mode 100644 skills/Testing & Security/better-auth-best-practices/SKILL.md delete mode 100644 skills/Testing & Security/checking-changes/SKILL.md delete mode 100644 skills/Testing & Security/claims/SKILL.md delete mode 100644 skills/Testing & Security/clerk-auth/SKILL.md delete mode 100644 skills/Testing & Security/component-refactoring/SKILL.md delete mode 100644 skills/Testing & Security/confidence-check/SKILL.md delete mode 100644 skills/Testing & Security/cpp-coding-standards/SKILL.md delete mode 100644 skills/Testing & Security/developing-with-fortify/SKILL.md delete mode 100644 skills/Testing & Security/feature-flags/SKILL.md delete mode 100644 skills/Testing & Security/flow/SKILL.md delete mode 100644 skills/Testing & Security/gdpr-data-handling/SKILL.md delete mode 100644 skills/Testing & Security/golang-testing/SKILL.md delete mode 100644 skills/Testing & Security/isms-audit-expert/SKILL.md delete mode 100644 skills/Testing & Security/mtls-configuration/SKILL.md delete mode 100644 skills/Testing & Security/pest-testing/SKILL.md delete mode 100644 skills/Testing & Security/pyrefly-type-coverage/SKILL.md delete mode 100644 skills/Testing & Security/python-anti-patterns/SKILL.md delete mode 100644 skills/Testing & Security/realphonevalidation-automation/SKILL.md delete mode 100644 skills/Testing & Security/receiving-code-review/SKILL.md delete mode 100644 skills/Testing & Security/requesting-code-review/SKILL.md delete mode 100644 skills/Testing & Security/security-audit/SKILL.md delete mode 100644 skills/Testing & Security/security-best-practices/SKILL.md delete mode 100644 skills/Testing & Security/senior-secops/SKILL.md delete mode 100644 skills/Testing & Security/senior-security/SKILL.md delete mode 100644 skills/Testing & Security/setup-web-tests/SKILL.md delete mode 100644 skills/Testing & Security/swift-protocol-di-testing/SKILL.md delete mode 100644 skills/Testing & Security/syncable-entity-testing/SKILL.md delete mode 100644 skills/Testing & Security/temporal-python-testing/SKILL.md delete mode 100644 skills/Testing & Security/test-driven-development/SKILL.md delete mode 100644 skills/Testing & Security/test-writer/SKILL.md delete mode 100644 skills/Testing & Security/tidb-test-guidelines/SKILL.md delete mode 100644 skills/Testing & Security/ui-review/SKILL.md delete mode 100644 skills/Testing & Security/verification-loop/SKILL.md delete mode 100644 skills/Testing & Security/verify/SKILL.md delete mode 100644 skills/Testing & Security/vitest/SKILL.md delete mode 100644 skills/Testing & Security/web-renderer-test/SKILL.md diff --git a/skills/Testing & Security/agent-authentication/SKILL.md b/skills/Testing & Security/agent-authentication/SKILL.md deleted file mode 100644 index 83a8e661..00000000 --- a/skills/Testing & Security/agent-authentication/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: agent-authentication -name: Agent Authentication -description: Step-by-step guidance for agent authentication. -category: Testing & Security ---- - -# Agent Authentication - -Support agent authentication workflows with clear steps and best practices. - -## When to Use - -- You need help with agent authentication. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/api-security-best-practices/SKILL.md b/skills/Testing & Security/api-security-best-practices/SKILL.md index a334418e..b22c9a8b 100644 --- a/skills/Testing & Security/api-security-best-practices/SKILL.md +++ b/skills/Testing & Security/api-security-best-practices/SKILL.md @@ -1,20 +1,43 @@ --- id: api-security-best-practices name: API Security Best Practices -description: Step-by-step guidance for API security best practices. +description: Review API designs for authentication, authorization, validation, rate limiting, error handling, and data exposure risks. category: Testing & Security +requires: [] +examples: + - "Review this API design for security gaps." + - "What API security best practices matter for this endpoint?" + - "Help me harden this public API surface." --- # API Security Best Practices -Support api security best practices workflows with clear steps and best practices. +Use this skill to review API designs and implementations for common security weaknesses before they become incidents. -## When to Use +## Start By Clarifying +- Who can call the API and from where. +- Which authentication and authorization model is in play. +- What data is sensitive or business-critical. +- Which endpoints are public, internal, privileged, or high-volume. +- What abuse scenarios are realistic for the system. -- You need help with api security best practices. -- You want a clear, actionable next step. +## Security Review Areas +- Authentication strength and token handling. +- Authorization checks tied to business rules. +- Input validation, payload size limits, and schema enforcement. +- Rate limiting, abuse resistance, and resource protection. +- Error handling that stays useful without leaking internals. +- Response shaping that avoids accidental data exposure. -## Output +## Good Output +- API-specific risks by severity. +- Missing controls or assumptions that need to be explicit. +- Safer default patterns for auth, validation, and responses. +- Verification ideas such as abuse-case tests or authorization checks. -- Brief plan or checklist -- Key recommendations and caveats +## Common Mistakes +- Validating syntax but not ownership or permissions. +- Treating internal APIs as automatically trusted. +- Returning more data than the caller actually needs. +- Weak or inconsistent error semantics around auth and validation. +- Missing rate limits on endpoints that are easy to abuse. diff --git a/skills/Testing & Security/backend-security-coder/SKILL.md b/skills/Testing & Security/backend-security-coder/SKILL.md index 45b4cee7..94f0fba6 100644 --- a/skills/Testing & Security/backend-security-coder/SKILL.md +++ b/skills/Testing & Security/backend-security-coder/SKILL.md @@ -1,158 +1,49 @@ --- id: backend-security-coder name: Backend Security Coder -description: Expert in secure backend coding practices specializing in input validation, authentication, and API security. Use PROACTIVELY for backend security implementations or security code reviews. +description: Design and review secure backend code with practical guidance on input validation, authentication, authorization, secret handling, and API safety. category: Testing & Security requires: [] examples: - - "Implement secure authentication for my API" - - "Review this backend code for security vulnerabilities" + - "Help me implement secure authentication for this API." + - "Review this backend code for security vulnerabilities." + - "What secure coding patterns should this backend endpoint use?" --- -## Use this skill when - -- Working on backend security coder tasks or workflows -- Needing guidance, best practices, or checklists for backend security coder - -## Do not use this skill when - -- The task is unrelated to backend security coder -- You need a different domain or tool outside this scope - -## Instructions - -- Clarify goals, constraints, and required inputs. -- Apply relevant best practices and validate outcomes. -- Provide actionable steps and verification. -- If detailed examples are required, open `resources/implementation-playbook.md`. - -You are a backend security coding expert specializing in secure development practices, vulnerability prevention, and secure architecture implementation. - -## Purpose -Expert backend security developer with comprehensive knowledge of secure coding practices, vulnerability prevention, and defensive programming techniques. Masters input validation, authentication systems, API security, database protection, and secure error handling. Specializes in building security-first backend applications that resist common attack vectors. - -## When to Use vs Security Auditor -- **Use this agent for**: Hands-on backend security coding, API security implementation, database security configuration, authentication system coding, vulnerability fixes -- **Use security-auditor for**: High-level security audits, compliance assessments, DevSecOps pipeline design, threat modeling, security architecture reviews, penetration testing planning -- **Key difference**: This agent focuses on writing secure backend code, while security-auditor focuses on auditing and assessing security posture - -## Capabilities - -### General Secure Coding Practices -- **Input validation and sanitization**: Comprehensive input validation frameworks, allowlist approaches, data type enforcement -- **Injection attack prevention**: SQL injection, NoSQL injection, LDAP injection, command injection prevention techniques -- **Error handling security**: Secure error messages, logging without information leakage, graceful degradation -- **Sensitive data protection**: Data classification, secure storage patterns, encryption at rest and in transit -- **Secret management**: Secure credential storage, environment variable best practices, secret rotation strategies -- **Output encoding**: Context-aware encoding, preventing injection in templates and APIs - -### HTTP Security Headers and Cookies -- **Content Security Policy (CSP)**: CSP implementation, nonce and hash strategies, report-only mode -- **Security headers**: HSTS, X-Frame-Options, X-Content-Type-Options, Referrer-Policy implementation -- **Cookie security**: HttpOnly, Secure, SameSite attributes, cookie scoping and domain restrictions -- **CORS configuration**: Strict CORS policies, preflight request handling, credential-aware CORS -- **Session management**: Secure session handling, session fixation prevention, timeout management - -### CSRF Protection -- **Anti-CSRF tokens**: Token generation, validation, and refresh strategies for cookie-based authentication -- **Header validation**: Origin and Referer header validation for non-GET requests -- **Double-submit cookies**: CSRF token implementation in cookies and headers -- **SameSite cookie enforcement**: Leveraging SameSite attributes for CSRF protection -- **State-changing operation protection**: Authentication requirements for sensitive actions - -### Output Rendering Security -- **Context-aware encoding**: HTML, JavaScript, CSS, URL encoding based on output context -- **Template security**: Secure templating practices, auto-escaping configuration -- **JSON response security**: Preventing JSON hijacking, secure API response formatting -- **XML security**: XML external entity (XXE) prevention, secure XML parsing -- **File serving security**: Secure file download, content-type validation, path traversal prevention - -### Database Security -- **Parameterized queries**: Prepared statements, ORM security configuration, query parameterization -- **Database authentication**: Connection security, credential management, connection pooling security -- **Data encryption**: Field-level encryption, transparent data encryption, key management -- **Access control**: Database user privilege separation, role-based access control -- **Audit logging**: Database activity monitoring, change tracking, compliance logging -- **Backup security**: Secure backup procedures, encryption of backups, access control for backup files - -### API Security -- **Authentication mechanisms**: JWT security, OAuth 2.0/2.1 implementation, API key management -- **Authorization patterns**: RBAC, ABAC, scope-based access control, fine-grained permissions -- **Input validation**: API request validation, payload size limits, content-type validation -- **Rate limiting**: Request throttling, burst protection, user-based and IP-based limiting -- **API versioning security**: Secure version management, backward compatibility security -- **Error handling**: Consistent error responses, security-aware error messages, logging strategies - -### External Requests Security -- **Allowlist management**: Destination allowlisting, URL validation, domain restriction -- **Request validation**: URL sanitization, protocol restrictions, parameter validation -- **SSRF prevention**: Server-side request forgery protection, internal network isolation -- **Timeout and limits**: Request timeout configuration, response size limits, resource protection -- **Certificate validation**: SSL/TLS certificate pinning, certificate authority validation -- **Proxy security**: Secure proxy configuration, header forwarding restrictions - -### Authentication and Authorization -- **Multi-factor authentication**: TOTP, hardware tokens, biometric integration, backup codes -- **Password security**: Hashing algorithms (bcrypt, Argon2), salt generation, password policies -- **Session security**: Secure session tokens, session invalidation, concurrent session management -- **JWT implementation**: Secure JWT handling, signature verification, token expiration -- **OAuth security**: Secure OAuth flows, PKCE implementation, scope validation - -### Logging and Monitoring -- **Security logging**: Authentication events, authorization failures, suspicious activity tracking -- **Log sanitization**: Preventing log injection, sensitive data exclusion from logs -- **Audit trails**: Comprehensive activity logging, tamper-evident logging, log integrity -- **Monitoring integration**: SIEM integration, alerting on security events, anomaly detection -- **Compliance logging**: Regulatory requirement compliance, retention policies, log encryption - -### Cloud and Infrastructure Security -- **Environment configuration**: Secure environment variable management, configuration encryption -- **Container security**: Secure Docker practices, image scanning, runtime security -- **Secrets management**: Integration with HashiCorp Vault, AWS Secrets Manager, Azure Key Vault -- **Network security**: VPC configuration, security groups, network segmentation -- **Identity and access management**: IAM roles, service account security, principle of least privilege - -## Behavioral Traits -- Validates and sanitizes all user inputs using allowlist approaches -- Implements defense-in-depth with multiple security layers -- Uses parameterized queries and prepared statements exclusively -- Never exposes sensitive information in error messages or logs -- Applies principle of least privilege to all access controls -- Implements comprehensive audit logging for security events -- Uses secure defaults and fails securely in error conditions -- Regularly updates dependencies and monitors for vulnerabilities -- Considers security implications in every design decision -- Maintains separation of concerns between security layers - -## Knowledge Base -- OWASP Top 10 and secure coding guidelines -- Common vulnerability patterns and prevention techniques -- Authentication and authorization best practices -- Database security and query parameterization -- HTTP security headers and cookie security -- Input validation and output encoding techniques -- Secure error handling and logging practices -- API security and rate limiting strategies -- CSRF and SSRF prevention mechanisms -- Secret management and encryption practices - -## Response Approach -1. **Assess security requirements** including threat model and compliance needs -2. **Implement input validation** with comprehensive sanitization and allowlist approaches -3. **Configure secure authentication** with multi-factor authentication and session management -4. **Apply database security** with parameterized queries and access controls -5. **Set security headers** and implement CSRF protection for web applications -6. **Implement secure API design** with proper authentication and rate limiting -7. **Configure secure external requests** with allowlists and validation -8. **Set up security logging** and monitoring for threat detection -9. **Review and test security controls** with both automated and manual testing - -## Example Interactions -- "Implement secure user authentication with JWT and refresh token rotation" -- "Review this API endpoint for injection vulnerabilities and implement proper validation" -- "Configure CSRF protection for cookie-based authentication system" -- "Implement secure database queries with parameterization and access controls" -- "Set up comprehensive security headers and CSP for web application" -- "Create secure error handling that doesn't leak sensitive information" -- "Implement rate limiting and DDoS protection for public API endpoints" -- "Design secure external service integration with allowlist validation" +# Backend Security Coder + +Use this skill when building or reviewing backend logic that touches authentication, authorization, input validation, secrets, or high-risk data flows. + +## Start By Clarifying +- What is the endpoint, service, or workflow supposed to do? +- Which inputs are untrusted? +- Which roles, identities, or permissions matter? +- What sensitive data or side effects are involved? +- What failure modes would be dangerous? + +## Secure Backend Priorities +- Validate and constrain untrusted input. +- Enforce authentication and authorization separately and explicitly. +- Use safe query and storage patterns for sensitive data. +- Fail securely without leaking secrets or internals. +- Keep dangerous operations observable and reviewable. + +## Implementation Lenses +- Input validation and allowlisting. +- Injection prevention for SQL, commands, templates, or downstream services. +- Authentication flow safety, session or token handling, and replay concerns. +- Authorization checks tied to business rules, not only route placement. +- Secret management, logging hygiene, rate limits, and abuse resistance. + +## Good Output +- Security risks relevant to the specific backend path. +- Secure implementation or refactoring guidance. +- Verification ideas such as abuse-case tests or authorization checks. +- Gaps where more context is needed before recommending a pattern. + +## Common Mistakes +- Treating authentication as authorization. +- Validating shape but not meaning or ownership. +- Logging secrets, tokens, or sensitive payloads. +- Trusting upstream systems too broadly. +- Returning detailed error information that helps attackers. diff --git a/skills/Testing & Security/better-auth-best-practices/SKILL.md b/skills/Testing & Security/better-auth-best-practices/SKILL.md deleted file mode 100644 index 3984bde2..00000000 --- a/skills/Testing & Security/better-auth-best-practices/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: better-auth-best-practices -name: Better Auth Best Practices -description: Step-by-step guidance for better auth best practices. -category: Testing & Security ---- - -# Better Auth Best Practices - -Support better auth best practices workflows with clear steps and best practices. - -## When to Use - -- You need help with better auth best practices. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/broken-authentication-testing/SKILL.md b/skills/Testing & Security/broken-authentication-testing/SKILL.md index b5ef47a2..411e0763 100644 --- a/skills/Testing & Security/broken-authentication-testing/SKILL.md +++ b/skills/Testing & Security/broken-authentication-testing/SKILL.md @@ -1,20 +1,44 @@ --- id: broken-authentication-testing name: Broken Authentication Testing -description: Step-by-step guidance for broken authentication testing. +description: Test authentication and session flows for common weaknesses such as bypasses, fixation, weak recovery, and token misuse. category: Testing & Security +requires: [] +examples: + - "How should I test this auth flow for weaknesses?" + - "What broken-authentication cases should I check?" + - "Review this login and session design for security test coverage." --- # Broken Authentication Testing -Support broken authentication testing workflows with clear steps and best practices. +Use this skill to design security tests for login, session, token, and account-recovery flows. -## When to Use +## What To Clarify +- Which auth flows exist: login, logout, refresh, reset, MFA, invitation, device trust, or session revocation. +- Which credentials or tokens are used and where they are stored. +- What permissions or account transitions happen after authentication. +- Which threat model matters most: account takeover, token theft, session abuse, or privilege escalation. -- You need help with broken authentication testing. -- You want a clear, actionable next step. +## High-Value Test Areas +- Authentication bypass or inconsistent enforcement. +- Session fixation, stale-session reuse, or weak logout invalidation. +- Password reset and recovery flow abuse. +- MFA downgrade, bypass, or weak recovery paths. +- Token rotation, expiration, replay, and storage assumptions. -## Output +## Good Output +- Abuse-case scenarios to test. +- Priority vulnerabilities by impact. +- Specific flow gaps or state-transition weaknesses. +- Safer design or verification recommendations. -- Brief plan or checklist -- Key recommendations and caveats +## Common Mistakes +- Testing only happy-path login success. +- Assuming logout truly invalidates all relevant sessions. +- Treating password reset as less sensitive than login. +- Forgetting old tokens, shared devices, or privilege changes after authentication. + +## Boundaries +- Do not pretend to run penetration tests automatically. +- Focus on test design, abuse scenarios, and review logic. diff --git a/skills/Testing & Security/checking-changes/SKILL.md b/skills/Testing & Security/checking-changes/SKILL.md deleted file mode 100644 index 171ce5c2..00000000 --- a/skills/Testing & Security/checking-changes/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: checking-changes -name: Checking Changes -description: Step-by-step guidance for checking changes. -category: Testing & Security ---- - -# Checking Changes - -Support checking changes workflows with clear steps and best practices. - -## When to Use - -- You need help with checking changes tasks. -- You want a clear quality checklist. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/claims/SKILL.md b/skills/Testing & Security/claims/SKILL.md deleted file mode 100644 index 88920de1..00000000 --- a/skills/Testing & Security/claims/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: claims -name: Claims -description: Step-by-step guidance for claims. -category: Testing & Security ---- - -# Claims - -Support claims workflows with clear steps and best practices. - -## When to Use - -- You need help with claims. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/clerk-auth/SKILL.md b/skills/Testing & Security/clerk-auth/SKILL.md deleted file mode 100644 index 2acadd32..00000000 --- a/skills/Testing & Security/clerk-auth/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: clerk-auth -name: Clerk Auth -description: Step-by-step guidance for clerk auth. -category: Testing & Security ---- - -# Clerk Auth - -Support clerk auth workflows with clear steps and best practices. - -## When to Use - -- You need help with clerk auth. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/component-refactoring/SKILL.md b/skills/Testing & Security/component-refactoring/SKILL.md deleted file mode 100644 index 1940ccd2..00000000 --- a/skills/Testing & Security/component-refactoring/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: component-refactoring -name: Component Refactoring -description: Step-by-step guidance for component refactoring. -category: Testing & Security ---- - -# Component Refactoring - -Support component refactoring workflows with clear steps and best practices. - -## When to Use - -- You need help with component refactoring tasks. -- You want a clear quality checklist. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/confidence-check/SKILL.md b/skills/Testing & Security/confidence-check/SKILL.md deleted file mode 100644 index 9afb8b9c..00000000 --- a/skills/Testing & Security/confidence-check/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: confidence-check -name: Confidence Check -description: Step-by-step guidance for confidence check. -category: Testing & Security ---- - -# Confidence Check - -Support confidence check workflows with clear steps and best practices. - -## When to Use - -- You need help with confidence check. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/container-security-auditor/SKILL.md b/skills/Testing & Security/container-security-auditor/SKILL.md index e58e284a..dcf020c4 100644 --- a/skills/Testing & Security/container-security-auditor/SKILL.md +++ b/skills/Testing & Security/container-security-auditor/SKILL.md @@ -1,43 +1,44 @@ --- id: container-security-auditor name: Container Security Auditor -description: Container security auditor for Security Advanced. Triggers on container security auditor patterns and best practices. +description: Audit container build and runtime setups for image hygiene, privilege boundaries, secrets exposure, and supply-chain risk. category: Testing & Security requires: [] examples: - - "Help me with container security auditing" - - "Set up container security auditor" -version: 1.0.0 -author: Jeremy Longshore + - "Help me audit this container setup for security issues." + - "What should I check in this Dockerfile and runtime config?" + - "Review this container deployment for privilege and secret risks." +author: "Jeremy Longshore " +version: "1.0.0" --- # Container Security Auditor -## Purpose - -This skill provides automated assistance for container security auditor tasks within the Security Advanced domain. - -## When to Use - -This skill activates automatically when you: -- Mention "container security auditor" in your request -- Ask about container security auditor patterns or best practices -- Need help with advanced security skills covering penetration testing, compliance frameworks, threat modeling, and enterprise security. - -## Capabilities - -- Provides step-by-step guidance for container security auditor -- Follows industry best practices and patterns -- Generates production-ready code and configurations -- Validates outputs against common standards - -## Example Triggers - -- "Help me with container security auditor" -- "Set up container security auditor" -- "How do I implement container security auditor?" - -## Related Skills - -Part of the **Security Advanced** skill category. -Tags: pentesting, compliance, soc2, gdpr, threat-modeling +Use this skill to review container images, Dockerfiles, and runtime configurations for practical security risk. + +## Audit Areas +- Base image trust and update posture. +- Build-stage versus runtime-stage separation. +- Privilege level, user identity, and Linux capabilities. +- Filesystem writes, mounted volumes, and secret exposure. +- Network surface, package footprint, and supply-chain risk. + +## Review Method +- Check what the image contains and why it needs to be there. +- Look for ways the container can do more than the workload requires. +- Review how secrets, credentials, and tokens enter the container. +- Separate image-hardening issues from orchestration/runtime issues. +- Highlight which findings are build-time, runtime, or operational. + +## Good Output +- Highest-risk findings first. +- Concrete hardening priorities. +- Questions about missing runtime context when needed. +- Verification ideas for image contents, privilege model, and secret handling. + +## Common Findings +- Running as root without justification. +- Oversized images with unnecessary packages or tools. +- Secrets baked into images or passed insecurely. +- Writable filesystems or broad mounts without need. +- Missing distinction between development convenience and production safety. diff --git a/skills/Testing & Security/cpp-coding-standards/SKILL.md b/skills/Testing & Security/cpp-coding-standards/SKILL.md deleted file mode 100644 index 7f8dc70b..00000000 --- a/skills/Testing & Security/cpp-coding-standards/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: cpp-coding-standards -name: C++ Coding Standards -description: Step-by-step guidance for c++ coding standards. -category: Testing & Security ---- - -# C++ Coding Standards - -Support c++ coding standards workflows with clear steps and best practices. - -## When to Use - -- You need help with c++ coding standards tasks. -- You want a clear quality checklist. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/cpp-testing/SKILL.md b/skills/Testing & Security/cpp-testing/SKILL.md index 82eb1400..5480e129 100644 --- a/skills/Testing & Security/cpp-testing/SKILL.md +++ b/skills/Testing & Security/cpp-testing/SKILL.md @@ -1,20 +1,40 @@ --- id: cpp-testing name: C++ Testing -description: Step-by-step guidance for c++ testing. +description: Design C++ tests with attention to ownership, lifetimes, undefined behavior risks, seams for isolation, and deterministic behavior. category: Testing & Security +requires: [] +examples: + - "Help me write tests for this C++ class." + - "What C++ edge cases should these tests cover?" + - "Review this C++ test plan for lifetime and ownership risks." --- # C++ Testing -Support c++ testing workflows with clear steps and best practices. +Use this skill to design tests for C++ code where memory ownership, lifetime, undefined behavior, and boundary conditions matter. -## When to Use +## Clarify First +- What level is under test: function, class, module, or integration seam. +- Which ownership and lifetime assumptions the code relies on. +- Whether concurrency, exceptions, or resource cleanup are involved. +- Which inputs or states are most likely to trigger subtle bugs. -- You need help with c++ testing. -- You want a clear, actionable next step. +## C++ Testing Priorities +- Cover boundary conditions and invalid states deliberately. +- Make ownership and resource expectations visible in the tests. +- Prefer seams that isolate behavior without hiding lifetime issues. +- Treat nondeterminism, time, and thread scheduling carefully. +- Use tests to expose invariants around construction, mutation, and cleanup. -## Output +## Good Output +- A scenario-based C++ test plan. +- Edge cases around ownership, cleanup, and invalid state. +- Suggestions for making code more testable without obscuring correctness risks. +- Warnings about brittle or UB-sensitive test approaches. -- Brief plan or checklist -- Key recommendations and caveats +## Common Mistakes +- Ignoring object lifetime and cleanup behavior. +- Testing only happy-path values in code with risky ownership semantics. +- Mocking away the exact resource behavior that matters. +- Letting undefined behavior slip through because the test passed once. diff --git a/skills/Testing & Security/developing-with-fortify/SKILL.md b/skills/Testing & Security/developing-with-fortify/SKILL.md deleted file mode 100644 index a4a00934..00000000 --- a/skills/Testing & Security/developing-with-fortify/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: developing-with-fortify -name: Developing With Fortify -description: Step-by-step guidance for developing with fortify. -category: Testing & Security ---- - -# Developing With Fortify - -Support developing with fortify workflows with clear steps and best practices. - -## When to Use - -- You need help with developing with fortify. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/feature-flags/SKILL.md b/skills/Testing & Security/feature-flags/SKILL.md deleted file mode 100644 index c97f0ac7..00000000 --- a/skills/Testing & Security/feature-flags/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: feature-flags -name: Feature Flags -description: Step-by-step guidance for feature flags. -category: Testing & Security ---- - -# Feature Flags - -Support feature flags workflows with clear steps and best practices. - -## When to Use - -- You need help with feature flags. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/flow/SKILL.md b/skills/Testing & Security/flow/SKILL.md deleted file mode 100644 index d0dba5f7..00000000 --- a/skills/Testing & Security/flow/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: flow -name: Flow -description: Step-by-step guidance for flow. -category: Testing & Security ---- - -# Flow - -Support flow workflows with clear steps and best practices. - -## When to Use - -- You need help with flow tasks. -- You want a clear quality checklist. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/frontend-code-review/SKILL.md b/skills/Testing & Security/frontend-code-review/SKILL.md index 9660b1a2..54c206c9 100644 --- a/skills/Testing & Security/frontend-code-review/SKILL.md +++ b/skills/Testing & Security/frontend-code-review/SKILL.md @@ -1,20 +1,41 @@ --- id: frontend-code-review name: Frontend Code Review -description: Step-by-step guidance for frontend code review. +description: Review frontend code for regressions, accessibility, interaction quality, state handling, and testability risks. category: Testing & Security +requires: [] +examples: + - "Review this frontend change for quality risks." + - "What should I look for in this UI code review?" + - "Check this component change for accessibility and regression issues." --- # Frontend Code Review -Support frontend code review workflows with clear steps and best practices. +Use this skill to review frontend changes with attention to user-visible regressions, accessibility, interaction quality, and testability. -## When to Use +## Review Priorities +- User-visible behavior and state transitions. +- Accessibility semantics, focus handling, and keyboard support. +- Error, loading, and empty-state behavior. +- State management clarity and unintended coupling. +- Testability and regression risk of the changed code. -- You need help with frontend code review tasks. -- You want a clear quality checklist. +## Review Questions +- What can break visually or behaviorally for users? +- Are important states and transitions represented clearly? +- Do labels, roles, focus order, and interaction patterns stay accessible? +- Is state flow understandable enough to test and maintain? +- Does the change introduce brittle selectors, implicit contracts, or hidden side effects? -## Output +## Good Output +- Highest-risk review findings first. +- Notes on regressions, accessibility, and state-handling concerns. +- Suggestions for tests that would reduce uncertainty. +- Concrete cleanup or simplification opportunities. -- Brief plan or checklist -- Key recommendations and caveats +## Common Mistakes +- Reviewing only styling and missing behavioral regressions. +- Treating accessibility as optional polish. +- Ignoring loading and failure states. +- Approving complex state wiring that is hard to reason about or test. diff --git a/skills/Testing & Security/frontend-testing/SKILL.md b/skills/Testing & Security/frontend-testing/SKILL.md index acc3cd05..4663612a 100644 --- a/skills/Testing & Security/frontend-testing/SKILL.md +++ b/skills/Testing & Security/frontend-testing/SKILL.md @@ -1,20 +1,46 @@ --- id: frontend-testing name: Frontend Testing -description: Step-by-step guidance for frontend testing. +description: Plan frontend tests across components, interactions, accessibility, rendering states, and browser-facing regressions. category: Testing & Security +requires: [] +examples: + - "Help me test this frontend feature." + - "What frontend tests are missing for this component?" + - "Review this browser and interaction test plan." --- # Frontend Testing -Support frontend testing workflows with clear steps and best practices. +Use this skill to design frontend tests that balance component behavior, interaction quality, accessibility, and browser-facing regressions. -## When to Use +## Clarify First +- What kind of frontend surface is involved: component, page, flow, form, or design-system primitive. +- Which behavior matters most: rendering, interaction, accessibility, state transitions, or integration with data. +- What should be tested at unit, component, integration, or browser level. +- Which regressions would matter most to users. -- You need help with frontend testing. -- You want a clear, actionable next step. +## Good Frontend Test Areas +- Rendering of important states and error conditions. +- User interactions and event handling. +- Accessibility semantics, focus behavior, and labels. +- Loading, empty, error, and success states. +- Cross-component behavior when state or routing matters. -## Output +## Testing Principles +- Prefer testing visible behavior over internals. +- Use accessible queries and user-facing outcomes whenever possible. +- Keep browser-level tests for workflows and regression-prone UI behavior. +- Avoid snapshot-heavy suites that are hard to interpret. -- Brief plan or checklist -- Key recommendations and caveats +## Good Output +- Suggested test layers by feature risk. +- Missing scenarios or state transitions. +- Accessibility and interaction checks worth adding. +- Warnings about brittle selectors or implementation-heavy assertions. + +## Common Mistakes +- Overusing snapshots instead of asserting meaningful behavior. +- Testing props and state plumbing rather than user-visible outcomes. +- Ignoring empty, loading, and error states. +- Treating accessibility as separate from normal frontend behavior. diff --git a/skills/Testing & Security/gdpr-data-handling/SKILL.md b/skills/Testing & Security/gdpr-data-handling/SKILL.md deleted file mode 100644 index 856e1bb5..00000000 --- a/skills/Testing & Security/gdpr-data-handling/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: gdpr-data-handling -name: GDPR Data Handling -description: Step-by-step guidance for gdpr data handling. -category: Testing & Security ---- - -# GDPR Data Handling - -Support gdpr data handling workflows with clear steps and best practices. - -## When to Use - -- You need help with gdpr data handling. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/golang-testing/SKILL.md b/skills/Testing & Security/golang-testing/SKILL.md deleted file mode 100644 index ba3b4652..00000000 --- a/skills/Testing & Security/golang-testing/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: golang-testing -name: Golang Testing -description: Step-by-step guidance for golang testing. -category: Testing & Security ---- - -# Golang Testing - -Support golang testing workflows with clear steps and best practices. - -## When to Use - -- You need help with golang testing. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/isms-audit-expert/SKILL.md b/skills/Testing & Security/isms-audit-expert/SKILL.md deleted file mode 100644 index 87473d4b..00000000 --- a/skills/Testing & Security/isms-audit-expert/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: isms-audit-expert -name: Isms Audit Expert -description: Step-by-step guidance for isms audit expert. -category: Testing & Security ---- - -# Isms Audit Expert - -Support isms audit expert workflows with clear steps and best practices. - -## When to Use - -- You need help with isms audit expert. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/javascript-testing-patterns/SKILL.md b/skills/Testing & Security/javascript-testing-patterns/SKILL.md index 6985ff56..6b09330b 100644 --- a/skills/Testing & Security/javascript-testing-patterns/SKILL.md +++ b/skills/Testing & Security/javascript-testing-patterns/SKILL.md @@ -1,20 +1,41 @@ --- id: javascript-testing-patterns -name: Javascript Testing Patterns -description: Step-by-step guidance for javascript testing patterns. +name: JavaScript Testing Patterns +description: Apply JavaScript-specific testing patterns for async behavior, timers, module boundaries, mocks, and browser-like environments. category: Testing & Security +requires: [] +examples: + - "What testing patterns fit this JavaScript module?" + - "Help me test this async JavaScript behavior." + - "Review these JS tests for brittle mocks and timer issues." --- -# Javascript Testing Patterns +# JavaScript Testing Patterns -Support javascript testing patterns workflows with clear steps and best practices. +Use this skill when JavaScript-specific behavior makes generic unit-test advice too shallow. -## When to Use +## Focus Areas +- Async flows, promises, retries, and race conditions. +- Timers, scheduling, and event-loop behavior. +- Module boundaries, imports, and side effects. +- Browser-like globals, DOM-adjacent behavior, or runtime assumptions. +- Mocking choices and how they affect trust in the test. -- You need help with javascript testing patterns. -- You want a clear, actionable next step. +## Useful Patterns +- Control time explicitly when timers or retries matter. +- Test observable behavior of async code, not incidental ordering details. +- Isolate side effects at module boundaries. +- Be cautious with mocks that rewrite too much of the runtime contract. +- Prefer patterns that keep tests readable under refactoring. -## Output +## Good Output +- JavaScript-specific test scenarios worth adding. +- Warnings about timers, globals, mocks, or async flakiness. +- Suggestions for cleaner seams around side effects. +- Notes on what should be tested synchronously versus with async orchestration. -- Brief plan or checklist -- Key recommendations and caveats +## Common Mistakes +- Letting fake timers or microtask behavior hide real bugs. +- Over-mocking modules until the test no longer reflects runtime behavior. +- Ignoring race conditions in code that awaits or schedules work. +- Treating DOM-like globals as harmless implicit state. diff --git a/skills/Testing & Security/libafl/SKILL.md b/skills/Testing & Security/libafl/SKILL.md index 1babf876..09ab1e6c 100644 --- a/skills/Testing & Security/libafl/SKILL.md +++ b/skills/Testing & Security/libafl/SKILL.md @@ -1,24 +1,40 @@ --- id: libafl name: LibAFL -description: LibAFL is a modular fuzzing library for building custom fuzzers. Use for advanced fuzzing needs, custom mutators, or non-standard fuzzing targets. +description: Guide advanced fuzzing design with LibAFL concepts such as harnesses, mutators, corpora, oracles, and execution strategy. category: Testing & Security requires: [] examples: - - "Help me build a custom fuzzer with LibAFL" - - "Create a custom mutator for LibAFL" + - "Help me design a custom fuzzer with LibAFL." + - "What should this LibAFL harness look like?" + - "How do I think about mutators and corpora for this target?" --- # LibAFL -LibAFL is a modular fuzzing library for building custom fuzzers. Use for advanced fuzzing needs, custom mutators, or non-standard fuzzing targets. +Use this skill for advanced fuzzing problems where the user needs design help beyond generic test generation. -## When to Use +## Focus Areas +- Harness design and target isolation. +- Corpus strategy and seed quality. +- Mutator choice and feedback signals. +- Crash or bug oracles. +- Determinism, reproducibility, and execution throughput. -- You need help with libafl. -- You want a clear, actionable next step. +## Design Questions +- What kind of target is being fuzzed: parser, protocol, VM, stateful service, or something else? +- What input shape and validity constraints matter? +- What signals indicate progress: coverage, comparison feedback, crashes, semantic failures? +- How expensive is one execution, and what is making it unstable? -## Output +## Good Output +- Suggested harness structure. +- Corpus and mutator guidance. +- Likely bottlenecks in throughput or signal quality. +- Ways to make failures reproducible and useful. -- Summary of goals and plan -- Key tips and precautions +## Common Mistakes +- Fuzzing through too much unstable environment at once. +- Weak oracles that only detect crashes and miss logical failures. +- Poor seed variety that limits exploration. +- Ignoring determinism and then chasing flaky results. diff --git a/skills/Testing & Security/mtls-configuration/SKILL.md b/skills/Testing & Security/mtls-configuration/SKILL.md deleted file mode 100644 index e10077d5..00000000 --- a/skills/Testing & Security/mtls-configuration/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: mtls-configuration -name: mTLS Configuration -description: Step-by-step guidance for mtls configuration. -category: Testing & Security ---- - -# mTLS Configuration - -Support mtls configuration workflows with clear steps and best practices. - -## When to Use - -- You need help with mtls configuration. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/performing-security-audits/SKILL.md b/skills/Testing & Security/performing-security-audits/SKILL.md index 422039ce..cd08e047 100644 --- a/skills/Testing & Security/performing-security-audits/SKILL.md +++ b/skills/Testing & Security/performing-security-audits/SKILL.md @@ -1,136 +1,55 @@ --- id: performing-security-audits name: Performing Security Audits -description: Thorough code review with focus on security, performance, and best practices. Use when reviewing code, performing security audits, checking code quality, or reviewing pull requests. +description: Perform security-focused code and design audits with clear severity, exploitability, and remediation guidance. category: Testing & Security requires: [] examples: - - "Review this code for security vulnerabilities" - - "Perform a security audit on this pull request" -author: awesome-llm-apps + - "Review this code for security vulnerabilities." + - "Perform a security audit on this pull request." + - "What are the highest-risk security issues in this change?" +author: "awesome-llm-apps" version: "2.0.0" --- -# Code Reviewer - -You are an expert code reviewer who identifies security vulnerabilities, performance issues, and code quality problems. - -## When to Apply - -Use this skill when: -- Reviewing pull requests -- Performing security audits -- Checking code quality -- Identifying performance bottlenecks -- Ensuring best practices -- Pre-deployment code review - -## How to Use This Skill - -This skill contains **detailed rules** in the `rules/` directory, organized by category and priority. - -### Quick Start - -1. **Review [AGENTS.md](AGENTS.md)** for a complete compilation of all rules with examples -2. **Reference specific rules** from `rules/` directory for deep dives -3. **Follow priority order**: Security → Performance → Correctness → Maintainability - -### Available Rules - -**Security (CRITICAL)** -- [SQL Injection Prevention](rules/security-sql-injection.md) -- [XSS Prevention](rules/security-xss-prevention.md) - -**Performance (HIGH)** -- [Avoid N+1 Query Problem](rules/performance-n-plus-one.md) - -**Correctness (HIGH)** -- [Proper Error Handling](rules/correctness-error-handling.md) - -**Maintainability (MEDIUM)** -- [Use Meaningful Variable Names](rules/maintainability-naming.md) -- [Add Type Hints](rules/maintainability-type-hints.md) - -## Review Process - -### 1. **Security First** (CRITICAL) -Look for vulnerabilities that could lead to data breaches or unauthorized access: -- SQL injection -- XSS (Cross-Site Scripting) -- Authentication/authorization bypasses -- Hardcoded secrets -- Insecure dependencies - -### 2. **Performance** (HIGH) -Identify code that will cause slow performance at scale: -- N+1 database queries -- Missing indexes -- Inefficient algorithms -- Memory leaks -- Unnecessary API calls - -### 3. **Correctness** (HIGH) -Find bugs and edge cases: -- Error handling gaps -- Race conditions -- Off-by-one errors -- Null/undefined handling -- Input validation - -### 4. **Maintainability** (MEDIUM) -Improve code quality for long-term health: -- Clear naming -- Type safety -- DRY principle -- Single responsibility -- Documentation - -### 5. **Testing** -Verify adequate coverage: -- Unit tests for new code -- Edge case testing -- Error path testing -- Integration tests where needed - -## Review Output Format - -Structure your reviews as: - -```markdown -This function retrieves user data but has critical security and reliability issues. - -## Critical Issues 🔴 - -1. **SQL Injection Vulnerability** (Line 2) - - **Problem:** User input directly interpolated into SQL query - - **Impact:** Attackers can execute arbitrary SQL commands - - **Fix:** Use parameterized queries - ```python - query = "SELECT * FROM users WHERE id = ?" - result = db.execute(query, (user_id,)) - ``` - -## High Priority 🟠 - -1. **No Error Handling** (Line 3-4) - - **Problem:** Assumes result always has data - - **Impact:** IndexError if user doesn't exist - - **Fix:** Check result before accessing - ```python - if not result: - return None - return result[0] - ``` - -2. **Missing Type Hints** (Line 1) - - **Problem:** No type annotations - - **Impact:** Reduces code clarity and IDE support - - **Fix:** Add type hints - ```python - def get_user(user_id: int) -> Optional[Dict[str, Any]]: - ``` - -## Recommendations -- Add logging for debugging -- Consider using an ORM to prevent SQL injection -- Add input validation for user_id +# Performing Security Audits + +Use this skill to review code, configuration, or architecture from a security-first perspective. + +## Audit Priorities +- Attack surface and trust boundaries. +- Authentication, authorization, and session handling. +- Input validation, output encoding, and injection risks. +- Secrets, credentials, and sensitive data exposure. +- Dependency, configuration, and deployment risks. + +## Review Method +1. Clarify what changed and what is exposed to users, services, or the internet. +2. Identify the most likely abuse paths before listing general best practices. +3. Separate critical exploitable issues from lower-risk hygiene concerns. +4. Explain why each finding matters, not just what rule it breaks. +5. Suggest the smallest effective remediation or mitigation. + +## High-Value Audit Lenses +- Can an untrusted actor reach this path? +- What input crosses trust boundaries? +- What permissions or secrets are involved? +- What happens when assumptions fail or inputs are malicious? +- Would logs, errors, or defaults leak more than intended? + +## Output Format +- Critical findings first. +- Each finding should include problem, impact, and suggested fix. +- Note assumptions and missing context when confidence is limited. +- Mention testing or verification steps if they would reduce uncertainty. + +## Common Security Findings +- Injection and unsafe interpolation. +- Missing authorization checks. +- Overly broad trust in external input or upstream services. +- Secrets or tokens handled insecurely. +- Risky defaults, missing rate limits, or weak validation. + +## Boundaries +- Do not pretend to run scanners or penetration tests. +- Prefer concrete findings, abuse scenarios, and remediation guidance over generic checklists. diff --git a/skills/Testing & Security/pest-testing/SKILL.md b/skills/Testing & Security/pest-testing/SKILL.md deleted file mode 100644 index 0f0f05b6..00000000 --- a/skills/Testing & Security/pest-testing/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: pest-testing -name: Pest Testing -description: Step-by-step guidance for pest testing. -category: Testing & Security ---- - -# Pest Testing - -Support pest testing workflows with clear steps and best practices. - -## When to Use - -- You need help with pest testing. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/property-based-testing/SKILL.md b/skills/Testing & Security/property-based-testing/SKILL.md index d8dbd53c..20e33748 100644 --- a/skills/Testing & Security/property-based-testing/SKILL.md +++ b/skills/Testing & Security/property-based-testing/SKILL.md @@ -1,408 +1,58 @@ --- id: property-based-testing name: Property Based Testing -description: Expert-level JavaScript testing skill focused on writing high-quality tests that find bugs, serve as documentation, and prevent regressions. Advocates for property-based testing with fast-check and protects against indeterministic code in tests. Does not cover black-box e2e testing. +description: Use property-based testing to find edge cases, challenge invariants, and strengthen test suites without overfitting examples. category: Testing & Security requires: [] examples: - - "Help me write property-based tests with fast-check" - - "Convert this unit test to a property-based test" + - "Help me find good properties for this function." + - "Convert this example-based test into a property-based test." + - "What invariants should I test for this algorithm?" --- -> **⚠️ Scope:** Testing functions and components, not black-box e2e. - -**🏅 Main objectives:** use tests as a way to... - -1. uncover hard to detect bugs -2. document how to use the code -3. avoid regressions -4. challenge the code - -**🔧 Recommended tooling:** `vitest`, `fast-check`, `@fast-check/vitest`, `@testing-library/*`, `@vitest/browser-playwright`, `msw` installed as devDependencies. -**✅ Do** try to install missing and relevant tooling, only recommend for `@fast-check/vitest` and browser testing. -**✅ Do** highly recommend user to install missing and relevant tooling. -**✅ Do** adapt yourself to missing tools. - -## File and code layout - -**✅ Do** mimic the existing test structure of the project when adding new tests - -**✅ Do** use one test file per code file - -**👍 Prefer** using `.spec.ts` extension (e.g., `fileName.ts` → `fileName.spec.ts`) and colocated with the source file if no existing test structure is present - -**✅ Do** put `it` within `describe`, when using `it` - -**👍 Prefer** `it` over `test` - -**✅ Do** name the `describe` with the name of the function being tested - -**✅ Do** use a dedicated `describe` for each function being tested - -**✅ Do** start naming `it` with "should" and considers that the name should be clear, as consise as possible and could be read as a sentence implicitly prefixed by "it" - -**✅ Do** start with simple and documenting tests - -**✅ Do** continue with advanced tests looking for edge-cases - -**❌ Don't** delimitate explicitely simple from advanced tests, just but them in the right order - -**✅ Do** put helper functions specific to the file after all the `describe`s just below a comment `// Helpers` stating the beginning of the helpers tailored for this file - -## Core guidelines - -**✅ Do** follow the AAA pattern and make it visible in the test - -```ts -it('should...', () => { - // Arrange - code; - - // Act - code; - - // Assert - code; -}); -``` - -**✅ Do** keep tests focused, try to assert on one precise aspect - -**✅ Do** keep tests simple - -**👎 Avoid** complex logic in tests or its helpers - -**❌ Don't** test internal details - -**👍 Prefer** stubs over mocks, the first one provides an alternate implementation, the second one helps to assert on calls being done or not -Why? Often, asserting the number of calls is not something critical for the user of the function but purely an internal detail - -**❌ Don't** rely on network call, stub it with `msw` - -**✅ Do** reset globals and mocks in `beforeEach` if any `it` plays with mocks or spies or alter globals -Alternatively, when using vitest you could check if flags `mockReset`, `unstubEnvs` and `unstubGlobals` have been enabled in the configuration, in such case resetting globals is done by default - -**👍 Prefer** realistic data for documentation-like tests -Eg.: use real names if you have to build instances of users - -**❌ Don't** overuse snapshot tests; only snapshot things when the "what is expected to be seen in the snapshot" is clear -Why? Snapshots tests tend to capture too many details in the snapshot, making them hard to update given future reader is lost on what was the real thing being tested - -**👍 Prefer** snapshots when shape and structure are important (component hierarchy, attributes, non-regression on output structure) - -**👍 Prefer** screenshots when final render is important (visual styling, layout) - -**✅ Do** warn developer when the code under tests requires too many parameters and/or too many mocks/stubs to be forged (more than 10) -Why? Code being hardly testable is often a code smell pinpointing an API having to be changed. Code is harder to evolve, harder to reason about and often handling too many responsibilities. Recommend the single-responsibility principle (SRP) - -**✅ Do** try to make tests shorter and faster to read by factorizing recurrent logics into helper functions - -**✅ Do** group shared logics under a function having a clear and explicit name, follow SRP for these helpers -Eg.: avoid functions with lots of optional parameters, doing several things - -**❌ Don't** write a big `prepare` function re-used by all tests in their act part, but make the name clearer and eventually split it into multiple functions - -**✅ Do** make sure your test breaks if you drop the thing supposed to make it pass -Eg.: When your test says "should do X when Y" makes sure that if you don't have Y it fails before keeping it. - -**👎 Avoid** writing tests with entities specifying hardcoded values on unused fields - -Example of test content - -```ts -const user: User = { - name: 'Paul', // unused - birthday: '2010-02-03', -}; -const age = computeAge(user); -//... -``` - -**👍 Prefer** leveraging `@fast-check/vitest`, if installed - -```ts -import { describe } from 'vitest'; -import { it, fc } from '@fast-check/vitest'; - -describe('computeAge', () => { - it('should compute a positive age', ({ g }) => { - // Arrange - const user: User = { - name: g(fc.string), // unused - birthday: '2010-02-03', - }; - - // Act - const age = computeAge(user); - - // Assert - expect(age).toBeGreaterThan(0); - }); -}); -``` - -**👍 Prefer** leveraging `fast-check`, if installed but not `@fast-check/vitest` - -**👎 Avoid** writing tests depending on unstable values -Eg.: in the example above `computeAge` depends on the current date -Remark: same for locales and plenty other platform dependent values - -**👍 Prefer** stubbing today using `vi.setSystemTime` - -**👍 Prefer** controlling today using `@fast-check/vitest` -Why? Contrary to `vi.setSystemTime` alone you check the code against one new today at each run, but if it happens to fail one day you will be reported with the exact date causing the problem - -```ts -// Arrange -vi.setSystemTime(g(fc.date, { min: new Date('2010-02-04'), noInvalidDate: true })); -const user: User = { - name: g(fc.string), // unused - birthday: '2010-02-03', -}; -``` - -**👎 Avoid** writing tests depending on random values or entities - -**👍 Prefer** controlling randomly generated values by relying on `@fast-check/vitest` if installed, or `fast-check` otherwise - -**✅ Do** use property based tests for any test with a notion of always or never -Eg.: name being "should always do x when y" or "should never do x when y" -Remark: consider these tests as advanced and put them after the documentation tests and not with them - -**👍 Prefer** using property based testing for edge case detection instead of writing all cases one by one - -**❌ Don't** try to test 100% of the algorithm cases using property-based testing -Why? Property-based testing and example-based testing are complementary. Property-based tests are excellent for uncovering edge cases and validating general properties, while example-based tests provide clear documentation and cover specific important scenarios. Use both approaches together for comprehensive test coverage. - -```ts -// for all a, b, c strings -// b is a substring of a + b + c -it.prop([fc.string(), fc.string(), fc.string()])('should detect the substring', (a, b, c) => { - // Arrange - const text = a + b + c; - const pattern = b; - - // Act - const result = isSubstring(text, pattern); - - // Assert - expect(result).toBe(true); -}); -``` - -**✅ Do** extract complex logic from components into dedicated and testable functions - -**❌ Don't** test trivial component logic that has zero complexity - -**👍 Prefer** testing the DOM structure and user interactions when using testing-library - -**👍 Prefer** testing the visual display and user interactions when using browser testing - -**👍 Prefer** querying by accessible attributes and user-visible text by relying on `getByRole`, `getByLabelText`, `getByText` over `getByTestId` whenever possible for testing-library and browser testing - -**✅ Do** ensure non visual regression of Design System components and more generally visual components by leveraging screenshot tests in browser when available -**✅ Do** fallback to snapshot tests capturing the DOM structure if screenshot tests cannot be ran - -## Guidelines for properties - -All this section considers that we are in the context of property based tests! - -**⚠️ Important:** When using `g` from `@fast-check/vitest`, pass the arbitrary **function** (e.g., `fc.string`, `fc.date`) along with its arguments as separate parameters to `g`, not the result of calling it. -Correct: `g(fc.string)`, `g(fc.date, { min: new Date('2010-01-01') })` -Incorrect: `g(fc.string())`, `g(fc.date({ min: new Date('2010-01-01') }))` - -**❌ Don't** generate inputs directly -The risk being that you may end up rewriting the code being tested in the test - -**✅ Do** construct values to build some inputs where you know the expected outcome - -**❌ Don't** expect the returned value in details, in many cases you won't have enough details to be able to assert the full value - -**✅ Do** expect some aspects and characteristics of the returned value - -**❌ NEVER** specify any `maxLength` on an arbitrary if it is a not a requirement of the algorithm -**👍 Prefer** specifying a `size: '-1'` if you feel that the algorithm will take very long on large inputs (by default fast-check generates up to 10 items, so only use `size` when clearly required) -Eg.: No `fc.string({maxLength: 5})` or `fc.array(arb, {maxLength: 8})` except being a string requirement - -**❌ NEVER** specify any constraint on an arbitrary if it is not a requirement of the arbitrary, use defaults as much as possible -Eg.: if the algorithm should accept any integer just ask an integer without specifying any min and max - -**👎 Avoid** overusing `.filter` and `fc.pre` -Why? They slow down the generation of values by dropping some generated ones - -**👍 Prefer** using options provided by arbitraries to directly generate valid values -Eg.: use `fc.string({ minLength: 2 })` instead of `fc.string().filter(s => s.length >= 2)` -Eg.: use `fc.integer({ min: 1 })` instead of `fc.integer().filter(n => n >= 1)`, or use `fc.nat()` instead of `fc.integer().filter(n => n >= 0)` - -**👍 Prefer** using `map` over `filter` when a `map` trick can avoid filtering -Eg.: use `fc.nat().map(n => n * 2)` for even numbers -Eg.: use `fc.tuple(fc.string(), fc.string()).map(([start, end]) => start + 'A' + end)` for strings always having an 'A' character - -**👍 Prefer** bigint type over number type for integer computations used within predicates when there is a risk of overflow (eg.: when running pow, multiply.. on generated values) - -Some classical properties: - -1. Characteristics independent of the inputs. _Eg.: for any floating point number d, Math.floor(d) is an integer. for any integer n, Math.abs(n) ≥ 0_ -2. Characteristics derived from the inputs. _Eg.: for any a and b integers, the average of a and b is between a and b. for any n, the product of all numbers in the prime factor decomposition of n equals n. for any array of data, sorted(data) and data contains the same elements. for any n1, n2 integers such that n1 != n2, romanString(n1) != romanString(n2). for any floating point number d, Math.floor(d) is an integer such as d-1 ≤ Math.floor(d) ≤ d_ -3. Restricted set of inputs with useful characteristics. _Eg.: for any array data with no duplicates, the result of removing duplicates from data is data itself. for any a, b and c strings, the concatenation of a, b and c always contains b. for any prime number p, its decomposition into prime factors is itself_ -4. Characteristics on combination of functions. _Eg.: zipping then unzipping a file should result in the original file. lcm(a,b) times gcd(a,b) must be equal to a times b_ -5. Comparison with a simpler implementation. _Eg.: c is contained inside sorted array data for binary search is equivalent to c is contained inside data for linear search_ - -## Guidelines for race conditions - -**✅ Do** write tests checking for race conditions and playing with resolution order — _automatically handled by `fast-check`_ — when an algorithm accepts asynchronous functions as input - -**✅ Do** leverage `fast-check` and its `fc.scheduler()` arbitrary to test asynchronous code depending on asynchronous functions - -Turn: - -```ts -it('should resolve in call order', async () => { - // Arrange - const seenAnswers = []; - const call = vi.fn().mockImplementation((v) => Promise.resolve(v)); - - // Act - const queued = queue(call); - await Promise.all([queued(1).then((v) => seenAnswers.push(v)), queued(2).then((v) => seenAnswers.push(v))]); - - // Assert - expect(seenAnswers).toEqual([1, 2]); -}); -``` - -Into: - -```ts -it('should resolve in call order', async () => { - await fc.assert( - fc.asyncProperty(fc.scheduler(), async (s) => { - // Arrange - const seenAnswers = []; - const call = vi.fn().mockImplementation((v) => Promise.resolve(v)); - - // Act - const queued = queue(s.scheduleFunction(call)); - await s.waitFor( - Promise.all([queued(1).then((v) => seenAnswers.push(v)), queued(2).then((v) => seenAnswers.push(v))]), - ); - - // Assert - expect(seenAnswers).toEqual([1, 2]); - }), - ); -}); -``` - -## Recommendation for faker users - -If using `faker` to fake data, we recommend wiring any fake data generation within `fast-check` by leveraging this code snippet: - -```ts -// Source: https://fast-check.dev/blog/2024/07/18/integrating-faker-with-fast-check/ -import { Faker, Randomizer, base } from '@faker-js/faker'; -import fc from 'fast-check'; - -class FakerBuilder extends fc.Arbitrary { - constructor(private readonly generator: (faker: Faker) => TValue) { - super(); - } - generate(mrng: fc.Random, biasFactor: number | undefined): fc.Value { - const randomizer: Randomizer = { - next: (): number => mrng.nextDouble(), - seed: () => {}, // no-op, no support for updates of the seed, could even throw - }; - const customFaker = new Faker({ locale: base, randomizer }); - return new fc.Value(this.generator(customFaker), undefined); - } - canShrinkWithoutContext(value: unknown): value is TValue { - return false; - } - shrink(value: TValue, context: unknown): fc.Stream> { - return fc.Stream.nil(); - } -} - -function fakerToArb(generator: (faker: Faker) => TValue): fc.Arbitrary { - return new FakerBuilder(generator); -} -``` - -Example of usage - -```ts -fc.assert( - fc.property( - fakerToArb((faker) => faker.person.firstName), - fakerToArb((faker) => faker.person.lastName), - (firstName, lastName) => { - // code - }, - ), -); -``` - -## Equivalence `fast-check` and `@fast-check/vitest` - -Example 1. - -```ts -// with @fast-check/vitest -import { it, fc } from '@fast-check/vitest'; -it('...', ({ g }) => { - //... -}); - -// with fast-check -import { it } from 'vitest'; -import fc from 'fast-check'; -it('...', () => { - fc.assert( - fc.property(fc.gen(), (g) => { - //... - }), - ); -}); -``` - -Example 2. - -```ts -// with @fast-check/vitest -import { it, fc } from '@fast-check/vitest'; -it.prop([...arbitraries])('...', (...values) => { - //... -}); - -// with fast-check -import { it } from 'vitest'; -import fc from 'fast-check'; -it('...', () => { - fc.assert( - fc.property(...arbitraries, (...values) => { - //... - }), - ); -}); -``` - -Example 3. If the predicate of `it` or `it.prop` is asynchronous, when using only `fast-check` the property has to be instantiated via `asyncProperty` and `assert` has to be awaited. - -```ts -// with @fast-check/vitest -import { it, fc } from '@fast-check/vitest'; -it.prop([...arbitraries])('...', async (...values) => { - //... -}); - -// with fast-check -import { it } from 'vitest'; -import fc from 'fast-check'; -it('...', async () => { - await fc.assert( - fc.asyncProperty(...arbitraries, async (...values) => { - //... - }), - ); -}); -``` +# Property Based Testing + +Use this skill to design tests around invariants, generated inputs, and edge-case discovery rather than hand-picking every case one by one. + +## When It Fits Best +- The behavior should always or never hold. +- The code accepts many valid inputs or combinations. +- Edge cases are hard to enumerate manually. +- Example tests already exist, but they are not enough to challenge the implementation. + +## Start By Clarifying +- What the unit under test guarantees. +- Which inputs are valid, invalid, or constrained by business rules. +- What would count as a real failure versus an irrelevant oddity. +- Whether the code depends on time, randomness, ordering, or async behavior. + +## Good Property Design +- Test stable invariants, not implementation details. +- Combine property-based tests with example-based tests. +- Prefer properties that explain behavior clearly enough for future readers. +- Use generated data to challenge the code, not to rewrite the production algorithm inside the test. + +## Common Property Patterns +- Output always satisfies a validity condition. +- Transformations preserve key structure or meaning. +- Two implementations agree on the same input. +- Doing an operation and then its inverse gets back to the original result. +- Reordering or combining inputs preserves a known property. + +## Failure Prevention +- Control unstable inputs like time, randomness, locale, and async scheduling when they would make failures noisy. +- Keep generators as broad as the real requirements allow. +- Avoid over-constraining inputs just to make tests pass more easily. +- Treat shrinking and reproducibility as part of the debugging workflow. + +## Good Output +- Candidate properties worth testing. +- Suggested input strategy and constraints. +- Warnings about unstable dependencies or false confidence. +- A balanced test plan that mixes properties with concrete examples. + +## Common Mistakes +- Using property tests where a few examples would explain the behavior better. +- Writing properties so vague they pass without testing much. +- Filtering generated inputs heavily instead of modeling valid inputs directly. +- Confusing exhaustive testing with property-based testing. diff --git a/skills/Testing & Security/pyrefly-type-coverage/SKILL.md b/skills/Testing & Security/pyrefly-type-coverage/SKILL.md deleted file mode 100644 index 0bb4c943..00000000 --- a/skills/Testing & Security/pyrefly-type-coverage/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: pyrefly-type-coverage -name: Pyrefly Type Coverage -description: Step-by-step guidance for pyrefly type coverage. -category: Testing & Security ---- - -# Pyrefly Type Coverage - -Support pyrefly type coverage workflows with clear steps and best practices. - -## When to Use - -- You need help with pyrefly type coverage tasks. -- You want a clear quality checklist. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/python-anti-patterns/SKILL.md b/skills/Testing & Security/python-anti-patterns/SKILL.md deleted file mode 100644 index 3a3d92dd..00000000 --- a/skills/Testing & Security/python-anti-patterns/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: python-anti-patterns -name: Python Anti Patterns -description: Step-by-step guidance for python anti patterns. -category: Testing & Security ---- - -# Python Anti Patterns - -Support python anti patterns workflows with clear steps and best practices. - -## When to Use - -- You need help with python anti patterns tasks. -- You want a clear quality checklist. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/python-testing/SKILL.md b/skills/Testing & Security/python-testing/SKILL.md index 0ac43d65..17accec5 100644 --- a/skills/Testing & Security/python-testing/SKILL.md +++ b/skills/Testing & Security/python-testing/SKILL.md @@ -1,20 +1,40 @@ --- id: python-testing name: Python Testing -description: Step-by-step guidance for python testing. +description: Design Python tests with attention to fixtures, parametrization, side effects, exceptions, and stateful runtime behavior. category: Testing & Security +requires: [] +examples: + - "Help me write tests for this Python module." + - "What Python test cases are missing here?" + - "Review this pytest-style test plan for fixture and state issues." --- # Python Testing -Support python testing workflows with clear steps and best practices. +Use this skill to design Python tests that are clear, maintainable, and sensitive to Python-specific failure patterns. -## When to Use +## Clarify First +- What is under test and what external state it touches. +- Which exceptions, side effects, or mutable state transitions matter. +- Whether fixtures, parametrization, or patching are helping or hiding problems. +- What behavior should remain explicit even if helper abstractions exist. -- You need help with python testing. -- You want a clear, actionable next step. +## Python Testing Priorities +- Make fixtures serve readability, not hide setup complexity. +- Use parametrization where it clarifies input variation. +- Be explicit about exceptions, warnings, and failure semantics. +- Control filesystem, environment, network, and time when they affect behavior. +- Watch for mutable shared state and order-dependent tests. -## Output +## Good Output +- Scenario-based Python test plan. +- Fixture or parametrization suggestions. +- Warnings about hidden state, monkeypatching, or brittle assertions. +- Missing failure or edge cases worth covering. -- Brief plan or checklist -- Key recommendations and caveats +## Common Mistakes +- Fixtures that hide more than they help. +- Overusing patching so the test stops reflecting real behavior. +- Missing assertions around exceptions or warnings. +- State leakage between tests through globals, caches, or mutable defaults. diff --git a/skills/Testing & Security/realphonevalidation-automation/SKILL.md b/skills/Testing & Security/realphonevalidation-automation/SKILL.md deleted file mode 100644 index 0e432b26..00000000 --- a/skills/Testing & Security/realphonevalidation-automation/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: realphonevalidation-automation -name: Real Phone Validation Automation -description: Step-by-step guidance for real phone validation automation. -category: Testing & Security ---- - -# Real Phone Validation Automation - -Support real phone validation automation workflows with clear steps and best practices. - -## When to Use - -- You need help with real phone validation automation tasks. -- You want a clear quality checklist. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/receiving-code-review/SKILL.md b/skills/Testing & Security/receiving-code-review/SKILL.md deleted file mode 100644 index 4e20d730..00000000 --- a/skills/Testing & Security/receiving-code-review/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: receiving-code-review -name: Receiving Code Review -description: Step-by-step guidance for receiving code review. -category: Testing & Security ---- - -# Receiving Code Review - -Support receiving code review workflows with clear steps and best practices. - -## When to Use - -- You need help with receiving code review tasks. -- You want a clear quality checklist. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/requesting-code-review/SKILL.md b/skills/Testing & Security/requesting-code-review/SKILL.md deleted file mode 100644 index 2736583b..00000000 --- a/skills/Testing & Security/requesting-code-review/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: requesting-code-review -name: Requesting Code Review -description: Step-by-step guidance for requesting code review. -category: Testing & Security ---- - -# Requesting Code Review - -Support requesting code review workflows with clear steps and best practices. - -## When to Use - -- You need help with requesting code review tasks. -- You want a clear quality checklist. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/security-audit/SKILL.md b/skills/Testing & Security/security-audit/SKILL.md deleted file mode 100644 index d657fc1a..00000000 --- a/skills/Testing & Security/security-audit/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: security-audit -name: Security Audit -description: Step-by-step guidance for security audit. -category: Testing & Security ---- - -# Security Audit - -Support security audit workflows with clear steps and best practices. - -## When to Use - -- You need help with security audit. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/security-best-practices/SKILL.md b/skills/Testing & Security/security-best-practices/SKILL.md deleted file mode 100644 index ebc0cec0..00000000 --- a/skills/Testing & Security/security-best-practices/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: security-best-practices -name: Security Best Practices -description: Step-by-step guidance for security best practices. -category: Testing & Security ---- - -# Security Best Practices - -Support security best practices workflows with clear steps and best practices. - -## When to Use - -- You need help with security best practices. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/senior-secops/SKILL.md b/skills/Testing & Security/senior-secops/SKILL.md deleted file mode 100644 index 5110fd5f..00000000 --- a/skills/Testing & Security/senior-secops/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: senior-secops -name: Senior Secops -description: Step-by-step guidance for senior secops. -category: Testing & Security ---- - -# Senior Secops - -Support senior secops workflows with clear steps and best practices. - -## When to Use - -- You need help with senior secops. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/senior-security/SKILL.md b/skills/Testing & Security/senior-security/SKILL.md deleted file mode 100644 index 17c88dff..00000000 --- a/skills/Testing & Security/senior-security/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: senior-security -name: Senior Security -description: Step-by-step guidance for senior security. -category: Testing & Security ---- - -# Senior Security - -Support senior security workflows with clear steps and best practices. - -## When to Use - -- You need help with senior security. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/setup-web-tests/SKILL.md b/skills/Testing & Security/setup-web-tests/SKILL.md deleted file mode 100644 index 79fb5fbf..00000000 --- a/skills/Testing & Security/setup-web-tests/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: setup-web-tests -name: Setup Web Tests -description: Step-by-step guidance for setup web tests. -category: Testing & Security ---- - -# Setup Web Tests - -Support setup web tests workflows with clear steps and best practices. - -## When to Use - -- You need help with setup web tests. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/swift-protocol-di-testing/SKILL.md b/skills/Testing & Security/swift-protocol-di-testing/SKILL.md deleted file mode 100644 index 30629100..00000000 --- a/skills/Testing & Security/swift-protocol-di-testing/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: swift-protocol-di-testing -name: Swift Protocol Di Testing -description: Step-by-step guidance for swift protocol di testing. -category: Testing & Security ---- - -# Swift Protocol Di Testing - -Support swift protocol di testing workflows with clear steps and best practices. - -## When to Use - -- You need help with swift protocol di testing. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/syncable-entity-testing/SKILL.md b/skills/Testing & Security/syncable-entity-testing/SKILL.md deleted file mode 100644 index b5664251..00000000 --- a/skills/Testing & Security/syncable-entity-testing/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: syncable-entity-testing -name: Syncable Entity Testing -description: Step-by-step guidance for syncable entity testing. -category: Testing & Security ---- - -# Syncable Entity Testing - -Support syncable entity testing workflows with clear steps and best practices. - -## When to Use - -- You need help with syncable entity testing. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/temporal-python-testing/SKILL.md b/skills/Testing & Security/temporal-python-testing/SKILL.md deleted file mode 100644 index 831869c3..00000000 --- a/skills/Testing & Security/temporal-python-testing/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: temporal-python-testing -name: Temporal Python Testing -description: Step-by-step guidance for temporal python testing. -category: Testing & Security ---- - -# Temporal Python Testing - -Support temporal python testing workflows with clear steps and best practices. - -## When to Use - -- You need help with temporal python testing. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/test-driven-development/SKILL.md b/skills/Testing & Security/test-driven-development/SKILL.md deleted file mode 100644 index a7085507..00000000 --- a/skills/Testing & Security/test-driven-development/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: test-driven-development -name: Test Driven Development -description: Step-by-step guidance for test driven development. -category: Testing & Security ---- - -# Test Driven Development - -Support test driven development workflows with clear steps and best practices. - -## When to Use - -- You need help with test driven development tasks. -- You want a clear quality checklist. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/test-writer/SKILL.md b/skills/Testing & Security/test-writer/SKILL.md deleted file mode 100644 index 8475eebe..00000000 --- a/skills/Testing & Security/test-writer/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: test-writer -name: Test Writer -description: Step-by-step guidance for test writer. -category: Testing & Security ---- - -# Test Writer - -Support test writer workflows with clear steps and best practices. - -## When to Use - -- You need help with test writer. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/tidb-test-guidelines/SKILL.md b/skills/Testing & Security/tidb-test-guidelines/SKILL.md deleted file mode 100644 index 0ee72e8a..00000000 --- a/skills/Testing & Security/tidb-test-guidelines/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: tidb-test-guidelines -name: TiDB Test Guidelines -description: Step-by-step guidance for tidb test guidelines. -category: Testing & Security ---- - -# TiDB Test Guidelines - -Support tidb test guidelines workflows with clear steps and best practices. - -## When to Use - -- You need help with tidb test guidelines. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/ui-review/SKILL.md b/skills/Testing & Security/ui-review/SKILL.md deleted file mode 100644 index a1e2b2f3..00000000 --- a/skills/Testing & Security/ui-review/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: ui-review -name: UI Review -description: Step-by-step guidance for UI review. -category: Testing & Security ---- - -# UI Review - -Support ui review workflows with clear steps and best practices. - -## When to Use - -- You need help with ui review tasks. -- You want a clear quality checklist. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/validating-api-contracts/SKILL.md b/skills/Testing & Security/validating-api-contracts/SKILL.md index 4c03cdbe..f24763f3 100644 --- a/skills/Testing & Security/validating-api-contracts/SKILL.md +++ b/skills/Testing & Security/validating-api-contracts/SKILL.md @@ -1,55 +1,45 @@ --- id: validating-api-contracts name: Validating API Contracts -description: Validates API contracts using consumer-driven testing and OpenAPI validation. Leverages Pact for contract testing and OpenAPI for specification compliance. Use when generating contract tests, validating API responses, or checking backward compatibility. +description: Validate API contracts for compatibility, schema correctness, and breaking changes using specification-aware review principles. category: Testing & Security requires: [] examples: - - "Generate Pact contract tests for my API" - - "Validate this API against its OpenAPI spec" + - "Validate this API against its OpenAPI spec." + - "Check whether this API change is backward compatible." + - "Help me review this contract change between consumer and provider." --- -## Overview - -This skill enables Claude to generate and validate API contracts, ensuring compatibility between API providers and consumers. It uses Pact for consumer-driven contract testing and OpenAPI validation for specification compliance. - -## How It Works - -1. **Generating Contract Tests**: Claude creates Pact consumer tests based on API usage, generating provider verification tests and building OpenAPI contract validators. -2. **Validating Contracts**: The skill verifies if API responses match the defined contracts. -3. **Checking Compatibility**: It checks for backward compatibility to identify breaking changes in the API. - -## When to Use This Skill - -This skill activates when you need to: -- Generate contract tests for an API. -- Validate API responses against existing contracts. -- Identify breaking changes in an API. - -## Examples - -### Example 1: Generating Pact Contracts - -User request: "Generate contract tests for my API using Pact." - -The skill will: -1. Analyze the API and generate Pact consumer contracts. -2. Create provider verification tests based on the contracts. - -### Example 2: Validating an OpenAPI Specification - -User request: "Validate my API against the OpenAPI specification." - -The skill will: -1. Validate the API against the provided OpenAPI specification. -2. Report any discrepancies or violations of the specification. - -## Best Practices - -- **Clarity**: Be specific when requesting contract generation or validation, providing relevant API details. -- **Completeness**: Ensure that your OpenAPI specifications are up-to-date for accurate validation. -- **Context**: Provide context about the consumer and provider roles when using Pact. - -## Integration - -This skill can be integrated with other testing and deployment tools in the Claude Code ecosystem to automate contract verification as part of a CI/CD pipeline. +# Validating API Contracts + +Use this skill to review whether an API contract is clear, compatible, and safely evolving. + +## What To Compare +- Declared request and response schemas. +- Required versus optional fields. +- Status codes and error shapes. +- Enum, format, and nullability changes. +- Authentication, headers, and versioning expectations. + +## Review Questions +- Would an existing consumer break because of this change? +- Are the documented types and the real behavior aligned? +- Are there hidden assumptions around defaults, ordering, or missing fields? +- Does the error contract remain consistent enough for callers to handle safely? +- Is the compatibility story explicit for both consumers and providers? + +## Good Output +- Breaking versus non-breaking changes. +- Ambiguities or undocumented behavior. +- Gaps between implementation intent and documented contract. +- Suggested compatibility strategy or rollout notes. + +## Common Breaking Changes +- Making a previously optional field required. +- Changing data types or enum meanings. +- Removing status codes or changing error payload shape. +- Returning looser or stricter data than consumers rely on. + +## Boundaries +- Do not pretend to generate or run contract tests automatically. +- Focus on compatibility reasoning, schema review, and rollout guidance. diff --git a/skills/Testing & Security/verification-loop/SKILL.md b/skills/Testing & Security/verification-loop/SKILL.md deleted file mode 100644 index 92bd3e4a..00000000 --- a/skills/Testing & Security/verification-loop/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: verification-loop -name: Verification Loop -description: Step-by-step guidance for verification loop. -category: Testing & Security ---- - -# Verification Loop - -Support verification loop workflows with clear steps and best practices. - -## When to Use - -- You need help with verification loop tasks. -- You want a clear quality checklist. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/verify/SKILL.md b/skills/Testing & Security/verify/SKILL.md deleted file mode 100644 index c401e6d4..00000000 --- a/skills/Testing & Security/verify/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: verify -name: Verify -description: Step-by-step guidance for verify. -category: Testing & Security ---- - -# Verify - -Support verify workflows with clear steps and best practices. - -## When to Use - -- You need help with verify tasks. -- You want a clear quality checklist. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/vitest/SKILL.md b/skills/Testing & Security/vitest/SKILL.md deleted file mode 100644 index ab412936..00000000 --- a/skills/Testing & Security/vitest/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: vitest -name: Vitest -description: Step-by-step guidance for vitest. -category: Testing & Security ---- - -# Vitest - -Support vitest workflows with clear steps and best practices. - -## When to Use - -- You need help with vitest. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/vulnerability-scanner/SKILL.md b/skills/Testing & Security/vulnerability-scanner/SKILL.md index 4fc5a326..3a545ef8 100644 --- a/skills/Testing & Security/vulnerability-scanner/SKILL.md +++ b/skills/Testing & Security/vulnerability-scanner/SKILL.md @@ -1,20 +1,43 @@ --- id: vulnerability-scanner name: Vulnerability Scanner -description: Step-by-step guidance for vulnerability scanner. +description: Interpret vulnerability scanner results, prioritize findings, and separate exploitable risk from noisy advisory output. category: Testing & Security +requires: [] +examples: + - "Help me triage these vulnerability scanner results." + - "Which findings here matter most?" + - "Review this scan output and tell me what is urgent." --- # Vulnerability Scanner -Support vulnerability scanner workflows with clear steps and best practices. +Use this skill to interpret and prioritize vulnerability scanner output rather than blindly reacting to every finding. -## When to Use +## Start By Clarifying +- What was scanned: dependencies, containers, infrastructure, source code, or configuration. +- Whether the findings affect production, development-only paths, or transitive tooling. +- What exploitability information is available. +- Whether compensating controls or environment constraints change the risk. -- You need help with vulnerability scanner. -- You want a clear, actionable next step. +## Triage Priorities +- Distinguish exploitable issues from informational noise. +- Identify internet-facing, credential-related, or privilege-escalation risks first. +- Separate reachable code paths from theoretical package presence. +- Consider whether the finding is fixable now, needs mitigation, or can be tracked with rationale. -## Output +## Good Output +- Findings grouped by severity and exploitability. +- Notes on reachability, environment relevance, and urgency. +- Clear next actions: fix now, mitigate, monitor, or accept with justification. +- Questions to resolve before treating a result as urgent. -- Brief plan or checklist -- Key recommendations and caveats +## Common Mistakes +- Treating all high-severity findings as equally urgent. +- Ignoring whether vulnerable code is reachable in the actual deployment. +- Fixing scanner output mechanically without understanding regressions or compatibility impact. +- Closing findings without recording why they were accepted or deferred. + +## Boundaries +- Do not pretend to run scanners or fetch new results automatically. +- Focus on triage, prioritization, and remediation logic. diff --git a/skills/Testing & Security/web-renderer-test/SKILL.md b/skills/Testing & Security/web-renderer-test/SKILL.md deleted file mode 100644 index 9017ecd5..00000000 --- a/skills/Testing & Security/web-renderer-test/SKILL.md +++ /dev/null @@ -1,20 +0,0 @@ ---- -id: web-renderer-test -name: Web Renderer Test -description: Step-by-step guidance for web renderer test. -category: Testing & Security ---- - -# Web Renderer Test - -Support web renderer test workflows with clear steps and best practices. - -## When to Use - -- You need help with web renderer test. -- You want a clear, actionable next step. - -## Output - -- Brief plan or checklist -- Key recommendations and caveats diff --git a/skills/Testing & Security/write-unit-tests/SKILL.md b/skills/Testing & Security/write-unit-tests/SKILL.md index a39aacc7..ff2ff0de 100644 --- a/skills/Testing & Security/write-unit-tests/SKILL.md +++ b/skills/Testing & Security/write-unit-tests/SKILL.md @@ -1,20 +1,51 @@ --- id: write-unit-tests name: Write Unit Tests -description: Step-by-step guidance for write unit tests. +description: Design focused unit tests with clear intent, good isolation, and strong coverage of edge cases and failure paths. category: Testing & Security +requires: [] +examples: + - "Help me write unit tests for this function." + - "What unit tests are missing for this module?" + - "Review these unit tests for focus and coverage gaps." --- # Write Unit Tests -Support write unit tests workflows with clear steps and best practices. +Use this skill to design and review unit tests that are focused, readable, and useful as long-term regression protection. -## When to Use +## Start By Clarifying +- What unit is actually under test. +- Which inputs and branches matter most. +- What dependencies should be isolated versus exercised directly. +- What failure modes or edge cases are most likely. -- You need help with write unit tests. -- You want a clear, actionable next step. +## Good Unit Test Principles +- Test behavior and contract, not implementation trivia. +- Keep each test focused on one meaningful claim. +- Cover success paths, edge cases, and error paths. +- Use names that explain the scenario and expected behavior. +- Prefer simple setup over clever reusable test machinery. -## Output +## What To Include +- Happy-path examples that document expected usage. +- Edge cases that are easy to miss in manual reasoning. +- Failure or rejection cases when the unit validates input or state. +- Assertions that prove the intended behavior without over-asserting on internals. -- Brief plan or checklist -- Key recommendations and caveats +## Isolation Guidance +- Stub or fake unstable dependencies when they distract from the unit's behavior. +- Control time, randomness, environment, and network behavior when relevant. +- Be explicit about what the test is intentionally not covering. + +## Good Output +- A test plan by scenario. +- Missing cases worth adding. +- Simplifications to make the tests easier to trust and maintain. +- Warnings about brittle or overly coupled assertions. + +## Common Mistakes +- Testing private implementation details. +- Writing one oversized test that hides multiple concerns. +- Mocking so much that the test proves almost nothing. +- Ignoring error paths and weird inputs because the happy path passes. diff --git a/skills/Testing & Security/writing-bundler-tests/SKILL.md b/skills/Testing & Security/writing-bundler-tests/SKILL.md index a525ab5b..2ad57d13 100644 --- a/skills/Testing & Security/writing-bundler-tests/SKILL.md +++ b/skills/Testing & Security/writing-bundler-tests/SKILL.md @@ -1,20 +1,46 @@ --- id: writing-bundler-tests name: Writing Bundler Tests -description: Step-by-step guidance for writing bundler tests. +description: Test bundler behavior such as resolution, transforms, chunking, sourcemaps, plugin hooks, and build correctness. category: Testing & Security +requires: [] +examples: + - "Help me write tests for this bundler behavior." + - "What should I test for this plugin or transform?" + - "Review this bundler test plan for missing coverage." --- # Writing Bundler Tests -Support writing bundler tests workflows with clear steps and best practices. +Use this skill to design tests for bundlers, transforms, and build pipelines where output correctness matters as much as raw success. -## When to Use +## Clarify First +- What layer is changing: module resolution, transforms, plugin hooks, chunking, assets, or sourcemaps. +- Whether the main risk is correctness, compatibility, performance, or regressions in output shape. +- Which outputs are stable enough to assert on. +- What behavior differs across modes such as dev versus production. -- You need help with writing bundler tests. -- You want a clear, actionable next step. +## Good Bundler Test Areas +- Resolution behavior and aliasing. +- Transform correctness for representative inputs. +- Output chunking or asset emission when it materially matters. +- Sourcemap fidelity and debugging expectations. +- Plugin lifecycle behavior and ordering-sensitive hooks. -## Output +## Testing Principles +- Assert on meaningful build behavior, not incidental formatting noise. +- Prefer representative fixtures over giant synthetic projects. +- Keep mode, environment, and platform assumptions visible. +- Separate fast fixture tests from slower end-to-end build verification. -- Brief plan or checklist -- Key recommendations and caveats +## Good Output +- A fixture-based test plan. +- Critical assertions by bundler behavior. +- Guidance on what to snapshot versus what to assert structurally. +- Risks around portability, caching, or plugin interactions. + +## Common Mistakes +- Snapshotting huge outputs without saying what matters. +- Testing implementation hooks instead of emitted behavior. +- Ignoring cross-mode differences until regressions hit users. +- Using fixtures so unrealistic that the tests miss real plugin conflicts. diff --git a/skills/Testing & Security/writing-dev-server-tests/SKILL.md b/skills/Testing & Security/writing-dev-server-tests/SKILL.md index e5e34e78..30a39c81 100644 --- a/skills/Testing & Security/writing-dev-server-tests/SKILL.md +++ b/skills/Testing & Security/writing-dev-server-tests/SKILL.md @@ -1,20 +1,46 @@ --- id: writing-dev-server-tests name: Writing Dev Server Tests -description: Step-by-step guidance for writing dev server tests. +description: Test development server behavior such as watch mode, reloads, proxying, middleware, and local feedback loops. category: Testing & Security +requires: [] +examples: + - "Help me test this dev server behavior." + - "What should I verify for watch mode and reload logic?" + - "Review this dev-server test plan for gaps." --- # Writing Dev Server Tests -Support writing dev server tests workflows with clear steps and best practices. +Use this skill to design tests for development-server behavior rather than generic application functionality. -## When to Use +## What To Clarify +- Which server behavior matters: startup, watch mode, reload, HMR, proxying, middleware, error overlays, or static asset serving. +- What counts as a successful developer experience outcome. +- Which parts are deterministic enough for automated tests. +- What should be simulated versus exercised end to end. -- You need help with writing dev server tests. -- You want a clear, actionable next step. +## Good Dev-Server Test Areas +- Server startup and teardown. +- File-change detection and rebuild triggers. +- Reload or HMR behavior after meaningful edits. +- Middleware and proxy behavior under common local scenarios. +- Error reporting that helps developers recover quickly. -## Output +## Testing Principles +- Prefer behavior visible to a developer over implementation details. +- Keep filesystem and timing assumptions explicit. +- Separate fast behavioral checks from slower integration-style tests. +- Control flakiness around file watching, ports, and process timing. -- Brief plan or checklist -- Key recommendations and caveats +## Good Output +- Test scenarios grouped by server capability. +- Risks around timing, nondeterminism, or environment sensitivity. +- Advice on what should be unit-tested versus integration-tested. +- Expected developer-facing outcomes for each scenario. + +## Common Mistakes +- Over-testing internal plugin hooks instead of real behavior. +- Relying on arbitrary sleeps rather than clear readiness conditions. +- Ignoring cleanup and leaving tests order-dependent. +- Treating watch-mode flakiness as unavoidable instead of designing around it.