Security Review Checklist (security_review axis)
This checklist backs the security_review axis of the production verification block.
It is the lightweight, skill-specific review required by
production-verification.md: a value of passed
is an AND claim across the five security-relevant dimensions of the axis
(input/output, credential, dependency, path, data-exposure).
Rules
passedrequires that every item in each of the five dimensions below is satisfied by skill-specific, auditable evidence. A green repository-wide scan (detect-secrets, supply-chain audit, CIsecurityjob) is not by itself sufficient (production-verification.mdstates this explicitly).- Mechanical checks in
scripts/backfill_security_review.pyare advisory signals. They cannot replace a recorded per-skill review. A skill is only apassedcandidate when it is clean on the mechanical checks and is explicitly reviewer-confirmed (the collector’s confirmation flag starts empty). - Every
passedrecord must name the reviewer, the assessment date, the assessed commit, and the exact files reviewed. These are recorded in the baseline overriderationaleandevidencefields (seescripts/tests/test_production_verification_baseline.pyfor the accepted shape). - A skill that fails any item, or whose applicability cannot be cleared, stays
not_verifiedwith the specific gap recorded in the baseline overriderationale.
Dimensions and items
1. Credential
- API keys, secrets, and tokens are read from environment variables (for
example
FMP_API_KEY,FINVIZ_API_KEY,ALPACA_API_KEY,ALPACA_SECRET_KEY) or command-line flags, never hard-coded as literal values in committed files. - Placeholder strings such as
export FMP_API_KEY='your-key'in help text are acceptable; a non-placeholder committed value is a fail. - Secrets are excluded from committed repositories (
.gitignore,.mcp.json,.envrc); no personal information or absolute user paths are committed.
2. Data exposure
- The skill does not print or log a resolved credential value to stdout, stderr, any log file, or any output artifact.
- Diagnostics that mention a credential reuse the shared redaction helpers
(for example the
_redact_keyfragment in the generated FMP client) so a leaked value is masked rather than emitted verbatim.
3. Path
- The skill does not invoke a shell (
shell=True,os.system, or a string command list) with attacker- or input-influenced arguments. - Subprocess calls and file paths derived from external input are validated or constrained before use.
4. Input/output and network
- Network calls set an explicit timeout; no call can hang indefinitely on an unresponsive provider.
- Non-2xx responses, empty/missing fields, and malformed payloads are handled fail-closed (rejected or surfaced as an error, never silently treated as a valid zero/success).
5. Dependency
- The skill ships a
skills/<id>/requirements.txtthat passespython3 scripts/check_skill_deps.py check(issue #330): no undeclared imports, no stale entries, and no optional entry without a fallback path plus absence test. - The declared dependencies are consistent with the repository supply-chain
audit (
docs/dev/supply-chain-security.md).
Recording evidence
A security_review: passed entry is added as a per-skill override in
docs/dev/production-verification-baseline.yaml (and mirrored in the skill’s
verification.security_review value in skills-index.yaml). The override
carries:
- a
rationaledescribing, per skill, what was reviewed and the fail-closed / credential / timeout / path facts that were verified; - an
evidencelist with non-empty entries such as the review date, the assessed commit, the reviewer, and the exact file or command inspected.
Keep evidence current as in production-verification.md “Keeping evidence
current”: when a skill’s credential handling, network code, dependencies, or
paths change, re-run this checklist and update the record, or reset the axis to
not_verified.