Signal Postmortem

Record and analyze post-trade outcomes for signals generated by edge pipeline and other skills. Track false positives, missed opportunities, and regime mismatches. Feed results back to edge-signal-aggregator weights and skill improvement backlog.

FMP Optional

View Source on GitHub

Table of Contents

1. Overview

Signal Postmortem records and analyzes the outcomes of trading signals generated by the edge pipeline, screeners, and other skills. It compares predicted edge direction against 5-day and 20-day realized returns, categorizes outcomes (true positive, false positive, missed opportunity, regime mismatch), and generates feedback for edge-signal-aggregator weight adjustments and skill improvement backlog entries.

2. When to Use

  • After a trade has been closed and you want to record the outcome
  • When reviewing a batch of signals that have reached their holding period (5 or 20 days)
  • To identify systematic false positive patterns from specific skills
  • To generate feedback for edge-signal-aggregator weight calibration
  • When building a skill improvement backlog from decision quality metrics
  • For periodic (weekly/monthly) signal quality audits

3. Prerequisites

  • Python 3.9+
  • FMP API key (optional, for fetching realized returns if not provided manually)
  • Standard library + requests for API calls
  • Input: signal records in JSON format (from edge-signal-aggregator or screener outputs)

4. Quick Start 

# Example: List signals ready for postmortem (5+ days old)
python3 skills/signal-postmortem/scripts/postmortem_recorder.py \
  --list-ready \
  --signals-dir state/signals/ \
  --min-days 5

5. Workflow

Step 1: Prepare Signal Records

Gather closed or matured signal records. Each record should include:

  • signal_id: Unique identifier
  • ticker: Stock symbol
  • signal_date: Date signal was generated
  • predicted_direction: LONG or SHORT
  • source_skill: Which skill generated the signal
  • entry_price: Price at signal generation (optional, for manual override)
# Example: List signals ready for postmortem (5+ days old)
python3 skills/signal-postmortem/scripts/postmortem_recorder.py \
  --list-ready \
  --signals-dir state/signals/ \
  --min-days 5

Step 2: Record Outcomes

Run the postmortem recorder to fetch realized returns and classify outcomes.

python3 skills/signal-postmortem/scripts/postmortem_recorder.py \
  --signals-file state/signals/aggregated_signals_2026-03-10.json \
  --holding-periods 5,20 \
  --output-dir reports/

For manual outcome recording (when price data is already available):

python3 skills/signal-postmortem/scripts/postmortem_recorder.py \
  --signal-id sig_aapl_20260310_abc \
  --exit-price 178.50 \
  --exit-date 2026-03-15 \
  --outcome-notes "Closed at target, +3.2% in 5 days" \
  --output-dir reports/

Step 3: Classify Outcomes

The recorder automatically classifies each signal into one of four categories:

Category Definition
TRUE_POSITIVE Predicted direction matched realized return sign
FALSE_POSITIVE Predicted direction opposite to realized return
MISSED_OPPORTUNITY Signal not taken but would have been profitable
REGIME_MISMATCH Signal failed due to market regime change

Classification rules are documented in references/outcome-classification.md.

Step 4: Generate Feedback Files

Generate feedback for downstream consumers:

# Generate weight adjustment suggestions for edge-signal-aggregator
python3 skills/signal-postmortem/scripts/postmortem_analyzer.py \
  --postmortems-dir reports/postmortems/ \
  --generate-weight-feedback \
  --output-dir reports/

# Generate skill improvement backlog entries
python3 skills/signal-postmortem/scripts/postmortem_analyzer.py \
  --postmortems-dir reports/postmortems/ \
  --generate-improvement-backlog \
  --output-dir reports/

Step 5: Review Summary Statistics

Generate aggregate statistics by skill, by ticker, and by time period:

python3 skills/signal-postmortem/scripts/postmortem_analyzer.py \
  --postmortems-dir reports/postmortems/ \
  --summary \
  --group-by skill,month \
  --output-dir reports/

6. Resources

References:

  • skills/signal-postmortem/references/feedback-integration.md
  • skills/signal-postmortem/references/outcome-classification.md

Scripts:

  • skills/signal-postmortem/scripts/postmortem_analyzer.py
  • skills/signal-postmortem/scripts/postmortem_recorder.py