Traceable data
Sources, dates, transformations, missingness, hashes, and frozen vintages remain attached to the result.
Research method
How each answer earns its confidence
Each investigation specifies the question, data, validation tests, uncertainty, failure criteria, and reproducible artifacts before a conclusion is accepted.
Short answer
Validation compares a model with pre-specified theoretical and empirical criteria; every conclusion remains conditional on the test design.
Purpose
Determine whether a result is reliable enough to inform a scoped allocation, risk, pricing, or governance decision.
Evidence required
Theoretical anchors, untouched data, uncertainty estimates, failure criteria, versioned artifacts, and reproducible execution.
Result
A supported, rejected, or revised claim with the reason and decision consequence stated explicitly.
Boundary
Passing validation does not prove that the model is the market or that future data will resemble the test.
Validation protocol
Five stages from question to published result
- 01
Frame a falsifiable question
Name the decision, state the claim, and define in advance what evidence would count against it. “Explore the data” is not a falsification criterion.
- 02
Engineer the evidence
Document selection, dates, transformations, missingness, hashes, and provenance. Separate refreshing data from reproducing an existing experiment.
- 03
Implement against anchors
Test numerical code against identities, limiting cases, convergence behavior, and independent methods—not merely stored fixtures.
- 04
Test the model
Use chronological holdouts, calibration tests, stress scenarios, uncertainty intervals, and multiplicity controls appropriate to the claim.
- 05
Publish the boundary
Report the result, uncertainty, failure modes, data vintage, source commit, and what decision changes. Preserve rejections and revisions.
Evidence standards
Three conditions for an inspectable result
Testable models
Theoretical identities, limiting cases, convergence behavior, holdouts, calibration, and stress tests define what can fail.
Bounded conclusions
Uncertainty, limitations, rejected claims, revisions, and the affected decision are reported together.
Reusable resources
Three checklists for analytical work
Model-validation checklist
Use before approving a model for decision support.
- Is the decision and model owner named?
- Are assumptions explicit and testable?
- Is there an independent theoretical or numerical anchor?
- Is leakage prevented by construction?
- Are calibration and residual behavior tested?
- Is uncertainty attached to the reported result?
- Are stress conditions and boundary cases included?
- Are failure criteria declared before comparison?
- Are limitations written in decision language?
- Is monitoring or revalidation defined?
Portfolio stress-test checklist
Use before accepting an allocation because it looks diversified.
- Stress expected returns and estimation error separately.
- Challenge correlation, not only asset volatility.
- Measure concentration by weights and risk contribution.
- Include liquidity, turnover, and transaction costs.
- Test chronological out-of-sample behavior.
- Replay dated crises without treating them as bounds.
- Include regime shifts and joint downside moves.
- Compare simple baselines to optimized portfolios.
- Report drawdown and recovery, not only volatility.
- State which risks remain outside the scenario set.
Research-reproducibility checklist
Use before calling an analytical result reproducible.
- Pin code to a commit or release.
- Record data source, vintage, transformations, and hash.
- Separate data refresh from experiment reproduction.
- Declare seeds and random-generator family.
- Document runtime and dependency requirements.
- Preserve executed artifacts and machine-readable outputs.
- Test a clean-clone reproduction path.
- Publish uncertainty and numerical error budgets.
- Link prose claims to the generating artifact.
- Keep rejected and revised claims discoverable.
Inspect the record
Go from principle to implementation
Full audit trail
The research record preserves how each conclusion changed.
Questions, rejected claims, revisions, data versions, source commits, notebooks, and known limitations remain available for inspection.