How organizations may submit systems for independent evaluation.
The Pilot Validation Program is the mechanism by which the SFR framework moves from proposed standard to independently-tested standard. Organizations that operate simulation systems may volunteer those systems for structured evaluation against the SFR classification criteria. The result is an evidence record — not a certificate, not an endorsement, and not a score.
The Pilot Validation Program is a structured mechanism for applying the SFR classification criteria to a real simulation system in a documented, reproducible manner. An organization submits a system. The evaluation is conducted using the Reference Test Methodology, drawing on the evidence required by Evaluation Inputs, and producing a determination under the Evaluation Process. The result is an evaluation record entered in the Results Registry.
Every evaluation conducted through the program tests the framework as much as it tests the system. Where two evaluators apply the same methodology to the same evidence and reach different conclusions, the discrepancy identifies a gap in the framework's clarity that must be resolved before the framework can advance. The program produces evidence about simulation systems and evidence about the framework simultaneously.
The framework cannot advance from Stage 1 (Community Review) to Stage 2 (Independent Evaluation) without at least one organization having submitted a system through this program. Pilot program participation is the concrete action that triggers Stage 2. See Stage 2 Readiness for the full advancement conditions.
The program produces evidence records. Evidence records are a public good for the standards development process. They are not commercial products.
Any organization that operates a simulation system may submit it for evaluation. There are no eligibility restrictions based on organization type, system type, or expected classification outcome. The program accepts systems across all three classification tiers — including systems that the submitting organization expects to classify as Surface-Level or Out-of-the-Loop.
Simulator manufacturers. Training institutions. University research programs. Motorsport organizations. Aviation training operators. Rehabilitation programs. Military training facilities. Any organization that operates or owns a simulation system and is willing to support an evidence-based evaluation.
Any simulation system across any classification tier. There is no requirement that the system be expected to classify as In-the-Loop. Surface-Level and Out-of-the-Loop evaluations contribute equally to the inter-evaluator agreement record and the methodology validation process.
The submitting organization must be able to support the evidence collection process defined in the Evaluation Inputs document. At minimum, Tier 2 architecture documentation should be available. Tier 1 telemetry is strongly preferred and required for an Insufficient Data determination to be avoided on any criterion.
For Stage 2 purposes, each system must be evaluated by at least two evaluators who were not involved in either the system's design or the SFR framework's authorship. Single-evaluator evaluations are accepted in the pilot phase but do not satisfy the inter-evaluator agreement requirement for Stage 2 advancement.
The evaluation follows the normative process defined in the SFR evaluation methodology documents. The following steps describe how the program applies those documents.
The submitting organization contacts the framework maintainers through the contact channel on this site, identifying the system and the organization's primary contact for the evaluation. No documentation is required at this stage — just the system identifier and organization name.
The submitting organization provides the Tier 2 documentation set: physics model specification, actuator specification, synchronization architecture, and reference point documentation. The Procurement Guidance document lists the exact documents required. This documentation is reviewed before scheduling an evaluation session.
Two independent evaluators are assigned. Independence is confirmed: neither evaluator may have been involved in the design of the system being evaluated, and neither may be an author of the SFR normative corpus. Evaluators apply the framework from the published documentation without guidance from the framework authors during the evaluation.
Evaluators apply the Reference Test Methodology in a structured test session. The test session must include the required inputs defined in Evaluation Inputs at minimum. Telemetry capture during the session is required for Tier 1 evidence. The session is conducted with both evaluators present, making independent assessments before comparing results.
Each evaluator completes an independent Evaluation Record for the system. Records are completed without discussion between evaluators after the test session ends. Determinations for each criterion (A, B, C) are recorded individually before the final classification is derived.
The two evaluation records are compared. Where determinations agree across all criteria, the classification is confirmed and entered in the Results Registry. Where determinations disagree on any criterion, the disagreement is documented in the Inter-Evaluator Agreement record and the framework authors review the discrepancy to identify methodological gaps.
Upon completion of an evaluation, the submitting organization receives:
Participation in the Pilot Validation Program contributes to the evidence infrastructure that advances the SFR framework toward formal standard status. The specific contributions are:
Every evaluation tests the methodology's repeatability and identifies gaps in criterion clarity. Evaluations that produce inter-evaluator agreement confirm that the methodology works as intended. Evaluations that produce disagreement identify exactly where it does not — which is the more valuable result for advancing the framework.
A completed evaluation produces a structured, evidence-based classification record. An organization that can reference an independently-conducted SFR evaluation in a procurement response, research publication, or training program specification is providing verifiable information about the system's structural properties — not a marketing claim.
The public Results Registry creates a growing body of classified systems. Organizations that reference SFR in procurement decisions benefit from a registry of evaluated systems that demonstrates the framework's practical applicability. Researchers benefit from classified study subjects they can reference with confidence. Each entry in the registry makes the framework more useful for everyone who uses it.
The framework cannot advance through assertion. It advances through use. Pilot program participation is use.
The Pilot Validation Program is not a commercial offering and not a certification path. It is the mechanism through which the SFR framework accumulates the independent evaluation record that standards advancement requires. The program is open to any organization willing to support a structured, evidence-based evaluation of their system — regardless of what classification that evaluation is expected to produce.