Transparent documentation of unresolved issues and pending questions.
This page publicly documents what the framework does not yet resolve, what questions remain open, what research is still needed, and where the current version has known limitations. Transparency about what the framework does not yet do well is as important as clarity about what it does. A proposed standard that claims to have no open questions has not been examined carefully enough.
These are specific issues in the evaluation methodology where the current framework documentation is incomplete, ambiguous, or known to produce inconsistent results in edge cases.
The evaluation methodology references telemetry capture and synchronization requirements but does not specify minimum acceptable sample rates for physics output, actuator feedback, or synchronization channel data. Without this, two evaluators assessing the same telemetry dataset may apply different quality thresholds and reach different Tier 1 evidence determinations. A supplementary technical specification is required before the evaluation methodology can claim full implementability.
Prerequisite for Stage 2The framework requires "independent" evaluators but does not define what qualifications, backgrounds, or competencies an evaluator must have to conduct a valid SFR evaluation. Without qualification criteria, the independence requirement has no enforceable meaning. An evaluator pool cannot be established without knowing what constitutes a qualified evaluator. This gap must be resolved before the Pilot Validation Program can operate with confidence.
Prerequisite for Stage 2Criterion B (Temporal Coherence) Pass and Fail conditions are defined qualitatively. In cases where synchronization latency is borderline — not clearly within or outside threshold — the current framework provides insufficient guidance to produce a consistent determination across two independent evaluators. The threshold values required to resolve this gap depend on instrumentation data that can only be collected through the Pilot Validation Program. The gap cannot be resolved before Stage 2 begins; it should be resolved through Stage 2 data.
Deferred to Stage 2The Governance Framework describes the custodianship model but does not identify a named institution or organization as the framework's governance authority. Without a named custodian, the framework cannot progress to formal ratification, the revision policy has no authoritative body to administer it, and the community review process has no defined decision-maker for Accepted or Declined dispositions. Identifying a custodian is the highest-priority governance gap.
Prerequisite for Stage 4These are research questions that the framework depends on but cannot resolve internally. They require empirical investigation by independent researchers working with classified systems.
The framework's classification tiers predict differential transfer of training — specifically, that In-the-Loop training produces meaningfully different transfer outcomes than Surface-Level training for high-stakes performance tasks. This prediction is grounded in neuroscience and physics, but has not been empirically validated in controlled studies using the SFR classification framework as the independent variable. Pilot studies using classified systems are needed to produce the first empirical evidence for or against this prediction.
Open Research QuestionCriterion C (Human Response Relevance) is defined in terms of whether delivered motion is within the relevant range of human vestibular system response. The framework does not specify numerical thresholds for what constitutes relevance. Establishing these thresholds requires biomarker studies in which participants are exposed to classified systems and vestibular response is measured. The Neurological Reserve and Compensation Demand framework proposes several candidate measurement dimensions.
Open Research QuestionThe Medical Risk Framework documents the hypothesis that Surface-Level simulation may present elevated risk for participants with compromised vestibular systems. The mechanistic pathway is described but the clinical outcomes have not been measured. Prospective studies with defined participant populations, classified simulation systems, and standardized outcome measures are needed to determine whether the hypothesized risk is clinically significant.
Open Research QuestionThe framework has not specified a minimum inter-evaluator agreement rate that would constitute validation of the evaluation methodology. Determining what agreement rate is sufficient requires both a theoretical standard (what does "acceptable repeatability" mean for a classification standard of this type?) and empirical data from the Pilot Validation Program. Both the threshold and the data are currently absent.
Prerequisite for Stage 2 CompletionThese are areas where the framework, by design or by current state, is not able to address the full complexity of the problem domain. These are not gaps to be filled — they are honest constraints to be acknowledged.
The SFR framework classifies simulation systems based on the physical fidelity of motion delivery — specifically, whether motion is derived from real-time physics computation and delivered through independent, causatively accurate axes. It does not evaluate visual fidelity, auditory fidelity, haptic fidelity beyond motion, or cognitive fidelity. A system could classify as In-the-Loop under SFR while having degraded visual rendering. The framework's scope is intentional and bounded — it addresses the dimension most consequential for neurophysiological training validity, but it is not a comprehensive simulation quality standard.
The three-tier classification system (In-the-Loop, Surface-Level, Out-of-the-Loop) treats each tier as a discrete category. Two systems that both classify as Surface-Level may differ substantially in their physics fidelity, motion range, or actuator accuracy — yet the framework assigns them the same classification. This is a deliberate design choice that prioritizes structural clarity over continuous scoring, but it limits the framework's ability to distinguish among systems within the same tier. Users who need finer discrimination within a tier should treat SFR classification as a necessary but not sufficient characterization of system quality.
The evidence hierarchy places manufacturer claims at Tier 4 — the lowest tier. The evaluation process document notes that Tier 4 evidence alone is generally insufficient to produce a Pass determination on any criterion. This means that systems for which only manufacturer documentation is available will frequently receive Insufficient Data determinations, even if the underlying system architecture would satisfy all three criteria. Manufacturers that want their systems to be classifiable under SFR must be prepared to support Tier 1 or Tier 2 evidence collection. This is a feature, not a flaw — but it means the framework has limited utility for systems with closed architectures.
These are questions that have been identified but not yet resolved — either because the resolution depends on community review feedback, on Stage 2 data, or on a governance decision that has not yet been made.
The framework defines Stage 2 advancement conditions (see Framework Validation) but does not specify exactly how many organizations must provide feedback, what proportion of feedback must be positive, or what quality of feedback is required. "Substantive engagement from relevant communities" is the stated condition — but substantive and relevant are not further defined. This question should be answered through community review itself: if the feedback record is thin and unrepresentative, Stage 2 advancement would be premature regardless of any numerical threshold.
Open — Community Review DependentThe citation guidelines and procurement guidance address how to reference SFR in research and procurement documents. They do not address how SFR classification should be handled in regulatory submissions, compliance documentation, or jurisdictional training standards. As national and international aviation, military, and medical training standards increasingly reference simulation fidelity, the framework needs guidance on how SFR classification interacts with existing regulatory frameworks. This guidance cannot be written until the framework has been reviewed by organizations operating within those regulatory contexts.
Open — Requires Regulatory EngagementThe Reference Test Methodology was developed primarily in the context of vehicle simulation (automotive, motorsport, aviation). The classification criteria are stated as domain-neutral, but the reference motion events (Steady-State Lateral, Transient Yaw Onset, Combined Axis, Limit-State Threshold) are vehicle-centric. Whether the methodology can be applied without modification to rehabilitation simulators, agricultural equipment simulators, or other non-vehicle domains has not been evaluated. Community feedback from non-vehicle simulation practitioners is particularly needed to resolve this question.
Open — Community Review DependentThis list is not exhaustive. Community review is the mechanism by which additional known issues are identified and added to this log. If you are aware of an issue not documented here, submit it as feedback.
A proposed standard that cannot identify its own limitations has not been examined with sufficient rigor. The issues documented on this page are not disqualifying — they are the honest inventory of what remains to be resolved before SFR v0.9 Draft becomes SFR v1.0 Formal Standard. Each item here represents either a clear prerequisite for advancement or a question that community review and validation program participation will help answer. Documenting them publicly is part of the framework's commitment to transparency.