Thesis/4-metrics-of-success/v1.tex

\section{Metrics for Success}

\textbf{Heilmeier Question: How will success be measured?}

Section 3 established the technical approach: compositional verification bridges discrete synthesis with continuous control. It will succeed because it leverages existing procedural structure, bounds computational complexity, and validates against industrial hardware. 

This section addresses the next Heilmeier question: How will success be measured? 

Success is measured by Technology Readiness Level advancement from fundamental concepts (TRL 2--3) to validated prototype demonstration (TRL 5). At TRL 5, system components operate successfully in a relevant laboratory environment. 

TRL advancement provides the most appropriate success metric. It explicitly measures the gap between academic proof-of-concept and practical deployment. This section explains why TRLs appropriately measure success. It then defines specific criteria for each level from TRL 3 through TRL 5.

Technology Readiness Levels provide the ideal success metric for work bridging academic proof-of-concept and practical deployment. 

Academic metrics—papers published or theorems proved—fail to capture practical feasibility. Empirical metrics—simulation accuracy or computational speed—fail to demonstrate theoretical rigor. TRLs measure both simultaneously.

Advancing from TRL 3 to TRL 5 requires maintaining theoretical rigor while progressively demonstrating practical feasibility. The system moves from individual components to integrated hardware testing. Two requirements constrain this progression. First: formal verification must remain valid throughout. Second: the proofs must compose as the system scales.

The nuclear industry requires extremely high assurance before deploying new
control technologies. Demonstrating theoretical correctness alone proves
insufficient for adoption; conversely, showing empirical performance without
formal guarantees fails to meet regulatory requirements. TRLs capture this dual
requirement naturally. Each level represents both increased practical maturity
and sustained theoretical validity, while TRL assessment forces explicit
identification of remaining barriers to deployment. The nuclear industry already
uses TRLs for technology assessment, making this metric directly relevant to
potential adopters. Reaching TRL 5 provides a clear answer to industry questions
about feasibility and maturity that academic publications alone cannot.

Moving from current state to target requires achieving three intermediate
levels, each representing a distinct validation milestone:

\paragraph{TRL 3 \textit{Critical Function and Proof of Concept}}

For this research, TRL 3 means demonstrating that each component of the
methodology works in isolation. Startup procedures must be translated into
temporal logic specifications that pass realizability analysis. A discrete
automaton must be synthesized with interpretable structure. At least one
continuous controller must be designed with reachability analysis proving
transition requirements are satisfied. Independent review must confirm that
specifications match intended procedural behavior. This proves the fundamental
approach on a simplified startup sequence.

\paragraph{TRL 4 \textit{Laboratory Testing of Integrated Components}}

For this research, TRL 4 means demonstrating a complete integrated hybrid
controller in simulation. All startup procedures must be formalized with a
synthesized automaton covering all operational modes. Continuous controllers
must exist for all discrete modes. Verification must be complete for all mode
transitions using reachability analysis, barrier certificates, and
assume-guarantee contracts. The integrated controller must execute complete
startup sequences in software simulation with zero safety violations across
multiple consecutive runs. This proves that formal correctness guarantees can be
maintained throughout system integration.

\paragraph{TRL 5 \textit{Laboratory Testing in Relevant Environment}}

For this research, TRL 5 means demonstrating the verified controller on
industrial control hardware through hardware-in-the-loop testing. The discrete
automaton must be implemented on the Emerson Ovation control system and verified
to match synthesized specifications exactly. Continuous controllers must execute
at required rates. The ARCADE interface must establish stable real-time
communication between the Emerson Ovation hardware and SmAHTR simulation.
Complete autonomous startup sequences must execute via hardware-in-the-loop
across the full operational envelope. The controller must handle off-nominal
scenarios to validate that expulsory modes function correctly. For example,
simulated sensor failures must trigger appropriate fault detection and mode
transitions, and loss-of-cooling scenarios must activate SCRAM procedures as
specified. Graded responses to minor disturbances are outside this work's scope.
Formal verification results must remain valid, with discrete behavior matching
specifications and continuous trajectories remaining within verified bounds.
This proves that the methodology produces verified controllers implementable on
industrial hardware.

Progress will be assessed quarterly through collection of specific data
comparing actual results against TRL advancement criteria. Specification
development status indicates progress toward TRL 3. Synthesis results and
verification coverage indicate progress toward TRL 4. Simulation performance
metrics and hardware integration milestones indicate progress toward TRL 5. The
research plan will be revised only when new data invalidates fundamental
assumptions. This research succeeds by achieving TRL 5: demonstrating a
complete autonomous hybrid controller with formal correctness guarantees
operating on industrial control hardware through hardware-in-the-loop testing in
a relevant laboratory environment. This establishes both theoretical validity
and practical feasibility, proving the methodology produces verified
controllers implementable with current technology.

This section answered the Heilmeier question: How will success be measured?

\textbf{Answer:} Technology Readiness Level advancement from 2--3 to 5. Each level demonstrates both theoretical correctness and practical feasibility through progressively integrated validation.

TRL 3 proves component-level correctness. Each methodology element works independently.

TRL 4 demonstrates system-level integration in simulation. Components compose correctly.

TRL 5 validates hardware implementation in a relevant environment. The complete system operates on industrial control hardware.

Achieving TRL 5 proves the methodology produces verified controllers implementable with current technology. The result is not merely theoretically sound but practically deployable.

Sections 2 through 4 addressed five Heilmeier questions. Section 2 established what has been done and what limits current practice. Section 3 explained what is new and why it will succeed. This section defined how to measure success.

Success assumes critical technical challenges can be overcome. Section 5 addresses what could prevent success. It explains how to respond when assumptions fail.