Thesis/4-metrics-of-success/metrics.tex

\section{Metrics for Success}

This research will be measured by advancement through Technology Readiness
Levels, progressing from fundamental concepts to validated prototype
demonstration. This work begins at TRL 2--3 and aims to reach TRL 5, where
system components operate successfully in a relevant laboratory
environment.
This section explains why TRL advancement provides the most appropriate
success metric and defines the specific criteria required to achieve TRL 5.

\oldt{Technology Readiness Levels provide the ideal success metric because
they explicitly measure the gap between academic proof-of-concept and
practical deployment---precisely what this work aims to bridge. Academic
metrics like papers published or theorems proved cannot capture practical
feasibility. Empirical metrics like simulation accuracy or computational
speed cannot demonstrate theoretical rigor. TRLs measure both dimensions
simultaneously.} \newt{TRLs measure the gap between academic
proof-of-concept and practical deployment, which is precisely what this work
aims to bridge. Academic metrics alone cannot capture practical feasibility,
and empirical metrics alone cannot demonstrate theoretical rigor. TRLs
measure both simultaneously.}Advancing from TRL 3
to TRL 5 requires maintaining theoretical rigor while progressively
demonstrating practical feasibility. Formal verification must remain valid as
the system moves from individual components to integrated hardware testing.

The nuclear industry requires extremely high assurance before deploying new
control technologies. Demonstrating theoretical correctness alone is
insufficient for adoption; conversely, showing empirical performance without
formal guarantees fails to meet regulatory requirements. TRLs capture this
dual requirement naturally. Each level represents both increased practical
maturity and sustained theoretical validity. Furthermore, TRL assessment
forces explicit identification of remaining barriers to deployment. The
nuclear industry already uses TRLs for technology assessment, making this
metric directly relevant to potential adopters. Reaching TRL 5 provides a
clear answer to industry questions about feasibility and maturity that
academic publications alone cannot.

Moving from current state to target requires achieving three intermediate
levels, each representing a distinct validation milestone:

\paragraph{TRL 3 \textit{Critical Function and Proof of Concept}}

For this research, TRL 3 means demonstrating that each component of the
methodology works in isolation. Startup procedures must be translated into
temporal logic specifications that pass realizability analysis. A discrete
automaton must be synthesized with interpretable structure. At least one
continuous controller must be designed with reachability analysis proving
transition requirements are satisfied. Independent review must confirm that
specifications match intended procedural behavior. This proves the
fundamental approach on a simplified startup sequence.

\paragraph{TRL 4 \textit{Laboratory Testing of Integrated Components}}

For this research, TRL 4 means demonstrating a complete integrated hybrid
controller in simulation. All startup procedures must be formalized with a
synthesized automaton covering all operational modes. Continuous controllers
must exist for all discrete modes. Verification must be complete for all mode
transitions using reachability analysis, barrier certificates, and
assume-guarantee contracts. The integrated controller must execute complete
startup sequences in software simulation with zero safety violations across
multiple consecutive runs. This proves that formal correctness guarantees can
be maintained throughout system integration.

\paragraph{TRL 5 \textit{Laboratory Testing in Relevant Environment}}

For this research, TRL 5 means demonstrating the verified controller on
industrial control hardware through hardware-in-the-loop testing. The
discrete automaton must be implemented on the Emerson Ovation control system
and verified to match synthesized specifications exactly. Continuous
controllers must execute at required rates. The ARCADE interface must
establish stable real-time communication between the Emerson Ovation hardware
and SmAHTR simulation. Complete autonomous startup sequences must execute via
hardware-in-the-loop across the full operational envelope. The controller
must handle off-nominal scenarios to validate that expulsory modes function
correctly. For example, simulated sensor failures must trigger appropriate
fault detection and mode transitions, and loss-of-cooling scenarios must
activate SCRAM procedures as specified. Graded responses to minor
disturbances are outside this work's scope\oldt{.}\newt{, as they require
runtime optimization under uncertainty that extends beyond the
correct-by-construction verification framework presented
here.}Formal
verification results must remain valid, with discrete behavior matching
specifications and continuous trajectories remaining within verified bounds.
This proves that the methodology produces verified controllers implementable
on industrial hardware.

Progress will be assessed quarterly through collection of specific data
comparing actual results against TRL advancement criteria. Specification
development status indicates progress toward TRL 3. Synthesis results and
verification coverage indicate progress toward TRL 4. Simulation performance
metrics and hardware integration milestones indicate progress toward TRL 5.
The research plan will be revised only when new data invalidates fundamental
assumptions. This research succeeds if it achieves TRL 5 by demonstrating a
complete autonomous hybrid controller with formal correctness guarantees
operating on industrial control hardware through hardware-in-the-loop
testing in a relevant laboratory environment. This establishes both
theoretical validity and practical feasibility, proving that the methodology
produces verified controllers and that implementation is achievable with
current technology.