This commit is contained in:
Dane Sabo 2025-07-30 19:03:44 -04:00
parent 90678a062e
commit fafa2db2a4
4 changed files with 266 additions and 77 deletions

View File

@ -1,14 +1,25 @@
---
tags:
- Ideas
---
## What are we doing:
Remember when Lance was talking about his monitor that would read a control system to see if a cyber-attack was taking place? This is in that vein.
The idea that I'm thinking of is that to tell if a control system is compromised, we should be able to look at the dynamics of the system to know if an attack is happening. A couple thoughts:
1. Sensor integrity: If we have a monitor that is an observer, this monitor should be able to know when the plant is diverging from what it would expect based on sensor values and it's model of the plant. When the error signal has a higher magnitude than expected, this is a sure fire sign that something might be wrong.
--- tags:
- Ideas --- ## What are we doing: Remember when Lance was talking
about his monitor that would read a control system to see if a
cyber-attack was taking place? This is in that vein. The idea
that I'm thinking of is that to tell if a control system is
compromised, we should be able to look at the dynamics of the
system to know if an attack is happening. A couple thoughts:
1. Sensor integrity: If we have a monitor that is an observer,
this monitor should be able to know when the plant is diverging
from what it would expect based on sensor values and it's model
of the plant. When the error signal has a higher magnitude than
expected, this is a sure fire sign that something might be wrong.
1. This is sensitive to maintenance problems however.
2. Monitor must be using data diodes and not connected to any outside sources.
2. A secondary, redundant control system: Assuming the perpetrator is tampering with signals, activate a secondary control system that a) latches the first system out of its control authority, and b) operates a safety shutdown mode. This controller doesn't need to be super fancy - it just needs to prevent damage.
## Why are we doing this:
This seems like a fun way to integrate control system math with cybersecurity. This is somewhat a more CIE topic, but actually does address some cybersecurity issues. This could also be implemented using KOs and the protected domains.
## Other details:
2. Monitor must be using data diodes and not connected to any
outside sources.
2. A secondary, redundant control system: Assuming the
perpetrator is tampering with signals, activate a secondary
control system that a) latches the first system out of its
control authority, and b) operates a safety shutdown mode. This
controller doesn't need to be super fancy - it just needs to
prevent damage. ## Why are we doing this: This seems like a fun
way to integrate control system math with cybersecurity. This is
somewhat a more CIE topic, but actually does address some
cybersecurity issues. This could also be implemented using KOs
and the protected domains. ## Other details:

View File

@ -1,22 +1,40 @@
#ERLM
# Goals and Outcomes
Review [[ERLM_Structure_of_Objectives_Page.pdf]]
_Begin with the end in mind_
_--- Steven Covey_
_You've got to be careful if you don't know where you are going, because you might not get there._
_--- Yogi Berra_
Review [[ERLM_Structure_of_Objectives_Page.pdf]] _Begin with the
end in mind_ _--- Steven Covey_
_Alice: Would you tell me, please, which way I ought to go from here?_
_Cheshire Cat: [smiling] That depends a good deal on where you want to get to._
_You've got to be careful if you don't know where you are going,
because you might not get there._ _--- Yogi Berra_
_What are you trying to achieve? Explain your outcomes using absolutely no jargon._
_Alice: Would you tell me, please, which way I ought to go from
here?_ _Cheshire Cat: [smiling] That depends a good deal on
where you want to get to._
The Goals and Outcomes of your proposal are your chance to set the vision for your research. It is important at the beginning of your research to make clear to the reader where you want to be at the end. The proposal is a plan for getting to that place, and everything after the Outcomes is about explaining where you start and what path you will take.
_What are you trying to achieve? Explain your outcomes using
absolutely no jargon._
The Goals and Outcomes of your proposal are your chance to set
the vision for your research. It is important at the beginning of
your research to make clear to the reader where you want to be at
the end. The proposal is a plan for getting to that place, and
everything after the Outcomes is about explaining where you start
and what path you will take.
An outcome is a thing aimed at or sought; it's a goal. That idea,
however, creates confusion since a research goal is the broad
purpose, and outcomes are specific statements about what you want
to be able to achieve. For our purposes, we will distinguish
between the research goal and the research outcomes, but both
will be contained in the Goals and Outcomes section of the
proposal.
An outcome is a thing aimed at or sought; it's a goal. That idea, however, creates confusion since a research goal is the broad purpose, and outcomes are specific statements about what you want to be able to achieve. For our purposes, we will distinguish between the research goal and the research outcomes, but both will be contained in the Goals and Outcomes section of the proposal.
## Research Goal
You must write the proposal with the reviewer in mind. The reviewer will want to know from the beginning what the proposal is about, so start every proposal with
You must write the proposal with the reviewer in mind. The
reviewer will want to know from the beginning what the proposal
is about, so start every proposal with
"The goal of this research is to ..."
@ -24,57 +42,192 @@ You can also say
"The purpose of this research is to ..."
This goal sets the vision for what you want to achieve; it is the circle on the map for where you want to go. Your goal should not be too broad or grandiose. Remember that your colleagues are trying to solve similar problems and know what can and cannot be achieved. Grand goals call for grand projects, and if the details of you project do not align with your goal, the reviewers will see the disconnect and decline funding. The goal should also not be too narrow or specific. Narrow projects are often overly constrained limiting inquiry, have solutions with limited impact, and lack vision. Researchers who lack vision often fail. They are unable to inspire teams, motivate performance, or create sustainable value.   The challenge, one that successful researchers often spend considerable time confronting, is to make the goal of the research "just right," thus creating a vision of a problem with sufficient intellectual merit and broader impact to be worthy of investigation.
This goal sets the vision for what you want to achieve; it is the
circle on the map for where you want to go. Your goal should not
be too broad or grandiose. Remember that your colleagues are
trying to solve similar problems and know what can and cannot be
achieved. Grand goals call for grand projects, and if the details
of you project do not align with your goal, the reviewers will
see the disconnect and decline funding. The goal should also not
be too narrow or specific. Narrow projects are often overly
constrained limiting inquiry, have solutions with limited impact,
and lack vision. Researchers who lack vision often fail. They are
unable to inspire teams, motivate performance, or create
sustainable value.   The challenge, one that successful
researchers often spend considerable time confronting, is to make
the goal of the research "just right," thus creating a vision of
a problem with sufficient intellectual merit and broader impact
to be worthy of investigation.
It can be challenging or even unwise to stuff a big vision into just one sentence, but you don't want to have it take up a page or even half of one. Keep the research goal one to three sentences in length. If you have to go longer than that, you probably don't have a goal that is clear enough to you, which means it will not be clear to the reader, or you are trying to over define your goal, in which case some of what you are saying might be part of the outcomes or impact.
It can be challenging or even unwise to stuff a big vision into
just one sentence, but you don't want to have it take up a page
or even half of one. Keep the research goal one to three
sentences in length. If you have to go longer than that, you
probably don't have a goal that is clear enough to you, which
means it will not be clear to the reader, or you are trying to
over define your goal, in which case some of what you are saying
might be part of the outcomes or impact.
While your goal begins your vision of what you want to achieve in
your research, it may not be tangible or concrete. Reviewers will
want and need to see a more clear picture of where your research
will go. It would seem, since the point of the proposal is to
explain what you want to do in your research, that next you
should lay out what you would do in your research. The trouble is
that, while the reviewer does care what you will do, they care
more where you will end up.  Think of them as the one with the
money --- they sometimes are, but often are not --- and they want
to know what it is they get for their investment. What are they
buying? What is the product of your effort? What will you
achieve? _These questions are not about the tasks you will
undertake}_, but rather about where you will be if the research
is successful and what you will be able to do that is new. The
answer to these questions are the research outcomes.
While your goal begins your vision of what you want to achieve in your research, it may not be tangible or concrete. Reviewers will want and need to see a more clear picture of where your research will go. It would seem, since the point of the proposal is to explain what you want to do in your research, that next you should lay out what you would do in your research. The trouble is that, while the reviewer does care what you will do, they care more where you will end up.  Think of them as the one with the money --- they sometimes are, but often are not --- and they want to know what it is they get for their investment. What are they buying? What is the product of your effort? What will you achieve? _These questions are not about the tasks you will undertake}_, but rather about where you will be if the research is successful and what you will be able to do that is new. The answer to these questions are the research outcomes.
## Research Outcomes
The research outcomes are brief, clear, concise statements of what you should be able to do if the research is successful. These are like mini-goals, but they are more specific. They are not necessarily steps along the way to a larger goal; rather, they are the sign posts that, if achieved, together would meet the overarching goal. The outcomes provide clarity and definition of your goal, and make clear what the specific things you want to be able to do if successful. This vision is something that you will return to throughout the proposal as you make connections between your efforts and how they will help you achieve your outcomes. Since you will be making these connections, the reader should be able to keep your vision in mind as they read, and you need to facilitate this by making each outcome sticky. Do this by making them simple, concrete, and credible. You do not need all of the traits, but do as many as you can.
The research outcomes are brief, clear, concise statements of
what you should be able to do if the research is successful.
These are like mini-goals, but they are more specific. They are
not necessarily steps along the way to a larger goal; rather,
they are the sign posts that, if achieved, together would meet
the overarching goal. The outcomes provide clarity and definition
of your goal, and make clear what the specific things you want to
be able to do if successful. This vision is something that you
will return to throughout the proposal as you make connections
between your efforts and how they will help you achieve your
outcomes. Since you will be making these connections, the reader
should be able to keep your vision in mind as they read, and you
need to facilitate this by making each outcome sticky. Do this by
making them simple, concrete, and credible. You do not need all
of the traits, but do as many as you can.
Well formulated outcomes should do the following:
- provide a clear purpose for the research;
- direct your choice for research activities;
- guide the assessment of the success of the research.
Each outcome should be a single sentence that starts with a verb --- remember you are stating what you should be able to do if the research is successful. Ideally, the verb should convey something verifiable. Verbs like 'understand'', 'know', 'comprehend', or 'make sense of' may describe something we want to achieve in research --- understanding is a general goal of all research. But, these are not observable or verifiable. Remember the eighth Heilmeir question asked "What are the mid-term and final 'exams' of the research?'' By making your outcomes observable and verifiable you setting yourself up to answer that question.
Each outcome should be a single sentence that starts with a verb
--- remember you are stating what you should be able to do if the
research is successful. Ideally, the verb should convey something
verifiable. Verbs like 'understand'', 'know', 'comprehend', or
'make sense of' may describe something we want to achieve in
research --- understanding is a general goal of all research.
But, these are not observable or verifiable. Remember the eighth
Heilmeir question asked "What are the mid-term and final 'exams'
of the research?'' By making your outcomes observable and
verifiable you setting yourself up to answer that question.
Examples of an outcomes written to varying degrees.
**Goal:** The goal of this research is to generate artificial light.
**Goal:** The goal of this research is to generate artificial
light.
| **Quality** | **Outcome** |
| ---------------- | -------------------------------------------------------------------------------------------------------------------- |
| Good | Use electricity to generate incandescent light from a wire filament. |
| Vague | Use electricity to make light. |
| Unmeasurable | Understand how incandescent light can be made with electricity. |
| Verbose | Use AC or DC electricity to generate light as a result of heating a wire filament to sufficiently high temperatures. |
| Not goal related | Determine the relationship between filament temperature and the characteristic of the light emitted. |
| **Quality** | **Outcome**
| | ---------------- |
--------------------------------------------------------------------------------------------------------------------
| | Good | Use electricity to generate incandescent
light from a wire filament.
| | Vague | Use electricity to make light.
| | Unmeasurable | Understand how incandescent light can be
made with electricity.
| | Verbose | Use AC or DC electricity to generate light
as a result of heating a wire filament to sufficiently high
temperatures. | | Not goal related | Determine the relationship
between filament temperature and the characteristic of the light
emitted. |
How many outcomes should you have? You want the reviewer to be able to remember your outcomes --- they should be able to recall your outcomes throughout the proposal--- so don't make too many, but don't make too few since that runs the risk of suggesting either a narrow scope for the research or overly broad outcomes that don't define the research sufficiently. Between three and five is about right. Less than three is too few --- the reviewer wonders what else you might be able accomplish. More than five is too much --- the reviewer thinks that you are over committing yourself. My preference is three outcomes, and there is room for more if you absolutely need them.
How many outcomes should you have? You want the reviewer to be
able to remember your outcomes --- they should be able to recall
your outcomes throughout the proposal--- so don't make too many,
but don't make too few since that runs the risk of suggesting
either a narrow scope for the research or overly broad outcomes
that don't define the research sufficiently. Between three and
five is about right. Less than three is too few --- the reviewer
wonders what else you might be able accomplish. More than five is
too much --- the reviewer thinks that you are over committing
yourself. My preference is three outcomes, and there is room for
more if you absolutely need them.
Presumably, your outcomes are not something that you were able to do before. That is, your research should extend the state of the art and redefine the limits of current practice. Who is the judge of the current art and limits? This depends upon the funding organization. If you are applying to the NSF, the state of the art may be much different from what a company sees as their state of the art. A large national funding agency sees their job as pushing the state of the art for the country, and you need to define an outcome that pushes the state of the art for the national science and engineering community. A small company may just want to innovate for their customers, and your research may be bringing new capabilities to the company and their customers but may not be as grand or far reaching it might be if you has applied, say, to the NSF. It is important to understand the organization to which you are applying and what their expectations are for what reasonable outcomes might be.
Presumably, your outcomes are not something that you were able to
do before. That is, your research should extend the state of the
art and redefine the limits of current practice. Who is the judge
of the current art and limits? This depends upon the funding
organization. If you are applying to the NSF, the state of the
art may be much different from what a company sees as their state
of the art. A large national funding agency sees their job as
pushing the state of the art for the country, and you need to
define an outcome that pushes the state of the art for the
national science and engineering community. A small company may
just want to innovate for their customers, and your research may
be bringing new capabilities to the company and their customers
but may not be as grand or far reaching it might be if you has
applied, say, to the NSF. It is important to understand the
organization to which you are applying and what their
expectations are for what reasonable outcomes might be.
---
# Peer Review: Goals and Outcomes 1
Offer comment using Acrobat's comment feature.  Address the questions below, which  focus whether the content meets the intention of the section, but you can say more about whatever else you think will help.   Since it can be hard to know what the writer intended, it is best to highlight parts you find confusing or that you may have had to read several times before understanding.  Some questions will ask you to summarize your understanding of what was written.  This will help the writer see how well their message has been communicated.
A note about spelling, grammar, and typos:  Writers should make a point to eliminate these errors from what they write.  Typos, for example, distract the reader and detract from your writing.  As editors, it is not your job to ferret these mistakes out.  Focus on the message and substance of the writing, and only comment on these annoyances if you just can't stand it.
Offer comment using Acrobat's comment feature.  Address the
questions below, which  focus whether the content meets the
intention of the section, but you can say more about whatever
else you think will help.   Since it can be hard to know what the
writer intended, it is best to highlight parts you find confusing
or that you may have had to read several times before
understanding.  Some questions will ask you to summarize your
understanding of what was written.  This will help the writer see
how well their message has been communicated.
A note about spelling, grammar, and typos:  Writers should make a
point to eliminate these errors from what they write.  Typos, for
example, distract the reader and detract from your writing.  As
editors, it is not your job to ferret these mistakes out.  Focus
on the message and substance of the writing, and only comment on
these annoyances if you just can't stand it.
_When you offer a criticism, offer a remedy so the writer can
improve their work._
_When you offer a criticism, offer a remedy so the writer can improve their work._
## Research Goal
The goal sets the vision for what you want to achieve; it is the circle on the map for where you want to be if the research is successful.
- Is the goal a clear statement about what the research would achieve if successful?  Explain why or why not.
The goal sets the vision for what you want to achieve; it is the
circle on the map for where you want to be if the research is
successful.
- Is the goal a clear statement about what the research would
achieve if successful?  Explain why or why not.
- Explain what you think the vision for the research is. 
- Is the scope of the goal of the research "just right'', not too grandiose or too narrow?  Explain how it could be made that way.
## Research Outcomes
The research outcomes should be brief, clear, concise statements of what could be done if the research is successful.  These are like mini-goals, but they are more specific.
- Are the research outcomes brief, clear, and concise statements of what should be able to be done if the research is successful?  Verify that the outcomes are not tasks or steps of the research.
- Explain how the outcomes, if achieved, meet the overarching goal.
- Is the scope of the goal of the research "just right'', not too
grandiose or too narrow?  Explain how it could be made that way.
## Research Outcomes
The research outcomes should be brief, clear, concise statements
of what could be done if the research is successful.  These are
like mini-goals, but they are more specific.
- Are the research outcomes brief, clear, and concise statements
of what should be able to be done if the research is successful? 
Verify that the outcomes are not tasks or steps of the research.
- Explain how the outcomes, if achieved, meet the overarching
goal.
- Are the research outcomes verifiable?  How so?
## Reader's Perspective
- In your own words, describe the goals and objectives.  Do you have a clear picture of where the PI wants his research to go?  Has the writer transmitted his message to you?
- Has the PI provided sufficient explanation about the goal and objectives?  Are they over/under explained?  Explain what pieces would be helpful to clarify the picture.
- In your own words, describe the goals and objectives.  Do you
have a clear picture of where the PI wants his research to go? 
Has the writer transmitted his message to you?
- Has the PI provided sufficient explanation about the goal and
objectives?  Are they over/under explained?  Explain what pieces
would be helpful to clarify the picture.

View File

@ -16,7 +16,7 @@ endif
badd +16 ~/.config/nvim/lua/custom/plugins.lua
badd +8 ~/.config/nvim/lua/custom/configs/lspconfig.lua
badd +6 custom/init.lua
badd +19 custom/language_specific_commands/markdown.lua
badd +22 custom/language_specific_commands/markdown.lua
argglobal
%argdel
edit custom/language_specific_commands/markdown.lua
@ -32,12 +32,12 @@ setlocal foldnestmax=20
setlocal foldenable
silent! normal! zE
let &fdl = &fdl
let s:l = 9 - ((5 * winheight(0) + 31) / 62)
let s:l = 22 - ((21 * winheight(0) + 28) / 56)
if s:l < 1 | let s:l = 1 | endif
keepjumps exe s:l
normal! zt
keepjumps 9
normal! 066|
keepjumps 22
normal! 06|
tabnext 1
if exists('s:wipebuf') && len(win_findbuf(s:wipebuf)) == 0 && getbufvar(s:wipebuf, '&buftype') isnot# 'terminal'
silent exe 'bwipe ' . s:wipebuf

View File

@ -15,13 +15,11 @@ ___________________________________________________________
## **Integrating Shielding into Nuclear Power Control**
### Goal:
The goal of this research is to develop machine learning
control algorithms for nuclear power applications with
strict safety guarantees.
### Outcomes:
If this research is successful, I will have accomplished the
following:
@ -34,53 +32,80 @@ following:
3. ??? <!TODO!>
### Impact:
Machine learning based systems have been shown to be more
efficient than typical PID based controllers, and are able
to learn more complex objective functions than a typical
controller can. The problem with these controllers though is
that they are often unexplainable. This is not acceptable
for high assurance applications, where slight perturbations
on inputs can yield wildly different outputs. Shielding can
solve this problem, helping ensure safety of ML based
controllers while not limiting their development or
construction.
efficient than typical PID based controllers, and are able to
learn more complex objective functions than a typical controller
can. The problem with these controllers though is that they are
often unexplainable. This is not acceptable for high assurance
applications, where slight perturbations on inputs can yield
wildly different outputs. Shielding can solve this problem,
helping ensure safety of ML based controllers while not limiting
their development or construction.
### Relevant Papers
[[safe-reinforcement-learning-via-shielding]]
[[evaluating-robustness-of-neural-networks-with-mixed-integer-programming]]
___________________________________________________________
## **Formally Verified Control of Reactor Systems**
## **Formally Verified Neural Network Control of Control Rod System**
### Goals:
The goal of this research is to use formal methods to ensure
that a neural network based control rod controller will never violate
safety guarantees of a reactor trip system.
The goal of this research is to use formal methods to ensure that
a neural network based control rod controller will never violate
safety guarantees of a reactor trip system. To do this, a
satisfiability modulo theory method will be applied to
exhaustively search the network for potential failure modes.
### Outcomes:
If this research is successful, I will accomplish the
following.
If this research is successful, I will have accomplished the
following:
- Build a neural network controller for real time control of a
control rod system.
- Formalize safety guarantees of shutdown margin in a
satisfiability modulo theory embedding
- Formally verify that the neural network based controller will
not violate any shutdown margin restrictions
### Impact:
SMT solvers and MILP formulations have been applied to neural
networks to ensure that the network is resilient to input
perturbations. I think we can expand this to more general
considerations of the state space, especially when there are a
relatively small number of states such as in power contexts. The
benefit of this system is that we would get closer to saying
neural network based systems can be high assurance for physical
systems.
### Related Papers:
[[reluplex-an-efficient-smt-solver-for-verifying-deep-neural-networks]]
[[evaluating-robustness-of-neural-networks-with-mixed-integer-programming]]
[[formal-verification-of-neural-network-controlled-autonomous-systems]]
___________________________________________________________
## **Temporal Logic Specifications for Autonomous Controller Synthesis**
## **Temporal Logic Specifications for Autonomous Controller Shield Synthesis**
(3)
### Goals:
If this research is successful, we will be able to generate
autonomous controllers that provably adhere to specifications
written with temporal logic automatically.
### Outcomes:
- Create an intermediary shield that mediates signals between an
optimal control system and the physical plant
-
### Impact:
### Related Papers:
[[enhancing-cyber-physical-system-dependability-via-synthesis-challenges-and-future-directions]]
[[safe-reinforcement-learning-via-shielding]]
___________________________________________________________
## **Formally Verified Runtime Monitoring and Fallback**