9.1 KiB
Wheelchair Cushion Testing Rig Destroys Itself
Introduction
Recently I have begun hosting a poker game. Poker is a beautiful game, as it connects people in conversation that is fulfilling and informative, as people discuss their life experiences. At my poker game, I learned about a somewhat comical cyber-physical system failure. A player at my poker game is a bioengineer who interned at Pitt's Wheelchair and Cushion Standards Group. She was responsible for configuring an immersion testing fixture to perform evaluations for a set of wheelchair cushions. She did not engineer the control program for the automated testing fixture, but soon found out one of her first days that the control program had a destructive mode of operation that was not intended.
Alex sought out the position at the Wheelchair and Cushion Standards Group as an summer internship that was conveniently located in the East Liberty neighborhood. Her job would be multifaceted: as part of a small laboratory group, he responsibilities would change upon the day and she would be expected to wear many hats. One of her first tasks was running experiments on sets of wheelchair cushions using an expensive test fixture.
Wheelchair Cushion Standards
Wheelchair cushions are a surprisingly sophisticated device, and far from only a piece of foam. Wheelchair cushions are critical for users who spend a large amount of time in their wheelchair, and an effective cushion can play a significant role in the overall health of the user. Cushions redistribute the weight of a wheelchair user evenly across their buttocks, while an insufficient cushion can create problems for wheelchair-bound individuals including ulcers, posture issues, and blood flow restrictions. These problems are even further magnified for individuals who have loss of feeling in their legs, as they can not detect the development of injuries until they are visibly apparent or manifest in greater health issues. For these reasons, regulatory standards exist for wheelchair cushions. Standards such as ISO 16840-2 exist to protect wheelchair users by providing a standard to which cushions can be tested. By using wheelchair cushions that meet these standards, users can have a degree of assurance that the product they're using will minimize risks associated with prolonged wheelchair use.
For a wheelchair cushion to pass this standard, it must be experimentally tested. This testing is commonly done by companies that specialize in standards testing. The sensors and equipment required to do ISO testing can be very expensive, and prohibitive for individual manufacturers of items like wheelchair cushions to create test fixtures of their own. Pitt's Wheelchair and Cushion Standards group does such testing for this particular wheelchair cushion standard using a testing rig that is described in the following image.
Testing Setup
!
This testing fixture consists of a hydraulic press with a specific CNC-machined wooden buttocks model attached. These buttocks integrate an array of pressure sensors at a speckling of locations in order to gather a holistic understanding of the distribution of pressure on the buttocks surface. A cushion is fixed to the lower part of the hydraulic press frame. The whole system measures two main quantities: the pressure distribution on the wooden buttocks, and the total displacement of the hydraulic press.
The positions of sensors on the wooden buttocks are of particular interest. These sensors are spread throughout the surface of the contoured shape. Ideally, these sensors should have similar values of pressure for a given load if the cushion is doing a good job supporting the load. These sensors have different heights relative to the displacement of the press. As a result, the sensors on the very bottom of the buttocks model first experience pressure, while the sensors on the sides of the buttocks eventually catch up in pressure measurement as displacement is increased.
The Failure
The testing procedure Alex was specifically performing when the failure occurred was a fatigue failure test. In this test, the buttocks would descend upon the cushion until a proscribed pressure is achieved on all sensors, hold for a certain amount of time, and then ascend until zero pressure is achieved again. This cycle is then repeated for a very large number of cycles. Performance of the cushion is then determined as to how the pressure distribution changes over time--more performant cushions will retain their cushioning properties for longer than less performant cushions.
One of the first cushions Alex was tasked with testing was a cushion that was described as being similar construction of a cheap air-mattress. Alex was instructed on how to start the test, where the E-Stop button lived, and what should be expected. Once the person instructing her left, the test began.
The press began depression the cushion with the wooden buttocks when the air-mattress-like construction gave only a whimper of support. The press continued to descend until the wooden buttocks were separated from the lower steel frame by only the thin rubber membrane of the cushion. At this point, the force on the lower sensors skyrockets while the circumferential sensors slowly accumulate load. The controller realizes that there is an extreme load on some of the sensors, however, and retracts the buttocks from the cushion. This safety feature was designed to protect the sensors in this exact event. Whoever designed this controller did not account for the fatigue testing mode, however, and once the buttocks were raised momentarily, the controller decided to return to the previous displacement. To Alex as an observer, the resulted in a violent ramming of the wooden buttocks into the steel frame, which repeated with cacophony as the air-mattress cushion sabotaged the testing fixture.
After the first repetition, the lower sensors in the buttocks were broken. Instead of accurately reading the pressure on the wooden model, the pressure sensors instead read a much lower value. This results in the test fixture applying more displacement to continually try to reach the optimal test pressure. The sensors continue to read nonsensically low values as they are repeated slammed into the metal lower frame as they are slowly disintegrating. Each time they make contact with the frame they return with an even lower reported value, resulting in an even higher force applied to the buttocks model with each cycle.
This whole process has happened in a time span of about 5 seconds. At this point, Alex realizes this is definitely not supposed to be happening and reaches for the E-Stop button. Before she could activate the E-Stop, the press descended the wooden buttocks with such ferocity that the wooden model splits in half as if the cushion itself were an axe. The ideal testing force is supposed to be around 400 lbf, but it was estimated that the fixture applied over a thousand pounds of force when the buttocks were destroyed. Finally, the E-Stop was activated before further damage was created.
Reflection
The part of this failure that makes it a cyber-physical failure is the fact that while a pressure based controller was implemented, it was never validated for a use-case where the cushion does not achieve a sufficient pressure on all of the buttock sensors. This case was realized when the buttocks made contact with the steel frame with this particular low quality cushion.
Another control was implemented in case of extreme pressure readings on the buttocks sensors, but that control also experienced a cyber-physical failure. The controller assumed a model of the sensors that would continuously report correct values even in the case of extreme loading. This assumption proved to be incorrect, which essentially nullified the over-pressure control once the model of the sensors was invalidated.
Fortunately, this comedy of errors did not injure anyone, but this failure incurred significant business costs. The wooden buttocks had to be replaced with a new model, which required expensive CNC machining. The damaged sensors had to also be replaced which were never cheap to begin with. And finally, the group paid Alex dozens of hours to recalibrate the system when the new sensors and buttocks model were installed, and to reperform all previous testing that was now invalidated by the broken sensors.
Cyber-physical system failures can be sneaky. It is not always clear while coding if a control will actually prevent a mitigated failure from actually occurring. Because of this opacity, other tools such as proof-based methods have to be used to ensure high-assurance. In this case, a proof could have been developed to determine that an unsafe force could not be applied to the lower sensors regardless of their reported value. By perhaps investigating this possible mode, the controller designer could be informed that only relying on the lower sensors as protection would have been insufficient to protect them. These proofs require extra effort to create, but if they prevent such expensive failures, it is worth it to create them.