Orion's lesson for IT: Test your systems, then test again (and again)
So how did NASA's engineers make this old technology work for them in the rigors of space
Testing. Testing. Testing.
"It was a very good flight," Matt Lemke, NASA's deputy manager for Orion's avionics, power and software team, said Tuesday. "It was as good as it could have gone.... The way we achieved that is by testing. We put our system together and we put it in the right environment and tested it in all directions to give ourselves the confidence it was going to work."
That encompasses the advice Lemke would give any CIO or enterprise IT shop. You don't have to have the latest and greatest technology. Sometimes, he said, it's more about saving money, using what already works and testing the heck out of it to make sure it does the job.
"I think you need to understand what you're trying to do and, in our case, it was reliable spaceflight," said Lemke. "It wasn't about the latest and greatest technology. It was what we could use for the least amount of money and the highest reliability to accomplish our mission. And that's what we did."
He added that NASA also spent a lot of time worrying over low-probability problems. Any risk was something to focus on.
"I think what we did right was designing and testing the system," said Lemke. "We worried over every detail, every anomaly. We understood what was happening every time we had a problem so when it was time to fly, those problems were past us."
Orion , NASA's next-generation spacecraft, is designed to carry astronauts into deep space. The space agency hopes it will take humans to an asteroid in the 2020s and to Mars and back in the 2030s. Orion is a critical component of NASA's plan to reach further into the solar system than ever before.
Flying without a crew n its first test flight, the spacecraft made two orbits around the Earth on Friday and flew 15 times higher than the orbit of the International Space Station. Despite traveling twice through the high radiation field of the Van Allen belts, Orion's three computers functioned perfectly. No crashes. No resets. No hiccups.
Lemke credited the older technology on board as one reason the computers performed flawlessly.
The spacecraft runs a flight computer that Honeywell International Inc. originally built for Boeing's 787 jet airliner. The computer uses IBM's single-core PowerPC 750FX processors, which were first launched in 2002.
NASA ruggedized the computer, which operates everything on the spacecraft, by building a thicker circuit board and hardware to decrease the effect vibrations have on it. NASA also added two redundant computers. That decreased the chances of all three computers failing at the same time to better than one-in-1.8 million missions.
Lemke said telemetry sent from Orion during its flight showed that the computers withstood the blasts of radiation.
Now, engineers are going through the data taken from the system while it was still on the ship that recovered it after splashdown in the Pacific ocean.
"As far as we can tell, the [computers] had no problems during the flight," said Lemke. "People are starting to look at all of that data in detail.... Now, we can see if there were any memory errors or any errors during Ethernet transmissions because of radiation. We're now looking at the next level of detail with the stored data. We'll look for any errors on any of those lines."
Engineers also are culling through data to figure out how well the new Time-Triggered Ethernet, or TTEthernet, system worked on the spacecraft. It was the first time the system, which offers a time-based guaranteed packet delivery, was used in space.
Orion used one basic Ethernet line for non-critical data transfers and three TTEthernet lines for all critical information, including navigation and instructions to fire the craft's engines.
Lemke and his team are trying to deduce how well all the data lines worked.
"That's one of the hardest things to figure out," he said. "We have multiple wires. Multiple planes of data. Did we have any kind of problem at all You have to really dig into the data to find out."
NASA engineers are also in the process of building the computers that will be used in Orion's next test flight, scheduled for 2018. Lemke said they're using the same PowerPC 750FX processors, while trying to make the computer easier -- and cheaper -- to build.
"It's the same chip, the same design," he explained. "It'll just be very small changes. We're trying to make sure we can build these computers with the least amount of money as possible. There were parts of these computers that used to have to be built manually by a technician. We want it to be more automated."
Engineers are trying to adjust the spacing in the circuit board so machines, instead of technicians, can build the boards.
From building to testing, the current computers cost more than $1 million each. Lemke said NASA hopes to cut the costs for them by 20%.