Friday, December 05, 2003

On ISO and how do you implement a "lessons learned" process in a very large R&D organization ?

I just came across the following tidbit: an internal document produced at the Mashall Space Flight Center as a result of a stand down week at NASA. The purpose was for NASA to look into the recommendations of the Columbia Accident Investigation Board. Nasa Watch has the full document here:

"One facet of this program is Retaining Knowledge."

"The ISO process, in its entirety may not be applicable to the continuous R&D efforts within the payload area."

"NASA processes are not well documented, and too many of them are labeled out of scope� under ISO9000, the only quality assurance program in use. ISO 9000 was designed to require documentation of all processes, and yet as implemented at NASA and MSFC it is hoop-jumping paperwork that simply adds to the confusion. Where processes are documented, the reasons behind these processes are often obscured and difficult to find, as may be the full documentation on the proper way to proceed."

" ISO 9000 has absolutely nothing to do with real safety, it just generates paperwork. Lack of technical safety expertise of safety personnel. Not enough engineering test and inspection; generally reviews are handled as just a review of paper. Lack of priority on safety issues; all safety issues are treated equally. Safety training is more bureaucratic and not related to practical knowledge of safety in a laboratory environment. Lessons learned are not incorporated from missions or programs. Each person is the bottom line of safety. Need double-checking by technical personnel"

"I don't think that we in MSAD have a useful, relevant database that can easily be referenced by PMs, LSEs, PI/PSs, Managers, or our support contractors. We don't take advantage of our organizational experience to increase our chances of future success."

" We are being engrained with the need to provide the elevator speech - the problem is that very often the elevator speech is all that we learn - and obviously all that we are able to communicate. As a leading edge community we should and must do better. ". . . impatience, the mother of stupidity, praises brevity . . ." Leonardo da Vinci, 1513. This is also spreading to the need for numerous dry runs prior to the visit of dignitaries. Knowing one's subject beyond the elevator speech level and the Powerpoint chart will led to refreshing spontaneity. This will probably be more appreciated."

"There seems to exist a lack of understanding by managers of probability theory governing events that had occurred during previous flights, events that seemingly did not produce serious consequences. This lack of understanding or misinterpretation led these managers to assume that future similar events would not produce serious flight safety issues or conditions "

At some point, NASA will be able to solve these problems. The issue remains though, is an ISO process needed for organizations that are in the business of cranking out single items or prototypes ? How does one keep knowledge in an organization that spread over several states and decades ? Can a large task manager of some kind do the job ? If so, how does one go about extracting the knowledge from different performances ? NASA has a lessons learned program, but how does one make sure that the task performed by somebody within the organization has a "link" of some kind to a particular past incident ? Maybe a Bayesian filter combining the tasks listed in the task manager with the appropriate lessons learned database entries could do the trick?