NASA Offers Airline Safety DataNASA on Monday released an intentionally scrambled, partly deleted version of the safety data it gathered from 24,000 interviews with airline pilots, making good on a promise to Congress to make public information that it said earlier this year would shake public confidence in the airlines and threaten their commercial interest.
But the agency released the data from the $11.5 million program in a format that made it difficult if not impossible for outsiders to analyze in search of trends, presenting the reports as documents rather than spreadsheets. And the NASA administrator, Michael Griffin, said his agency had no plans to do additional work with the material, which he sought to disown in a conference call with reporters.
“It’s hard for me to see any data here that the traveling public would care about or ought to care about,” he said. “But it’s also not for me to prescribe what others may care about. We were asked to release the data and I said that we would, and I’ve done that.”
As released, the survey data no longer linked pilot reports to the type of plane the pilot flew, the pilot’s experience level or other particulars. NASA said the reason was to maintain the anonymity of respondents.
In a sometimes testy exchange with reporters, Dr. Griffin said that it was never NASA’s intention to do anything more than test methods of data gathering. The project, called the National Aviation Operations Monitoring System, sought to uncover safety problems by surveying pilots, rather than waiting for them to make anonymous reports, or gathering information from “black boxes” or similar sources.
But Representative Bart Gordon, Democrat of Tennessee and chairman of the House Committee on Science and Technology, who extracted the promise from Dr. Griffin to release the information, said that the thousands of pages of “redacted” and “disaggregated” data put on the Internet on Monday was “a start but not a satisfactory start.”
Mr. Gordon said the database, which also includes 5,000 interviews with pilots other than airline pilots, was so extensive that there must be safety insights to be gained from analyzing it.
“Like penicillin or other types of discoveries, it’s not what you went in looking for,” he said. “I think we’re going to find some things that will help us.”
Mr. Gordon and Representative Brad Miller, Democrat of North Carolina, who is chairman of the investigation and oversight subcommittee, pledged to push NASA further. Mr. Miller said that “if 80 percent of the pilots they ask agree to sit still for a half-hour survey, voluntarily, my conclusion is the pilots had something they wanted others to know about.”
“This is now 3 years old, and it’s been dumped, unanalyzed and scrubbed of much of the useful information,” Mr. Miller said.
Dr. Griffin did not say whether the project had yielded any useful insight into how to conduct surveys, which NASA had expected at one point would be extended to mechanics, flight attendants and others. He expressed frustration about his agency’s problems in bringing the project to a close. It was, he said, “intended to have a beginning, a middle and an end, and we seem to be unable to end it, which is a bit frustrating because we don’t have the money to continue it.”
“Researchers being funded by the United States government will always have a strong belief that their research work should be extended ad infinitum,” he said.
But Jon A. Krosnick, a professor at Stanford who helped with the survey design, said in a letter to the Science Committee on Dec. 17 that the project had been terminated before it could be expanded beyond pilots to other personnel important to safety.
NASA’s discomfort with the study it paid for appeared to arise partly from the differing approaches of engineers, who run NASA and the Federal Aviation Administration, and the social scientists who conducted the survey. The engineers tend toward gathering numbers from computers; the social scientists interview people. At the F.A.A., officials have said that the reports in the survey were not sufficiently detailed to be useful. Laura J. Brown, a spokeswoman, said Monday that the survey had gathered “hangar talk, perceptual data” that her agency would probably find hard to integrate with its other data sources. But, she said, “We do intend to see how it can fit with the rest of our data.”
NASA Gives Glimpse of Air Safety SurveyNASA grudgingly released some results Monday from an $11.3 million federal air safety study it previously withheld from the public over concerns it would upset travelers and hurt airline profits. The data reflects hundreds of cases where pilots flew too close to other planes, plunged from altitude or landed at airports without clearance.
NASA published the findings contained in 16,208 pages but did not provide a roadmap to understand them, making it cumbersome for any thorough analysis by outsiders. Released on New Year's Eve, the unprecedented research conducted over nearly four years relates to safety problems identified by some 25,000 commercial pilots and more than 4,000 private pilots interviewed by telephone.
The results from commercial pilots appeared to reflect in part at least 1,266 incidents in which aircraft flew within 500 feet of each other, generally considered a near miss; at least 1,312 cases where pilots suddenly dropped or climbed inadvertently more than 300 feet in flight; and 166 reports of pilots landing without clearance at an airport with an active control tower. The Associated Press matched the data to the questionnaire that was used to interview pilots and was obtained separately by the AP.
The data also reflected 513 reports of hard landings and 4,267 cases of aircraft hitting birds.
Because NASA scrambled the data, it was impossible to determine whether multiple pilots might be reporting the same incidents, and a key expert said the numbers appeared inflated. NASA also did not present the data so researchers could project survey results to overall safety trends.
The data that NASA released was "intentionally designed to prevent people from analyzing the rates properly and are designed to entrap analysts into computing rates that are much higher than the survey really shows," said Jon Krosnick, a Stanford University professor and survey expert who helped design the project for NASA. He urged NASA to release more of the data needed for a better analysis.
Citing people familiar with the research, the AP reported earlier that the data showed events like near-collisions and runway interference occur far more frequently than previously recognized.
The data was based on interviews with about 8,000 pilots per year from 2001 until the end of 2004. NASA Administrator Michael Griffin said Monday the survey was poorly managed and told reporters the traveling public shouldn't care about the data.
"It's hard for me ... to see any data here that the traveling public would care about or ought to care about," Griffin said.
Griffin dismissed suggestions NASA chose to release the data late on New Year's Eve, when the public is distracted by holidays and news organizations are thinly staffed.
"We didn't deliberately choose to release on the slowest news day of the year," Griffin said.
NASA drew harsh criticism from Congress and news organizations for keeping the information secret. Rejecting an AP request under the Freedom of Information Act, NASA explained that it did not want to undermine public confidence in the airlines or hurt airline fortunes.
Griffin later overruled his staff and promised Congress he would release at least some data by the end of the year.
NASA's survey, the National Aviation Operations Monitoring System, was intended to see whether it could help identify problems and prevent accidents. Survey planners said it was unique because it was a random survey with an 80 percent response rate and it did not rely on pilots to voluntarily report safety incidents.
Griffin said NASA never intended to analyze the data it collected, but planned to pass its methodology to the aviation community.
Pilots were asked how many times they encountered safety incidents in flight and on the ground, such as near-collisions, equipment failure, runway interference, unruly passengers or trouble communicating with the tower.
Griffin outraged some NASA employees by criticizing the project and saying its methodology was not properly verified. Survey experts who worked on it said they used state-of-the-art industry techniques and carefully validated it.
NASA releases data on pilot surveyNASA grudgingly released some results yesterday from an $11.3 million federal air safety study it previously withheld from the public over concerns it would upset travelers and hurt airline profits.
more stories like thisIt published the findings in a format that made it cumbersome for analysis by outsiders.
The unprecedented research conducted over nearly four years relates to safety problems identified by some 29,000 pilots interviewed by telephone.
Earlier characterizations from people who have seen the results said they would show that events like near-collisions and runway interference occur far more frequently than previously recognized. The data was based on interviews with about 8,000 pilots per year from 2001 until the end of 2004.
The NASA website shows formatted, printed reports that the space agency scrubbed to ensure none of the pilots who were promised anonymity could be identified. The 16,208 pages were posted as NASA officials began a telephone news conference, allowing no time to look at the material and ask them questions about it. NASA did not provide documentation on how to use its data, nor did it provide keys to unlock the cryptic codes used in the dataset.
NASA administrator Michael Griffin told reporters the agency typically releases information in Adobe System's portable document format, known as pdf, which presents the information on formatted, printed pages. But there are dozens of reports available from NASA's website about other subjects in Microsoft's Excel data format, which would permit researchers to conduct a meaningful analysis more easily.
Griffin said NASA wanted to ensure that no one modified the survey results and circulated false data as NASA's research product. He said even inexpensive optical character recognition software could convert the formatted reports. Such software can risk introducing errors in the data as it performs these conversions.
"We've gone the extra mile with this data, and well beyond our original intentions," Griffin said.
He dismissed suggestions that NASA chose to release the data late on New Year's Eve, when the public is distracted by holidays and news organizations are thinly staffed.
"We didn't deliberately choose to release on the slowest news day of the year," Griffin said.
NASA drew harsh criticism from Congress and news organizations for keeping the information secret. Rejecting an Associated Press request under the Freedom of Information Act, NASA explained that it did not want to undermine public confidence in the airlines or hurt airline fortunes.
Griffin later overruled his staff and promised Congress that he would release at least some data by the end of the year.
NASA's survey, the National Aviation Operations Monitoring System, was launched to see if a massive pilot survey would help pinpoint problems and prevent accidents. Survey planners said it was unique because it was a random survey, with an 80 percent response rate, that did not rely on pilots to take the initiative to report problems but rather reached out and interviewed them.
Griffin said NASA never intended to analyze the data it collected, but rather they planned to pass on its methodology to the aviation community.
He said he had only looked at a few results, but that, "It's hard for me . . . to see any data here that the traveling public would care about or ought to care about."
Pilots were asked how many times they encountered safety incidents such as near-collisions, equipment failure, runway interference, trouble communicating with the tower and unruly passengers.
Griffin outraged some NASA employees by saying the project had been poorly managed and its methodology not properly vetted. Survey experts who worked on it, however, said they used state-of-the-art industry techniques and carefully validated the results.
NASA's handling of the matter prompted a congressional investigation and investigations by its inspector general and by a union representing NASA workers.
Jon Krosnick, a Stanford University professor who helped design the project for NASA, said the released data were inadequate and "intentionally designed to prevent people from analyzing the rates properly and are designed to entrap analysts into computing rates that are much higher than the survey really shows."
Federal Aviation Administration officials in a recent telephone interview questioned the project's results showing more safety incidents than the FAA's own data.
"It's just something that were going to have to try and understand," Peggy Gilligan, FAA deputy associate administrator, said. "We are always interested in any kind of safety data, but we always want to look at it in terms of its quality, its quantity and how we're going to use it and what assumptions underlie it."
She noted NASA's interview questions didn't track specifically with FAA report language and said pilot responses were their subjective views over 30- to 90-day time frames.
FAA officials also said multiple pilots could have seen the same events, which Krosnick said should easily be taken into account with corrective tools that were designed to make sure the survey did not skew totals.
The Air Transport Association of America, an organization representing major airlines, responded to the NASA release with a statement praising US skies as the safest in the world and describing NASA's study "as not designed to capture real-time, verifiable data."
Representative Bart Gordon, Democrat of Tennessee, chairman of the House Science and Technology Committee, urged NASA to finish reviewing the data for further release as soon as possible.
