A Long-Standing, Successful Aviation Safety Reporting System

Air Line Pilot, January 2001, p.28
By Capt. Thomas A. Duke (Ret.)

or more than 10 years now, British Airways has been compiling reports and data from its employees on safety problems. Working closely together, the company, the British Air Line Pilots Association (BALPA), and other employee representatives devised a computer program called BASIS (British Airways Safety Information System) and have continually fine-tuned it and tweaked it for possible ways to improve flight safety.

Early last July, I happened to be in the neighborhood of London Heathrow, so I leaned on the editor of the British Airways quarterly safety magazine, Flight Deck, for a visit to see how a well-established program that is the model for U.S. airlines’ Aviation Safety Action Plans and Flight Operations Quality Assurance programs is doing. I was treated like a "Royal" (though I doubt if it had anything to do with my name).

Safety Services at British Airways requires both active and passive flightcrew participation.

Every aircraft in the airline’s 280-airplane, 11-model fleet is equipped with blank "Air Safety Reports" with a 40-item list of mandatory reporting events (see sample, page 29). In addition, all but a few of the airline’s airplanes are equipped with a quick access recorder (QAR) that can easily be pulled daily for Flight Operations analysts to monitor for "exceedances."

In the United States, flight data recorder (FDR) information is customarily available only when investigations take place.

British Airways developed the Special Event Search and Master Analysis (SESMA) in the 1970s. Safety Services uses SESMA to publicize data (which BALPA deidentifies) to recognize safety trends and to foster improvements. BASIS and SESMA are complementary, but distinctly different, safety programs.

Safety Services compiles this information into meaningful and valuable data and analysis and publishes a monthly report in a pamphlet called Flywise. Thus, both management and flight crews are informed about the latest perils in the system and are motivated to keep up the good work and avoid what has happened to others. When watching for trends and ways to improve safety, British Airways has solid data with more than 800 Air Safety reports (ASRs) and SESMA inputs a month to draw from.

The information is quite public but available only for flight safety purposes. By following the reporting rules and allowing the intrusion of QAR monitoring, British Airways flight crews are participating in the solution, not suffering from the unintended evil of errors. Anonymity is controlled. Self-incrimination is not a worry. If a report or exceedance reveals a need to "talk to or counsel" a flight crew member, the phone call comes from a BALPA union representative, and effective positive countermeasures are mediated. If the civil authorities become involved because of an external report, British Airways handles it judiciously. It is a trusted promise. Essentially, peer pressure has replaced fear, and safety improvements are eagerly sought.

The Flight Safety Foundation’s Flight Safety Digest, July-September 1998, listed only four U.S. carriers participating in trial FOQA programs with the help of some FAA funding. More than 35 non-U.S. carriers are gathering data similar to British Airways’ SESMA program. Resolving the legal use of FOQA information remains contentious to this day in the United States. Instead of continuing to maintain a traditional safety enforcement mindset, both the FAA and the airlines could benefit from the additional safety information and the reduced need for inspectors, litigators, and persons to handle FOIA requests from the press or researchers.

A British Airways pilot fills out an Air Safety Report form (see example above and right) immediately after the mandatorily or voluntarily reported event and turns it in to the next station dispatcher, duty engineer/officer, or agent. The fact that reporting 40 types of events or incidents (see Table 1 for list) is mandatory takes most of the guesswork out of the decision to submit. The bigger incentive for reporting an event is the QAR or an assertive first officer who knows the rules, too.

British Airways pilots file more than 400 ASRs each month, 10 to 15 a day, about one per pilot per year. How often a pilot fills out an ASR varies, depending on which aircraft the pilot flies and how numerous the fleet or destination problems are. I spoke to a captain who last filed one 6 years ago.

Safety Services has about six analysts—pilots, flight engineers, and statisticians—who first assign a risk assessment category rating based on a judgment of frequency and severity of occurrence:

* The designation severe is assigned to a rare incident requiring the highest priority for resources and action; it generally involves a State investigation.
* The designation high is assigned to incidents of significant concern that take priority over other incidents—they may involve a State investigation but Safety Services usually investigates them.
* The designation medium is assigned to incidents that require the attention and action of a line department; the probability of recurrence distinguishes them from low.
* The designation low is assigned to an incident of low concern that normally requires no further action but a line department may investigate it.
* The designation minimal refers to incidents that are a matter of only statistical interest.

For some special studies, the analysts assign a risk assessment number between 1 and 100 for each category. They would assign a rating of 100 when they find an imminent danger of tragedy, such as near fuel exhaustion or a hard TCAS (RA, or resolution advisory). The analysts assign "1" (minimal) rating when the potential for a major accident is extremely remote, such as a minor flight plan error corrected enroute and reported in the interest of safety—many safety barriers are designed in the system between a reported safety event and tragedy. The risk-assessment numerical rating system is based on European Space Agency and nuclear safety practices and has been very helpful in assessing safety improvement or deterioration trends. The system allows the airline to take action based on incident data before similar but very infrequent serious accidents (surprises) occur.

The system seems to work. The 1–100 risk assessment system indicates that the degree of risk per 10,000 departures has remained steady since 1993, at the same time that departures increased 6 percent and pilots’ ASRs increased 140 percent. This demonstrates that the overall risk to the operation has been reduced. According to Flywise, March 2000, "this has been a direct result of the application of best safety management system practices, in particular in the areas of leadership, planning, training, reporting and investigating, monitoring, and audit and review." [Note that "enforcement and compliance" do not appear and that openness and leadership are emphasized.]

To prevent recurrence of safety problems, the risk assessment team, based on its judgment, assigns a status to ASRs for action, as follows:

* Active investigation—when the team does not know or fully understand what actions are required.
* Action required—when the team knows what actions are required but they have not yet been taken.
* Action monitored—when the effect of required actions is being monitored.
* Report monitored—when required actions have been made without the need for a line department or Safety Services investigation; rates of occurrence are monitored.

The line department most involved in identifying the cure for the problem and having the authority to make changes investigates incidents, as follows:

* The manager of Air Traffic Services reviews events caused by blocked runways and wake turbulence events and the causes of go-arounds and all other ATC incidents.
* Flight Technical Services reviews the circumstances of TCAS events.
* All medical illness and injury events are reported to British Airways Health Services.
* The frequency and seriousness of birdstrike events at each location are monitored for significant trends.
* Aircraft Performance Engineering analyzes flight data recordings.
* The level of nuisance/false GPWS warnings is under constant review for significant trends.
* Fleet Management constantly reviews RTO events for significant trends.
* Fleet Management reviews handling events for significant trends.
* Line management reviews ground incidents for preventive action.
* Safety Services monitors weather events for significant trends.

Flywise lists the monthly data results about 30 days after the events occur. Each type of aircraft is analyzed by number of reports, with a total of each type of event, and a total of risk categories. For example, Flywise gave mini-narratives of the three medium events it reported in January 2000 (see Table 2): while the airplane was parked, a refuel panel door was found partially detached from aircraft; during climb, an altitude bust of 300 feet occurred; and during descent, an altitude bust of 470 feet occurred. Each narrative included preventive action taken, which included special monitoring of ground handling and preflight checks, adding cases to a special human factors study, and notifying ATC of the investigation of a mandatory CAA report.

This is an example of a full report of an event from a different fleet, further deidentified: Taxi-out: Runway incursion. Crew mistook takeoff clearance. Whilst [crew was] taxiing out, ATC asked if the aircraft was ready for departure from point B on Runway 26L. The crew advised that [they] could depart from Point C. ATC accepted this. The handling pilot got into a mindset that takeoff clearance had been given. The NHP noticed too late, and the aircraft had crossed the holding point. Neither crew[member] has ever been involved in anything similar, and there is no reason to believe this was anything other than an isolated incident. (Medium)

With more than 400 ASRs entered each month into the BASIS computer program, Safety Services can develop many different reports that measure success and failure and can then direct where most of the prevention efforts should go. For example, in Flywise, March 2000, charts show ATC to be the hottest high-risk subject in 1993, 1995, 1997, and 1999, indicating that this is an area in which British Airways could "exert sufficient influence." TCAS was added as a topic in 1998 and adds more urgency to ATC as a topic, as TCAS reports came in third, behind ATC and pilot handling/airmanship, in 1999.

An investigation of a recent British rail tragedy revealed that management had failed to respond to numerous previous similar incidents. In reaction, Flywise, January 2000, reported that Safety Services had taken a 12-month look at all high-severity ASRs to assess potential outcomes by accident category for all the world’s recent hull loss experience. Here the BASIS data showed that reports of potential midair collisions were most numerous by a large margin. Safety Services was able to document many management-initiated safety improvements. Although most of the British Airways fleet is equipped with TCAS, the airline cannot directly control many matters, including the quality and coverage of ATC, other operators’ TCAS-modification standards, timely investigations, and the mix of airline and general aviation traffic.

The remaining top 12 categories for high-severity accident potential, based on ASRs in the BASIS files, were in-flight fires, control loss from technical causes, runway excursion landing, runway excursion takeoff, smoke/fumes, communications loss/midair, hard landing, controlled flight into terrain, runway collision, control loss from nontechnical causes, and fuel exhaustion. Current high-interest items like fuel tank explosions and ground collisions appeared rarely in the ASR database. FOD on the runway or downstream damage from tire explosions were not on the list of high-severity ASR events associated with worldwide hull losses.

The British Airways ASR (ASAP) reporting system involves intensive labor and human analysis, but SESMA is almost all automated, except for inputting exceedance parameters. At the end of each aircraft’s service day, maintenance personnel remove the QAR disc and turn it in to Flight Operations. The data are all entered into the computer and automatically scanned for the exceedance information that fits ongoing preprogrammed projects. Between 400 and 500 reports "pop up" each month. Nearly all are very minor. Individual narratives are not publicized in the monthly Flywise, but frequent trend analyses are. The British Airways safety publication, Flight Deck, also publishes them (see Table 3).

Flywise, February 2000, provides a good example of SESMA information that is useful for all flight crews. In October 1999, the data showed "the largest improvements were in go-arounds, heavy landings, and deep landings [beyond the touchdown zone]. Most fleets shared this improvement, but the B-757 had no change and the A320 was slightly worse. Most of the A320 events were caused by external factors (ATC and weather), but there were three incipient rushed approaches —one was continued, but two, sensibly, were abandoned."

The analysis also commented on rushed/unstabilized approaches at certain identified airfields where exceedances were on the rise. They were added to a list of destinations requiring special briefings on avoiding rushed approaches.

The pilots involved in SESMA events are identified by code, and only their union, BALPA, has the key. The airline and BALPA have entered into an agreement that stipulates that "evidence from a flight data recorder alone will not constitute a basis for any disciplinary hearing or action." If another report identifying a deviation pops up in Fleet Management, the contact for further information requests is a BALPA representative, usually a pilot qualified in the same aircraft. If needed, SOP interpretations are retrained, and simulator training is available free of incrimination. Peer pressure and safety improvements are the ally of safety.

The knowledge that the QAR is aboard the aircraft has helped reduce the divergences in SOP performance. In some cases, the SOPs themselves have been improved. Some wear-and-tear items on aircraft have also benefited. At some airports, departure procedures and routes were found to be impossible to fly in certain wind and temperature conditions, and ATC has been notified. Arguing with the documented facts is difficult. The SESMA program is also used in the British Airways engine-health program and for monitoring continuing aircraft performance, all to the ultimate benefit of flight deck crews.

One glaring question this author posed to the analysts was, "When does a low or minimal monitored event attract enough attention to merit some accident prevention measures?" In other words, suppose that a repeated identical minimal problem shows up in a severe accident scenario somewhere in the world. They are normally scored at a "1" risk level each. Do they ever accumulate enough interest to receive the attention of a high or severe/imminent-danger event?

With the current paradigm for prioritizing safety action items concentrating on high-visibility fatal and hull loss accidents, the small stuff that invariably needs improving too often gets left by the wayside. History shows that future accidents often involve neglected issues.

The BASIS analysts have struggled with that and have upgraded many repetitious minimal-risk events to require management investigation and action. British Airways is proud that the number of reports has increased from 2,500 per year in 1992 to more than 8,000 a year in 1999, but the number of severe- and high-risk events has greatly diminished. This indicates that this voluntary and open reporting system has greatly improved air safety, according to British Airways Safety Services.

Perhaps the most salient point to make about long-standing, successful aviation safety reporting and analysis systems was made by British Airways’ present head of safety, Capt. Roger Whitefield, when USA Today interviewed him in May 2000: "This is probably the most important safety management tool any airline could have. If you don’t have it, you don’t know what your operation is doing.

"Instead of worrying about how the data could be used against the airline," he said, British Airways "believes that the data and its efforts to improve safety protect it from legal action. Our legal position is that if we had these tools and didn’t use them, we’d be liable."