Training a Stronger Lens on Aviation Safety Risk

A MITRE-developed prototype is revolutionizing the way the Federal Aviation Administration (FAA) identifies and examines aviation safety events.

Known as Aviation Risk Identification and Assessment, or ARIA, the prototype is a computer system that automatically analyzes radar and other surveillance data about flights in U.S. airspace in near-real time. Using algorithms that examine factors such as speed, altitude, and trajectory, ARIA identifies situations that may constitute safety issues. It then prioritizes them for deeper investigation.

MITRE's work on ARIA stemmed from a recognition that increasingly complex operations in the National Airspace System (NAS) require a deeper, more nuanced approach to identifying safety concerns. Air traffic is increasing. Aircraft performance and equipage are growing more sophisticated.

Additionally, increasing numbers of unmanned aircraft systems—or drones—now fly in the NAS, and the burgeoning commercial space industry is introducing still more novel vehicles into the nation's airspace.

While existing regulations, compliance checks, and safety data collection and analysis efforts have produced the safest decade of air travel in the nation's history, the FAA is taking steps to address these changing conditions and extend this stellar record.

ARIA is a major part of that effort.

Focusing Investigators' Attention Where It Matters Most

MITRE began evaluating an initial, test version of ARIA in February 2019. For six months afterwards, our researchers collaborated with subject matter experts from the FAA's Quality Assurance unit to refine the algorithms ARIA is built on.

For every safety event ARIA identified, FAA personnel assigned a score designating the event's severity. Our researchers then refined the algorithms to identify and automatically score all risk events and rank them in order of severity for further review.

Although it's still in the testing phase, ARIA should provide substantial safety benefits to the FAA—and the flying public—when it goes live in spring 2020.

"Before ARIA, FAA personnel and contractors had to hand-review roughly 200,000 potential safety events each year, which took about 50,000 staff hours," says MITRE project leader Jon Parker.

ARIA makes that process far more efficient, he explains. "For example, in 2017, FAA investigators found 7,000 compliance violations. Only a fraction of them were ultimately categorized as medium- or high-risk events.

"ARIA sorts safety events and puts the rare—but extremely important—high-risk events at the top of the pile. And it will do this in near-real time."

In addition to prioritizing safety events, ARIA provides FAA investigators with a more complete safety picture.

The system assesses traffic that occurs in regions with radar and other surveillance coverage. A piece of the puzzle missing until now is data from flights operating under "visual flight rules," many of which involve small personal aircraft.

With data from those flights, ARIA provides investigators with a more comprehensive view of potential risk in the system.

Integrating Analysis with Compliance

ARIA enables the FAA to take a more risk-based approach to safety analysis. This goes beyond traditional processes, which focus primarily on compliance with existing rules and regulations.

"With ARIA, it's becoming possible to evaluate risk alongside compliance to form a more complete risk picture," Parker says.

"For example, if the operation is compliant with the rule, but we still observe risk, that may indicate the rule is ineffective at achieving the desired level of safety," explains Wally Feerrar, manager of MITRE's Transforming Aviation Risk Management portfolio. "On the other hand, if the operation is noncompliant, but no risk is detected, that may be an indication that the rule warrants re-evaluation.

"By associating measures of risk with adherence to specific rules or regulations, the FAA can both identify risky behaviors as well as evaluate its own rules and regulations to determine which ones warrant review, revision, or simply greater emphasis."

"ARIA is a game changer," adds Mike Balder, National Quality Assurance Manager for the FAA. "It will enable us to change the conversation around safety and focus on identifying and mitigating the most significant risk in the system.

"We will have a 360-degree view of air traffic risk in the NAS and provide targeted, actionable information to the operation and support organizations to make air travel even safer in the future." 

—by Marlis McCollum