Article

Real safety and capacity gains at Heathrow from artificial intelligence initiatives

NATS digital tower simulation

The introduction of an artificial intelligence (AI)-based airfield monitoring system, via the NATS Digital Tower Laboratory, at London/Heathrow could help the airport reclaim up to 20% of lost capacity caused by low cloud and reduced visibility.

Heathrow is the world’s most capacity-constrained airport – delays and disruptions there caused by bad weather have a knock-on impact throughout the continent and beyond, so any improvement in the resilience of the airport during times of low visibility has a strategic value to the entire European network.

The airport is impacted by a lack of visibility 12 to 15 days of the year, when the top of the control tower disappears into low cloud, for example. Such events typically last for between 30 and 90 minutes; as Heathrow is working to about 99% of its capacity, that means an impacted hour will affect the schedule for the rest of the day. It also has a significant impact on the European network.

But AI could provide a surprisingly swift and significant uplift in the airport’s resilience performance.

UK air navigation service provider (ANSP) NATS has been working on an AI programme within a purpose built Digital Tower Laboratory (see box overleaf), located inside the Heathrow control tower, a 87-metre structure which provides commanding views of the airport and surrounding landscape but which can also disappear into low cloud, even when the runways below are clear.

The lab is transforming the picture that controllers have on critical airside areas of the airport, improving safety and capacity while reducing workload in fair weather and foul.

The programme comprises a network of ultra HD 4K cameras integrated within an AI and machine-learning platform called AIMEE, developed by Canada-based Searidge Technologies. Aircraft and ground vehicles equipped with transponders are tracked by airport surveillance radar (ASR) and GPS location data. Radar data is received from four different radar heads, to give a maximum coverage of the airfield, while further tracking data is available from multilateration position identification. At the heart of the platform is Searidge’s Hold-Line Surveillance System (HLSS), developed to ensure runway and taxiway operations can be safely monitored in International Civil Aviation Organization (ICAO) ‘VIS 2’ visibility conditions, when some or all of the manoeuvring area is no longer visible to controllers from the tower cab.

The enhanced view of the airfield area is displayed on a ninescreen video wall, each screen comprising a 55-inch ultra HD (4K) display, with ground vehicles and aircraft identified by real-time smooth labelling.

The AIMEE platform interprets the images, tracks the aircraft and then informs the controller when the aircraft has successfully cleared the runway and its associated protected areas. The controller then makes the decision on whether or not to clear the next arrival.

“For the controllers it’s a sharp contrast to the real-world view they have from the control tower – they come downstairs and they have this ultra-high-definition image, really sharp and with that extra layer of data over the top of it. Then at night time the low-light performance of the 4K cameras is better than the human eye so you can actually see the colours of the aircraft paint scheme and paint markings on the runway which you can’t see from the control tower. That positive effect means you instantly have an engagement to the technology.”

One key element of the AI-based system is the software which smooths the update rates – the ASR/GPS and multilaterationderived labels have a one-second update rate while the 4K cameras run at 25 Hertz, or 25 frames per second, so the software has been developed to ensure the labelling sticks with the target aircraft despite the differences in update rates.

But implementing any new technology at such a busy hub – especially a new technology which pushes the boundaries of human/machine interactions – is an extraordinarily complex task. There is no quiet period in which to slowly introduce the new system and the controllers need to receive exactly the right amount of information at the right time – which has meant NATS has had to undertake some innovative, lateral thinking to bring the lab into operation.

“If you want to be able to deploy these systems in a rapid, sequential way then it needs to be done in a very intuitive manner so the controllers will be able to easily interact with it” says Taylor. “We did not want something that required a huge amount of training and was a sudden change to work practices. We wanted something to enhance their current experience and reduce workload.”

In a conventional ATM equipment procurement programme an engineering-focused requirement spec is developed, a tender is sent out to industry, replies are compared, and the new solution is tested and then put through a site-acceptance process. But with the digital lab, Searidge engineers sit directly alongside the end users and the product evolves according to the input of the controllers. So, for example, cameras were placed near the runway exits in such a way as to avoid an aircraft obscuring the view of an exit. But controllers did not need or want to see individual footage of each runway-exit camera. So the engineers developed a system which provides live prompts to the controller when an event occurs – such as the aircraft tail clearing the runway.

Interested? Subscribe!

Never miss out on the latest developments from EUROCONTROL

That is what the controllers monitor every day in fine weather and that is what the AI and camera network has been configured to deliver in all weather conditions.

NATS, Searidge and the UK Civil Aviation Authority (CAA) have also set up an innovation team to help develop a new regulatory framework to assist in developing appropriate standards for such AI airport operations technologies and procedures, and to validate the results of the initial test phase.

“The AI model can operate day or night and that’s part of what the validation process is showing: that the performance regardless of whether it’s day or night conditions is equally good,” says Andy Taylor. “So potentially we will have benefits in reduced workload as the system will undertake much of the monitoring work as it scans the entire airfield constantly, providing feedback 25 times faster than the most up-to-date, multilateration, ground movement radar.”

According to Taylor the controllers’ reaction to the new system has been enthusiastic.

“When they see the Heathrow operation running live, in the lab downstairs, and can compare and contrast it exactly to what they’ve just seen from inside the control tower, it is a powerful contrast. We’re getting great feedback and they’re engaged.”

For NATS, the AI digital tower lab journey is just starting. Once the technology has been validated the ANSP will start looking at using the system for both main and contingency operations at Heathrow. 

“We don’t see that building a typical tower is necessary when you have the digital capabilities that you can bring together now,” says Taylor. “At Heathrow we have had a contingency facility in place for over 10 years, but it has no windows and no camera surveillance which means it has a 20% less capable operation than the main tower. What we would like to do is upgrade that facility and make it 100% capable. In fact with the cameras and integrated AI system we are developing here there is the possibility of having a contingency system which has more capability than the current, conventional tower. That’s true not just for Heathrow but any airport tower.”

NATS has already begun work on planning the next applications of the digital tower lab, developing new capabilities – many focused on developing human-machine interface applications – based on a long list of operational constraints that impact airports around the world. The work is developing on an application-based approach, says Andy Taylor, which is helping to change the long-term asset cycle approach to capability upgrades, one individual system at a time, to one closer to an open ecosystem approach more like a smart phone than a traditional ATM systems procurement strategy.

“NATS looks at the challenge of airport capacity in the same way as EUROCONTROL – in the current growth market airports are a considerable potential constraint to the network so we have to do whatever we can to make our airport operations as efficient as possible,” says Taylor. “I think we have to get as much efficiency out of what we currently have while optimising any future investment in aviation capacity by ANSPs, airports and governments. We’re all in this together and we all have to play our part.”

The digital tower laboratory timeline

Work on the NATS Digital Tower Laboratory at Heathrow Airport began in 2018. From January to March 2019 the programme started with a series of real-time, non-operational data collection/validation trials which involved the monitoring of more than 50,000 arriving aircraft to ensure the accuracy of the system, with the results of the project presented to the Civil Aviation Authority (CAA). This will be followed by a deployment phase, which will see added cameras around the airfield and integration of the data output on controller’s screens.

Read our full Skyway issue

Learn more about the new era for aviation powered by AI and our role in it.