A few years ago I was talking to a very large airport, trying to understand how Splunk’s technology could help. Typically, for new customers we assist with IT and Security issues before addressing the business use cases. In this case however, the airport insisted we focused on a burning issue: How to understand runway occupancy in order to find efficiencies.
It was estimated that its most important assets, the runways, were close to 100% capacity. It could take almost a decade to build a new runway, so how would it remain competitive with other large airports until then? One thing I’ve learned; there’s always a solution with Splunk!
Occupancy of each flight on the runways
The only data available came from the Advanced Surface Movement Guidance and Control Systems (A-SMGCS). This is a standard system used by airports and airlines to provide situational awareness of aircrafts and vehicles. There are industry standard solutions for this data, but these are limited in terms of analytics.
A-SMGCS captures the position of the aircraft relative to the control tower, altitude, the ATC call sign of an aircraft and more. The format of the data given to me by the airport was in the format of very large XML files collected from an ESB:
The XML format wasn’t a problem since Splunk will index this data like any other. However, knowing the position of the aircraft relative to the control tower, did not tell me how long it spent on the runway, or if the aircraft was even on the runway at all.
Only answer would be to apply spatial analysis of the data. I created a simple custom command in python (“positionaircraft”), which based on coordinates relative to the control tower (positioned at 0,0), calculates if the aircraft intersects specific polygons (also defined as coordinates relative to the control tower). This is known as a Point In Polygon (PIP) algorithm. The obvious polygons are the runways, but you could define at any point, polygons of other parts of the airfield.
sourcetype=radar TransportationRegistrationIdentification=HBIPS | parseXML _raw | eval x=tonumber(""+XSGN+XPOS) | eval y=tonumber(""+YSGN+YPOS) | positionaircraft xField=x yField=y outField=position
I then used the transaction command to identify the continuous length at which a plane intersects the same polygon, giving me the time on the runway:
sourcetype=radar TransportationRegistrationIdentification=HBIPS | parseXML _raw | eval x=tonumber(""+XSGN+XPOS) | eval y=tonumber(""+YSGN+YPOS) | positionaircraft xField=x yField=y outField=position | lookup flightdetails AircraftRegistration as TransportationRegistrationIdentification OUTPUT ArrivalOrDeparture, CarrierCode, FlightNumber, FlightStatus, AODBUniqueID | search position=*runway* | transaction TransportationRegistrationIdentification, position, ArrivalOrDeparture maxspan="10min" | eval takeoff_time=strftime(_time,"%Y-%m-%d %H:%M:%S") | fields - _time | table TransportationRegistrationIdentification, takeoff_time, TrackSizeClass, position, ArrivalOrDeparture, CarrierCode, FlightNumber, FlightStatus, AODBUniqueID, duration | sort - TrackSizeClass, duration
NOTE: There are now some Spatial-Mapping Apps available for Splunk.
Distance between aircrafts
To calculate the time-distance of the aircrafts as they land/takeoff, I used the same principle: Define a small polygon somewhere on the runway and calculate the difference in time between the last event for one plane that intersected the polygon, with the first event for another plane.
Paths that aircrafts took on the tarmac
Lastly, it was important for the Airfield Operations teams to know if some aircrafts were misbehaving on the tarmac, as pilots sometimes block other planes, turn off at wrong places and get lost. A single mishap can have a cascading impact on the whole day’s operations. To combat this, I converted the coordinates of the plane that were relative to the control tower, to global cartesian coordinates (latitude, longitude).
sourcetype=radar transportationRegistrationIdentification=HBIPS | parseXML _raw | lookup flightdetails AircraftRegistration as TransportationRegistrationIdentification OUTPUT ArrivalOrDeparture, CarrierCode, FlightNumber, FlightStatus, AODBUniqueID | eval xPosition= tonumber(XSGN+XPOS) | eval yPosition=tonumber(YSGN+YPOS) | streamstats window=2 current=false first(XPOS) as previousXPOS, first(YPOS) as previousYPOS by TransportationRegistrationIdentification | eval foo1=previousYPOS-yPosition | eval foo2=xPosition-previousXPOS | trigonometry outField=heading function=atan2 foo1 foo2 | trigonometry outField=pi function=pi test | eval heading=heading*(180/pi) | eval offset=sqrt(pow(xPosition-previousXPOS,2)+pow(yPosition-previousYPOS,2)) | eval distance=sqrt(pow(xPosition, 2)+pow(yPosition, 2)) | trigonometry outField=bar function=atan2 xPosition yPosition | trigonometry outField=degrees function=degrees bar | eval bearing=(degrees+360)%360 | trigonometry outField=coordinates function=positionaircraft distance bearing | table TransportationAtcCallSign, TransportationRegistrationIdentification, TransportationIataTypeCode, TrackSizeClass, CarrierCode, FlightNumber, AODBUniqueID, ArrivalOrDeparture, bearing, coordinates, XP*, XS*, YP*, YS* | rex field=coordinates "(?P<lat>[^,]+),(?P<lon>.*)"
Finding even the slightest efficiencies can be worth millions to an airport. So, how long did it take to do this? About two days, and the radar data generated only a few GB per day. That’s not a bad return on investment!
If you would like more details about this use case, ping me in the comments below or on Twitter at @lemastergui - I’m happy to share my configuration. You may also like to catch up on Gatwick Airport’s webinar on how it uses Splunk to optimise passenger experiences by analysing terminal, road and rail data.
Thanks for reading!