Virgin Australia is using the Kafka stream processing platform to power a growing range of customer experience features, from automated rebooking of flights to journey and baggage tracking tools in its app.

Integration platform lead Nick Mouton told a Confluent data streaming world tour event in Sydney this week that the airline replaced “batch-oriented legacy middleware and messaging technologies used to enable data flows between systems” with Confluent Kafka.

A 4impact case study suggests Kafka has been in place at Virgin for several years, underpinning a capability called the flight state engine or FSE, which Mouton described as “probably our first major use case” for Kafka.

The case study notes that FSE pre-dates this, but was re-platformed from “end-of-life Oracle SOA [service-oriented architecture] technology” to Kafka.

FSE, according to the case study, is used as an “authoritative view of flight status”, taking in data from “dozens” of in-house and third-party systems, reordering it, and then using it for operational planning and communicating updates to customers.

“The FSE, as the name suggests, is a state machine that sanitises and coordinates changes to flight timings and movements,” Mouton told the Confluent event.

“It’s a stream processor at heart. It streams updates from hundreds of systems – aeroplanes, airports – [taking] all these different sources of information and sanitising and sometimes reordering them because they don’t always come in in the order that you’d expect. 

“Some of those [source] systems have spotty internet or … could be in an outback airport, [and] it doesn’t have the most reliable delivery mechanism, so we need to be able to put those messages into order and make them useful to other downstream systems.”

Kafka has since been used to underpin a range of other passenger-facing applications.

One of these is the “journey tracker” in the Virgin Australia app. This, according to Mouton, is “a live tile that … keeps our guests up to date with the latest information on their journeys, communicating gate changes, delays and other relevant information in the app”.

“Another key feature built on event streams is the baggage tracking tool,” he added.

“Users of that mobile app now have their bags checked and registered to them automatically. 

“What happens is guests can now receive baggage updates as soon as their bags are put on the plane, and then carousel information when it’s ready to pick up on the other side.

“We find that’s very performant, to the point where when [the phone] shakes, your bag’s coming up [for collection].”

Mouton raised a fourth use case for event data streaming – what he called the “automated passenger recovery” system, or APR.

“If a flight gets cancelled, that’s when the APR kicks in,” he said.

Cancellation events are sourced from the airline’s reservation system.

“We use that data pipeline to determine when our guests have been disrupted,” he said.

“If they’ve been disrupted, we can automatically repatriate them on a new flight, and it’ll actually send notifications to you to say, ‘Hey, we rebooked you on this flight, is that OK? 

“Then, you can make some of your own decisions from there.”

One use case not mentioned – but illustrated in a recent video – targeted improvements to the process used to generate a final passenger manifest and deliver it to a flight crew prior to departure.

The video suggests that the process runs an average of 40 seconds faster, “which has a material effect on how long it takes to get a flight in the air.”

Supporting “immediate action”

Mouton said that a big part of the reason for switching from batch to real-time data event processing was to improve operational and customer experience.

“The key objective was to achieve real-time visibility and responsiveness across operations,” he said.

“Previously, events such as flight changes, baggage handling, and passenger check-ins could take a significant amount of time to process and communicate to downstream systems. 

“We wanted to modernise that process and provide instant visibility to empower our operational teams and help them take immediate actions.”

Mouton noted the importance of “immediate action” when it came to serving customers.

“They don’t want to wait in line to figure out what’s happening at the airport,” he said. “They just want to know [now].”

Mouton said the selection of Kafka also supported Virgin’s ability “to leverage cloud-native architectures”, although he declined to elaborate further on this when asked.

He added that an emerging use case for Kafka “is providing some data pipelines for our analytics department.” 

“They’re sourcing real-time transactional data that’s currently very siloed in various systems, and I think that Confluent Kafka could be a key tool in that space,” he added.