Case Study : Big Data solution for real time customer insights at leading Airline

Large Australian Airline with over 30,000 employees and more than 6000 daily flights.


In the Airline industry booking data is the essential driver for many business decisions and is an important input in many operational and sales processes. To provide the best service to customers the Airline requires this data and associated customer insights in near real time to be provided to the greater business community.

The objective was to build a platform that can provide data from various different operational data sources, analyse this data in near real time and provide the information to business systems.

Some data structures contain more than 3000 fields that are updated over 1 million times per day with peaks of over 80 transactions per second.


The solution was built in line with a typical Bigdata architecture. It is based on the Hadoop Ecosystem hosted on the AWS cloud platform Elastic Map Reduce and a number of peripheral systems allowing ingestion and event driven distribution of data to business systems in near real time.

The platform ingests data from a variety of data sources. The main source is the Amadeus Booking system that provides a live feed of booking updates in a continuously ingested stream. Other real time data feeds include Flight updates and Customer Profile information. In the future more Data Sources will be added with the aim to enrich the data and to provide the ability to perform cross domain analytics.

Processing is based on Clustered computing models using the Apache Hadoop ecosystem. Spark Streaming is used for long running, streaming applications that process up to 500 transactions per second. Transient clusters use the capabilities of code driven cloud deployments to generate the infrastructure on the fly and shutdown once the analytics job is complete. This allows cost effective and flexible processing for complex analytical jobs with large amounts of complex data.

Data is stored predominantly in clustered relational database systems and simple storage solutions that allow structured as well as unstructured data to be stored.

Data is served to operational and business systems that require access to real time customer insights. The Data API exposes the data via an API Gateway to internal system or third party partners.

An Event API provides the means to notify interested systems of changes in the underlying data. This way systems can make full use of the real-time nature of the data and the analytical capability.


Fusion Professionals provided technical leadership to make this project a successful implementation. Our staff provided Solution Architecture, Development Management, established development processes and standards as well as implemented large parts of the framework.


Fusion Professionals is very experienced in providing streaming Bigdata platforms based on Hadoop and Spark frameworks. Our technical ability and architectural excellence helps customers move to the modern, integrated analytics platforms for 21st century business models.


The delivered solution is a platform rather than a standalone system. It is an extendable data analytics platform that will allow the airline to process large amounts of data, process machine learning experiments as well as do the heavy lifting in complex analytics processes that interact directly with operational systems.

Here are some of the highlights:

  • Near real time analytics capability
  • Bigdata processing platform
  • Architecture framework
  • Event driven API interface to socialise real time customer insights
  • Fully cloud hosted
  • Continuous deployment of infrastructure and applications
  • Operational architecture for monitoring and management

Cloud hosted platforms can provide extreme processing power at relatively low costs. This can be achieved with on demand deployment and automatic scaling of environments. Spot pricing can be used to create large processing clusters at a fraction of the cost of inhouse analytics platforms.


Information cycles are becoming faster with customers expecting personalised service experiences. This is only possible if data is analysed fast and made available to the business units in real time:

  • Real time customer insights
  • Better service ability through faster data cycles
  • Ability to engage with customers in a more personalised customer experience
  • Improved profitability through better customer sentiment

Whilst this sort of business model is very cutting edge today in only a few years it will be a must for all companies to know their customers and their daily interactions with the company in real time.


This bigdata platform is designed to perform complex processing in extremely short time frames. Here are some of the benchmarks achieved during testing and in production:

  • Over 500 transactions per second
  • 15-20 second latency from business transaction to insights delivered
  • Average processing time within the system 2-3 seconds
  • Over 900 API calls per second (testing limit)
  • High Availability with inbuilt Disaster Recovery
  • Auto Scaling and Auto Healing
  • Deployment of complete environment in under 50 minutes

Start your journey into the 21st century data processing today and let us talk to you about your requirements to move your business closer to your customers.


Fusion Insights

Building a system that houses your organisation’s data can be daunting, especially now that data acquisition is growing rapidly. The…


Human-to-machine communication has not yet been perfected, but enterprises are already beginning to integrate this groundbreaking technology into their operations,…


Fusion Professionals has signed a partnership agreement with MapR Technologies, provider of the industry’s leading data platform for AI and…


“Big data is at the foundation of all of the megatrends that are happening today, from social to mobile to…


In recent years data volumes have been increasing dramatically. This has created major challenges for traditional analytics platforms in terms…


With the increasing volumes of data that can be cost effectively stored in the cloud, comes increasing responsibility. The current…


With the advancement of technology and abundance of data your business receives on a daily basis, companies are now in…


Fusion Professionals held its annual Fusion Summit last Thursday the 18th of October at the Rag and Famish Hotel in…


The Client is one of major NSW government departments providing services to public. The Department had been experiencing performance issues…


Though its conception dates back to 1979, containers made their mark as much needed, major technology assets in 2000. Digital…


Objective The intelligent mobile app-based lending system is a new field, blending recent technical developments in mobile phones and Artificial…


Our Client is a well-known Australian freight logistics company, operating in railway freight and shipping.  The company embarked on a…


Data warehouse management and data analytics always had the challenge to decide what data to store and for how long…


Cloud computing is becoming a preferred storage platform for IT managers and organisations in general. In Australia alone, 31 percent…


Serving your customer in the best possible, most efficient way should always be the major goal of any organisation. The…


Moving out from proprietary software seems like a daredevil act, considering the possible data security issues some open source databases…


The Challenge Complex IT environments can pose significant technical risk that, if not managed adequately, have the potential of major…


Fusion Professionals has signed a partnership agreement with Waterline Data ( ) the leading provider of Information Catalogs and…


Most people do not like change. As much as possible, they want things to stay the same that is why,…


Regardless of your infrastructure whether you are running in the cloud or on-premise, there will always be a need to…