Fusion Professionals and Australian research and advisory firm BigInsights, recently hosted an industry breakfast to discuss how organisations can capitalise on the Big Data movement and deliver real time business insights through data analytics.
Keynote speakers included Achim Drescher, Principal Data Architect, from Fusion Professionals and Raj Dalal, Founder and Principal of BigInsights. Achim presented the evolution of data analytics and what to expect with Big Data in the next five years. Mr Dalal provided insights into the catalyst of the data-driven change and delivered examples of organisations leading the way in Big Data.
BUT WHAT IS BIG DATA?
Big Data comes in the form of both structured and unstructured datasets that are available in a continuous stream. These large volumes of data represent opportunities and threats for organisations, used wisely they can provide near real time data insights, used frivolously they can become data swamps and very expensive to maintain and analyse. Big Data provides the potential for organisations to act faster and provide better service to customers.
To keep a competitive advantage, organisations must adapt to a more personalised customer experience by using Big Data wisely. With new technology offering faster insights and processing ability, now is the optimum time to leverage the capabilities of data.
THE 3RD WAVE OF DATA ANALYTICS
Over 30 years ago, businesses were driven by people where data was collected and stored by departments. Newspapers, television and billboards were the main means of connecting with customers. With the information age, data warehouses brought operations, data insights and departments together with computers and technology such as social media and Smartphones have continued the wave. This is gathering momentum empowered by cloud platforms and the emergence of Artificial Intelligence. In the next 5 years, real time customer analytics driven by Big Data will be common practice.
Early innovators such as Alibaba, Airbnb and Uber have all developed large data models to welcome the third wave of data analytics. Fusion Professionals foresee that in 3-5 years, all organisations will be using similar data models.
For example, airline systems ingest booking data in real time with up to 3,000 fields (flights, people, locations etc.) and up to 2,000 records of repeating structures. With an average of 20 transactions per second and up to 80 transactions per second in peak times, airline booking systems require a comprehensive architecture including a Data Lake feed by multiple data streams, a structured service layer, API, data visualisation tools and data replication to process this volume of information.
The below infographic is a high-level architecture example of an airline booking. This is an overview of structural requirements and architecture needed for Big Data.
WHY NOW?
Australia has more users on Facebook compared to the United States and also ranks in the top three countries when it comes to mobile broadband usage. From the rise of the humble smartphone, every click made, every swipe taken has been stored. The challenge for organisations today, is how to effectively manage this continuous stream of data to inform business decisions. The early innovators are using data science to deliver hyper-personalised customer experiences. Increasing competition and consumer behaviour is forcing companies to look to data for competitive advantage. More so now than ever, data analytics is the key to digital transformation.
The opportunity for data analytics lies in customer intimacy, operational excellence (predictive maintenance and fraud reduction) and product excellence. 70% of data analytics projects have centered around the customer experience, but it’s now evolving with a need and continued emergence of Artificial Intelligence (AI) to support the ‘next generation’ platforms.
An example of one organisation spearheading operational excellence through Big Data is Lineage Logistics in the United States. They identified their warehouse was running out of space for new orders. For space efficiency, using Big Data, they evaluated stock movements over time and thus, reconfigured the warehouse racks to have three common spaces instead of one (i.e. tall, medium and small racks). As a result, they saw a 25% increase in warehouse density at 10% the cost of building a new warehouse.
Another example is Barclays Africa who have been using machine learning and data analytics, underpinned by Big Data, to increase customer engagement. With 12 million customers in 13 countries, their challenge was predicting when customers used their ATMs. They implemented a propensity model capability, which was developed as part of an end-to-end solution that determined the optimum time to contact a customer. With this model, they could predict the shortfall in bank accounts based on customers’ historical spending pattern. They alerted customers via an SMS on the next steps to take so they didn’t default on their account.
SUCCESS FACTORS OF A BIG DATA STRATEGY
But what does this mean for SME’s and how can they use Big Data? Fusion Professionals have developed the following factors that need to be taken into consideration to successfully implement Big Data and Data Lakes:
Clearly defined vision for data and retention;
Data should be of value;
Effective visualisation of large data volumes;
Variety of data types and subject areas (structured/unstructured);
Consider the velocity of data flows and the retention period;
Volume of data (cost vs benefit); and
Veracity – accuracy of data.
WHAT’S NEXT?
By 2020, Big Data and Data Lakes will be common practice for all organisations. Not only to store data, but search and visualise data in a more seamless way with real time capability. This is what the future holds for the next 3-5 years.
“The best way to get started, is to get going!” Contact Fusion Professionals today to discover how we can optimise your platform and leverage Big Data to drive better business insights. Call us on +61 2 8569 0272 to find out more.
