Increasingly technology scholars and analyst the world over seem to be in agreement around what is (are) the most important technology trends shaping our world Very easily the four mega-trends that everyone seems to agree is at play are Cloud, Social, Mobility and Big Data.
I would say there has been a lot of discuss around three of these trends in our locale – Cloud, Social and Mobility, what has not yet start to be main-stream in conversation and dialogue is Big-data. Some may argue it’s not near or that it’s still far from us but some recent occurrences in our country seem to challenge this notion and point to the fact that the time of Big Data is upon on us than we think. But maybe the first challenge is the understanding of what Big Data means? [pull_quote_right]I would say there has been a lot of discuss around three of these trends in our locale – Cloud, Social and Mobility, what has not yet start to be main-stream in conversation and dialogue is Big-data. [/pull_quote_right]
The term big data was coined by the sciences like astronomy and genomics which first experienced data explosion. According to Mayer-Schonberger & Cukier, in the real sense of the word, there is no rigorous definition of big data; one simplistic way to think of it is: big data refers to things one can do at a large scale that cannot be done at a smaller one, to extract new insights or create new forms of value, in ways that change markets, organizations, the relationship between citizens and governments and more. In 2012, Gartner defined it as follows: “Big data is high volume, high velocity, and/or high variety information assets that require new forms of processing to enable enhanced decision making, insight discovery and process optimization.[i] To address the challenge of data haven grown so large that it no longer fit into the memory that computer use for processing, engineers came up with new processing technologies that lets you manage large quantities of data than before and more importantly the data need not be structured (in tidy rows or classic database tables).
Back to the recent local occurrences, the recent spate of air-craft mishap in our country for example got me thinking could big data have helped? In my thinking probably yes, what if our aviation industry and its regulators had capabilities that is able to use predictive tools (basically using mathematical models and historic data) to determine before a flight takes up to a near perfect prediction that a flight may most likely go wrong mid-air? A model that marsh data that includes weather conditions (like wind and storm, etc.), data created by jet engines in real time, sensors collecting data on the surrounding environment (temperature, humidity, air pressure, etc.), air-craft maintenance data, number of hours clocked, etc. regulators are therefore able to ensure compliance and safety in real-time by setting up all the compliance rules & safety criteria and validating them against a streaming data set comprised of data from flight sensors, flight management system, vertical navigation system, etc. The system is built to trigger alerts, in real-time, if there is a potential compliance breach or a safety concern.
And another occurrence that comes to mind is the banking crisis that we have only truly/somewhat started to come out of, again maybe if the regulators had capabilities to simulate in a near to real time the sensitivity of each bank’s Capital Adequacy Ratio to moving data feeds like (both external & internal); Sectorial Risk tied loan-book exposure, Cash Reserve Ratio, Foreign Exchange Rate movement, Consumer Price Index, Loan Account Velocity, Asset/Collateral Cover & Quality, Quality & Sophistication of Loan Management Process & Organization, Frequency of Approach to Re-discount Window, etc. Maybe, just maybe they could have saved the day by knowing the best time to stop the systemic hemorrhage possibly with the right ‘policy-dosage’.
Beyond this situational scenarios, increasingly enterprises are finding new use for big data that was hitherto not possible. Before now when people think of big data, they think of social media and Internet sites but this is changing very quickly due to advances in technologies which is bringing to mainstream and within affordable reach the discuss and relevance of big data in enterprise and government. Advances in technologies like in-memory technologies that allow for processing of millions of rows of data in seconds, increased sophistication of analytics software and tools allowing for deeper/greater data visualization and Self-service BI Technology. These coupled with the always connected trend powered by two of the 4 technology mega trend (Cloud & Mobility) mobility fueled by proliferation of mobile devices.
Would it be interesting to a CEO of a bank when the banking relationship starts to wane with a particular customer segment; say after a set number of complaint and complaint type, in a particular branch category/location running a particular product based on their social profile (tribe, creed, current status in life, network, etc.)? Assuming he gets to have an early-warning signal just as that threshold was about to be reached. This is powerful and business value in that it ensure that truly organization get to know something is happening and needs immediate attention before it truly happens and we are left with situational analysis as to causes. Would it also be interesting to be able to generate Liquidity Risk of millions of customers on the fly from their millions of cash flows whilst doing real-time sensitivity analysis using data like exchange rate movement, interest rate, etc.? A practical example and common scenario that bankers would easily relate to in this locale is the concept of “Follow the Money”, consider a situation where all inflow across channels and across locations can be consolidated for the source of the fund to bubble up a new customer to capture just purely be able to leverage big data technology and analytics tools.
A similar scenario is where a large and unusual deposit into a customer’s account triggers real time alert to the relationship team and contact centre of the moment of truth opportunity to make a new product offer real time. Increasing the opportunity to trap and lock the funds in the bank and deepen the bank’s share of the customers’ business. And you can extrapolate the value of this to other relevant banking scenarios like real time and granular transactional risk-based pricing, real time interaction of data across multiple channels to detect fraud threat as it happens real time in order to provide live responses/action, enhancement of data quality/data cleansing, etc. And lastly, with the increasing IT Consumerization trend, customers are being preconditioned and stereotyped to expect high level of personalization in the channel interaction with their bank. Reality, most of our banks are non-starters when it comes to leveraging big data for personalization of customer experience.
Another scenario that would largely benefit from Big Data is the increasing need for Business automation and integration as a result of the increasing complex status most of our businesses have started to attain, most especially our Financial Services and Telco sector. Whilst there is nothing new about business automation and integration, but leveraging cloud computing has greatly enhanced the extensibility of what is possible. For the techno at mind, it simply means we can now expose or call Application Programming Interphase (APIs} securely (whilst selectively gating access to more sensitive data) at scale combining hundreds millions of search terms/data to produce answer in near real time system automation. In lay terms it means our traditional system automated processes now have the ability to be more intuitive and more granular in decisioing and hinging closer to age long desire of business to unlock value from its already sunk huge investment in technology. Imagine for a moment a real life scenario when you have been on a trip for a while and wanted to place a call to loved ones and suddenly discovered we have been cut-off by our mobile operator? Would it not have been nice if the system before it went ahead to cut off your line that you are above your contract level, was able to pull several data and churn same to arrive at a decision, data like the average monthly spend of the customer, credit record of the customer, frequency of roaming, previous customer contact with the service/call centre, customer profile (status, preferences, sentiments, etc.). Other relevant telco opportunity leveraging big data and Internet of things include;
Saving the best for last. As I was finalizing this piece, I asked that a colleague help me peer-review my write-up and his profound comment was that the place where big data can unlock the biggest value is in public sector; and that whilst government are typically a little further back in technology maturity they have also been known to leap-frog the fastest and to scale. Beyond trends, some practical examples that government can use to leverage big data to better execute on their mission.
A good use is in criminal investigation – Microsoft published recently a case study on Thailand’s Department of Special Investigations (DSI), and its use of big data to dramatically accelerate and improve the accuracy of its investigations into criminal cases. Leveraging better BI and data-mining tools the DSI was able to reduce complex and manual processes and establish a system that could automatically notify personnel of suspicious persons or activities related to criminal cases. For example, when many foreign criminals pour into the country and all travel to the same location, or when there is a noticeably large sum of money being transferred into the country. The most impressive about DSI’s new big data implementation is the dramatic impact that it’s had on investigations overall. Before, conducting a traditional investigation could take as many as two years to search for tips, gather, and analyze data. Today, with the Microsoft big data solution, the DSI is able to conduct investigations in as few as 15 days, thanks to the automated processes, reduced complexity and more accurate insights afforded by the new system. If you’d like to learn more about this impressive case study
The key to achieving greater judicial efficiency is unlocking data so that it can be easily shared, analyzed, and acted upon. Today’s case management solutions break down jurisdictional and organizational barriers to ensure that every stakeholder has an up-to-date view of case information from any location, and can update and distribute that data as needed. It’s about getting attorneys, judges, and law enforcement personnel on the same page, through one integrated solution that not only facilitates more efficient collaboration, but enables high-level business intelligence analysis through powerful analytics. Decision-makers need access to historical justice data in order to identify hidden trends and make better, more informed decisions. We’ve seen the power of case management at work within organizations like the County Commissioners Association of Pennsylvania (CCAP). CCAP was facing resource-constraints while operating several disparate case management solutions across its various courts, jails, and corrections offices, which severely limited information sharing and required redundant data entry at each location. To address these challenges and become more efficient, CCAP partnered with Microsoft to build a Unified Case Management system on Microsoft Dynamics CRM that brought stakeholder data together within one system, empowering users to easily search, share, and act upon all of the information at the organization’s disposal. CCAP leaders are now able to extract better insights due to more accurate dashboards and reporting tools, and with manual data entry kept to a minimum, personnel are free to focus more of their time on pursuing justice and keeping citizens safe.
Other big challenges that can be tackled by big data include; using sensors to better understand phenomena such as weather, pollution, or traffic patterns to analyzing massive sets of “nanodata” to model the societal effects of policy, to the fast and low-cost mapping of the human genome to deliver better health outcomes.
Some persons may say all these is but another hype from the tech world, agreed some of it may end of being particularly when approached from just a technology purchase/nice to do IT Project standpoint (hmmm, nice to do IT Project is actually something I like to write about soon). But there are already references to real tangible value:
In 2003 Oren Etizioni needed to fly from Seattle to Los Angeles for his younger brother’s wedding. He bought online his plane ticket months before online on the premise that it would be cheaper. When he got on the flight he was curious to validate that he got a good bargain, so he asked couple of people around him the cost of their ticket to his surprise he paid more than everyone around him. Back from his trip Etzioni was determined to figure out a way for people to know if a ticket price they see online is a good deal or not.
An airplane seat is a commodity: each one is basically indistinguishable from others on the same flight. Yet the prices vary wildly, based on a myriad of factors that are mostly known only by the airlines themselves. Using a sample of 12,000 price observations that was obtained by “scraping” information from a travel website over a 41-day period, Etzioni created a predicted model that handed its simulated passengers a tidy savings. The model had no understanding of why, only what. That is, it didn’t know any of the variables that go into airline pricing decisions, such as number of seats that remained unsold, seasonality, or whether some sort of magical Saturday-night-stay might reduce the fare. It based its prediction on what it did know: probabilities gleaned from the data about other flights. “To buy or not to buy, that is the question,” Etzioni mused. Fittingly, he named the research project Hamlet.
The little project evolved into a venture capital – backed startup called Farecast. Microsoft snapped up Farecast for around $110million, and integrated it into the Bing search engine. By 2012 the system was making the correct call 75% of the time and saving travelers, on average, $50 per ticket.
In conclusion, the world of data is changing in a big way, and customer expectations are changing right along with it. Big Data gives you the tools to make sense of all your collected data. It challenges you to view your business in new ways. And it gives you a basis to power innovation. What this write up tries to do is to illicit thoughts as to the various scenarios of use of Big Data both in the enterprise and individual lives.
At Microsoft our approach to big data is to; “Enable insights into ‘Big Data’ to all users wherever they are”, taking Big Data to a billion people because everyone can make smarter decisions based on data. We believe big data should be in the hands of the people closest to your business who are moments away from that next big idea. Our approach is simple – we enable organizations of any size to access data of all types, whether structured or unstructured, big or small. We empower end users to easily analyze their data with familiar tools like Excel. And we offer a complete data platform for IT to scale insights across their organization with Enterprise-class security and data governance. Microsoft Big Data gives you the power to take action. You can gain a flexible platform and accessible tools to discover, connect, and deliver new insight.
Big Data is here today, within your reach. Isn’t it time to get started?
Olayinka Oni is seasoned IT professional with experience spanning consulting and the banking industry, he is currently the Chief Technology Officer of Microsoft Nigeria and he writes from Lagos
[1] Booz Allen Hamilton [1] Laney, Douglas. “The Importance of ‘Big Data’: A Definition”. Gartner. Retrieved 21 June 2012.
