Big Data, Big Business, is the market ready?

Source: The Wall Street Journal

Last April 18^th perhaps the first Big Data specific initial public offering was made by Splunk Inc., a company that developed a software which helps companies to analyze data, but not any data, it’s ‘machine data’ as they call it. At the IPO, the company was valued at $3.28 billion, selling at a revenue multiplier of 28! For instance, Google trades at a revenue multiplier of 5 which goes to show how excited the market is about this Big Data technology. Splunk raised $229.5 million, and at the NYSE close, its stock price jumped 109%. The software that companies such as Splunk offer, help businesses to manage their increasing amounts of data and in this way avoid data inflation. Apparently, the market is getting very excited about this new IT trend, but are companies really prepared to handle these loads of information?

It seems that what firms are requiring nowadays are data scientists. It’s not only about having the engineering skills to build complex mathematical models to process the data, but also about being able to shape the data in order to get a story from it. Once these skills are fulfilled, a deep understanding of the business is also needed in order to be able to ask the right questions from which to make the data work for them. Big Data will give decision makers more information on which they can rely on, but just having more information may overwhelm them and it will not mean they will be able to use it in the right way towards the business objectives.

It is clear now that there will be a rising demand for these data scientists, however the supply side does not seem to be moving at the same rate. Universities still do not teach courses which prepare people for this, and they can hardly be found at recruitment agencies. Nevertheless, just as 30 years ago IBM started a generation of Cobol programmers, it seems as though market forces will lead to a generation of data scientists to cope with the massive amount of everyday data that is being accumulated.

More about this can be found in: 

http://on.wsj.com/Jq8okx

http://bitly.com/I506fo

What actually is Big Data?

New technology and innovation often bring about new terminologies. With Big Data, this is exactly the case. But what does Big Data really mean?

It appears that so far there is no standard definition for the term Big Data. A search reveals that various explanations have evolved over time.

■       In 2009, Adam Jacobs described Big Data as “Data whose size forces us to look beyond the tried-and-true methods that are prevalent at that time” in his interesting article “The pathologies of Big Data” (http://queue.acm.org/detail.cfm?id=1563874) Jacobs argues that getting stuff into databases is easy, but getting it out (in a useful form) is hard; the bottleneck lies in the analysis rather than the raw data manipulation.

■       In 2011, IBM, which has the Big already in its nickname "Big Blue" in turn focuses on the three V’s on its definition of Big Data

• Volume – Big Data comes in one size: large. Enterprises are awash with data, easily amassing terabytes and even petabytes of information.
• Velocity – Often times-sensitive, Big Data must be used as it is streaming into the enterprise in order to maximize its value to the business.
• Variety – Big Data extends beyond structured data, including unstructured data of all varieties: text, audio, video, click streams, log files and more. (http://www-01.ibm.com/software/data/bigdata/)

IBM is one of the pioneers of bringing Big Data analyses to their customers. I highly recommend taking a look at their eBook titled “Understanding Big Data”

(https://www14.software.ibm.com/webapp/iwm/web/signup.do?source=sw-infomgt&S_PKG=500016891&S_CMP=is_bdebook1_bdmicro)

■       Recently, the McKinsey Global Institute, the research arm of McKinsey and Company pointed out that no specific threshold can be set for amounts of data to be accounted for as Big Data by saying: “Big Data” refers to data sets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze. This definition is intentionally subjective and incorporates a moving definition of how big a data sets needs to be in order to be considered as Big Data - i.e., we don’t define Big Data in terms of being larger than a certain number of terabytes (thousands of gigabytes). We assume that, as technology advances over time, the size of data sets that qualify as Big Data will also increase. Also note that the definition can vary by sector, depending on what kinds of software tools are commonly available and what sizes of data sets are common in a particular industry. With those caveats, Big Data in many sectors today will range from a few dozen terabytes to multiple petabytes (thousands of terabytes). The consultancy also provides insides into the financial opportunities associated with the topic. Check out their report.

(http://www.mckinsey.com/Insights/MGI/Research/Technology_and_Innovation/Big_data_The_next_frontier_for_innovation) 

What do all these definitions have in common? They highlight that existing approaches to collecting, handling and analyzing data no longer help companies to gain a competitive advantage. In contrast, new approaches are needed to take into account the exponential speed of change. It seems that Big Data calls for

     a) Radical thinking

         and

     b) Willingness to deal with uncertainty

We will investigate these points further and keep you posted!

BIG DATA - Overview

This blog is a project for INFORMATION TECHNOLOGY & INNOVATION with Professor Kiron Ravindran at IE Business School.  It has been created by the following IMBA (April '12) student group members (Nationality): Beatriz (Colombia), Matan (Israel), Jaime (Peru), Tom (USA), Denny (Germany) & Vikas (India).

With this project we want to communicate about "BIG DATA", an emerging technology and its application in business. In weekly blog posts we will compile articles about Big Data and give our opinions about this IT topic. The Blog will also be a way of sharing information about Big Data with other users interested in the subject. It will be updated weekly from today 12/04/2012 to the end of June 2012. 

So stay tuned and we look forward to reading your comments!

"LIKES!" That’s what we expect anxiously whenever we post a new picture on our Facebook accounts. Ever wonder how many such likes, comments or tags, on how many pictures or videos are saved and available for us and our friends to see every second, minute, day, month or even a year? And now think of all those 800 or so million users on Facebook thinking exactly the same way as you are right now. Well that’s Big data we are talking about, isn’t it?

When we, the mundane people are having a hard time comprehending the logic behind storing such massive amounts of data, it is worthwhile sparing a thought for establishments who need to do so in order to remain in business. Now, also imagine, what if the top modelling agencies of the world were willing to spend millions of dollars in order to scout the next Tyra Bank or Gisele Bundchen from the mediocre photographs on such social networking sites? What? Now it’s totally worth it?

Through this blog, we set out to explore the virtual worlds where our lives are stored in - Big Data. The terabytes, exabytes or even zettabytes of data that’s collected, stored and consumed. In the upcoming posts, we will also try to touch upon the various aspects of these consequential and/or random yet potentially highly useful information (Big Data) being collected from all walks of life.