Data Driven Decisions in Business

All companies, big or small, generate some form of data: if you’re selling products, you have data pertaining daily sales, inventory/stocks, purchase orders, etc; if you’re selling a service, you have data regarding service time, customer feedback, employee productivity, etc.; all companies generate operational data such as resources used, supply chain logistics, leaves taken by employees, energy usage, and much much more. However, only the smart ones know that data is powerful, and can be used to achieve both the conventional and the exceptional.

Figure: Daily Statistics of Data Creation as per 2010, taken from GOOD

Big data has recently become a buzzword, but the analysis of data to find meaningful patterns have existed for a long time. The reason why the concept of big data is now at the forefront is due to improved capabilities that are enabling businesses to use this data in heretofore unimaginable ways. Let’s take retail for example. Before, retail stores used to have manual records of sales from which very little could be deduced other than the obvious seasonal demand fluctuations. As retail stores started to expand to form giants, the need arose for software for fast checkouts, stock management, accounts management, etc. Retail started to move online and virtual stores began to thrive. With the digitalization of business processes, suddenly there was an information flood. Retail giants started to store this data to keep track of movements in stocks and finances, and they found that this data could be used to trace a multitude of other nitty details that could come in very handy in unconventional ways. For example, Costco used big data to pinpoint and warn consumers who bought stone fruits that were, as informed by their supplier, possibly contaminated. Thus they were able to avoid widespread panic, and give attention to those affected. Empowered by big data, Amazon tracks consumer behavior on their website to recommend products that they are likely to buy, which is impossible with physical stores.

The application of big data is relevant to all types of industry. Big data has been solving industry-specific challenges even in low potential application areas, as shown in the diagram below as per 2013 data taken from Wilson Lucas’s slides.

Figure: Potential Big Data Opportunity in Various Industry Verticals (2013)

We have seen and heard the wonders of the application of big data, but…

What is big data really?

Big data has come to be defined by the 3 Vs: volume, velocity, and variety of data.
Volume: Volume is the most apparent characteristic of big data. Every single day of business generates data, which means that the data is increasing at an exponential rate. Your business must have the hardware/cloud to meet these storage needs.
Velocity: Big data usually comes in at high frequency at any given moment. For example, sensors at a train station as people swipe and enter keep track of passengers in real time. Traffic to a new episode of Game of Thrones on an online streaming site is tracked real time and generates viewership statistics.
Variety: Big data is a combination of structured and unstructured data. Structured data follows specific rules and are relatively easy to analyze in big volumes. For example, serial numbers, dates, codes, sizes, etc. Unstructured data, on the other hand, does not follow any rule and is haphazard. For example, social media tags, descriptive reviews, vlog reviews, etc.

Now that we know the characteristics of big data, it is important to understand that on its own, big data is just enormous quantities of data. It is only through the analysis of this data that your business will find paths to increase productivity, cut costs, decipher trends, and exploit opportunities in a strategic manner.

So, the question is…

What can you do to start making data-driven decisions?

There are a number of steps that can help you get started.

Step 1: Find Your Data Sources

This infographic by Kapow shows the variety of data sources for any business. Once you know your sources, you are ready for the next step.

Step 2: Start Storing Information Using the Right Technology

Before you can start storing information, you have to assess your storage needs. There are 7 key factors to consider for big data storage.

1. Storage Capacity
   When assessing capacity needs, keep in mind that much of the daily data generated may be useless after a certain time, so this data will be deleted thus freeing up space. On the other hand, some data is so critical that you’re going to be backing it up, thus using double space. Don’t fret too much about optimum space if your capacity need is low, it is quite cheap to entail cloud services on public networks.
2. Object Storage vs. File Storage
   While object storage is preferred for complex and high volumes of data, file storage is a better option for smaller volumes of data. Read this comparison of object and file storage systems to have an understanding of what best suits your business.
3. Information Lifecycle Management
   The data should be structured in a way that data is deleted when it no longer serves a purpose. When using cloud services that are easily scalable, businesses often make the mistake of buying more space instead of recycling the existing space, which ends up in higher costs.
4. Data Privacy
   Make sure that the storage privacy policies comply with privacy laws.
5. Investments in SSDs
   SSDs have been growing in capacity while becoming cheaper over time. This is why investments should be made sensibly with the anticipation that in a year or two, there will be more cost-effective alternatives that will turn current investments to scrap.
6. Compression
   Compression of data can reduce space requirements while drastically increasing transmission speeds.
7. Cloud Service Providers
   For small and medium size (in terms of data needs) businesses, cloud services such as Amazon, Google, and Microsoft are cheaper and easily scalable. For companies that have a very high volume/sensitive data, having in-house storage systems is recommended.

Step 3: Analyze Data Intelligently

While technology these days makes it possible to analyze all the data in storage, it is smarter to know specifically what you are looking for, and analyzing the data accordingly. Don’t let the abundance of data make you lose sight of what you are looking for.

Step 4: Bring in Management that Champions Data Driven Decision Making

Perhaps this should be the first step before embracing big data technology. The right kind of leadership should be in place that uses data, rather than intuition, to make strategic decisions. They should have the vision to perceive market trends and developments and use the data at hand to seize opportunities when they present themselves. They should lead data scientists by asking the right questions, and guide them in finding answers that translate to higher profit margins. It is often easy to misinterpret the data when you want to justify a decision already made. Instead, base your decisions on careful study of the underlying relations.

Now that you know the benefits, it’s time for you to embrace big data technology, and get that one up from your competitors! Because, face it, in a few years, only those that know how to tame the tides of data will surf the waves of a highly satisfied customer base, lower operational costs, innovative products that delight, and profit maximization.