The Myth of Hidden Treasure: Why Data Exploration Can Be Foolish if done without focus.

By using use case focus and business questions, you'll be able to explore data in a way that will reveal the true insights - without being lured by large data sets in your organization.

Myth Buster: Exploring data will reveal hidden insights

Mythbuster: Exploring data will reveal hidden insights

  1. What is data exploration?
  2. Myth: Explore Data to find actionable insights
  3. How to go about data exploration

There’s a famous saying that goes, “knowledge is power.” And while that may be true in some cases, it’s not always the case when it comes to data. In fact, some data can be misleading. That’s why it’s important to be careful when exploring data – you don’t want to end up fooled by hidden treasure. This post will discuss the myth of finding hidden treasure using data exploration.

1. What is data exploration?

Exploration is one of the first steps in data preparation. It’s a way to get to know data before working with it. Through survey and investigation, large datasets are readied for deeper, more structured analysis.

By exploring your dataset, we need not learn about its structure, which wastes time and effort. When preparing your dataset for analysis, consider what types of exploration will be most beneficial for your project and use the resources available to you to achieve your goals.

2. Myth: Explore Data to find hidden treasure

There is no magical treasure hidden in data – only valuable insights that can be used to improve your business. Many small businesses make the mistake of blindly exploring data without any plan or goal. For example, if you are a FinTech startup or medium business with a data lake worth a few years, it is common to assume that there must be some value in those data sets.

There are three reasons why this type of data is unlikely to lead to hidden treasures:

  • It takes a very long time to explore.
  • Insights will fail to derive any outcomes or dollars.
  • ROI can be zero.

Hence, the approach of exploring data to find hidden treasure doesn’t work.

3. How to go about data exploration

Even though we are now aware that exploring data to find hidden treasures will not work, that does not mean that we leave large data sets alone. When a company approaches us with data sets to explore, the first question we ask is, to what end is this data explored?

Using a use case can direct the focus to find specific insights from data. For example, suppose the focus is to predict fraud better. In that case, a fraud alert system can be developed using historical transaction data, which adds monetary value by reducing risk. Or if it is possible to figure out a routing mechanism for transactions to increase the success rate.

And the next step will be to form multiples hypothesis about existing business problems. Focusing on a small subset of data depending on the use case makes it possible to clean only the necessary data, not the entire database.

Hence, the result of this approach would be:

  • Find insights or fail fast
  • Fewer resources and faster method
  • Lesser time needed
  • Feature Engineering: Possibility to derive complex features with further data mining

Conclusion

In the end, you may have realized that the key to successful analytics is not just having a data set but also knowing how to use it. With a focused approach, it will be easier to find the right business question, apply the right analytics solution, and derive insights that will drive ROI and faster results.

close

Related Posts

Great SQL doesn't make an awesome analyst Great SQL doesn't make an awesome analyst

SQL is a critical language for database administrators and data analysts, but that doesn't mean you ...

Mythbuster: Only Data Scientists Can Analyze Data Mythbuster: Only Data Scientists Can Analyze Data

Many people think you need to be a data scientist to extract insights from it. This can’t be furth...

Mythbuster: The best analysts are the fastest order takers Mythbuster: The best analysts are the fastest order takers

Often, an analyst is a person who extracts necessary values from the database and produces values an...