An Honest Approach to Your Data

“But it seems so simple!” responded the executive across the table from me. His disbelieve worn like a mask upon his face, a rose color beginning to blossom in his cheeks. He was surprised at my estimate of how long it would take to get him a reliable, accurate and timely dataset and rightfully so. Just a few weeks earlier, he had met with a team of good-looking, personable people from a data visualization vendor who had assured him that their product would “plug into” his data and provide him with everything he needed and more. In fact, they had professed the ability to find things he was not yet thinking about and alert him to actions our customers would be taking in the future. They had promised him a crystal ball.

Just as the snake-oil salesmen of times long passed, the sales portion of a vendor relationship is very different from the actionable part. And just as in the past, my friend across the table from me was being fed some very accurate information in a not-so-accurate context. Take a few minutes on Google and you will find any number of companies out there who have services or products that can magically take your data and transform it quickly and easily into valuable insights. They are not lying, but they are definitely leaving something out. In order for that data to easily be transformed into insight, it must first be organized, categorized and cleaned to fit into a model that will facilitate its journey through these amazing tools and services. Quite frankly, if your data is crap, you’re insights will be crap; end of story.

For guys like me, living in the real world of data management, there is a constant barrage of attacks from both inside and outside the organization on how to manage and deliver data. I won’t cry about it because honestly it is my job to educate those around me to the reality of that data to insight evolution. It’s an ever-present challenge to remind those who consume data for insights that nothing comes easy and if it does it should be highly suspect. These tools and systems around us that will allow us to stand up machine learning, data visualizations, master data management solutions, data profiles, reports and regression models all rely on data that is fairly well organized and clean. Additionally, if you feed a bunch of data to some new system without any prior knowledge of what that data represents, no technology is going to figure that out for you.

The sweet spot at the intersection of human knowledge and technology advancement is where you get the most “bang for the buck”. Setting expectations up front that there is a level of effort around sorting through and understanding your data, cleansing data anomalies, devising mechanisms for reliable and timely delivery will go a long way to creating a healthy partnership between your business leaders and your data organization. Data is currently the most important and valuable asset in any organization. In recent years, the amount of data we have access to has exploded and this makes the importance of thoroughly evaluating vendor promises that much more critical. And if no one else is going to say this, I will; just because you are on the business side and not the technology side of things that is no excuse not to understand the complexities of a data practice in your organization.

Just as culture and leadership is everyone’s responsibility in today’s successful organizations, so too is an understanding of data and what it means to that success. There was a day when requesting a report and a couple weeks or months later it dropped on your desk was acceptable and even expected. But we now exist in a world where decisions are made on millions of data points in seconds. Cars are almost to a point at which automated driving is ubiquitous, billions of sensors across the planet are feeding data to massive data farms constantly and the size of data stores for companies like Facebook and Google have gone beyond the realm of a human’s capacity to understand. We just start taking terms like “Exabyte” for granted as just words now. They mean nothing other than large to most people. These are all things that can lead to apathy of understanding with regards to data. If you look at something that requires Einstein’s brain to unravel you throw up your hands and say let the experts deal with it. They want that, believe me. It’s like when you bring the car to the mechanic and you don’t know jack about cars. That header replacement makes a lot of sense. Why? ; Because this dude with a bunch of black crap under his finger nails just told you so.

First off, give yourself some credit and realize that most of what we are talking about in the data world does not take a PhD to figure out. Where is your data? What does it represent? How clean is it? How much is there? What questions do I want it to help me answer? Who can help me with this? When you ask a few questions you can begin to get an overview of your situation rather quickly and speak intelligently about your data. On top of that, you can pose the same questions to the next slickster-salesman through the door.

“This all looks great Chuck but what do we do about the data issues we have?” You ask.

“Does your product automatically identify and resolve those issues too?”

That glassy-eyed, slack-jawed, silent response is the one that tells you to move on to the next vendor! Oh, and on the inside of your organization, give your data folks a break. My guess is that they are not lazy or deceptive. They are being honest and that is what you really need right now. You need to be honest about your data. You need honesty about the state of that data and you need to be very honest and realistic about what it will take to get you from raw data to valuable insights. Lay the foundation and keep the lines of communication open, otherwise your crystal ball will look more like a bowling ball. Create a data maturity curve and set some expectations around what milestones you need to tackle before the machine learning stage of things and align those with realistic timelines.

In the end, honesty in your data journey is how you will be successful. It is tempting to fall for the promise of an easy path because there are organizations out there that have a big head-start on you. Be okay with that and focus on your goals to move you to the next strategic milestone. Remember, if you could just plug something into your data and get insights, why hasn’t everyone done it?

© 2017 by The BI Brewery | Business Intelligence Insights. Proudly created with