Machine Learning Versus Traditional Analytics

“I suppose it is tempting, if the only tool you have is a hammer, to treat everything as If it were a nail.” – A Maslow.

Consider this quote in the context of a machine learning practitioner. Your work colleagues tell everyone about what you can do, your family tells all their friends, you’re proud of yourself for it and talk about it at parties. You can extend this metaphor towards a company. You’re an industry leader; it’s how you market yourself and it’s your niche that you’ve proclaimed so loudly. The risk is that machine learning is now your hammer, you need to use it to support your value proposition. If your problem isn’t a nail, you shape it into a nail. It must be a nail!

I’ve personally been here and had conversations with many senior managers in multiple industries with this mindset. I once tried to determine yearly flight patterns using a neural net. Although this resulted in great predictive power, the final results still needed to be plotted. I could have made the exact same plots with existing data. I argued my machine learning model helped determine the most important variables affecting flight behaviour, but everything it told me was common sense and could have been found using correlative statistics. I was arguing with myself, trying to justify my means of using machine learning. I spent 3-4 times longer on the problem than I needed to.  I now realise there are machine learning niche companies acting in a similar manner – justifying why they used such a complex model when a simpler approach was possible.

I now metaphorically view data driven problems as an apple tree. To pick an apple, first, we need to reach it, so we need to make a ladder. The ladder in this case, is the data being stored in a single centralised location. This also assumes the right types of data is being generated. Now using this ladder, we can reach the fruit of the tree. The low hanging fruit requires the least effort to pick and brings quick value. Often this just requires scatter plots, histograms and asking the right questions. This alone is enough to revolutionised entire industries. As time goes on, we can use machine learning to get those higher hanging fruit, but in a mature and justifiable manner.

It’s takes humility and maturity to understand and act upon all of this, but such maturity with how to use machine learning, will be the foundation of pioneering data analytical companies. I’d like to end on the following quote:

“The average human is born with 5 senses, but only requires sight in a life-threatening situation. The average data analyst has thousands of senses. Which senses do they truly need to get the job done?” – Luke Ginn


Luke Ginn
Data Scientist and Applied Mathematician
, Interlate