I love analytics, and it truly revolutionizes the way I do business, but there is unfortunately a downside. Data collection is everywhere.

What I’m going to be talking about today is Big Data in our everyday lives. I’m going to avoid all the technical babble so the average person can understand what a large part it plays in shaping our lives.

I have plenty of moral objections to certain types of data collection, but the bottom line is – in my industry it’s use is unavoidable.  If I didn’t offer analytics as a service, someone else would. Also, if I didn’t offer it, I wouldn’t be serving my clients, and by extension their customers, to the best of my ability. So the bottom line is, it’s here to stay.

Introduction:

Every day, regardless of if we intend to, each of us is gradually adding to mountains of data being collected on us. When you are driving in your car, or flipping through movies on Netflix – there is a computer out there, somewhere, keeping tabs. Most of this data may not have your name attached to it, but it’s still an eery thought that someone is going to look at it. Today I’m going to be looking at a couple of ways we interact with ‘Big Data’ and may not even realize it.

Netflix Analytics

Like any other billion dollar company, Netflix takes great care in monitoring it’s products performance.

One of it’s best features is it’s movie prediction algorithm, which according to Wired Magazine, drives 75% of all of it’s traffic. In a sense, this means that 3 out of 4 people on Netflix don’t pick what they watch – the data collected from the other 25% not using this feature pick for them. Weird, huh?

While that isn’t too surprising, some of the things they keep stock of might be.

From my research in their company tech blog, and other sources on the web I’ve been able to gather that they track things like:

• How often you pause what you watch, and if you rewind:
  If a show is great, you pause less or are sleeping. If a large volume of people replay a certain part, it might be really good or hard to understand.
• Drop off rate for shows:
  If 3 million people stop watching after 1 episode, there’s something wrong with the content.
• The device you watch Netflix on:
  If more people watch on Apple then Android, you can bet they take priority with advertising and development.
• How you scroll through the feed.
  Do certain colors or pictures make you slow down? Is it the title that got you? Netflix needs to know this data for marketing and User Experience research.

And that is just a portion of what is available on the web about their data collection… What they collect that we don’t know about might in fact be more troubling. Unlike most data, I imagine theirs isn’t anonymous because they need it linked to a profile for suggestions.

Traffic Analysis

Have you ever wondered how your smartphone knows there is going to be traffic? Or how it predicts that there is going to be traffic on a certain road? The answer: There are millions of little data sensors roaming around the streets – our cellphones.

By tracking our GPS data in real-time while we use the service, companies like Google can tell by our phones whether we’re stuck in traffic. Furthermore, by amassing this data, they can tell other drivers if road conditions are harsh by comparing your speed to the data collected on millions of other people who drove that very same road.

Beginning in 2009, Google turned to crowd-sourcing to improve the accuracy of its traffic predictions. When Android phone users turn on their Google Maps app with GPS location enabled, the phone sends back bits of data, anonymously, to Google that let the company know how fast their cars are moving. – HowStuffWorks

For some odd reason, I know quite a bit about bees. Traffic analysis isn’t far off from how bees learn and communicate by using a thing called Associative Learning.

Associative learning is the process by which an association between two stimuli or a behavior and a stimulus is learned. […] In operant conditioning, a certain behavior is either reinforced or punished which results in an altered probability that the behavior will happen again. – Wikipedia

In English: Our phones are stimuli that indicate that we hit traffic at a certain time, and Google’s data network acts as a ‘Brain’ and interprets this to mean a higher probability of it happening again in those conditions.

Shopping Analysis:

As if your TV habits and location data wasn’t enough, there are also large corporations keeping tabs on what you buy.

Walmart, for example, is one of the first pioneers of Big Data. With a whopping $482.2 billion dollars in sales in 2015 alone, they certainly have enough money and customers to work with.

So what is Walmart doing with all this data?

• Predicting sales trends, and their association with events.
  Some events may be obvious, like chips during the Super Bowl. But with all this data, they have found weird trends. Like for example, Strawberry Poptarts sell 7 times more than usual during a Hurricane.
• Building databanks bigger than the internet.
  By only 2004, Walmart had 460 terabytes of data stored, which at the time was 2 times more data than the whole internet. Data collection has came leaps and bounds since 2004, and you could only speculate how much they have now.
• We can only speculate, but probably treating humans like science experiments.
  Walmart has the data to know things like how packages affect our buying habits, and how we walk around the store. Treating customers like rats in a maze, some stores measure their customers walking and linger time using their security cameras to make ‘Heatmaps‘ which show hot areas customers frequently linger in. Walmart is most likely doing this research. Walmart sells it’s shelf space in 3 foot blocks, and I imagine the shelf space in ‘hot’ areas costs more.

Closing Thoughts:

Is data collection really that horrible? Not usually. It can do amazing things for our daily lives, and the science behind it is pretty interesting.

However – there is no clear line between ‘good’ and ‘bad’ data collection at this point in time, which is where the problem lies. All of this data is collected in a perfectly legal manner, and we all agreed to it when signing up to one of the many services that collect data. If there were companies abusing the data, we wouldn’t even know about it. Most terms of service (Like Facebook especially) allow them to keep that stuff secret.

Oddly enough, Facebook released some of it’s data research and was highly criticized for it. Whenever anyone tells me it’s unethical, I remind them that FB had no legal obligation to even tell us about it, because we signed away those rights on sign up. So I give the company respect for being honest.

This is a small portion of a continued series on Big Data, Analytics, and Marketing. If you would like to follow along with this blog, sign up to the newsletter:

[mc4wp_form id=”758″]