Issue 12  June 29 2021

Gamification of apps and services and their consequences have been a disaster for the human race.

Well, OK, maybe not that bad. But they aren't helping to make the world a better place, that's for sure.

Gamification is a concept companies use to help drive engagement. Unfortunately, it only drives people to adjust their behaviors to generate rewards. Because that's how humans work. When you measure anything, especially metrics for rewards, we humans make an effort to maximize our score.

But that alone isn't making us stupider. It is our inability to perform critical thinking when we come across some random piece of code we find on the internet, promising to solve our problem.

Let's look at Kaggle as an example. As someone still in the beginner-to-intermediate phase of their data science journey, I spend a fair amount of time on Kaggle. Like many websites, Kaggle offers rewards for engagement. At first glance, these categories and levels sound great! But after some time, you come to understand how people will do anything they can to earn high rankings by taking shortcuts.

Let's look at the Housing Prices competition. This is meant to be a practice competition for new users to get familiar with Kaggle itself. But the data itself is fairly comprehensive and offers a challenge to data scientists of all levels. The goal is to take sample housing price data and build a predictive model.

As you would expect, Kaggle offers a way for users to publish and share their code. Here's one example, where a user published their notebook, claiming they scored in the top 0.3% of entries at the time (two years ago). The notebook is filled with a lot of useful regressing and exploratory data analysis techniques. It's quite good.

Except for the part where the user took a shortcut.

In section 19 of the notebook there is this comment: "Combine train and test features in order to apply the feature transformation pipeline to the entire dataset".

And this is where critical thinking comes into play. Many examples for this competition all include the combination of training and test data in order to build a predictive model. Chances are one person used this method and subsequent users, novices to data science, all follow along believing this is a standard technique.

But it's not. It's lazy, and likely leads to your model having a higher score due to data leakage.

Still, it's a good notebook and has value in many areas. Much more value than this notebook, where the user built a perfect submission by finding the answers online. There's no real explanation for this code, so perhaps they did this in an effort to test their models without hitting the limit of ten submissions a day.

Or perhaps they just wanted to get a perfect score. Because upvotes.

And this is the issue I see. Someone new to data science might believe these techniques are standard. And there isn't anyone at Kaggle offering instruction on "right" versus "wrong", this is something we users need to learn for ourselves. Oh sure, you can think this is where the Kaggle community would come to the rescue, and all pitch in to decide what is good, bad, or wrong.

I think we've seen enough of these websites over the past two decades to understand that none of us is as dumb as all of us. Thinking a community of users produces the most accurate information is a fool's folly. All this does is produce information a handful of loud voices believe to be correct.

Humans are bad at this. And gamification isn't helping to make things better.