Data Entropy — More Data, More Problems?

<p>&ldquo;It&rsquo;s like the more money we come across, the more problems we see&rdquo; Notorious B.I.G</p> <p>Webster&rsquo;s&nbsp;dictionary defines Entropy in thermodynamics as a measure of the unavailable energy in a closed thermodynamic system that is also usually considered to be a measure of the system&rsquo;s disorder.</p> <p>In Information Theory, the concept of information entropy was introduced by&nbsp;Claude Shannon&nbsp;in 1948 and represents for a random variable the level of &ldquo;surprise,&rdquo; &ldquo;information&rdquo;, and &ldquo;uncertainty&rdquo; related to the various possible outcomes. Some nice reads for my math nerds out there (here&nbsp;and&nbsp;here).</p> <p>In a broader context, Entropy is the tendency of things to become disordered over time and for general disorder and uncertainty to increase in a system or environment.</p> <p>If you are a data practitioner in today&rsquo;s flourishing data ecosystem and are asked by a peer or a business stakeholder to describe your data platform, I imagine you would use a combination of the following words: modern / cloud-based, modular, constantly evolving, flexible, scalable, secure, etc. But between you and I, you know that you&rsquo;d like to throw: messy, unpredictable, chaotic, expensive and disorganised into the mix as well.</p> <p><a href="https://towardsdatascience.com/data-entropy-more-data-more-problems-fa889a9dd0ec">Read More</a></p>