Data Entropy — More Data, More Problems?
<p>“It’s like the more money we come across, the more problems we see” Notorious B.I.G</p>
<p>Webster’s dictionary defines Entropy in thermodynamics as a measure of the unavailable energy in a closed thermodynamic system that is also usually considered to be a measure of the system’s disorder.</p>
<p>In Information Theory, the concept of information entropy was introduced by Claude Shannon in 1948 and represents for a random variable the level of “surprise,” “information”, and “uncertainty” related to the various possible outcomes. Some nice reads for my math nerds out there (here and here).</p>
<p>In a broader context, Entropy is the tendency of things to become disordered over time and for general disorder and uncertainty to increase in a system or environment.</p>
<p>If you are a data practitioner in today’s flourishing data ecosystem and are asked by a peer or a business stakeholder to describe your data platform, I imagine you would use a combination of the following words: modern / cloud-based, modular, constantly evolving, flexible, scalable, secure, etc. But between you and I, you know that you’d like to throw: messy, unpredictable, chaotic, expensive and disorganised into the mix as well.</p>
<p><a href="https://towardsdatascience.com/data-entropy-more-data-more-problems-fa889a9dd0ec">Read More</a></p>