Class Imbalance: From SMOTE to SMOTE-N

In the previous story we explained how the naive random oversampling and random oversampling examples (ROSE) algorithms work. More importantly, we also defined the class imbalance problem and derived solutions for it with intuition. I highly recommend checking that <a href="https://medium.com/@essamwissam/class-imbalance-from-random-oversampling-to-rose-517e06d7a9b" rel="noopener">story</a> to ensure clear understanding of class imbalance. In this story, we will continue by considering the SMOTE, SMOTE-NC and SMOTE-N algorithms. But before we do, it’s worthy to point out the two algorithms we considered in the last story fit the following implementation framework: <ol> <li>Define how the algorithm takes data belonging to class X that needs Nx examples and computes such examples by oversampling</li> <li>Given some ratios hyperparameter, compute the number of points that need to be added for each class</li> <li>For each class run the algorithm then combine all newly added points together with the original data to form the final oversampled dataset</li> </ol> For both the random oversampling and ROSE algorithms it was also true that to generate Nx examples for class X the algorithm does the following: <ul> <li>Choose Nx points randomly with replacement from the data belonging to class X</li> <li>Perform logic on each of the chosen points to generate a new point (e.g., replication or placing a Gaussian then sampling from it)</li> </ul> It holds that the rest of the algorithms we will consider in this story also fit the same framework. SMOTE (Synthetic Minority Oversampling Technique) Thus, to explain what SMOTE does we only need to answer one question: What logic is performed on each of the Nx randomly chosen with replacement examples from class X in order to generate Nx new examples? <a href="https://towardsdatascience.com/class-imbalance-from-smote-to-smote-n-759d364d535b">Read More</a>