Catastrophic Forgetting (degrades model performance)
Catastrophic forgetting occurs when a machine learning model forgets previously learned information as it learns new information.
This process is especially problematic in sequential learning scenarios where the model is trained on multiple tasks over time.
Catastrophic forgetting is a common problem in machine learning, especially in deep learning models.
Example
A sentiment judgment task. We fine-tune the model to give sentiment results instead of sentences, and it works.