In this blog post, I’ll introduce the process of creating personalized & animated memes using Deforum and a fine-tuned Stable Diffusion (SD) model as a style transfer tool and present some examples of memes that I have made by personalizing videos.
Just before starting, if you still have not read my blog about how I have used finetuned SD models to generate meme images, please check out the link below:
Creating personalized memes using fine-tuned text-to-image models
Memes have become an integral part of our online culture, serving as a means of expressing humor, satire, and social…
You can check the entire code on my GitHub page here, which I will be referring to.
Step 1: Model Fine-tuning
The first step is fine-tuning the SD model. I picked SD1.5 and downloaded the weights of the model from here. And then I ran the training algorithm locally as my PC has an RTX3080 with 10GB VRAM which is enough.
Be careful while picking the images for your fine-tuning, try to have different backgrounds and facial expressions. The images I used are as the followings.