Creating Personalized Animated Memes Using Fine-tuned Text-to-image Models
<p>In this blog post, I’ll introduce the process of creating personalized & animated memes using <a href="https://github.com/deforum-art/deforum-stable-diffusion" rel="noopener ugc nofollow" target="_blank">Deforum</a> and a fine-tuned <a href="https://github.com/CompVis/stable-diffusion" rel="noopener ugc nofollow" target="_blank">Stable Diffusion</a> (SD) model as a style transfer tool and present some examples of memes that I have made by personalizing videos.</p>
<p>Just before starting, if you still have not read my blog about how I have used finetuned SD models to generate meme images, please check out the link below:</p>
<h2><a href="https://medium.com/@berkanzorlubas/creating-personalized-memes-using-fine-tuned-text-to-image-models-66acfb3a2950?source=post_page-----37a45de4c7c3--------------------------------" rel="noopener follow" target="_blank">Creating personalized memes using fine-tuned text-to-image models</a></h2>
<h3><a href="https://medium.com/@berkanzorlubas/creating-personalized-memes-using-fine-tuned-text-to-image-models-66acfb3a2950?source=post_page-----37a45de4c7c3--------------------------------" rel="noopener follow" target="_blank">Memes have become an integral part of our online culture, serving as a means of expressing humor, satire, and social…</a></h3>
<p><a href="https://medium.com/@berkanzorlubas/creating-personalized-memes-using-fine-tuned-text-to-image-models-66acfb3a2950?source=post_page-----37a45de4c7c3--------------------------------" rel="noopener follow" target="_blank">medium.com</a></p>
<p>You can check the entire code on my GitHub page <a href="https://github.com/berkanz/meme-personalization-via-Deforum" rel="noopener ugc nofollow" target="_blank">here</a>, which I will be referring to.</p>
<h1>Step 1: Model Fine-tuning</h1>
<p>The first step is fine-tuning the SD model. I picked SD1.5 and downloaded the weights of the model from <a href="https://huggingface.co/runwayml/stable-diffusion-v1-5" rel="noopener ugc nofollow" target="_blank">here</a>. And then I ran the training algorithm locally as my PC has an RTX3080 with 10GB VRAM which is enough.</p>
<p><em>Be careful while picking the images for your fine-tuning, try to have different backgrounds and facial expressions. The images I used are as the followings.</em></p>
<p><a href="https://betterprogramming.pub/creating-personalized-animated-memes-using-fine-tuned-text-to-image-models-37a45de4c7c3"><strong>Read More</strong></a></p>