Generative Image Dynamics Summary
<p>Recently a new Google Research has been published under the name of Generative Image Dynamics which seems astonishing to everyone because of Interactive motions.</p>
<p><strong>This research paper presents an approach of modelling an Image space to Dynamic space.</strong></p>
<p><img alt="" src="https://miro.medium.com/v2/resize:fit:700/1*SDceiVfBQkS5ySoDmCD6Ag.png" style="height:420px; width:700px" /></p>
<p>Image-space prior on scene dynamics that can be used to turn a single image into a seamless looping video or an interactive dynamic scene (<a href="https://generative-dynamics.github.io/#demo" rel="noopener ugc nofollow" target="_blank">link</a>)</p>
<p>Here We’ll quickly summarise the approach and methodology used by Google Researchers in this paper.</p>
<p>Methodology used by in this paper consists of 4 main stages which are as follows:</p>
<ol>
<li><strong>Data Collection: </strong>Data consists of motion trajectories extracted from Real Video Sequences containing natural, oscillating motion such as trees, candles, and clothes blowing in the wind which can also be seen from <a href="https://generative-dynamics.github.io/#demo" rel="noopener ugc nofollow" target="_blank">official demo</a>.</li>
<li><strong>Model Training</strong>: Given a single image, the trained model uses a <a href="https://medium.com/@rk4082262341/what-is-frequency-co-ordinated-diffusion-sampling-d00e0e563280" rel="noopener">Frequency-coordinated diffusion sampling process</a> to predict a per-pixel long-term motion representation in the Fourier domain. This representation is referred to as a Neural Stochastic Motion Texture in paper.</li>
<li><strong>Motion Representation</strong>: This Neural Stochastic Motion texture can be converted into dense motion trajectories that span an entire video which we are seeing demo are motion trajectories.</li>
<li><strong>Application</strong>: Along with an Image-based Rendering module, these motion trajectories can be used for several downstream applications. For instance, they are turning still images into seamlessly looping dynamic videos or allowing users to realistically interact with objects in real pictures.</li>
</ol>
<p><a href="https://prasantdixit.medium.com/generative-image-dynamics-summary-18668ab92259"><strong>Learn More</strong></a></p>