CS 180: Project 5 - Fun With Diffusion Models!

Tobias Worledge, Fall 2024

Introduction

In this project, I explored using denoising models to generate images based on text descriptions and initial images. I then built a denoising model specifically for the MNIST dataset.

Implementing the Forward Process

I began by implementing a function that could add a defined amount of noise to a given image. Here are a few examples of increasingly noisy images of the Campanile.

Original (t=0)

t=250

t=500

t=750

Here were my attempts to denoise these images using a Gaussian filter. It's pretty rough!

Denoised t=250 (kernel=5, sigma=4)

Denoised t=500 (kernel=5, sigma=5)

Denoised t=750 (kernel=5, sigma=3)

Now using a denoising model, here are some results with one-step denoising!

Onestep Denoised t=250

Onestep Denoised t=500

Onestep Denoised t=750

Now let's try iterative denoising!

Original

Iterative Denoised

Onestep Denoised

Gaussian Filter

If we try to denoise starting with an image of pure noise, we get images like this!

Image to image translation!

i_start=1

i_start=3

i_start=5

i_start=7

i_start=10

i_start=20

i_start=1

i_start=3

i_start=5

i_start=7

i_start=10

i_start=20

Hand-drawn and Web Images

Pixeldog

i_start=1

i_start=3

i_start=5

i_start=7

i_start=10

i_start=20

Stickfigure

i_start=1

i_start=3

i_start=5

i_start=7

i_start=10

i_start=20

Normaldog

i_start=1

i_start=3

i_start=5

i_start=7

i_start=10

i_start=20

Inpainting

Campanile

i_start=1

i_start=3

i_start=5

i_start=7

i_start=10

i_start=20

Dog

i_start=1

i_start=3

i_start=5

i_start=7

i_start=10

i_start=20

Campanile Rocket

i_start=1

i_start=3

i_start=5

i_start=7

i_start=10

i_start=20

Flip Illusions

Example 1 of campfire old man

Example 2 of campfire old man

Snow village and a waterfall

Low/High Pass Filter Illusions

Skull waterfall

Rocketship Pencil

Hipster Barrista

Part B of the project

Examples of noise levels

Number 0

Number 1

Number 4

Number 5