We've all seen images where when you zoom in you realize that it's made up of many smaller images and harkens to some greater theme. While cool, I found the problem of actually creating this mosaic quite interesting so this weekend project is me doing so. The dream was to replicate starry night made of galaxies but we'll see it didn't quite work out that way!
I'll go through my steps for solving this problem by creating a mosaic of UCSD's beloved CSE bear! For those that don't know, right outside CSE at UCSD, there is a large stone bear art piece!
The first step is just importing your target image, this is easily done using OpenCV.
The next step is to tile the base image into all the spots where new images will replace the original pixels. I opted to specify square tiles of a given number of pixels on each side and then cropped the image to the largest region occupied by the specified tile size.
This looks extremely similar to the original image as the number of pixels removed is quite small in the scale of the image. In the case of this image and the prior sizes we have tiles!
Now that we have a tile size in the original image we can either make our entire dataset of sample images match the tile size or scale up the original image such that the original tile size maps to the dataset image size. I opted to do the latter to maintain resolution and not have to resize many thousands of images.
I opted to use the cifar dataset as there is a wide range of colors and textures that I believed will replace the original tiles well! For my convenience I aggregated all the images into one large list.
To replace every tile with an appropriate sample image we need a means of comparing the tile to the dataset. Initially I had the idea of using a mean pixel color metric and finding the most similar example in the dataset but I found that even for a small tile the texture across the tile is important to making a realistic looking image. I opted instead to do a pixel-wise comparison of each tile against the entire dataset, which is simply the L2 norm of the difference between the tile and sample image. To aid in speeding this process up, I used the multiprocessing library in Python.
While this is quite slow it gets the job done! If you have any ideas to make it faster let me know! The result is quite nice for our bear!
As mentioned before, my original idea was to create a rendition of Starry Night using galaxies. Using one of the astroNN datasets I attempted to do this.
The problem is the diversity of images and the textures from images of galaxies is not enough to have decent replacements for the original tiles. Other people have solved this problem with tinting and other image augmentations to make them work but I did not want to do this.
I opted to try the same image but now with cifar and the results were MUCH better. Even with a coarser tile arrangement it still looks quite nice.
I really enjoy the art style that you get from different mosaics like this, and plan to try out other dataset and image combinations. If you have a request let me know and I may just run it!