Photo Mosaics in R

Thu, 13 Feb 2020 00:00:00 +0000

Harrison Schramm, CAP, PStat, is a Senior Fellow at the Center for Strategic and Budgetary Assessments.

In this short piece, I’m going to discuss a fun photography project I did over the winter using R. I’m also going to touch on some of the implications of the R license, which underlies our entire ecosystem, but we don’t usually think about that often.

I’ve been a dedicated useR for the past 4 years. I started by using R for all the things that I previously did with spreadsheets - a great way to learn your way around the magrittr and dplyr packages. From there, I Replaced my word processor and slide software with markdown. All the while, I’ve been increasing the amount of time I spend on graphics, particularly within the ggplot2 construct, as well as color scales provided by the ggsci package (there are Simpsons and Futurama color palettes)! At some point over the summer, my interest in developing graphics began to inspire my R-tistic side, which I wrote about in INFORMS/Analytics magazine this summer.

All of last year, I worked on a project titled ‘Mosaic’. When we finished our report, my coauthors asked for suggestions on cover art, I naturally suggested we create a photo mosaic. While there are both free and commercially available solutions, my first choice, of course, was to find an R-centric solution. The advantages to using R for this project (as well as other things) is that it allows for the creation of bespoke solutions; in other words, I don’t want just any photo mosaic, but rather one that has the attributes that I want.

After a quick stack overflow / CRAN search, I found the RsimMosaic package which gave me the tools I was looking for.

Making a photo mosaic

Basically, the tools in this package take a base image and replace each pixel with a tile. If properly chosen, a close-up view will focus on the tiles, but at a distance the base image will emerge. While this is simple enough, there are a few thing to consider.

First, the dimension of the resulting image will be (in pixels) approximately the base image expanded by the tiles; for instance, if the base is 150x150, and the tiles are 30x30, the resulting image will be 4500 x 4500. I (somewhat embarrassingly) discovered this by having my R instance return a cannot allocate vector of size 9GB error ¹.

Second, there is an R-tistic balance between the size of the base image and the size of the tiles; some experimentation is necessary. If you have an extensive library of tiles (I had over 600 in this instance), it is possible - but ill advised - to try to adjust their sizes manually. Fortunately, the package has a utility for doing this. However, there’s a catch; not all pictures have a base resolution that is amenable to being scaled down.

Building a photo mosaic is really an R-tistic thing to do. The key is to collect a library of tiles that will allow sufficient diversity so that the program can make good contrast choices. For example, pictures of ships and airplanes are heavy in blue tones, etc.

Making a tile library

Once you have found your tiles, you will want to resize them. For example, I have found in most images I’ve been working with that a 30 x 30 (pixel) tile is a good size, balancing resolution with ‘mosaic-ness’. However, this is not the size of most raw images, and while you can resize them in MS Paint, this is a painstaking process. Fortunately RsimMosaic provides a handy method: createTiles. It’s almost perfect for my application.

Here’s the bit about the license

Because R and it’s packages are distributed under the GPL license, you have the ability to adjust the functions that are in packages. This is straightforward; if you want to adjust a function foo, you can assign it a new name foo2 <- foo and then fix(foo2) (or use a script / markdown) to make changes. You can of course simply edit the original function, but I do not recommend this as it can cause confusion on subsequent sessions.

Remember in the preceding paragraph where I said it was almost perfect? Some tiles have base sizes that cannot be coerced to rectangular shapes. The method that comes with the package simply generates an error. What is useful when generating ~600 tiles is to have a list so that I know the one that threw an error ².

But because we can do whatever we want with existing functions, consider the following modification:

createTiles2 = createTiles

fix(createTiles2)

{add the following directly after line 15:}

print(filenameArray[i])

And voila! Fast and artistic! I used this list to find the tiles that could not be resized, and removed them from my tile library.

Making the Mosaic

Now, for the part where we make the mosaic. The command:

composeMosaicFromImageRandomOptim("RLogo_small.jpg", "RMosaic.jpg", "OutputPathGoesHere")

generates the following:

is it surprising that an accomplished professional would admit to such a silly mistake in public? I don’t think so! We would all be better off if we were as open with our success as our failures; first to prevent others from wasting precious time with similar mistakes, and second to show that if we are going to work at the cutting edge, we are all in a sense students.↩
other programmers might suggest a tryCatch() environment, and I did think of that. Here was a case where I wanted something that worked fast, vice something that worked well.↩

Photography on R Views

Photo Mosaics in R

Making a photo mosaic

Making a tile library

Here’s the bit about the license

Making the Mosaic