Flash based image mosaic
I have always thought image mosaics were pretty neat, not just because of the way they look, but because of the processes that are used to create the mosaic. With a better grasp on what is possible with images using the BitmapData class in Flash and a plethora of images available via the Flickr API I decided to try developing a Flash based image mosaic application.
Step 1 - Mapping the colours
This process is pulled directly from my one of my Pixel Fun samples. The concept is straight forward, load an image and then determine the dominate colour and grey scale value for each predefined block of the image. Sounds simple but as anyone who has tried to program massive amounts of looping in Flash will tell you, a nasty warning about a script slowing your movie will pop up pretty quickly if you do not handle things properly. In this case I have an image that is 400 pixels wide by 300 pixels high. With 10 x 10 blocks, it means I need to loop over 1200 blocks, 100 times each, that works out to 120,000 iterations. I used some trickery with the setInterval() method to break the iterations into smaller chunks, slow down the Flash engine and allow it to perform all those iterations without exploding.
Step 2 - Loading and processing the Flickr images
Photo mosaics require a lot of photos to sample, in order find photos that match the dominate colours determined in step 1. Thankfully Flickr has an API that allows developers to tap into their database of images. The trick in this step is loading the images from Flickr into the BitmapData class. A constraint on the BitmapData.draw() method is that any image in the source object must exist in the same domain as the swf. Thankfully my knowledge of Coldfusion saved the day. I created a cfm page that takes the URL of the image to load from Flickr, read the file and writes out the image header and data, thus allowing the image to 'exist' in the same domain.
Step 3 - Tiling the image
Speed was my enemy during this process. Originally in step 2 I loaded each image, determined the dominate colour value, saved that information in an array and then dumped the image. During this step I would use the array to find the images I needed and then reload each image back into Flash. The load/unload/reload process was much slower than I wanted so I did some testing to come up with a better solution. The solution I chose was to merge each image into an instance of the BitmapData class that could contain all the photos (the right side of the image in step 2), then by saving the x/y coordinates of the merged image I could easily copy the pixels from the large source bitmap into the mosaic as needed. It turns out this process has a technical term, blitting. It works like a charm and sped up the tiling process by an incredible amount. I used this same concept to create the mosaic. Instead of having 1200 movieclips to represent each tile in the mosaic (which brings the Flash player to it's knees as I discovered), I have one large bitmap that I merge the photos into.
Step 4 - Previewing tiles
Nothing overly exiting here except the finished product. As you roll over the mosaic the pointer's x/y position is tracked to determine which tile in the larger bitmap you are over. Using an associative array, the correct tile data can be found with no looping required, and the original source image block, with the RGB and grey scale values is displayed along with the mapped image and it's RGB and grey scale values, click the tile and you can view the original full size image in Flickr.
End Result
As they say 'a picture is worth a thousand words'.
Update: I have converted the Flash mosaic into a standalone AIR application you can run from your desktop. You can download the application here. If you are interested in a web based version, please use the contact page to reach me.
Next steps
Hopefully as time permits I can extend this project out to include:
- Source image uploading
- Allowing the user to enter the sample photo search criteria
- Zoom in/out capability
- Downloading the finished mosaic
Jul 13, 2006 at 11:58 AM
Caught your blog on FITO forum. Looking fwd'd to future posts. Best of luck - mL
Jul 25, 2006 at 4:35 PM
Just came across this Shockwave version of an interactive mosaic. Really neat.
Aug 03, 2006 at 4:41 PM
Very cool. I have a project I am looking for someone to help me develop. I have an event coming up where attendees will be constructing a tile mosaic the day of the event of a predesigned image. Each tile placement gets associated with a "wish" from a participant and an ID number. There is then a digital representation of the image that allows viewers to mouse over the image and see the "wishes" submitted by all the participants. It is mosaic meets wishing tree....
A couple of questions:
1) Does this tool you developed let you pull images from a specific flickr photstream, or is it all of flickr?
2) Would you be interested in helping design the site I explained above? It would be similar to yours above, but would not need to process the image for photos, and would like to a separate page displaying a database entry rather than pull images from flickr....
Let me know, and nice work! I like it!
Jan 09, 2007 at 12:23 PM
looks great! looking forward to see additional features: source image uploading, and also example where you pull taged images from some other database, not flickr.
Jun 21, 2007 at 12:48 AM
Exillent work.
Is there any possibility to use Flash Action Scripts which enables client user to mix or overlap two images (by accessing swf file through my web page) at his/her own machine and send it back to my server in the merged JPEG, GIF, BMP, PNG etc. format.
Please feel comfortable to contact me at my email address for further clarification on my request.
thanks & regards
Dec 05, 2008 at 10:40 AM
Interesting project, and the technical side is very cool. Unfortunately the output is not very visually appealing. I think to produce better output, you'd need a more advanced algorithm instead of just 'average color per block'.
Though I'm not sure if these more advanced algorithms are possible in flash, since even just average color is already very computationally intensive. Still, don't give up, you've made good progress!
Jan 03, 2009 at 3:27 PM
What about this:
www.mosaicmaniac.com ?
Jan 19, 2009 at 11:38 AM
I saw something like this at Silverlight showcase.
Clicking on the tile zooms in the original image.
Although a littlt obscure, clicking again on the image name in the lowe left corner, you can execute your own javascript by replacing handleDocumentClick(url).
I think it might require a silverlight plugin though.
http://mosaicmaniac.com/?c=DeepZoom&;i=art/dz/horse2/DeepZoomProject.xap
Jan 19, 2010 at 7:12 AM
Great job!