I have been meaning to write a post about looping over large data sets in actionscript for some time. After reading Jesse Warden's post on parsing lots of data in Flash I figured it was time. As someone who does a lot of work integrating Flash and Flex apps with databases I routinely run across cases where I need to loop over a long list of items and do some work on each item. The actionscript virtual machine has always had issues with looping for long periods of time (long being a relative term). How many times have you seen this.
A script has executed for longer than the default timeout period of 15 seconds.
Or better yet, you get the beachball of death (if you're on a Mac).
Jesse does a good job of explaining the issue and presenting some work arounds. I was even introduced to new term from his post, green threads. I have never tried the getTimer method however I plan to in the near future. It seems to make a lot of sense and I really like the idea of controlling how long the process can run.
In my own experience I have used a slightly different approach to get around looping over multiple frame heads while trying to execute methods. I don't use a loop, at least not a traditional one. It is quite similiar to the Builder pattern but I also add an Observer pattern into the mix. I like to call it the Worker Bee pattern only because it reminds me of a bee hive where the worker bees go off, do their work and when done come back for more, over and over and over again . I create set of "worker bees" that can be given some task to perform outside of the hive (loop). When they have finished their task they notify the hive and then wait to be given a new task.
This solution came out of a problem I faced when building my flash based photo mosaic application. Each mosaic can contain 6400 quadrants. Each quadrant is assigned an image based on how closely it's color matches that of an input image. To get a decent mosaic you need hundreds or thousands of input images to get good color matches. In my case I use about 5000 input images. Some quick math .... 6400x5000=32,000,0000, yes that's right, 32 million loops. Can you see the flash plugin starting to smoke? There was no way I could run that many loops to create the mosaic. The only way to do it was to break things down into smaller chunks.
I tried a strategy similiar to the one outlined in Jesse's post with a single 'builder' that would be given a quadrant and told to find a match and then return it. This worked but it was slowwww. I wanted the process to be as fast as possible. After some tinkering I found I could run multiple builders at the same time. The trick then was to keep all the builders busy and know when all of them had finished working. The concept in a nutshell, is to create a small army of workers who can be given a task, when the task is done they report back and then get back in line for more work. When all the work is done, everyone can cleanup and go home.
At this point I'm sure some actual code will help to make things clear. The first step is to setup some workers to do the work. In this case I create an array and populate it with 10 workers. Each worker fires two events which are listened for.
_tilers = new Array();
var tiler:Tiler;
for (var i:int=0;i<10;i++)
{
tiler = new Tiler();
tiler.id = "tiler_" + i;
tiler.addEventListener(TilingEvent.MATCHED_TILE, tileMatched);
tiler.addEventListener(TilingEvent.SKIPPED_TILE, tileSkipped);
_tilers.push(tiler);
}
The next step is to get them working. This snippet starts the process
var len:int = _tilers.length;
for (var i:int=0;i<len;i++)
{
loadTile(_tilers[i]);
}
And this method assigns each tiler a task. It also checks the length of the array being 'looped' over. Once the length is zero it checks each tiler to see if it is busy. Once all the tilers are no longer working things can continue.
private function loadTile(tiler:Tiler):void
{
//get the next tile and look for a colour match
tiler.busy = false;
if (_model.quadrants.length>0)
{
var quadrant:QuadrantVO = _model.quadrants.splice(idx,1)[0] as QuadrantVO;
tiler.getTile(quadrant, _diff);
}
else
{
//check to see if all the chips are done processing
//if no then wait until finished
var len:int = _tilers.length;
var done:Boolean = true;
for (var i:int=0;i
{
tiler = _tilers[i] as Tiler;
if (tiler.busy == true){
done = false;
break;
}
}
if (done)
{
//go do whatever you need
}
}
}
Finally, handle the matced and skipped events that get fired from the tilers, once the event has been dealt with, make a call back to the method above with the freed up tiler so it can do some more work.
private function tileMatched(event:TilingEvent):void
{
var tile:TileVO = event.tile;
//do whatever you need with the tile
//then tell it to get some more work
loadTile(tile);
}
private function tileSkipped(event:TilingEvent):void
{
var tile:TileVO = event.tile;
//handle the fact it was skipped
//go get some more work
loadTile(tile);
}
One thing you likely noticed is that nowhere in the code is there a 'for loop' to iterate over the quadrants array. Instead of using a traditional for loop I leverage the tilers notifying when they have finished their work to simply pop (or slice) off the next chunk that needs to get processed. This is super scalable, it works for arrays of any length 100, 10000, 100000, it doesn't matter, by not using a loop statement you escape all the issues around a loop timing out.
Does it run as fast as just hammering over 'x' items? Well no, but if 'x' gets into the thousands then speed doesn't matter because things just stop working. What's better, going 200mph for 20 minutes and having your engine seize or going 100mph for as long as you want. If you need to travel for very long then obviously it's case 2.
That's all I've got to say on this. A little longer than I expected but hopefully it's helpful.
Derrick Grigg is a Rich Internet Application (RIA) freelance contractor based in Toronto, Canada. He specializes in architecting and developing applications using a variety of technologies, most notably Flash, Flex and Coldfusion.
