Perform distributed image processing, such as resizing, using an IPython cluster

I’m working on a project that needs to process a lot of images on a daily regular basis.
Each processing task makes use of SciPy and PIL and is fairly cpu and time intensive.

I thought I’d share a gist of the image resizing code and a highly simplified client.
Uses IPython.parallel to distribute processing onto a bunch of nodes.

(the production version will use an ipcluster in ssh mode)

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s