Rethinkdb 1.11, python and queries

Just did a short discovery session in python to figure out if Rethinkdb 1.11 can do some of the queries I’m after.

Wanted to to perform queries that filter

  • by field
  • by nested field
  • by values within a range
  • if value exists in embedded array
  • any combination of those

More info:

https://github.com/diffused/sbf-python-rethinkdb-play

 

Advertisements

Perform distributed image processing, such as resizing, using an IPython cluster

I’m working on a project that needs to process a lot of images on a daily regular basis.
Each processing task makes use of SciPy and PIL and is fairly cpu and time intensive.

I thought I’d share a gist of the image resizing code and a highly simplified client.
Uses IPython.parallel to distribute processing onto a bunch of nodes.

(the production version will use an ipcluster in ssh mode)