Map Reduce with Node.js and Mongo

Working on my pet project I ran into an issue where the data coming for mongo needed to be prepped for client-side view.

Traditionally, if it was an RDBMS, we could use a “group by” query.

In Mongo, we either have to use the dev release 2.1 that includes the aggregation framework (and then make sure that mongo driver for node.js supports aggregate commands), or use recommended map-reduce approach with stable mongo release.

I tried running a “group” command with mongo 2.1.1, but something didn’t quite work. So then I went for an alternate approach – writing a map-reduce function.

My data set looks like this (hashtags with dates and counters):
{ “count” : 1, “tag” : ” #style”, “tag_date” : “6/20/2012″ }
{ “count” : 1, “tag” : ” #centralpark”, “tag_date” : “6/20/2012″ }
{ “count” : 1, “tag” : ” #tisch”, “tag_date” : “6/20/2012″ }
{ “count” : 2, “tag” : ” #dogs”, “tag_date” : “6/20/2012″ }
{ “count” : 1, “tag” : ” #familytime”, “tag_date” : “6/20/2012″ }
{ “count” : 1, “tag” : ” #rhonyc”, “tag_date” : “6/20/2012″ }
{ “count” : 5, “tag” : ” #believe”, “tag_date” : “6/20/2012″ }
{ “count” : 1, “tag” : ” #blogher12″, “tag_date” : “6/20/2012″ }
{ “count” : 1, “tag” : ” #youthinkwerekidding”, “tag_date” : “6/20/2012″ }
{ “count” : 1, “tag” : ” #pokemonconquest”, “tag_date” : “6/20/2012″ }

And I need it to look like this:

{tag: {date:count}, {date:count} … }

In other words, have all dates and counters aggregated for each tag.

Here’s the map-reduce functions:

And here’s how we call it from the app:

Note: ‘inline’ parameter MUST be specified

And the results returned from the DB:

{ “_id” : “#10thingsilove”, “value” : { “value” : [ { "tag_date" : "6/21/2012", "count" : 1 }, { "tag_date" : "6/22/2012", "count" : 11 }, { "tag_date" : "6/25/2012", "count" : 2 } ] } }
{ “_id” : “#10worstfeelings”, “value” : { “value” : [ { "tag_date" : "6/25/2012", "count" : 1 }, { "tag_date" : "6/26/2012", "count" : 3 } ] } }

Great resources that helped me make this work:

http://benbuckman.net/tech/12/06/understanding-mapreduce-mongodb-nodejs-php-and-drupal

http://cookbook.mongodb.org/patterns/pivot/

Leave a Reply