Map Reduce with Node.js and Mongo

Working on my pet project I ran into an issue where the data coming for mongo needed to be prepped for client-side view.

Traditionally, if it was an RDBMS, we could use a “group by” query.

In Mongo, we either have to use the dev release 2.1 that includes the aggregation framework (and then make sure that mongo driver for node.js supports aggregate commands), or use recommended map-reduce approach with stable mongo release.

I tried running a “group” command with mongo 2.1.1, but something didn’t quite work. So then I went for an alternate approach – writing a map-reduce function.

My data set looks like this (hashtags with dates and counters):
{ “count” : 1, “tag” : ” #style”, “tag_date” : “6/20/2012” }
{ “count” : 1, “tag” : ” #centralpark”, “tag_date” : “6/20/2012” }
{ “count” : 1, “tag” : ” #tisch”, “tag_date” : “6/20/2012” }
{ “count” : 2, “tag” : ” #dogs”, “tag_date” : “6/20/2012” }
{ “count” : 1, “tag” : ” #familytime”, “tag_date” : “6/20/2012” }
{ “count” : 1, “tag” : ” #rhonyc”, “tag_date” : “6/20/2012” }
{ “count” : 5, “tag” : ” #believe”, “tag_date” : “6/20/2012” }
{ “count” : 1, “tag” : ” #blogher12″, “tag_date” : “6/20/2012” }
{ “count” : 1, “tag” : ” #youthinkwerekidding”, “tag_date” : “6/20/2012” }
{ “count” : 1, “tag” : ” #pokemonconquest”, “tag_date” : “6/20/2012” }

And I need it to look like this:

{tag: {date:count}, {date:count} … }

In other words, have all dates and counters aggregated for each tag.

Here’s the map-reduce functions:

And here’s how we call it from the app:

Note: ‘inline’ parameter MUST be specified

And the results returned from the DB:

{ “_id” : “#10thingsilove”, “value” : { “value” : [ { “tag_date” : “6/21/2012”, “count” : 1 }, { “tag_date” : “6/22/2012”, “count” : 11 }, { “tag_date” : “6/25/2012”, “count” : 2 } ] } }
{ “_id” : “#10worstfeelings”, “value” : { “value” : [ { “tag_date” : “6/25/2012”, “count” : 1 }, { “tag_date” : “6/26/2012”, “count” : 3 } ] } }

Great resources that helped me make this work:
http://benbuckman.net/tech/12/06/understanding-mapreduce-mongodb-nodejs-php-and-drupal
http://cookbook.mongodb.org/patterns/pivot/

One thought on “Map Reduce with Node.js and Mongo

  1. Pingback: Tanya Nam » Blog Archive » You know what we did last summer?

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s