Technology thoughts, how-to tutorials
My latest tech interests are node.js/mongo on (fashion), followers of certain brands – whatever you want.
I’m hoping to put it into a good demo soon and share on my blog, and meanwhile, I’m posting a couple of introductory presentations I did on my node/mongo/heroku stack:
It’s been very busy in the office lately, with a variety of projects going on with my accounts, and I really missed posting to my blog. Aside from work, there has been also a ton on things happening in my personal life, which are all awesome. I created a separate personal blog for that which I’m not ready to share yet, but might do at some point in the future. And then also I traveled for fun and business.
So as busy as the past few weeks have been, I feel lucky and fulfilled to be working on a lot of different things, and have so many things happening in life too. Here I wanted to talk about work a bit more.
Since the beginning of the year I’ve been working on 2 large projects, a whole handful of smaller ones, and also new business/strategy. Each of those is so distinctly different, and aside from occasional hassles and mini-annoyances, I enjoy the work. My team completed or just about to complete:
As you can see, this range of work really stretches your potential and makes it very interesting (although creates time challenges). And I love it. Thinking of this made me remember a Show&Tell talk I gave last year about what a typical day in a life of a Technical Director looks like (here’s the link to the deck file). Oh, and I don’t like to brag, but as of January 1st of this year I was promoted to a Sr. Technical Director (I think titles are only somewhat relevant because I kick ass no matter what the job is called :) but still feels awesome, nevertheless!
Today I had an honor of speaking at the career panel for CS students at Brooklyn College CUNY.
My friend Suzanne invited me to participate a few weeks ago, and I immediately thought it was a great opportunity, especially since the topic of careers in tech is near and dear to my heart, and I loved being in college and wanted to see what students these days are about.
There were quite a few topics discussed today, mostly focussed on students graduating and entering the job market. A few questions were really interesting and related to today’s hot topics, so I wanted to highlight and recap them here.
Q: What is your understanding of the future or trends in the technology field? What direction is the industry heading in and where are opportunities opening up?
It would actually be funny if someone can predict the future – or we would all be rich. Coincidentally, I just re-watched an old Soviet movie really popular with kids during my childhood, it’s called “Guest from the future”. The movie was made in the early 80s, and there’s a time machine that allows people to travel in time, and in the future circa 2080’s people “flip” in flying little cabins instead of using cars, and have devices that can read anyone’s mind. But if these things will actually happen in the future, and when – nobody knows for sure.
The best thing we can do is to get solid fundamental education, stay curious, and know how to adapt quickly. Also find out who the experts are in your area of interest (be it mobile, connecting tech with physical devices, artificial intelligence, etc.) and talk to them. A great discussion with smart people is priceless, and sometimes you can even find yourself thinking about something that addresses needs of people, and seeing it used widely in the near future.
Q: How is the technology industry currently for women and/or minorities? Do you have any particular advice for women and/or minorities interested in technology?
Technology industry is a great place to be for women and minorities! I did not personally face any prejudice or discrimination either in school or my career, and actually feel that the opposite is true – tech is a very welcoming place to be right now, with lots of jobs and openings and developer bootcamp programs encouraging women to apply. Technology field is very fair and based on merit and actual work you do – so being great is the key, regardless of your background and who you are.
My advice for anyone (men or women) would be the same – do your best, learn as much as you can, excel at what you do, be professional and a good communicator, and the rest will fall into place.
Q: Please talk about graduate school. How necessary or important is it for the field?
My answer would be based on my own experience (so might not apply to everyone), and I really didn’t want to spend time in grad school, but rather get as much hands-on, real-world experience as I could. So no grad school for me. Other panelists suggested to consider grad school if you know already you want to focus on a certain area of your field, or if your employer is subsidizing your education in some way. Agreed on both counts.
Q: What skills, training, classes or experience are important for someone interested in the technology career field?
Some panelists provided very focused answers, targeted to web development with technologies at hand (HTML/CSS). I disagree. Technologies du jour may come and go, but your basic fundamental understanding of how things work, and how and why new technologies were created will serve you long term.
For example, Ruby is a very popular language to know today. There are developer schools specializing in teaching Ruby and making you into a web developer in as little as few months. It is great. But just to know Ruby, its syntax and how to create web pages, is only scratching the surface. If you really understand why Ruby was created in the first place, what MVC is and how it was implemented with Rails, what are the limitations of Ruby that are better done in other languages, you’ll go way further than majority of other self-proclaimed developers.
If you know your fundamental stack and principles, you can program anything from a spiffy web app, to a mobile app, to a tiny-sized app that you put into a watch or send to the moon… So don’t limit yourself to one technology, and stay curious and keep learning every day. Skillshare is a great place to learn new stuff, including non-tech subjects like business, project management, art, etc.
Big thanks to Suzanne, Brooklyn College CS society who organized the event, and my fellow panelists, who each offered a unique perspective and shared their experience and advice. It was a great honor!
Last night I stopped by the NYC Rails meetup group, which is now hosted at the Flatiron School, where students with no previous programming background learn the fundamentals of web development during the intense 3-month program. One of my friends is in the program right now, so it was great to chat with her and learn about it from someone who’s there almost 24/7 :)
Every week, a group of students present a project that they created to the fellow students and anyone attending the meetup. Yesterday 3 students demoed the app they called Octomaps, that shows locations of collaborators for a given Github repo on a map view. It was a pretty cool demo, especially given the fact that the students were just a few weeks into the program.
The app required using Github and maps API, database integration, some front-end presentation code, then deployment to Heroku. But above all just coming up a cool idea and not being afraid to dive in and implement in only a few days was very admirable.
Aside from great effort from students, major props go to Avi Flombaum, who runs the school and teaches most of the classes. I only was there for an hour, but it was obvious that he instills lots of enthusiasm and fosters curiosity (both essential for a web developer, I think), in addition to sharing the knowledge. Very impressive!
If you’re into web development, Ruby or just want to find out what you could learn in just 3 months, you should definitely stop by one of the meetups. They run through end of April, and next week is supposed to be an interesting group lunch ordering app. Seamless, watch out!
So my Insights team had a challenge with timestamp data on tweets. Their reporting system was giving them data in users’ timezones and it was not consistent (since timezones are different for different users) and bad for reporting. Their question to me was whether we can have all timestamps converted to the same timezone (preferably EST since the team is here).
The data is actually already available through an API call, and the only task is to convert it to a readable local timezone format.
Here’s how data is coming back from the API call, in UTC zone:
"created_at": "Wed Jun 01 12:53:42 +0000 2011"Python code:
#import datetime
from datetime import datetime
from datetime import timedelta
clean_timestamp = datetime.strptime(obj['created_at'],
'%a %b %d %H:%M:%S +0000 %Y')
offset_hours = -5 #offset in hours for EST timezone
#account for offset from UTC using timedelta
local_timestamp = clean_timestamp + timedelta(hours=offset_hours)
#convert to am/pm format for easy reading
final_timestamp = datetime.strftime(local_timestamp,
'%Y-%m-%d %I:%M:%S %p')
Had another great question in the M102 forums a few days ago, so wanted to share it, since I didn’t know the exact answer, and Dwight graciously clarified it for everyone.
Question:
My initial thinking went like this:
The user info is stored in the system.users database (where all user credentials are stored), so with replica sets it would be replicated across all nodes and allow access to data from any node. And since the info would be copied across all nodes, I don’t think it’s possible to have different admins for different nodes.
Dwight’s explanation with example:
The auth information replicates, so you either have authorization for the set as a whole, or not. The auth is per database (with authorization on the ‘admin’ database implying you can access all).
So you could do something like this:
$ mongo –host abc8
use mydb
db.system.users.find()
^C
$ mongo –host abc9
use mydb
rs.slaveOk()
db.system.users.find()And you should see the same user information (assuming the secondary is caught up to the replication time of the user additions).
The above example is for a standalone replica set; if you are sharded you would connect to the cluster through mongos. Once again your credentials are for the whole cluster, per db.
Additional resources and notes:
More information on authentication: http://docs.mongodb.org/manual/tutorial/control-access-to-mongodb-with-authentication/
Note that for replica sets you’ll need to use –keyFile option to specify configuration file that will be used to authenticate between members of the replica sets: http://docs.mongodb.org/manual/administration/replica-sets/#replica-set-security
One of the benefits of being a TA for M102 course is that you get to learn even more by answering questions. Because some questions are fascinating.
Yesterday we had a very interesting case that puzzled even those folks who are usually able to answer anything in the forum. There was some virtual head scratching (or at least I’m imagining it), and we decided to ask the 10gen team if they can shed some light on this.
Here’s the scenario, where we are starting with an empty collection:
db.test.update({_id : "Jane"}, {"count" : 1}, true)
db.test.update({_id : "Jane2"}, {$set: {"count" : 1}}, true)
db.test.find()
{ "_id" : ObjectId("510aa62cbb530b24318c93d4"), "count" : 1 }
{ "_id" : "Jane2", "count" : 1 }
As we see, in both update statements, the “upsert” option is set to true.
The result of the first statement is a document with an _id field assigned by default by MongoDB, even though we specified _id:”Jane”.
However, the result of the second statement is the document with the _id that we specified – so it’s a different outcome when we use “$set” operation.
And here’s the explanation (from MongoDB docs):
If the argument includes only field and value pairs, the new document contains the fields and values specified in the argument. If the argument includes only update operators, the new document contains the fields and values from argument with the operations from the argument applied.
Link: http://docs.mongodb.org/manual/applications/update/
Then Andrew also explained in the discussion forum:
The behavior of the update operator depends on what is the second positional document. If you put in set operator or push operator, then the db will insert a document that has both the fields that are set in the selector and is the result of applying your pushes and sets.
So this behavior is expected, but still makes a very interesting case. Live and learn, my friends!
I have a quick exciting announcement! I wrote about taking 10gen’s M101 and M102 online courses last fall, and about excellent quality of their videos and content.
Well, after the course completion, I was happy to learn that I got to be one of the top graduates for M102 (MongoDB for DBAs track). I did pretty good in M101 but mis-submitted one of the answers by checking an option that was too similar to the correct answer, but was incorrect.
10gen asked top graduates if they would like to offer assistance in the next round of classes, and of course I said that I’d love to help, since I’ve been really impressed by the company, their product and the education portal they put together. A few conversations later, they invited me to participate in the M102 Spring course as a TA (teaching assistant).
I’m helping out by answering questions in the discussion board and clarifying details on home work and quizzes. It’s really cool that some other discussion participants are so knowledgeable and willing to help other fellow students out.
If you’d like to learn more – 10gen Education blog is a great resource where Andrew Erlichson, 10gen’s VP of Education, writes about how they created the courses, cost and equipment involved, and even some interesting stats on the Fall semester (about 19% of students enrolled graduated with scores 65% or higher, for both M101 and M102). Check it out.
And of course, definitely sign up for the online classes – such invaluable in-depth content, and anyone who successfully completes them will get cool certificates, just like I did :P
A couple of months ago I wrote about a great iniative from Coding for GOOD, where applicants from any background, even without prior programming experience, could take online classes and submit their projects for a chance to get a cool job in LA!
The other day Robbyn from GOOD Worldwide, reached out to me on Twitter letting me know that the finalists have been chosen! So exciting! And what’s really cool is that one of the finalists is from NYC – Ada Ng from Brooklyn. Definitely read about the finalists and their projects on the GOOD website.
And check back tomorrow, because the last round (a weekend hackathon) has been completed, and the winner is being chosen today and will be announced tomorrow! Who do you think it will be? In any case – I think it’s a really big achievement for all finalists, big congrats to each of them and kudos to GOOD for organizing this. Can’t wait to find out who the winner is and follow their brand new career gig at GOOD!
I wrote about setting up a cron job on my work machine to process some files in a shared Dropbox folder.
After a while, old files will start to accumulate there and I’d want to delete them, by adding another command to the crontab.
The criteria for deleting files in my case are:
– File is older than 5 days
– File has an extension .csv (keep other files like readme)
– Do not delete a few example .csv files
So HGH the HGH Unix command will look like a pipe that first finds the files, then deletes them.
find X -exec rm {} ;
It’s pretty easy to find all .csv files older than 5 days:
find /path/to/files/*.csv -mtime +5
And then, to exclude certain CSV files that I want to keep, I used -not and -and operators to specify filenames (test.csv, test_out.csv and input_example.csv):
find /path/to/files/*.csv
-not -iname test* -and -not -iname input* -mtime +5
Tanya Nam 