Memorable ID Sentences

I stumbled upon the really neat idea of converting integers into memorable sentences on the Asana blog. The crux of the problem is that when giving a user a random number, it is likely that they will get it wrong, or it will be annoying to repeat each of the numbers to a customer service representative. So, instead of providing them with a number, how about providing them with a sentence instead? For example, the following numbers and sentences are "equivalent" to each other:

2474137653 == 20 exultant leopards strode intensely
1934320125 == 16 excited platypuses babbled wearily
4225612387 == 33 flippant snakes traveled sadly

I won't go into the details of how it works, as it is covered in detail in the original blog post and the code below, but I thought I'd share my implementation of the idea.

The original work uses 32 bit integers, and I implemented it pretty much as it was described. However, in order to allow for the potential extension of this approach to other types of numbers (shorts, longs, etc.) I created a base class that does all of the "heavy lifting" followed by a subclass that merely provides it with the necessary bits of information to make it work. So, if you felt like it, you can create longer sentences for longs, or use fewer words for shorts, etc.

And before you wonder where you can get the code, it's available as a gist and for your viewing pleasure is included here.

2 Weeks With The Kinesis Advantage Keyboard

For the past two weeks I've been working with the ergonomic Kinesis Advantage keyboard. For those of you who don't know what it its, well, the picture pretty much sums it up... it's a keyboard with a rather unorthodox design meant to be highly ergonomic. The basic premis is that it forces one to use the correct hand placement when typing because all of the keys are recessed into the keyboard itself. This way, one "naturally" has all fingers below the wrist and minimizes the risk of getting every programmer's worst enemy, RSI (lack of sleep, lack of caffeine, bugs in code, etc. are not far behind.)

First Impressions: RTFM

I must say, this is the first keyboard that I've ever used for which I actually read the instruction manual. As I work on a Mac, the first thing that I had to do was convert the keyboard layout for the Mac keyboard layout -- not very hard, just a simple key combination that is on page one of the instructions. Hats off to the folks at Kinesis to actually supply extra keys to switch out the default Windows keys for to the ones that correspond to the Mac layout. They are even nice enough to include a little plastic doohickey that makes the removal of the keys a snap. In addition, the keyboard comes with some foam pads that can be attached to the keyboard to make a comfortably padded area for your wrists. After that initial setup (and reading about how to access the various functions), I was ready to go.

Although you are free to make your own judgement, stylistically, the keyboard is no where near as sleek as the Apple keyboards (or the Microsoft mobile keyboard). It's most definitely an imposing piece of hardware that can make even the most spartan desk look like Times Square on New Years Eve. I don't fault them for this, since there really isn't another way to have keys recessed in a keyboard without having a big keyboard. (Although, they could potentially shave a couple millimeters from various locations.) Also, by no means is this a quiet keyboard. If you have 10 people using these simultaneously, you'd probably think you were working on a assembly line in an early 1900s factory. But, I guess that's the price you pay for for keeping RSI at bay.

Day 1: Wow, I can't type

The first thing that I noticed, and you probably will to if/when you use a Kinesis keyboard for the first time, was that I couldn't type. The enter key was not by my left pinky. The delete key is now accessed using my left thumb. Boy, oh boy, was this going to be an adventure. My words-per-minute dropped down to probably about 10 (it's a good thing I had no immediate pieces of code that I had due). I was hunting and pecking as if I were back in middle school trying to make that little turtle draw a doughnut in Logo (those were the days!).

Day 2: Things are looking up

There was definite improvement in my typing skill on the second day. I now could type most things without having to double check them on the keyboard. That being said, auto-correct and auto-complete were still my best friends. One thing that you'll also become accustomed to is random passerbys gawking at your keyboard. You'll undoubtedly come up with some sort of witty remark as to why you are using this thing like, "It's how I ensure that no one asks to use my computer."

Two Weeks In: I can type again!

While the first couple of days were certainly rough, I am back to my original typing speed for most things. The placement of the arrow and bracket keys, however, are not ideal for the amount that I use them. Due to the layout of the keys, the largest drawback to using this keyboard is that you are forced to have two hands on the keyboard at all times - it's pretty difficult to, for example, use your mouse with one hand and the keyboard with the other. Of course, having the keys separated and recessed into the keyboard is the point of this keyboard.

Wrap Up

Am I keeping the keyboard? Yes. By and large, it is the most comfortable keyboard that I have worked with. Although the first couple of days were rough, it was worth fighting through the pain and getting accustomed to it. I am currently at the point where I sometimes have trouble using a regular keyboard due to the differences in layout. So, go ahead, treat yourself (and your wrists) to the "La-Z-Boy" of keyboards and enjoy the wondrous stares you'll receive at the office when you show up with your Kinesis keyboard. For those of you who don't want to restrict the fun of typing to your hands (or want to feel like a drummer in a rock band), the even keyboard comes with an optional foot-pedal that can be mapped to commonly used keys!

Celebrating with Lasers

To celebrate 2.5 weeks at our new startup, we visited the machine shop in the Computer Science and Artificial Intelligence Laboratory (CSAIL) at MIT, and used the laser cutter to create a first pass at a sign for our company. Seeing that we are working in some co-working space at the Harvard Business School Innovation Lab (i-Lab), being able to hang up a sign above our desks, is, well, a good thing.

With the lasers changed from stun to cut, we placed some acrylic in the machine, and hit the green "Go" button. Since our settings weren't right the first time around, we had to do a couple of passes for it to cut all the way through the plastic, but it didn't take long until we could take all the bits and pieces and glue them together. Of course, talk is cheap, so, how about a quick movie? Yes, you say? Ok, here it is.

As we build this company up from three people to, hopefully, plenty more, I'm going to try my best to keep this creative, fun-loving, culture alive so that work isn't just work, but also interspersed with fun (and lasers!).

(JSON) Serializing (SQLAlchemy) Objects

A common task when building a web application (or REST API) is to take some data from a database and then ship it over the wire in some serialized format. Although the concepts of this post apply to pretty much any sort of serialization task, I am going to be using python and the SQLAlchemy to illustrate my current preferred solution.

The first thing that I tried for this was to add a to_json() method to all of my SQLAlchemy models. So, as an example, a User model may look something like

from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

class User(Base):
    id = Column(Integer, primary_key=True)
    first_name = Column(String, nullable=False)
    last_name = Column(String, nullable=False)

    def to_json(self):
        return dict(id=self.id,
                    first_name=self.first_name,
                    last_name=self.last_name)

And for simple situations this works perfectly. However, let's add a little twist to this. Now let's assume that our IDs are not auto_increment integers, but some binary value (e.g. a UUID of sorts) and that we also have a field that contains the user's date of birth (dob). The problem we face now is that we can't just return the binary value for the ID and the DateTime object for the date of birth, because python's JSONEncoder doesn't know what to do with those. So, now, we have a class that looks something like this:

import uuid

from sqlalchemy import Column, DateTime, Integer, String
from sqlalchemy.types import BINARY
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

class User(Base):
    id = Column(BINARY(16), primary_key=True, auto_increment=False)
    first_name = Column(String, nullable=False)
    last_name = Column(String, nullable=False)
    dob = Column(DateTime)

    def to_json(self):
        return dict(id=uuid.UUID(bytes=self.id).hex,
                    first_name=self.first_name,
                    last_name=self.last_name,
                    dob=self.dob.isoformat())

Will this work? Assuming there are no bugs in my code, then yes, this will most definitely work. However, as I see it, there are two main flaws with this solution:

1. In most decent sized projects you will end up having quite a few models. So, this means, that for every model (and many of them will have quite a few more attributes (database table columns) than the aforementioned User with only 4 fields. Thus, you end up having to write this to_json() method with all of the attributes over and over again, wasting both time and increasing the chance of a bug.
2. If you want to change the format of any of the values, say, you moved away from UUID4s for the IDs for all of your models to UUID1s, you have to go through every to_json() method and make the appropriate changes. Again, a waste of time and highly error prone.

As such, here is the solution that I have come up with that is working well so far. It is based on these two threads (1, 2) in StackOverflow but instead of using mixins, I create a separate Serializer class that takes the object to serialize as a parameter. You'll see why I do this shortly.

First, let's define the serializer. Don't be scared, this serializer only has to be created once (and you can just copy and paste it), and after that serializing any object becomes a piece of cake! Trust me!

import dateutil.parser

class JsonSerializer(object):
    """A serializer that provides methods to serialize and deserialize JSON 
    dictionaries.

    Note, one of the assumptions this serializer makes is that all objects that
    it is used to deserialize have a constructor that can take all of the
    attribute arguments. I.e. If you have an object with 3 attributes, the
    constructor needs to take those three attributes as keyword arguments.
    """

    __attributes__ = None
    """The attributes to be serialized by the seralizer.
    The implementor needs to provide these."""

    __required__ = None
    """The attributes that are required when deserializing.
    The implementor needs to provide these."""

    __attribute_serializer__ = None
    """The serializer to use for a specified attribute. If an attribute is not
    included here, no special serializer will be user.
    The implementor needs to provide these."""

    __object_class__ = None
    """The class that the deserializer should generate.
    The implementor needs to provide these."""

    serializers = dict(
                        id=dict(
                            serialize=lambda x: uuid.UUID(bytes=x).hex,
                            deserialiez=lambda x: uuid.UUID(hex=x).bytes
                        ),
                        date=dict(
                            serialize=lambda x, tz: x.isoformat()
                            deserialize=lambda x: dateutil.parser.parse(x)
                        )
                    )

    def deserialize(self, json, **kwargs):
        """Deserialize a JSON dictionary and return a populated object.

        This takes the JSON data, and deserializes it appropriately and then calls
        the constructor of the object to be created with all of the attributes.

        Args:
            json: The JSON dict with all of the data
            **kwargs: Optional values that can be used as defaults if they are not
                present in the JSON data
        Returns:
            The deserialized object.
        Raises:
            ValueError: If any of the required attributes are not present
        """
        d = dict()
        for attr in self.__attributes__:
            if attr in json:
                val = json[attr]
            elif attr in self.__required__:
                try:
                    val = kwargs[attr]
                except KeyError:
                    raise ValueError("{} must be set".format(attr))

            serializer = self.__attribute_serializer__.get(attr)
            if serializer:               
                d[attr] = self.serializers[serializer]['deserialize'](val)
            else:
                d[attr] = val

        return self.__object_class__(**d)

    def serialize(self, obj):
        """Serialize an object to a dictionary.

        Take all of the attributes defined in self.__attributes__ and create
        a dictionary containing those values.

        Args:
            obj: The object to serialize
        Returns:
            A dictionary containing all of the serialized data from the object.
        """
        d = dict()
        for attr in self.__attributes__:
            val = getattr(obj, attr)
            if val is None:
                continue
            serializer = self.__attribute_serializer__.get(attr)
            if serializer:
                d[attr] = self.serializers[serializer]['serialize'](val)
            else:
                d[attr] = val

        return d

Now, assuming there are no bugs in the code above from when I adapted it from our production code, you can create a serializer for your User object by simply doing something like this:

class UserJsonSerializer(JsonSerializer):
    __attributes__ = ['id', 'first_name', 'last_name', 'dob']
    __required__ = ['id', 'first_name', 'last_name']
    __attribute_serializer__ = dict(user_id='id', dob='date')
    __object_class__ = User

The best part is, for any new object that you create, all you have to do is create one of these serializers and you are good to go. No more writing of to_json() in each model. And to get it to do some serialization, just do:

my_json = UserJsonSerializer().serialize(user)

As it currently stands, this can be used as a mixin, and we could add JsonSerializer as one of the parent classes for our User model. The trouble with going that route, is that you can't pass arguments to the serializer class. For example, in our system we store all dates as UTC formated dates, but need to convert them to the local timezone of the current user. As the serializer currently stands, there is no way to pass it a timezone parameter. To do this, our JsonSerializer has a constructor that takes a timezone parameter that is then used in the serialization of dates. So, for example:

class JsonSerializer(object):
    
    ... all code that was here before ...

    def __init__(self, timezone):
        self.tz = timezone

Make sense? As an added benefit, we can also add more serializers to our default list of serializers in the constructor. For example, let's say our User object references a list of Email objects and we want to serialize that as well. So, first we'd create an EmailJsonSerializer just like we did for the User, but then add this email serializer to to the serializers in Users. Ok, that was a bit convoluted, so here is what I mean:

class EmailJsonSerializer(JsonSerializer):
    __attributes__ = ['user_id', 'email']
    __required__ = ['user_id', 'email']
    __attribute_serializer__ = dict(user_id='id')
    __object_class__ = Email


class UserJsonSerializer(JsonSerializer):
    __attributes__ = ['id', 'first_name', 'last_name', 'dob', 'emails']
    __required__ = ['id', 'first_name', 'last_name']
    __attribute_serializer__ = dict(user_id='id', dob='date', emails='emails')
    __object_class__ = User

    def __init__(self, timezone):
        super(UserJsonSerializer, self).__init__(timezone)
        self.serializers['emails'] = dict(
            serialize=lambda x:
                [UserEmailJsonSerializer(timezone).serialize(xx) for xx in x],
            deserialize=lambda x:
                [UserEmailJsonSerializer(timezone).deserialize(xx) for xx in x]
        )

Now when we call the serializer, it will not only serialize the contents of the User object, but also the contents of any and all Email objects associated with it (assuming you set up the one-to-many relationship properly in your SQLAlchemy models).

Again, while I used SQLAlchemy models to illustrate this pattern, this can work for pretty much object going to and from any type of serialized data. Happy coding!

To Cubicle, Or Not To Cubicle

Many companies have either already adopted (Google, Facebook, Dropbox, Path, Locu, etc.) or are experimenting with (e.g. athenaHealth) the open office layout for their development teams. As we start thinking about the office environment at our startup (ACT.md), we are weighing the options of an open layout compared with a "traditional" Office Space-esque cubicle layout. We're currently only a small team so it's less of an issue, but when we start to grow, this can become a serious topic of conversation.

As with most things, there are pros and cons to both options. Joel Spolsky points out that developers need to have a quiet location where they can bang out their code. Nothing is worse than when you are in the middle of writing some code (or trying to find that deeply buried bug) and someone comes over to you and interrupts your train of thought. Once that train leaves the station, it takes several minutes (at least it does for me) before I'm back in the groove and making the same mental connections I was making earlier. If those interruptions happen multiple times a day, that's quite a few minutes that are wasted just trying to get back on the proverbial horse. Not to mention, it may lead to the introduction of bugs or other errors caused by not being in that "coding nirvana" state of mind.

On the flip side, cubicles, and other such designated quiet rooms, isolate people. As many have said, proximity and interaction yield innovation (take a look at the first couple of chapters about the "adjacent possible" and "liquid networks" in Where Good Ideas Come From for a decent overview). Innovation is exactly what every (software) company strives for. Without innovation, products become dated, poor design decisions are made, and ultimately the product (or even company) may fail. Even more business oriented books such as businessThink, make a point of saying that companies need to "create curiosity." Employees, no matter their level, need to ask questions about why things are being done the way, and be able to provide alternate (possibly better) solutions. Something that is markedly harder in the traditional office that is ripe with grey dividers and neon overhead lights.

I've had the "pleasure" of having worked in both environments. During my graduate career I was predominantly in a 6 person "office" that was setup like an open layout -- there were 6 desks, 2 large windows, and everyone could see everyone (and the sun outside!). At my postdoc it was the exact opposite. I was in a 2 person office with harsh neon lighting, while many of the other employees were working in cubicles in the middle of the building with hardly any access to any natural light (take a look at this regarding the effect of natural light on building occupants). If it weren't for the weekly group meetings (or trips to the coffee machine), I doubt most people would have interacted with each other. If it takes effort to do something, most people won't be bothered to exert the effort. Oh, and worst of all, there were hardly any whiteboards that we could use to sketch out ideas.

So, what is better, a quiet space for each developer where there are little distractions or a more free-form layout where interaction is encouraged? While some may beg to differ, I prefer (and our startup will adopt) the open layout. It allows for easy reconfiguration of desks for flexible team arrangement, encourages collaboration, creates an open feel that makes even a small office look and feel much larger, and reduces office politics by eliminating things like, "Who gets the cube by the window?" Yes, there need to be rooms designated for meetings and for occasional quiet work for those that are in the "zone", but those are the exception, not the rule. Importantly, we also want to foster a collaborative community where everyone feels comfortable asking anyone a question about anything -- there is no such thing as a stupid question.

That being said, I urge anyone thinking of switching to the open layout to make the best investment they can make for each of their devs, a great pair of headphones. A pair of $100 headphones (I personally use, and love, my Audio Technica M50s) are not only cheaper than a single set of cubicle walls, but also create that "quiet" space for each employee without isolating them from their colleagues. As an added benefit, it's cool to tell your potential employees that when they start working at your company they will get a free pair of headphones -- all perks are good perks.

So, be like Peter Gibbons in Office Space and knock down those dividers and start innovating!

Context Managers Are Your Friend

Anytime there is built in support for functionality that reduces the potential for bugs (and number of lines of code), while simultaneously improving code legibility, that functionality should be used with reckless abandon. Similar to function decorators that can be used to elegantly wrap functions with additional functionality, context managers allow often repeated blocks of code to be elegantly written.

Let's take a common database transaction use-case as an example.

try:
    db.insert(some_pbject)
except:
    db.rollback()
else:
    db.commit()

... do some stuff ...

try:
    db.insert(some_other_pbject)
except:
    db.rollback()
else:
    db.commit()

As you can see, there is nothing special about the (pseudo) code above, as it simply is some code to insert some objects into a database. There isn't really anything wrong with above code, but using context managers we can do this in a much more elegant way:

from contextlib import contextmanager

@contextmanager
def transaction(db):
    try:
        yield
    except:
        db.rollback()
    else:
        db.commit()

with transaction(db):
    db.insert(some_pbject)

... do some stuff ...

with transaction(db):
    db.insert(some_other_pbject)

Ok, ok, that's about the same number of lines of code than the original, but now, every time you want to commit something to the database, you never need to write that try-except-else block to make sure that things are either committed or rolled back - you merely need to use the with transaction(db) statement and you're good to go! And since you only wrote the commit logic once, if there is a bug, it only needs to be fixed in one location. Similarly, if after you've written a bunch more code you decide that you want to log every time there is an exception, you simply have to add the logging code in the except block in your transaction function.

Now that you've gotten a taste of the power of context managers, you will probably want to know more about how they actually work. As there is a great explanation here and here, I'm not going to cover that in this post. You may also check out the Python docs on the @contextmanager.

Create A Netflix-like Page for Your Movies

A while back I undertook the project of converting all my DVDs to files that I could store on a computer. Although having all movies in a single directory is very convenient for 'movie night', it's not great for browsing. Admittedly, there is no way to replicate the feeling of actually browsing DVDs sitting on a shelf, I decided to create a simple webpage with all of the movie cover images along the lines of what Netflix and Amazon have. Of course, the result isn't identical (nor did I feel like spending the time to make it absolutely perfect), but here is what it basically looks like (and yes, our movie collection is eclectic).

In case anyone wants to do the same thing with their movies, you can check out the script that I put together that generates this page. One big caveat, it assumes that your movie files are named 'nicely' and are of the form MOVIE_NAME (YEAR).avi (e.g. Battle Los Angeles (2011).avi). The year is optional, however. Once you have the script, you can run it as follows:

python moviepagemaker.py [movie_dir] [output_dir]

In case you don't feel like going over to github to take a look at the code, here it is in full. Before you complain, this isn't exactly the best code that I have ever written (and it was written while watching Drive on a Saturday night), but it should be good enough to get the job done.