“Those who don’t know history are destined to repeat it.”

I know the title is a bit alarming, but that was my first thought after reading @bryce‘s latest post “The Rising Generation“.

Briefly, he mentioned a 25 y/o asking a question on Quora about how was life before everyone had a cell phone and no one talked a lot or texted in public areas. Bryce also say that the new entrepreneurs, like the ones in yesterday’s Y Combinator Demo Day have different expectation, understanding and perceived value of technology than any other that has come before them.

I’m 30. Not too far from the 25 y/o who asked the question (although I was about 15 when cell phone started to penetrate Israel quickly and spread like wild fire). In most companies I worked at I was usually the youngest (or second youngest) in the company for quite some time. Most of my co-workers used to be (or are) between 9-15 years older than me and are still surprised of my knowledge of things from the past.

I’m a bit of a history buff when it comes to computers, science and technology (but also to general history). That’s why I do know what a ZX Spectrum is (and I don’t know how it reached my house when I was about 7 but I had the chance to play with it). I know how VMS systems work (long story form the Army :-) ). When I was six I did play with an Apple IIc my brother got for his Bar Mitzvah and really liked Captain Goodnight and Karateka. I do know how a modem sounds (and can even detect by sound what is the connection speed. Tiiii Taaa TiiTaTa TiiiTa – Yay 28KBPS! ). Heck, I even ran a BBS and was a node on FidoNet when I was 14.

That might be a bit unusual for most people my age that are into computers (and maybe even for older ones) but that’s not too different from people that hear Pink Floyd, Led Zeppelin, Bach and Mozart. While these composers and bands did not exist when I was born or when I was a child the music kept on going. I and a lot of other people both older and younger do know it, hear it and enjoy it. It even inspires some to go on and create new things.

Knowing a bit of the past and how it related to an idea or thought you have can give you a much better appreciation to the things others have done or to the things others advice you to do. After all, specifically in the computers and internet industry, we are all “standing on the shoulders of giants”. It can also give entrepreneurs some much needed perspective on how things were, how things are and how it should be.

So, in addition to Bryce’s hope that “they can rise high enough to meet the emerging opportunities generations before them have made possible”, my 2 cents are to also learn a bit from the past. Take a minute for some history and try to figure out how things were before you push on forward. If they won’t they are destined to repeat it.

Python Implementation of Twitter’s Snowflake Service

A while back Twitter announced the Snowflake service. Snowflake is a unique ID generator that is fast and generate 64bit integer unique ids that are “roughly sortable”. That is, newer ids are bigger than older ones, up to a certain point.
The service was originally written in Scala (which runs on the JVM) and has a Thrift interface, which means you can talk to it from almost any thinkable programming language.

The project was shared on GitHub.

Personally, I don’t really like the JVM. It’s rather bloated in memory terms and can make quite a mess when you need to fine tune it to low memory environments. Also, the Snowflake service code is rather simple and rarely allocate a lot of new objects, which means allocation wise, its rather fixed.

I’ve re-implemented the service in Python using the same Thrift interfaces for both testing as well as being able to run it on low memory environments without the need to fine tune the JVM.

This implementation is rather naive and doesn’t work too much around CPython’s Global Interpeter Lock (GIL) so it yields much less IDs per second than the Scala implementation, however you can compensate for it by running multiple processes.

You can grab the service code from here: https://github.com/erans/pysnowflake

I’ve also written a very simple Python client (it should support connecting to multiple Snowflake services, but the current version disregards this) which I only tested with PySnowflake (the Python server I created). I didn’t test it against the original Scala service.

You can grab the Python client code here: https://github.com/erans/pysnowflakeclient

While I do use some of this code in production, it is far from being fully tested and checked and I would use it as a reference or study it well and load test it before deploying it.