First moment of truth in recruiting

I read this post and liked it:

So what does all this have to do with recruiting? A lot actually. An candidate will begin to form an idea of who you are as an employer over time, your employer brand, through articles, tweets, consumer ads…etc….maybe even a friend who already works there.

When she walks into your office for the first time, all this could come crashing down or on the contrary, it could live up to the hype.

Take a minute and think about all the candidate touchpoints from inital reachout to 1st day of work? What are they? What happens in each? Is your brand being reflected in each one? Do you even have an employer brand?

I also think that, for the recruiting process, all those details are very important, and there is a huge opportunity in those terms for recruitment. Of course, it should be sincere. Just trying to look cool does not make you cool.

Nice try, but no

Nice try, Tony, but no

4 years ago, I did two interviews for positions on Dublin with a difference of a few days.

My first interview was done in a Hotel’s bar, as I was just arriving from my plane. It was for a position on a small startup company. The ones conducting the interviews were technical people (CTO and technical lead). I had a phone interview with them a couple of days before. It was funny to each a sandwich while I was hearing about the role and the company history. I also had to do a couple of programming exercises on paper. They were interesting, and I felt like I was collaborating, not like doing an exam. After the interview (which was not very long), they said that they’ll be making me an offer, and send me the details by email (the CEO, which was in charge of it, was sick that day). So –they ask me — do you want to see the office? So I went to see the office. Then it was — do you want to see the new office? An I saw the new office. Then we went to the pub, when we were joined by a couple of people working on the company. After that, they invite me for dinner, and then we went to another pub. All the “post-interview” process was “off the record” (or so they said, and if I had done something really inappropriate they could have reconsidered, but it was quite relaxed). We talked about a lot of stuff, mostly not related to the job (including things like Star Trek and other nerd subjects) and everyone was quite smart and amicable. It was an enjoyable evening, really.

The second interview was for a more established company. The company was locate in a business park. I had an interview initially with an HR person, then with the manager, a technical interview (which included some coding), and then again with the manager. The technical interview was interesting, but the rest was a quite standard interview process. Nothing fancy or particularly bad, but a lot of the kind of questions that you feel are just not very relevant. At the end, the manager asked me if I’ll be interested in coming back to do another interview. Please note that I was coming from Spain, which mean two 2.5 hours flight, plus getting to the airport, etc, etc.

I guess I don’t really have to tell that I accepted the offer from the first company (Jolt Online) and I was quite happy while I was there (unfortunately, I closed two years ago. You know, startups). Probably that’s quite extreme example of this kind of “Wow factor”, and I think that bringing someone to the pub is not necessary for a job interview (Nice touch, though). But most of the interview processes are, well, quite dull.

For example, I had a lot of problems with phone interview times and reschedules. With scheduling interviews at reasonable times (meaning reasonable times for someone that is already working somewhere else). With interviews that are supposed to take 1 hour and take way more. With questions that requires “aha moments” and doesn’t look remotely similar to what real work looks like. Or with processes that are just one interview after another, with no clear view on when will it end.

They have never been deal breakers for me, to be honest. Most of the time you understand that it’s the way it is. But I think that anyone that tries to be careful about all those details and sincerely works on delighting the interviewees can get a huge advantage.

Again, this needs to be sincere. This needs to reflect who you truly are, or it won’t work. There are already to many places pretending to be cool, looking for ninja rockstar developers. But there are also a lot of nice places that are not showing their best (and real) face.

Being cool is really about believing in it

Being cool is really about believing in it

Some characteristics of the best developers I worked with

I had a conversation last November on the PyConEs, when I was on a conversation stating that I am working with truly brilliant people in DemonWare, and then someone asked me: “Do you have problems agreeing in  what to do? Normally great developers have problems reaching consensus on tech discussions”. My answer something like: “Well, in my experience, truly awesome developers know when to have a strong argument and they usually are ok reaching an agreement in a reasonable time”.

So, I wanted to, as sort of follow-up, summarise what are the characteristics that I’ve seen in the best developers I’ve been lucky to work with. This is not a list I am making on “what’s my ideal developer”, but more a reflexion on the common traits I’ve seen on my experience…

    • Awesome developers are obviously smart, but that’s not typically shown as bursts of brilliance, solving really difficult issues with “aha!” moments. In my experience, genius ideas are rarely required nor expressed (though they surely happen once in a blue moon). Instead, great developers are consistently smart. They present solutions to problems that are reasonable all the time. They find and fix typical bugs with ease. They struggle with very difficult problems, but are able to deal with them. They are able to quickly present something that will make you say “Actually that’s a nice point. Why didn’t I think about this?”. They do not typically present something ingenious and never heard of, but deliver perfectly fine working ideas over and over, one day after another.  Their code is not full of mind blowing concepts, but it is logical, clean and easy to follow most the time (and when’s not, there is a good reason). They are able to remove complexity and simplify stuff, to a degree that it almost look easy (but it’s not)
Normally brilliant people on real life do not come with crazy great ideas out of nowhere

Brilliant people on real life do not come with insanely great ideas out of nowhere

  • They keep a lot of relevant information on their minds. They are able to relate something that is in discussion with something that happened three months ago. They seem to have the extraordinary ability of getting out of the hat some weird knowledge that is applicable to the current problem.
  • While they have a passion for coding, it is not the only thing in their lives. They have hobbies and interests, and they don’t usually go home in the weekends to keep working on open source all day, though they may occasionally do.
  • They love to do things “the right way”, but even more than that, they love to make things work. This means that they will use tools they consider inferior to achieve something if it’s the best/most convenient way. They’ll complain and will try to change it, but deliver will be more important that being right. They have strong opinions about what language/framework/way of doing stuff is best, being that Python, Ruby, Haskell, PostgreSQL, Riak or COBOL, but that won’t stop them knowing when it’s important to just stop arguing and do it.
  • They are humble. They are confident most of the times, but far from arrogant. My impression is that they don’t think that they are as awesome as they truly are. They will want to learn from everyone else, and ask when they have questions. They will also catch new ideas very fast. They are also friendly and nice.
  • Communication is among their best skills. They are very good communications, especially, but not limited, about tech issues. They may be a little social awkward sometimes (though this is not as common as stereotypes portrait), but when they have the motivation to express some idea, they’ll do it very clearly.
  • In some of the truly remarkable cases, they’ll be able to fulfil different roles, when needed. I mean different roles in the most broad sense, basically being able to be what’s needed for that particular moment. Sometimes they’ll have to be leaders, sometimes they’ll be ok being led. They’ll know when a joke is the proper thing to do and when to remain formal. They’ll be the person that helps you with a difficult technical question, or the one that will tell you “you’re tired, just go home and tomorrow it will be another day”
  • And they’ll have a great sense of humour. I know that almost everyone thinks that they have a good sense of humour. That’s not totally true.

Again, this is sort of a personal collection of traits based in my experience and on what I consider the best developers I’ve been honoured to work with. Any ideas?

My concerns with Bitcoin as a currency

Today I retweeted this brilliant tweet:

So, to start the year, I’ve decided to share some of my thought on the bit coin issue, and some of the problems I see. As I am not an economist, I’m not going to go into the deflation / long term scenario. For what I know, that’s very bad, but as that can lead to a deep economic conversation, one I don’t really want to get into, as I lack of the required knowledge, I’m going to concede that. Let’s imagine that bitcoin, from the macroeconomic point of view is absolutely sound. Even in that case, my impression is that it is not very safe from the user point of view. These are “social problems“, more than “tech problems“.

(I am also going to assume that it is cryptographically sound, as I don’t have any reason to think is not)

One of the main problems the system have is that you are entirely on your own to safe your bitcoins / wallets. I guess some people don’t perceive this as a “real problem“, but as someone that can be considered tech-savy, the perspective of a virus, a hardware problem or a missing password that can make disappear my money forever  is really worrying. Even a common problem like transferring money from a dead person (unfortunately, everyone gets to that point) can be impossible if not planed in advance. A Bitcoin wallet (which can be reduced to a private key, a sequence of bits that should be secret) associated to all your Bitcoins can be gone or inaccessible in seconds. Accidental deletion, hardware problems, a malicious virus … Yes, there are countermeasures to this, like backups (if you’re reading this and you don’t have a backup in place, PLEASE DO), but the sad truth is that most of the people out there does make regular backups.

Gone in 10 minutes

Gone in 10 minutes

The single most important quality of any currency is trust. I trust that, if I have money in Dollars or Euros, they are not going to be vaporised for a stupid reason like a failing hard drive. All you need is some horror stories of people loosing all their savings on Bitcoin because there is a virus out there, and non-tech-wavy people will be scared, loosing trust on the currency.

Of course, this scenario can be avoided by an intelligent move. Hey, I don’t have my Euros with me in cash because of these problems. I put them in the bank! Awesome. I can move all my bitcoins to a bank, and interact with my money in the usual way, like credit cards, getting some from time to time from the ATM (in this case, a virtual online ATM). But, in this case, what’s the point of  Bitcoin? If I relay on a bank, I am using the currency exactly as I am using Dollars, Euros or Sterling Pounds (and the banks will charge accordingly). It could have some small benefits, like getting the money out of the bank to transfer it to someone else in an easier fashion than with a traditional currency (especially for small amounts), but I doubt it will be different enough or advantageous enough to justify using Bitcoin instead of regular currencies for most people.

I must say that Casascius coins are gorgeous

I must say that Casascius coins are gorgeous

Another insidious problem I can see is privacy. Bitcoin is pseudonymous, meaning that all the transactions are public, but there is no association between a wallet and someone. I don’t see that as reassuring, as getting to know that wallet A belongs to person B is definitively not a extremely difficult operation. In case Bitcoin was popular, there will be a lot of transaction, and most people would use a couple of wallets at most, for convenience. If you need to send goods to someone, for example, it won’t be that difficult to associate the wallet that pay for the goods with the person receiving the goods. Again, this can be obscured and some people will use complex schemas to hide who they are, but in a typical operation, I’d say that most people wouldn’t care too much about it, just as they don’t care at the moment with a credit card.

Ok, so you manage to know that person B is behind wallet A. Now you can track all the activity of wallet A (because it is public) and use it for whatever you want. A lot of wallets will be simply obvious what they are (known shops), so for example that will be a great way of  “directed marketing”. For example, Amazon could know that you have a contract with Vodaphone that looks like a mobile contract. Now you’ll get “directed information” of all the million offers that Amazon has about mobile products. Great, now you have more spam in your inbox. The data mining implications are incredible.

Of course, any purchase that you don’t necessarily what to share with the world can be exposed. And it’s there, publicly available, forever. If you move to a different wallet and move your bitcoins around, hey, that’s registered, so you can’t hide unless you transfer all the money out of the system, and then exchange it back, to a new wallet(s) that, this time, hopefully won’t be discovered. Plus all the inconveniences of doing so, of course.

Of course, there are ways of dealing with it. Using a lot of wallets, circulating the money among them (and hoping this is safe enough, as there could be advanced methods of detection for common uses). Being aware of what information is being shared. But, seriously, are we expecting everyone that just wants to use a currency to make common operations to add all that overhead and knowledge? I think that’s asking too much.

As the objective of a currency is to be used as means of payment, to be exchanged often, I think that these problems are in the way of considering Bitcoin as a currency replacement that can get some real traction in the world. The potential risks are quite big, and not well understood for a lot of people at the moment. Of course, these problems are at the moment less important that the fact that Bitcoin is used at the moment as an investment / speculation product, making the exchange rate so volatile that using Bitcoin as a currency is currently unviable. But assuming that Bitcoin can leave this state behind, I still see these issues in the way of becoming a viable currency.

I am not an expert in this subject, so if I am mistaken at some point, let me know. Comments welcome :-P

“Blog is dead” and the change in information consumption

I read this post about the “Blogs are dead (we’ll not really/but they are)” thing…

Sometime in the past few years, the blog died. In 2014, people will finally notice. Sure, blogs still exist, many of them are excellent, and they will go on existing and being excellent for many years to come. But the function of the blog, the nebulous informational task we all agreed the blog was fulfilling for the past decade, is increasingly being handled by a growing number of disparate media forms that are blog-like but also decidedly not blogs.

Instead of blogging, people are posting to Tumblr, tweeting, pinning things to their board, posting to Reddit, Snapchatting, updating Facebook statuses, Instagramming, and publishing on Medium. In 1997, wired teens created online diaries, and in 2004 the blog was king. Today, teens are about as likely to start a blog (over Instagramming or Snapchatting) as they are to buy a music CD. Blogs are for 40-somethings with kids.

It is curious, because I’ve never used blogs (or RSS, for what matters) as a way of “reading the news“. I think all the alternative ways of communication (Twitter, Reddit, Facebook, etc) are very good for talking about the now.

Twitter is great, but a timeline is really focused in what’s going on today. That’s great, but I can easily miss articles, because, for any low number of people you follow, you simply need a lot of time to be “updated”. And is hugely biased over people tweeting a lot. Again, is a great tool. It’s fast and it’s great for provoking conversation. And incredibly funny. I use it everyday, but it is not a good tool for deep thoughts, or for learning.

Facebook? It’s oriented to friends and family. It’s great to keep in touch, to share photos, but it is oriented on the personal side.

Reddit or Hacker News? Great curating content and discovery, but I lack control over what is displayed there. I can’t follow someone that I (not a community) find interesting.

Those are all great tools for discovering. For sharing content. But I still value a lot my personal RSS selection, when I have been selecting channels for years. I think is (so far) the best way of following articles that aspire to talk about something more permanent than this week sensation. I have an automatic reading list of “things I don’t want to miss“.

It is probably true that blogs (and RSS syndication) are not “hot” and that now teens start using other tools, and public spotlight is in other technologies. I’m fine with that, I understand that the rest of the tools are fantastic and fill different needs. But I still think that blogs are the best posible tool at the moment for the kind of deep exposition of ideas that I still like, including the strongly personalised selection of my sources. 

I’ve also argued that they are good for creating a community of readers and the comments on blogs typically add value to the original article. I know that I am on a minority here, but I think that the comments on blogs belong with the article, not on a external service.

I guess that I think that just because something is not “the next hot thing in tech”, we shouldn’t be treat it like it is dead.

Agile important bits

There is a lot of Agile talking and I think it has reached a point where it is, if not standard, at least a common way of doing software. But, even if there is a lot talking about Agile methodologies, and companies telling that the are doing Agile, are they really doing it? I’m not so sure. When relating to Agile, I always come back to the source, which is the Agile manifesto. I really like its simplicity. Let me copy it here

We are uncovering better ways of developing software by doing it and helping others do it.Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

The first time I read it, I must say I wasn’t impressed. Yeah, sure… Great values, dude. But after spending more time, I come to see that as a really good set of values that, in mi opinion, work really well for software development. In practice, I think there are some of those values that are somehow “forgotten” over the day to day operations. I’ve been thinking about what are the ideas that, again in my opinion, are the ones I consider the best way of implementing those values. Consider them my personal “Agile highlighted parts”

Never-ending learn

Well, maybe not in 21 days, but that's the spirit...

Well, maybe not in 21 days, but that’s the spirit…

In my opinion, the single word that is capital in software development is “LEARN”. The same thing applies to Agile. Agile is about being constantly learning. Change (which is also a very important word) is just a consequence of this learning process, the outcome. Because we learn how to do things better, we change our way to work. Because we truly understand the problems of the customer, we can develop what the customer needs (which may not be what the customer has in mind in the first place). This learning process should not view restricted to the developers. It is also applicable to the rest of the people involved, including the customer. If the customer is not willing to learn in the process and refuses to accept feedback, then the process is much more difficult. That could be the single most important risk in Agile, as that will make the collaboration difficult and adds a lot of friction. Even in products aimed for mass consumption, this process can be present one way or another. For example, recently there has been a lot of discussion about iOS 7, and how consumers have learned how to use a touch screen and elements that were present on previous versions to help usage are no longer needed as average consumer know now what it is about. Refusal of learning can also be a problem in Agile. There are people that do not like the constant effort and the change in mindset that it implies.

Team centric view.

The team in Agile is king. When I say “a team” I am not referring to a list of people put together on the same building working on the same product. A team is much more than that. It is people working towards the same goal, but also effectively working close together and helping each other. Just as a soccer team needs players with different abilities, so does software teams. The people working on the same product will have to be, with high probability, multidisciplinary. That has not been usually the case on software companies, where you have the testing department in one side, the design department in another, and the R&D department detached from the rest of teams, communicating through big documents and formal meetings. That creates the need of strict interfaces, and negotiation between the parts to agree how to exchange information.

Some development talk is full of wibbly-wobbly... timey-wimey... stuff

Most development talk is full of wibbly-wobbly… timey-wimey… stuff

Instead, a team will try to learn and improve how to work and exchange information. Asking constantly, “how can I make the work of X easier?”. To be able to do that, you have to know and understand what are the problems that X will face, and the only way of knowing that is to work closely with them. That is more difficult that it looks. We developers are usually very “machine centric”, tending to try to fix everything with a script, or adding a new feature that complicates the system. It is not simple to learn about how and why other “non-techies” people (sales, designers, etc) are doing the things they are doing the way they are doing them. We prefer to keep running out scripts and our UNIX commands. We talk our techie talk of threads, classes, recursion and functional programming. But the learning of the”domain knowledge” is really what makes the difference between a good team and a great one. And that knowledge can only be achieved constantly learning from one another. Both the developers understanding the real problems of the customer, but also, in some cases, the customer understanding how the team works and what is possible on a certain amount of time. Creating a good team is very important and challenging. But also accepting that it is formed by different individuals, with different capabilities, strong and weak points, is probably one of the most difficult parts in any organisation. I always think that the most challenging task is to deal with people, which, no matter how “good cultural fit” is achieved, will be individuals different from any other one. Acknowledging it and being able to make everyone on the same page is difficult, but capital for highly successful teams.

Interact in a constant, but structured fashion

While interaction is really important, some balance need to be achieved with interrupting ongoing work. Agile is not about “hey, we can change all at any point”. I know, the name is a little misleading. It about knowing that you are going to change stuff, and deciding what at certain points. One of the details that I found out more efficient in that are sprints and stand up meetings. Both are tools to structure the conversation while providing constant feedback.

Stand up meetings

Doing stand up meetings in the best way is more difficult that it looks. It needs to be kept very simple, fast, and with as little noise as possible. Basically each participant need to say, keeping it simple: What did I do yesterday? What I am going to do today? Are there any problems in the way? And listen carefully to the rest of the people to be aware of the work being done on the team, and to see if you can give support to anyone (during the meeting or later, if more than a few minutes are necessary). Being actually standing up and in a different room helps, as you tend to keep things up to the point, eliminating distractions. One interesting part is not the meeting itself, but the time that previously is used to structure your mind into what you’re going to say. That helps a lot in planning and in keeping you focused, as everyday you’ll have to explain what have you done. Focus is paramount in software development. Another important thing is to keep that as a strong habit. There are always moments when it looks like no one is saying something new, and it feel as redundant. If the meeting is all about the same things for a long time, then is clearly a problem (your tasks are not as small as they should). But the benefits in terms of constant feedback and focus are not achieved until some time. I also think that managers/product owners should be present on the meeting, and probably participate actively. Remember, it is part of the learning process and it’s a very good opportunity to show how the team is working and what are the management tasks. A proper star up meeting reduces the need to interrupt the work of the day with typical “what are you working on?” questions, and detects very fast problems and blockers. It structures communication to channel it.


Well, it does not scream "sustainable pace", but still...

Well, it does not scream “sustainable pace”, but still…

While the standup meetings structures the communication between the team, Sprints structure the communication between the team and “the external world”. The idea of the sprint is to produce something that can be shown, and ideally works, even if it’s small and not complete. That gives feedback and then the goal for the next sprint can be decided. While having a simple objective for a sprint can be good, I don’t think is necessary. A simple grouping of tasks with no particular meaning together may well be the objective. I don’t like the word “Sprint”. I think is not fortunate, as it will not give the idea a long term race, but a strong burst in effort. Software projects are long, and teams should find a comfortable development speed, or the quality of the result will suffer. I’d prefer something like “stage” (using bicycle race metaphor), because the main idea is that they should have a sustainable pace. The sprint objective is not a contract signed by blood of your firstborn. It is a reasonable objective that the team honestly thinks can be achieved.  Let me get back to  couple of ideas there. What can be achieved on a sprint is something that can only be decided by the team. They are the only ones with the knowledge of what is possible and what is not. Estimations are that, assumptions. A typical problem in software development is unrealistic deadlines by management, that will be used as weapons against the developers, and the natural response from developers is to produce “safe” estimations, much bigger that they should, so they can be protected. This is not a good thing, and it is a reflection of distrust. The way to overcome that is to treat estimations as that, and try to improve them, without punishment for  mistakes. Let me rephrase it: Estimations will be wrong. Often. The objective should be to improve them, and to not be wrong by a huge margin. Dividing work in small tasks will help, but that’s an inherent problem.

Make it work

I consider myself a pragmatist. At the end, everything should be done not for the sake of it, but because it helps towards an end. As with a lot of good ideas, people has fallen too often in “Agile cargo cult”, just following processes without thinking why, or without analysing who are the people on charge of doing it. Again, analysing and learning what’s going on is important, to enable the best concepts applied to a particular team.

Wrath of Khan and failure

SPOILERS AHEAD: I will comment a couple of things about Star Trek Into Darkness, so, if you haven’t seen it, and don’t want to know some plot details, you may not want to read this. Just watch the movie and come back, I’ll be here waiting ;-)

Oh, there were plenty about Star Trek II: The Wrath of Khan (and a little of Space Seed), but being a movie from 30 years ago, I doubt they can be considering spoilers.

so 80s

so 80s

Star Trek II: The Wrath of Khan is, no doubt, a fan favourite. It has a great combination of a powerful story, iconic moments, connection with the TV series roots, and that extra Shakespearean touch that keeps everything together. But I want to focus in a particular aspect, one of the stronger themes than runs through the movie.

And that is to grow old. The fact that in particular (here Admiral) Kirk is not young any more is presented through the whole movie. He becomes an unexpected father. He needs to use reading glasses.

But, over all, he has to face the consequences of his previous actions. Especially failed ones.

Kirk starts the movie as that invincible fleet captain that has faced all possible risks and dangers (“Risk… risk is our business! That’s what this starship is all about…”) and has been triumphant on each and every occasion. Getting as far as cheating on Kobayashi Maru (a training mission designed to present failure to cadets) because, well, he simply don’t believe in non-win situations. During The Original Series, basically every single opportunity to take the bold approach to fix a problem is taken, with amazing success.

But, just as the movie starts, we discover that there has been something unexpected during this time. Khan and his crew were left at the end of Space Seed on a harsh planet, basically so they can get a challenge worth of genetically engineered supermen. At the same time, the rest of the universe won’t have to worry about their conquest aspirations. But there was a mistake. That challenge has mutated into a nightmare, even for men like Khan. Enterprise crew (and Starfleet, for what matters) just forgot about the issue, moving to the next exciting adventure the following week. But the truth is that it has been a disaster. Kirk failed without noticing it.

Ok, so Kirk goes to fix the problem and saving the day. He is even able to use some tricks and cheat Khan a couple of times, giving the impression that he is defeated, but managing to save the day at the eleventh hour. As usual, on the other hand.

I have been -- and always will be                             -- your friend... Live. Long. And.                             Prosper.

I have been — and always will be — your friend…

But, then the final moment arrives. Kirk truly faces a real non-win situation. Spock sacrifices himself to save the full ship, in a move that Kirk couldn’t anticipate. And here is the genius on the script. Spock dies. He truly does! There are no tricks, no last minute saving, no technobabble that reverts death. Spock is dead, he is no more, has ceased to be, expired and gone to meet his maker. For real. For the next 2 years (until the next Star Trek movie) Spock was totally dead. And this is extremely powerful from the point of view of drama. It is not common at all for a big budget movie, and it was a big bold move for Star Trek.

Because Kirk faces the moment he has feared for his whole life. The moment of failure. Yes, sure, it’s not a total failure, the ship has been saved. But he has been able during his whole career to end every adventure making a comment with Spock and McCoy at the bridge without a scratch. Death is for Red Shirts and aliens, not for him or close friends.

And that’s the main point of it. Failure really sucks. It hurts a lot. That’s why no one wants to fail. No matter how many times we say “failing is the first step in succeeding“, or how we repeat the mantra of “fail often, fail early“. Fail small sucks less, but still sucks. But it is unavoidable. Kirk was able to deal with it because, well, he is a fiction character. But in our lives, the truth is that we fail constantly. And the way of dealing with it is not denying that fact, or cheat our own Kobayashi Maru, living in a “I don’t believe in non-win situations” bubble. The way of doing it is trying to make each failure the seed of a new success, to learn from our errors, to keep us motivated to better ourselves. Even if it hurts every single time.

That’s one of the main reasons why Star Trek Into Darkness didn’t really move me inside. While there are some superficial similarities/homages, the character of John Harrison does not have any particular meaning for Kirk or Spock. He does not represent any personal failure for them or haunt from the past. And the end is yet-another-protagonist-almost-die-to-be-saved-by-a-gimmick, lacking the deep meaning of The Wrath of Khan.

Python Wizard

Elton+John+Pinball+WizardEver since I was a young boy,
I typed on keyboards
From bash commands to Java
I must have code them all
but I ain’t seen nothing like him
In any Hackathon
That nice, nerd and shy kid
Sure codes great Python!

He stands like a statue,
Becomes part of the machine.
Lots of comprehensions
always writing clean
right code indentation
dicts used the most
That nice, nerd and shy kid
Sure codes great Python!

He’s a coding wizard
There has to be a twist.
A coding wizard,
S’got such a supple wrist.

How do you think he does it?
I don’t know!
What makes him so good?

ain’t got no distractions
semicolons or brackets
Nice packaged modules
produced everyday
Functional programing
when it fits the best
That nice, nerd and shy kid
Sure codes great Python!

I thought I was
The system admin king.
But I just handed
My hacker crown to him.

Even on my favorite system
He can beat my best.
Opens the text editor
And he just does the rest
He’s got crazy vi fingers
no IDE at all
That nice, nerd and shy kid
Sure codes great Python!

Requests per second. A server load reference

As there seems to be a lot of misconceptions about what Big Data, there are also not really a good baseline to know “how much is high load”, specially from the point of view of people with not that much experience in servers. If you have some experience dealing with servers, you will probably know all this. So, just for the sake of convenience, I am going to do some back-of-the-envelope calculations to try to set a few numbers and explain how to calculate how many requests per second a server can hold.

We are going to use RPS (requests per second) as the metric. This measures the throughput, which is typically the most important measure. There are other parameters that can be interesting (latency) depending on the application, but in a typical application, throughput is the main metric.

Those requests can be pure HTTP requests (getting a URL from a web server), or can be other kind of server requests. Database queries, fetch the mail, bank transactions, etc. The principles are the same.

I/O bound or CPU bound

There are two type of requests, I/O bound and CPU bound.

Everything you do or learn will be imprinted on this disc. This will limit the number of requests you can keep open

Everything you do or learn will be imprinted on this disc. This will limit the number of requests you can keep open

Typically, requests are limited by I/O. That means that it fetches the info from a database, or reads a file, or gets the info from network. CPU is doing nothing most of the time. Due the wonders of the Operative System, you can create multiple workers that will keep doing requests while other workers wait. In this case, the server is limited by the amount or workers it has running. That means RAM memory. More memory, more workers.[1]

In memory bound systems, getting the number of RPS is making the following calculation:

RPS = (memory / worker memory)  * (1 / Task time)

For example:

Total RAM Worker memory Task time RPS
16Gb 40Mb 100ms 4,000
16Gb 40Mb 50ms 8,000
16Gb 400Mb 100ms 400
16Gb 400Mb 50ms 800
Crunch those requests!

Crunch those requests!

Some other requests, like image processing or doing calculations, are CPU bound. That means that the limiting factor in the amount of CPU power the machine has. Having a lot of workers does not help, as only one can work at the same time per core.  Two cores means two workers can run at the same time. The limit here is CPU power and number of cores. More cores, more workers.

In CPU bound systems, getting the number of RPS is making the following calculation:

RPS = Num. cores * (1 /Task time)

For example:

Num. cores Task time RPS
4 10ms 400
4 100ms 40
16 10ms 1,600
16 100ms 160

Of course, those are ideal numbers. Servers need time and memory to run other processes, not only workers.  And, of course, they can be errors. But there are good numbers to check and keep in mind.

Calculating the load of a system

If we don’t know the load a system is going to face, we’ll have to make an educated guess. The most important number is the sustained peak. That means the maximum number of requests that are going to arrive at any second during a sustained period of time. That’s the breaking point of the server.

That can depend a lot on the service, but typically services follow a pattern with ups and downs. During the night the load decreases, and during day it increases up to certain point, stays there, and then goes down again. Assuming that we don’t have any idea how the load is going to be, just assume that all the expected requests in a day are going to be done in 4 hours. Unless load is very very spiky, it’ll probably be a safe bet.

For example,1 million requests means 70 RPS. 100 million requests mean 7,000 RPS. A regular server can process a lot of requests during a whole day.

That’s assuming that the load can be calculated in number of requests. Other times is better to try to estimate the number of requests a user will generate, and then move from the number of users. E.g. A user will make 5 requests in a session. With 1 Million users in 4 hours, that means around 350 RPS at peak. If the same users make 50 requests per sessions, that’s 3,500 RPS at peak.



A typical load for a server

This two numbers should only be user per reference, but, in my experience, I found out that are numbers good to have on my head. This is just to get an idea, and everything should be measured. But just as rule of thumb.

1,000 RPS is not difficult to achieve on a normal server for a regular service.

2,000 RPS is a decent amount of load for a normal server for a regular service.

More than 2K either need big servers, lightweight services, not-obvious optimisations, etc (or it means you’re awesome!). Less than 1K seems low for a server doing typical work these days.

Again, this are just my personal “measures”, that depends on a lot of factors, but are useful to keep in mind when checking if there’s a problem or the servers can be pushed a little more.

1 – Small detail, async systems work a little different than this, so they can be faster in a purely I/O bound system. That’s one of the reasons why new async frameworks seems to get traction. They are really good for I/O bound operations. And most of the operations these days are I/O bound.

Make beautiful Python code (talk at PyCon IE ’13)

Another year, another amazing PyCon. I guess I repeat myself, but I keep being impressed about the quality of the talks and the friendly, vibrant atmosphere. It is always a pleasure to spend some time with people interested in code and technology… There was also an increase in the number attendees, and quite a lot students. I said that on Twitter, but Python Ireland, you guys rock.

Of all the talks I attend to, I’d like to comment two that were especially interesting. The first was one of the keynotes, PRISM-as-a-Service: Not Subject to American Law, by Lynn Root. All this think is pretty scary when you think about it. Definitively worth a read. The other one was The Clean Architecture in Python, by Brandon Rhodes, about ways of designing code and make them data-centric.

I also gave a talk, and other than a problem with the project that made me rush a little, I think it went good. Just in case you’re interested, here are the slides. Here is also the PDF version with notes.

Oh, and another thing. there are launching the pyLadies Dublin group this wednesday 15th October, so if you’re interested, show up.


UPDATE: Added slides for Brandon Rhodes talk

Rockstar programmer and Rockstar teams

There has been some discussion about the so-called Rockstar Programmer. You know, that awesome engineer (also called 10x engineer) that can produce what 10 other, average engineers can.

This post by Scott Hanselman[1] fueled some discussion on Hacker News. What has been overseen about the original post is that he advocates about 10x teams.

That resonates a lot, because I think that we should agree that, while there is people with potential to be ninja programmers, that’s not something that can be achieved without the proper care on the environment.

A good team is one that reinforces the good points of their members while hiding away (or at least mitigating) their weaknesses. It makes not sense to talk about a guru engineer that can come in a parachute in a project (any project), replacing an average Joe on a 10 members team, and simply double their output! And, after a while, she’s probably take her umbrella and fly away to the sunset (to a better payed work, one can only imagine)

If we use a low-level warp bubble around that code, we could reduce its gravitational constant, making it lighter to push!

If I use a low-level warp bubble around that code, I could reduce its gravitational constant, making it lighter to push for the rest of the team!!

Real life just doesn’t work like that. Everything is a little more complex. A toxic team or project can be beyond salvation. A regular programmer can achieve a lot just by giving some motivation and direction. A great engineer can be disastrous working in a particular area.

Do you want to see someone transformed from an x programmer to a Nx programmer? Just take a look on the same engineer the first day in a new job and then again after a whole year. The first day she’ll have to ask lots of questions. After a while, she’ll be committing patches, and, later, she’ll reach a cruise speed much much faster than the first couple of days. Or… maybe she is an x programmer, and during the first days she was a x/N programmer. Mmmm….

I also like how Haselman he approaches the subject talking about “titles” and “loud programmers”.  The Rockstar Engineer idea is more a recruitment-marketing issue. It is used to hype possible hires: “Hey, we are a Rockstar company and we are looking for Ninja developers. Maybe you’re an Stellar Programmer”. It has been so used that it doesn’t mean anything anymore. I’ve even seen a job offer asking for a Python Admiral[2]. It is currently more a way of signalling spam than any other thing. [3]

But there is still the myth of “the 10x programmer”, not as a way of describing that there are obviously people more productive than other (and who can reach high notes), but taking for granted that is mostly a characteristic of the programmer itself, while the truly stelar results are achieved mostly when the environment is the  adequate. A lot of the great results in a team is not a magical increase in productivity by some gifted individual, but more the constant improvements in the good directions or good vision to focus in what’s truly relevant. A single good developer can move quite fast in the good direction not because she’s wildly more productive but because she has a clear view and focus.

Because an average programmer can be at least a 5x programmer when the proper details fall in place, and a great developer can be a 1/5x programmer in the wrong place.

1 – He’s also referencing this very interesting article by Shanley.

2 – Mmm, I have two offers, in one they got good salary, decent benefits. But the other one is offering a fleet command. Tempting.

3 – Not to talk about salaries, of course. We talk and talk about 10x programmers, I’d love to see some place offering 5 times an average salary.