Visual Programming and Mental Constructs


I saw yesterday live the Apple keynote on the WWDC. I am far from an Apple developer, but I use OS X and iOS everyday, and I’m interested on new stuff. There was a full section devoted to developers, which is great (well, it’s supposed to be a developer’s conference, after all), and, arguably, the most interesting stuff on that part (for a developer’s perspective) was the release of a new programming language, Swift.

It was announced with an (irrelevant) comparison with Python in terms of speed (I actually have plans to write a post about “why Python is not really slow“, but I digress), as well as a lot of other details that (IMO) are completely pointless in terms of what makes a good or bad programming language.

Most pointless benchmark ever

Most pointless benchmark ever

I am generally skeptic about the announcement of new languages. Almost as much as new web frameworks. Sure, it adds a new flavour, but I’m not that sure about real advancement in tech. Creating a new language, full with proper “clean and beautiful” syntax is not really that difficult. The difficult part is to create a vibrant community behind it, one that loves the language and works to expand it, to push the boundaries of current tech, to make amazing applications and tools, to convince other developers to use it and to carry on the torch. The target of a language are developers. “End customers” couldn’t care less about how the guts of their products are done. “Ruby sharp? Whatever, I just need that it help us increase our sales

Interestingly enough, languages get a lot of character from their communities, as they embed their values on the relevant modules and tools. A great example of that is “The Zen Of Python“. There’s nothing there about whitespaces, list comprehensions or classes, but it reflects a lot of the ideas that are common on the Python world, values of the Python Community. Using a language is not just writing code, but also interacting with other developers, directly or even just reading the documents and using the APIs.

As mandatory as honor

As mandatory as honor

Obviously, Apple is a very special situation, as it can force developers to use whatever they like for their platform. Hey, they managed to create an Objective-C ecosystem out from nowhere, which is impressive. For what is worth, they can even tailor a language for their platform, and not to worry about anything else. iOS is a platform big enough for devs to have to learn the language and official IDE and use it. And I am pretty sure that in this case it will be an improvement over the previous environment.

But the one part that I am most skeptic about is the “visual programming” stuff. One of the “wow” announcements was the possibility of creating “playgrounds”, to show interactively the results of the code. That means that, for example, a loaded image will be available, or that a graph can be displayed showing the results of a function. And that’s the part that I’m not really that sure that is interesting or relevant at all.

Does it look cool? Absolutely. May it be interesting once in a while? Sure. But I think that’s the kind of process that, in day to day operation, is not really that useful in most kinds of programming.

Programming, more than anything else, is creating a mental image of code. Code can be a very complex thing. Especially on a big application. But normally we don’t need to keep the whole code in our mind. We only have to keep certain parts of it, allowing to focus in a problem at a time. That’s the main principle behind modules, classes and other abstractions. I can use OS calls to open a file, to draw some pixels on the screen, or to make a call to a remote server. All of that without having to worry about file systems, graphic drivers or network protocols. And I can also use higher level modules to search on files, create 3d models or make HTTPS calls.

And the amazing power of programming is that you are coding on the shoulders of giants. And on the shoulders of regular people. And on the shoulders of your co-workers. And on your own shoulders. That’s a lot of shoulders combined.

But a lot of that process deals with the unavoidable complexity of the interaction. And being able to move from an abstracted view to a more specific one, to look inside and outside the black box, is crucial. It may not be evident, but the mental process of programming deals a lot with that sudden change in perspective. This is one of the reasons of multiparadigm being a useful thing. Because you can move between different abstractions and levels, using the proper one on each case (especially for leaky ones).

And there are lots of those processes that are not easily represented with graphs or images. They are constructs on your mind: loops, flexible structures, intuitions on the weak points of an algorithm, variables changing values, corner cases… Showing all intermediate results may be detrimental to that quick change in perspective. Too much information.

There has been experiments with visual programming, trying to represent code as visual blocks in one way or another, since a long time ago (at least 25 years). They are useful in certain areas, but they are far from a general solution. There are also interactive notepads to allow easy display of graphs and help with the interactivity. iPython Notebook is an excellent example (and a very similar idea to the playground). But, again, I feel that those are specialised tools, not something that is that useful in most programming contexts.

I’m just skeptic. All of this doesn’t necessarily means that Swift is bad, or that those tools are wrong. Maybe the new X-Code will have a lot of amazing tools that will help create fantastic applications (I still don’t like IDEs, though). There are already people checking the docs and giving a try to the new language.  But I think that it has to show up how good or bad it is for itself, and by the developers that decide to use it. So far, it is just an announcement. I just feel that most that was said on the keynote was not relevant to determine whether it’s a good working environment or not, but was just a gimmick. Yes, obviously these kind of announcements are publicity stunts, but in this particular case it looks especially so.

Looks cool, but is not particularly relevant to how the mental process of programming works or what makes a language good.

Hmph. Visual blocks. Heh. Excitement. Heh. A Developer craves not these things.

Hmph. Visual blocks. Heh. Excitement. Heh. A Developer craves not these things.

The amazing forgiveness of software


One of the things I like most about developing software is the fact that you can recover from most mistakes with very few long term impact.

Bugs are unavoidable, and most of the people involved on programming deeply understands that is something we all live with.  So,  there’s no hard feelings, once you find a bug, you fix it and immediately move on. Not only no one thinks that you’re a bad developer because you write bugs, but typically the impact of a bug is not that problematic.

Yes, there are some bugs that are just terrible. And there’s always the risk of losing data or do some catastrophic operation on production. But those are comparatively rare, and with the proper practices, the risk and damage can be reduced. Most on the day to day operation involves mistakes that have a much limited effect. Software development is more about being bold and move fast fixing your mess, than it is to play safe (within limits, of course).

This can affect production data! Show warning sign!

This can affect production data! Display warning sign!

Because the greatness of software is that you can break it on purpose and watch it explode, and then fix that problem. In a controlled environment. Without having to worry about permanent effects or high costs. And a good test is the one that ambushes the code and try to viciously stab it with a poisonous dagger. The one that can hurt. So you know that your system is strong enough against that attack. And then iterate. Quickly. It’s like having a permanent second chance to try again.

Not every aspect of live is that forgiving. I guess that a lot of doctors would love to be able to do the same.

Do you mind if we start the trial again and the jury forgets everything, your honor?

Do you mind if the jury forgets everything and we start the trial again, your honor?

First job in a startup considered harmful


Well, at least is not ideal from my point of view…

At the moment there seems to be a lot of hype about startups. And why not? They are the places where the cool stuff happen. Filled with purpose, excitement, high stakes, fantastic teams, growth opportunities and the rare chance of maybe becoming a multibillionaire at a young age. I’ve worked in big and small companies (including startups), and I definitively prefer to work on smaller ones. You’re impact is bigger, the team works closer, way less corporate BS, etc…

But, while I think that working on a startup is exciting, and a great career move, I don’t think is a great first job. When starting a career on tech, a better place to start is an established company, at least for one or two years. Why? Simple because if you want to be a rebel, you have to know what to rebel against.

 

How else are are you going to know how to use a rubber chicken with a pulley in the middle?

How else are are you going to know how to use a rubber chicken with a pulley in the middle?

Established companies have a lot of bureaucracy, process, rules and middle managers. They are boring, that’s for sure. But there is one thing that they have than an startup has not. A proved method of doing stuff. A running operation. It can be dysfunctional, inefficient, stupid o just crazy, but they are being able to earn money with it. While the stupid processes seems to be there with the sole purpose of annoying workers, they are there for a reason. Maybe not the best one, but it is great learning experience to navigate through them.
Also, within all that craziness there are some steps that make sense. Something that has been refined after years of experience and a lot of work and it’s there for a reason. On every department there is some one that is actually brilliant and knows everything around. These people are excellent mentors. Not only on tech, but also on “the business world” and career-wise. We tend to think when we start that everything older than 5 years is obsolete. Talking to someone that is 25 or 30 years older gives a lot of perspective. They had gone through a lot, and their experiences are very valuable.
And, obviously, because that will make feel some of the pain on what a big company is. When I was a freshman out of college, I’ll just accept things that will drive me crazy today. Once you think about that you can effectively remove the pain points knowing why, and with actual experience on them. It’s the equivalent of measuring some software to discover the bottlenecks and then concentrate on them, instead of just making wild assumptions. Without real work on the field, it’s all premature optimisation.

Moving in the opposite direction, from small company to a big one can be absolutely horrendous. It is extremely difficult to perceive an increase in bureaucracy and processes as an improvement. Even in the few cases when it is. Having facing it in advance helps accepting it.
To be able to recognise how big companies operate and what are the real reasons why you don’t want to work there is a great experience when working in a startup. Of course, a small company have its challenges. But if your aim is to improve what big companies are doing (and that should be the goal of anyone working on a startup), how do you expect to do it without knowing it beforehand?

Online community biases


There are a lot of discussion online about a huge number of different topics. That’s fantastic news, I’d love to had a learning tool that powerful when I was in school. To share some of my interests, and have other people to talk about “cool stuff” and learn from them. Online communities have speed up personal and technological growth intensely, allowing people from around the world to share knowledge and to feel close. But, on the other hand, these kind of communities get naturally and subtlety biased. While this is normal, and probably unavoidable, anyone participating should be aware that the so-called “real world”, or even the community as a whole, is not a perfect extrapolation of it.

Totally scientific data, properly labelled

Totally scientific data, properly labelled

It is quite spread the idea of the “1% rule” over the Internet. A1% of the community will be the most active, driving the discussion, generating the subjects that will be provoke discussion, etc. ~10% of the people will collaborate, comment, retweet, add their impressions… And the rest will just consume it and learn from it. This distribution seems to be present in any community big enough. It makes sense, there’s only a very limited number of people that can be creators (I’ll call them leaders), there is a bigger group of people willing to spend time and effort collaborating (I’ll call them participants), and then the rest that are interested, but not willing to spend a lot of time (I’ll call them consumers).

But, here is the interesting part. The 1% is not a perfect representation of the whole.In fact, it can (and normally will) be pretty biased. That’s something quite natural. After all, leaders are different from the majority of the community, or they won’t be leaders. But other than their tendency to stand up, to speak up, they can have a lot of significative biases.

For example, a clear example of that are so-called “hardcore gamers”. While the statistical profile of a “gamer” (someone that enjoys video games from time to time) is very very broad, the “hardcore gamer community” is the most vocal. The discussion about games is centred into big AAA games (GTAs, CoDs…), and, to a lesser degree, to big casual games (FarmVille, Candy Crush…) and interesting indie experiments (Gone Home). The main idea someone will get is that “gamers” are mainly young, male and like to play for hours, when that’s not a good statistical representation of the community. Keep in mind that 45% of players are female, and a third have over 35 years. There are discussions about “what is a game and what is not” (meaning, “I’ve decided that you’re not playing games, OK?”), entire genres that are often ignored by everyone, and a general perception on what “real gaming” should be. A very good indication of that are the recent rants against microtransactions[1]. Sure, they feel wrong for a lot of people that is used to get a whole game for a price, and play it all. But I’m afraid that a lot of people right now spend a small amount of time playing and they just don’t feel like committing to a game, and Free to Play model present advantages to that kind of player.

I really don't see the point denying that this a game. It may be a BAD game, though.

I really don’t see the point denying that this a game, even if is a BAD one.

I am not arguing that biases are good or bad. Some will be good, because will bring focus to a chaotic community, some will be bad because will represent a minority that think they are the only “real” members of the community. Probably each of us will have a different opinion about which ones are positive or negative. What I am trying to say is that they are unavoidable.

Let me focus in development, as is the one community that I am most interested in. In the general online developers community, there are some biases that I think are quite strong, and probably not perceived from leaders and participants (after all, it mostly resembles them).

The community is young. This is clearer in the participants group than in the leaders one, after all, wisdom and insight are a good qualities for being a leader, and those comes mostly with age. With youth comes new views to change the world, but also naïveté and inexperience.

It is driven mostly by Americans (and foreigners living in the US, to a certain degree), not only by the strong position US has in tech, but also because the online lingua franca is currently English. In particular, it is very centred into Silicon Valley because is where the most discussion-driven companies of the world are based. Both well established companies and start-ups.

The most talked technologies are web tech (both front-end and back-end), with mobile apps in a second place. There is comparatively few discussion about desktop applications (which are the basis of everyday work), and even less on areas like embedded systems or commerce backends (including banks).

All those biases (there are more, of course, but just to limit to these three) work together in ways that some times are curious. Like assuming that most people are able to earn a Silicon-Valley-level salary, or that access to a computer (or even worse, Internet) in your teens is granted. Also, grammar errors are unforgivable mistakes a lot of times that should be pointed (and forget about things like transcript conference talks). Products are only relevant when they’re launched in the US, and everyone went to an american High School (which, as depicted on media and comments seems to be the Worst.Place.Ever.). That hardware come, out of the blue, from time to time, so we can run software faster. Of course, I’m exaggerating, but not by much.

I may have a biased idea on how US High School is.

I may have a biased idea on how US High School is.

I don’t know, from my point of view, given that I don’t share a lot of those biases (I can’t honestly consider myself “young” anymore, I am a Spaniard living in Ireland, and I spent half of my career working on non web technologies[2]), sometimes I get baffled by online discussion, especially the ones that talk about the community (as opposed to the tech, which is a different issue). My main concern with all the system is that everyone (participants and consumers in particular) will assume that every single issue raised by leaders can be translated directly into the general community.

Just to show an example, there is a lot of discussion about what makes a great developer and the proper strategies to hire them. Some ideas that could work for hiring a young front-end developer in San Francisco may not work as well on other places, for different technologies. There is always discussion about being a founder in different countries, and, as you can imagine, the experiences are quite different. Different legal system, different business cultures, etc…

Being in contact with communities where you’re talking to what in many aspects are your peers is absolutely fantastic, You can relate to them in a lot of things. That’s why you’re part of the community. Heck, I learn a lot everyday. But we also need to take some distance some times, be a little critic on some subjects and try to adapt what we learn to our particularities. Because  chances are you’re  biased in a different way than the leaders and participants of the community.

1.I don’t like the microtransactions thing, but I just think that there is a business case for it.

2. Embedded software related to satellites and industrial control systems.

Rockstar programmer and Rockstar teams


There has been some discussion about the so-called Rockstar Programmer. You know, that awesome engineer (also called 10x engineer) that can produce what 10 other, average engineers can.

This post by Scott Hanselman[1] fueled some discussion on Hacker News. What has been overseen about the original post is that he advocates about 10x teams.

That resonates a lot, because I think that we should agree that, while there is people with potential to be ninja programmers, that’s not something that can be achieved without the proper care on the environment.

A good team is one that reinforces the good points of their members while hiding away (or at least mitigating) their weaknesses. It makes not sense to talk about a guru engineer that can come in a parachute in a project (any project), replacing an average Joe on a 10 members team, and simply double their output! And, after a while, she’s probably take her umbrella and fly away to the sunset (to a better payed work, one can only imagine)

If we use a low-level warp bubble around that code, we could reduce its gravitational constant, making it lighter to push!

If I use a low-level warp bubble around that code, I could reduce its gravitational constant, making it lighter to push for the rest of the team!!

Real life just doesn’t work like that. Everything is a little more complex. A toxic team or project can be beyond salvation. A regular programmer can achieve a lot just by giving some motivation and direction. A great engineer can be disastrous working in a particular area.

Do you want to see someone transformed from an x programmer to a Nx programmer? Just take a look on the same engineer the first day in a new job and then again after a whole year. The first day she’ll have to ask lots of questions. After a while, she’ll be committing patches, and, later, she’ll reach a cruise speed much much faster than the first couple of days. Or… maybe she is an x programmer, and during the first days she was a x/N programmer. Mmmm….

I also like how Haselman he approaches the subject talking about “titles” and “loud programmers”.  The Rockstar Engineer idea is more a recruitment-marketing issue. It is used to hype possible hires: “Hey, we are a Rockstar company and we are looking for Ninja developers. Maybe you’re an Stellar Programmer”. It has been so used that it doesn’t mean anything anymore. I’ve even seen a job offer asking for a Python Admiral[2]. It is currently more a way of signalling spam than any other thing. [3]

But there is still the myth of “the 10x programmer”, not as a way of describing that there are obviously people more productive than other (and who can reach high notes), but taking for granted that is mostly a characteristic of the programmer itself, while the truly stelar results are achieved mostly when the environment is the  adequate. A lot of the great results in a team is not a magical increase in productivity by some gifted individual, but more the constant improvements in the good directions or good vision to focus in what’s truly relevant. A single good developer can move quite fast in the good direction not because she’s wildly more productive but because she has a clear view and focus.

Because an average programmer can be at least a 5x programmer when the proper details fall in place, and a great developer can be a 1/5x programmer in the wrong place.

1 – He’s also referencing this very interesting article by Shanley.

2 – Mmm, I have two offers, in one they got good salary, decent benefits. But the other one is offering a fleet command. Tempting.

3 – Not to talk about salaries, of course. We talk and talk about 10x programmers, I’d love to see some place offering 5 times an average salary.

Sort of the same expression when looking not commented code.

Vim speed is not really the point


500px-Vimlogo.svg

I am a Vim user. And a Vim fan.

I was fiddling around for some time, you know, just knowing the basics, because it is convenient to do small changes on servers without having to worry about installing anything. Just the basics, insert mode, some search, save,

and a couple more things. Then, around two years ago, I decided to give it a try as my main editor, after watching a presentation of a co-worker (my boss, actually) about how to use Vim (he has been using Vim for around 20 years)

At the beginning, it is quite difficult, to be honest. You have to relearn everything about editors. But I was doing OK after one or two weeks, so I kept using it. I was also forcing myself into improving my usage, reading about it and learning tricks…

Then, after a while of using it and read a lot of instructional material (I cannot recommend “Practical Vim” by Drew Neil strongly enough. It’s a FANTASTIC book), everything started to make sense. Let’s be serious, the problem with Vim is not exactly that is “difficult” per se, it’s that it is so alien to any other text edition experience, that you have to forget everything that you know about how to edit text. That’s why I don’t agree that the Vim learning curve is a myth. Because, when we first heard of Vim, we already have 10+ years of using things like Word, NotePad or Gmail to write text. We need time to assimilate how to edit text “the Vim way”. And that takes time to assimilate, as your muscular memory works against you.

And, yes, the Vim way is awesome. But it is awesome not for the reason that usually someone will think at the start. There is the perception that Vim is awesome because is fast. It is not.

Continue reading

In defense of resting


I have been watching recently some documentaries about software development, including the classic Triumph of the nerds (available in YouTube in three episodes, 1, 2 and 3) and Indie Game: The Movie. They are both very good  and I’d recommend them not only to developers, but to people interested in technology and/or entrepreneurship in general.

But they are very good exponents into something very present on the software scene, which is presenting crunch mode, working insane hours, in some sort of glamourised way. It is part of the usual storytelling and, and probably, part of the hard work -> ??? -> profit logic.

Let me told you something. When I was starting my career, on my first long term job, we once had a very strong deadline. This made us work in crunch mode for a long time (around 2 months). That meant working around 12 hours or more per day, 6-7 days a week. The very last day (a Sunday), I started working at 9:00 AM and went home the Monday at 6:00 PM, only stopping for eating something quick and going to the toilet. The rest of the team did similarly.

Continue reading

ffind: a sane replacement for command line file search


Screen Shot 2013-03-26 at 22.53.13
I tend to use the UNIX command line A LOT. I find it very comfortable to work when I am developing and follow the “Unix as IDE” way. The command line is really rich, and you could probably learn a new different command or parameter each day and still be surprised every day for the rest of your life. But there are some things that sticks and gets done, probably not on the most efficient way.

In my case, is using the command `find` to search for files. 95% of the times I use it, is in this form:

find . -name '*some_text*'

Which means ‘find in this directory and all the subdirectories a file that contains some_text in its filename’

It’s not that bad, but I also use a lot ack, which I think is absolutely awesome. I think is a must know for anyone using Unix command line. It is a replacement for grep as a tool for searching code, and works the following way (again, in my 90% usage)

ack some_text

Which means ‘search in all the files that look like code under this directory and subdirectories that contains the text some_text (some_text can be a regex, but usually you can ignore that part)

So, after a couple of tests, I decided to make myself my own ack-inspired find replacement, and called it ffind. I’ve been using it for the last couple of days, and it integrates quite well on my workflow (maybe surprisingly, as I’ve done it with that in mind)

Basically it does this

ffind some_text

Which means ‘find in this directory and all the subdirectories a file that contains some_text in its filename’ (some_text can be a regex). It has also a couple of interesting characteristics like it will ignore hidden directories (starting with a dot), but not hidden files, it will skip directories that the user is not allowed to read due permissions  and the output will have by default the matching text in color.

The other use case is

ffind /dir some_text

Which means ‘find in the directory ‘/dir’ and all the subdirectories a file that contains some_text in its filename’

There are a couple more params, but they are there to deal with special cases.

It is done in Python, and it is available in GitHub. So, if any of this sounds interesting, go there and feel free to use it! Or change it! Or make suggestions!

ffind in Github

ffind in Github

UPDATE: ffind is now available in PyPI.