October 28, 2008

Stop Judging Resumes: Virtuously Virtual Hiring Practices

by with 4 comments

In my own experience there’s been very little relationship between the
quality of a resume and the eventual usefulness of a developer. I’ve seen guys
with great work history, references, advanced degrees, numerous publications,
and so on, and yet their presence proved less valuable than their absence.
Meanwhile some of the most rewarding engineers I’ve worked with introduced
themselves with nothing more than a simple letter.

At a previous company I worked with in the dot-com era, we created an epic
test for long distance interviews for a Perl programmer/ Linux sysadmin role.
It consisted of questions that a veteran hacker would maybe know 80 or 90% off
the top of his head, and exactly which man pages to lookup for another 10%.
Cute stuff like “How can you rm a file named -rf?” and “Name
3 things you can accomplish at a GRUB prompt.” We would arrange a designated
time and email the applicant the test. They had one hour (which we would pay
them for) to return it. The test was so long and specific there was no hope of
completion if you needed Google’s help for a large portion of the answers. The
feedback from many applicants was elaborately negative.

These days our process is more to the point. If we’re considering brining
someone on staff, we start by giving them some work to do. We find detachable
development tasks that will further the SpiderOak cause, send them a minimal
set of instructions, and let them run with it. It’s usually something
smallish, 1 – 3 days at most. As an all telecommute team, we’re already
accustomed to giving code feedback. When they’re done, they send us a bill and
we send them a review.

Sometimes we give several people the same task. The results often show an
obvious contrast of strengths and weaknesses across several applicants, and it
conserves the (sometimes scarce) resource of development tasks that don’t
require detailed knowledge of core SpiderOak source code. Sometimes we’re not
sure after the first task so we give more.

I’m sure there are big corporate HR departments who would be astonished to
learn that the best predictor of a developer’s usefulness might be an ability
to complete development tasks.

Comments
  1. I'm not looking for a SysAdmin job, but I'm tempted to apply just to see the test. It sounds like a lot of work has gone into it.

  2. So why don't you show us the test?

    If it's just trick questions, that just depends on whether the person has heard the answer before or not. If you've heard it, you get it right instantly.

    Are any questions substantive about what administrators actually do?

    You also need to keep in mind — it's like tying your shoes. Everyone ties their shoes, but if you asked someone a test question about the steps of the algorithmic process, they'll probably need more than an hour to explain it.

    I administer Linux boxes, but I rarely think about step-by-step processes. If you asked me to actually explain what I did step-by-step I'd probably flounder helplessly because I don't consciously think about it any longer. I'll also write scripts whose purpose is that I do NOT have to think about it any longer. I might know some obscure tar options or iterative bash loop syntax, but I wrote a script 10 years ago and never looked at it again because I don't NEED to know that kind of detail when I can copy the script. I haven't changed my .cshrc since 1993 and doubt I remember what it does, but it sure has everything I need.

    A real professional will automate his work so he doesn't have to think on a low level.

  3. OK I can name three things you can do at the Grub prompt without resorting to Google (which _might_ mean that I'm the type of person you're looking for even if I do have a degree) but what does your company do?

    (Yes I know, the good candidate always does research but you've got a great podium here so a small hint can't hurt too much right? I mean I was almost tempted to try even even though I'm currently preoccupied at the moment just to see whether you would turn me down straight away…)