Most comments here are misunderstanding
these puzzles as some awful alternative to leetcode, but as someone who remembers puzzling over these on the T, still learning to code, it was a very different world when ITA was using these.
These appeared during the post-dotcom bust pre-startup boom period of 2004-2008. If you look at CS enrollment rates of the time you’ll see they were at record lows. Software paid well, but not that well and most engineering jobs were really bad ones resembling Dilbert comics. The culture of treating engineers well, giving them good tools, snacks, unlimited pto didn’t happen until later.
At the time people solving hard problems and keeping up with changes in programming languages and tech were doing it because they loved it. Nearly everyone doing tech was doing it because they where obsessed and passionate about it. This is because if you weren’t there where much more pleasant ways to make about the same money.
These ITA puzzles were a way of signaling “hey! we’re a company of people that love to code and work on hard puzzles too! If you’re the type of person that sees puzzles like this and can’t get it out of your head for the rest of the day, you should check us out!”
These puzzles were not a leetcode like gates (in fact they very explicitly were NOT required for applicants), but rather a way of advertising
to programmers that ITA was not just another boring big corp that would suck your passion for coding away.
Absolutely right! While we did suggest that applicants try to solve one of these in whatever language they liked, the main purpose was to attract curious and smart people. And it worked! These puzzles brought us a ton of great candidates along with lots of smart people who weren’t looking for a job, but just wanted to know how their solution to a given problem stacked up. It wasn’t a proxy for 2019 whiteboard hazing; these were simpler times...
-- Dave, ITA cofounder and creator of some of the puzzles, including my personal favorite, “Rebus”.
Dave is also known in some circles for coding Crash Bandicoot, a much-loved game of the period. I wonder of anybody remembers it today, and plays it on emulators.
Yes, the people who were still in tech were obsessed and passionate about it. And then companies like this insisted we act like trained seals, spending days working on irrelevant puzzles for the honor of being maybe possibly considered for a job. And then to top off that shit sandwich, they sprinkle in these wonderful bacon bits:
"When implementing the server, aim for scalability and robustness. (Many submissions fail due to lack of robustness!) Your submission should include a description of the steps you took towards those two goals. Keep in mind that the client may be buggy, or even malicious. For example, if a client connects to the server and sends an infinite stream of the byte 'X' with no line break, the server should deal with this case gracefully. Please do not use an existing networking framework (e.g., Twisted or asyncore for Python, ACE for C++, etc.) to implement the server."
Not only no, but fuck no. I settled for a job in the IT section at a bank working with COBOL and EJBs because of this shit, until I'd finally had enough, co-founded a company, moved to San Francisco and implemented a SANE hiring process that treated candidates like professionals. Most of us did, so I guess in a way these horrible and misguided hiring practices were partly responsible for the next resurgence, because we'd finally had enough.
Same. I had to do this in first-year CS, and got pretty surprised to find out I'd be working with senior backend engineers who've never heard of sockets. Concerns at this layer don't come up much, but when they do, the set of people available to help gets a lot smaller.
What I've found worked well is to have a conversation about something they've built in the past. Ask them questions about it, like why certain architectural decisions were made, what constraints they had to work under, what issues came up, and how they dealt with them. I've even hired without seeing a single line of code. You of course have to change tactics when hiring junior devs, and would probably want to see SOME code in such a case, but regardless, just talking about technical subjects they claim knowledge about is usually enough to discover how deep that knowledge goes. If they have any code at all in the wild, I'll look at it to get a fuller picture of their abilities, but I won't require it.
If I want to see code, I'll tell them beforehand that I want to build a little project together with them (less than 50 LOC) that does this one trivial thing, and that they can write beforehand if they want to. Then when they come in, we either build it together if they haven't done it beforehand, or we change the specs if they did. And no, no bonus points for doing it beforehand. That's just so that they can feel more comfortable in the codebase if they feel it's necessary (being nervous in the interview and all). I'm most likely not going to compare their code to others; I'm only interested in whether their code meets the company standards.
I've also fired many people. It's not hard to do, nor does it cost much.
So many of these recruitment processes are predicated on the principle of "what if someone lies to us?" for fear of hiring the dud. But in reality, it's so hard to bullshit someone knowledgeable that the actual risks are tiny.
Yes, I've encountered people who couldn't code. I've had such resumes passed to me by people (usually non-technical, or recruiters) telling me how awesome this person is (because they don't know enough about the subject matter to tell the difference). But it takes very little conversation about any topic one is knowledgeable in to sniff out someone's bullshit.
Think of it this way: You have a leak in your house and hire a plumber. The guy comes and tells you that you need this and that. How do you know he's telling the truth? Now imagine that you're a plumber who's been asked by a friend to sit in on the conversation. How long do you think it would take before you can confidently call bullshit on what he's saying?
If YOU know your shit, you can tell when other people don't. And if you DON'T know your shit, you're going to have a hard time no matter how much code they throw at you.
Ah, of course. My personal experience from several employers has been that people who believe that they can judge programmer skills by listening to their bullshit end up hiring bullshit artists. I have also worked with programmers who could easily spin a tale that covers all your questions but still couldn't write a single working line of code. Coincidentally, they were masters at explaining why the project is delayed.
I'm in Europe, so it's not easy to fire wrong hires after trial period, but I have never even seen anyone admit that they made a wrong hiring decision.
As an interesting case, one company I worked at hired an architect who had failed their rather trivial coding test but interviewed well. That person, together with company management, ended up driving away several competent engineers.
I won't work any more for employers who don't require a programming test. Your experience may be different.
I'm also in Europe now (Germany). If you haven't figured out by the end of the trial period that the person can't do their job, it's not he who's the problem.
It has also been my experience (20+ years) that sociopaths and egotists (the ones who will try to bullshit you beyond a few ham-fisted attempts) are in such a minority that they're not worth basing your interview process around.
The only cases of hiring engineers who couldn't code that I've ever heard of directly from colleagues over my entire career (that weren't third-hand re-tellings) have been where he either was not interviewed at all, was only interviewed by someone non-technical, or was hired by someone non-technical over the objections of someone technical.
> These puzzles were not a leetcode like gates (in fact they very explicitly were NOT required for applicants), but rather a way of advertising to programmers that ITA was not just another boring big corp that would suck your passion for coding away.
However, from the linked page:
> To be considered for an interview, you may need to solve one of the programming puzzles below.
In fairness, they used the word "may". But I wouldn't call that "explicitly NOT required".
I applied to ITA in 2008ish and was require to complete one of these, something very similar to instant search. Needless to say I failed because the question isn't about your ability to program, it is just if you know what a suffix-tree is.
These are fun puzzles but terrible interview questions.
Like a few other commenters have mentioned, I had a great time trying to work through these problems having seen them in ads on the Red Line in the mid to late 2000s.
For me, these puzzles were what got me to change careers; at that point I was in a non-technical job and didn't have any CS training. Still, I saw these puzzles and was hooked. Figuring out how to even set up a basic coding environment was part of the puzzle for me.
I spent a few weeks on "Strawberry Fields", eventually getting to a brute force solution that passed all the tests. I was about to send it in, and happened to meet one of the ITA folks at a social event. After a few minutes of talking to this gentleman, I realized he was in a completely different intellectual class from me, and that I had to do a lot better than hacking together a solution to a single optimization problem if I really wanted to work in this industry.
I persisted. I kept solving whatever coding problems I could find and rung by rung, climbed my way into some interesting and challenging software and data roles. I bombed plenty of whiteboard interviews along the way.
It sounds funny to write down, but it is not an understatement to say that those ads on the T changed my life!
I remember reading these on the T on my way to and from my first job in Boston back in maybe 2009. Back then it felt like you had to have a PhD to work at google and it was cool to know there were companies that just wanted curious people who liked solving puzzles with code.
I don’t see the question I remember pondering, which was simpler than these but more fun to think through on a short subway commute— something like “Create the longest possible string of connected movie names, like ‘Independence Day of the Dead Man Walking...’”
I think the key here is "old". It's their OLD hiring puzzles. They worked because they did them first, in a new an interesting way.
So much stuff on the internet (and business in general) thinks that "hey, it worked for someone - it must be a new paradigm!"
The people who wrote those puzzles would never think of writing them as incentives to join their company now - because that idea has long-since sailed.
So much of the crap part of hiring (and business in general) is because it involves taking something unique and interesting that someone has done, and rehashing it until it's a hollow, shallow, uncreative version of its original effect!
ITA used to advertise puzzles on the MBTA (public transit) in Boston and I emailed in a solution when I was a university student. Looking back, it seems like an effective way to find candidates with a sense of curiosity for problem solving.
I eventually got an offer, but chose another job. They were acquired by Google soon after, and I sometimes regret not taking it!
I don't know. The wordplays, yes maybe, but the "build a chat" one is pretty interesting. What kind of buffer will you use, how do you handle errors, how did you test, which socket library did you use and why can tell a lot about how you work alone with clear but not well-defined objectives.
It depends on how you want to understand language.
The point of a hiring process is to exclude candidates that are legitimately not suitable for the role.
A reasonable contextual interpretation of the grandparent comment is that the hiring process will end up excluding suitable candidates for illegitimate reasons.
Whether they should have to slavishly spell this out in a message board comment is a hot topic.
All of these seem to require a significant investment of time before the interview process gets kicked off, which is uninclusive because it biases the hiring process against candidates whose time outside of the office is consumed by taking care of their children. (Stereotypically, it’s borderline age-ism too because candidates in their 30s and 40s are much more likely to have children compared to people in their mid-twenties.)
>All of these seem to require a significant investment of time before the interview process gets kicked off, which is uninclusive because it biases the hiring process against candidates whose time outside of the office is consumed by taking care of their children.
You have to be kidding me. It also is biased against people whose time is spent playing video games or who do nothing at all.
Some jobs are meant for people at a certain point in their career. Sometimes you don't get to have everything at the same time.
People like you would argue it would be better to shut down hospitals than let doctors work long hours because those long hours might make women not want to do the job.
It filters out anyone without a few hours/days/a week to spend interviewing at your company (those without access to good hardware at home, those with familial or other after hours commitments, etc)
The act of dissecting a candidate's qualifications and mapping them to what a company is searching for is always going to come off as appearing to be uninclusive to one group or another unless you are naive to believe every single human being contains the same skill set, current ability, and potential.
Maybe today when every company would love you to do a series of coding exams to vet you for their thousands of competing positions, but in 2007/2008 a fun brain teaser to chew on and solve in your spare time was novel, fun, and a good way to attract people who liked solving tough problems without just filtering through resumes.
I just checked my gmail account - found that I had an interview with ITA Software in 2005. The task was about finding palindromes in the dictionary. Man, I bombed that one really hard. The language barrier was one thing, but lack of practice was another. Nowadays we get a lot of support from a plethora of sites, other engineers and many companies train people to take interviews which helps another way too. I got into a large Common Lisp project 5 years later anyway, so no bad feelings.
These problems were very late, mostly after ITA had got into developing an airline ticket-booking application that had to interact with badly managed airline companies who thought Java was actually a good idea. Extreme algorithm design skills were not so valued then, and a person who liked that kind of thing wouldn't enjoy interfacing with corporate servers.
I was hired at ITA in 2002 on the strength of solving one of their puzzles.
The thing about the puzzles was that, once you start thinking about one, you want to see what a solution would look like, and the only way to find out is to solve it. Having solved it, you want to know how it compares to others'. So you send it off, even if you aren't really interested in moving cross-country.
In my case, the puzzle was "add-a-gram": given a word list, find the longest sequence of words in which each word after the first is an anagram of the previous word, with one letter added.
I thought about it for a day or two until I realized I had a workable strategy. Then I sat down at my laptop, checked the clock, and started coding. Exactly two hours later, I had a program that, after reading in the dictionary, was a one-page, deeply-nested loop that exited when it found the answer.
It ran in ten seconds. How could I know if that was any good? I sent it off to ITA. The next day I got a call. We talked a little about setting up a phone screen, but she let slip that their solution was faster than mine. That could not stand. I spent the day inverting the control flow, with memoizing, and got a 10x speedup for less than 2x as much code, and sent that off.
I went on to interview. At the interview, they assigned a programming task that was a simplified version of the puzzle. I worked there for three years, until I got fired for being depressed. (I found treatment five years later.) I don't remember much of it because I was sleep-deprived, from having toddlers who would only sleep in the daytime. But I encountered there the most intelligent people I have ever known. Most were musicians, on the side.
I had been there three months before I thought to check their solution. My second version was faster. (Bitwise operations FTW!)
While there, I encountered a problem that I have used in conducting dozens -- maybe over a hundred -- interviews: Undergraduates solve it in two minutes, graduates in five, MSs in 20 or more, PhDs either never, or as fast as an undergrad. Every single person who has solved it used the same hand gesture in describing the solution.
Sometime later, ITA were bought up by Google, and I got some cash for stock. Taking care to owe only long-term capital gains did no good because Google paid cash, so we all ended up owing "alternative minimum" taxes as if we had made all the money in one day, not over N years. (The same thing happened when my next employer was bought by IBM.) Alternative minimum taxation is a terrible thing.
The version of the puzzle I was assigned during the interview is one we sent to candidates between phone screen and scheduling an interview, at my next employer. If it ever took more than an hour from problem statement to solution, it was too long; although I heard of people taking 12+ hours. Some sent code that ran 12+ hours, and was wrong. (Our best ran in <1 ms, which no candidate matched.) Some candidates posted their solutions online, after being rejected, but every posted solution was really bad, so that was OK. Cheaters disqualified themselves.
In 2010 I solved the "bitvector" problem in the posted article, during two weeks of sharply heightened intelligence I had while adjusting to the new antidepressant. To this day I marvel at that code. But by then, there was noplace to send it. Google took down the puzzles a little later.
I sometimes wonder what Carl went on to do, after he left ITA.
It is a file distribution problem, copying a file to a large number of destinations through a slow link. At ITA, the file was the airlines' seat availability record, that really indicated which price points on each flight could be offered for sale right then.
I always tell them that undergraduates solve it quickly, but that never seems to help MSes.
There was a similar problem, back in the 80s, that made starting an Occam program on a network of transputers very, very slow, that was solved the same way.
These appeared during the post-dotcom bust pre-startup boom period of 2004-2008. If you look at CS enrollment rates of the time you’ll see they were at record lows. Software paid well, but not that well and most engineering jobs were really bad ones resembling Dilbert comics. The culture of treating engineers well, giving them good tools, snacks, unlimited pto didn’t happen until later.
At the time people solving hard problems and keeping up with changes in programming languages and tech were doing it because they loved it. Nearly everyone doing tech was doing it because they where obsessed and passionate about it. This is because if you weren’t there where much more pleasant ways to make about the same money.
These ITA puzzles were a way of signaling “hey! we’re a company of people that love to code and work on hard puzzles too! If you’re the type of person that sees puzzles like this and can’t get it out of your head for the rest of the day, you should check us out!”
These puzzles were not a leetcode like gates (in fact they very explicitly were NOT required for applicants), but rather a way of advertising to programmers that ITA was not just another boring big corp that would suck your passion for coding away.