I once worked at a tech-focused games company (sadly no longer existing) which had the best interview process I’ve ever seen.
They had this massive programmer test. It was something like 40 pages long, hardcopy, double-sided. With answers needing to be written on the test generally in essay format; no computer allowed (and so no real coding involved; it was all about concepts and not about syntax). It covered everything; math, physics, logic, sound, calculus, hardware, etc. The thing was so long that we estimated it’d take a really strong applicant about four hours to complete the whole thing if they somehow were experts on all the topics simultaneously.
We informed the applicants about the test’s length upfront; that it was designed to be a 4 hour test, and that they were only going to get to spend one hour and that we absolutely didn’t expect them to know everything (or even most things) on it. That the point was for them to pick the questions they were most interested in and to only work on those ones.
And then we never formally scored the test.
Instead, we basically used it as a conversation-starter for the interview which immediately followed the test, using it as a springboard to talk about their understanding and skills and why they had selected to do one section over another, or about particular questions which they’d provided interesting (or problematic) answers to.
The “here are a whole lot of questions, answer whichever ones you want, you can’t answer them all and we’re not scoring you on how many you get right” approach always felt kind of ‘fair’ to me in a way that pure interviews or coding questions never did, for me.
(caveat: I wasn’t hired using that test; I had been hired as a junior long before, based on my portfolio and a short interview, and the folks who did get hired via taking the test may not have liked it as much as I did. But I loved that it didn’t hang everything on a single gimmick code question the way I’d seen from a whole lot of other companies)
> caveat: I wasn’t hired using that test; I had been hired as a junior long before, based on my portfolio and a short interview
That's always the catch. The founders and initial employees of a company never had to go through the crappy hiring process that the company adds later as it gets bigger. And then we claim that the tests make tech a "meritocracy" and "objective", but those at the top are exempt from the tests.
On a related note, evaluating the quality of tests is even more challenging than writing them.
I recently tried interviewing for a growing small company, who nullified the initial good impression with a multiple choice test about certain technologies they were using, administered over the web by some test company.
The best questions were "which of these four functions or classes do you use to do $common task with $framework", testing documentation speed reading skills rather than actual coding skills; other questions were ambiguous, debatable or completely wrong.
>The best questions were "which of these four functions or classes do you use to do $common task with $framework", testing documentation speed reading skills rather than actual coding skills; other questions were ambiguous, debatable or completely wrong.
Sounds like testgorrilla. I swear they have a poor markov-chain based AI generating the questions.
Modern not-quite-AI might be able to figure out reasonable questions from high quality source material (e.g. StackOverflow questions) and synthesize wrong answers, but competent human reviewers are still needed and this is probably the weak point of the process.
This is all true, although note that I was not a founder or initial employee; I definitely arrived during the second half of the company's decades-long lifespan. The test was just developed years after I was hired, and I was not involved in its development in any way (except for once taking it to act as a baseline for what a senior programmer's answers might look like).
I just thought it was a really neat and kind idea to let the applicant choose what question or questions they wanted to discuss and in what order, instead of forcing them to answer a single pre-defined high-pressure coding question and hang the whole interview on that.
The least I would do is politely decline such an interview. I would be sorely tempted to give them the choice between letting me use a keyboard or hearing from an ADA lawyer.
I mean, obviously exceptions were frequently made for folks with special needs or just personal preferences. I’m aware of many times that was done for both situations.
I assure you that I’m not a monster and I would really appreciate it if you’d assume positive intent in such replies.
That isn't obvious at all, you explicitly say 'no computers allowed' and yeah, I consider a forty-page handwritten essay a sadistic way to find out more about a computer programmer.
I don't think you're a bad person for it, and ok, good to know you're willing to make accomodations, but my reaction to being asked to do this would be sharply negative. The keyboard is inherent to the profession.
Assuming you read the essay it is impossible to not judge the applicant on their penmanship. A difficult skill with absolutely no bearing on the qualification for the job.
I urge anyone reading this to not offer a 40 page handwritten essay to software developers in the first place. It's an awful idea.
I have explicitly and repeatedly pointed out that individual applicants each engaged with only a very small portion out of a very large test — the entirety of which had its questions printed across 40 pages.
There was obviously no “sadistic 40 page handwritten essay”; individuals engaged with whatever individual questions they wanted to, and would normally write their answers into the bits of blank space left in between the questions. I’d estimate that a normal applicant who answered ten to twenty questions would normally write substantially less than what would constitute a single page of text in total.
Again, I’m not a monster. I’d really appreciate it if you’d assume positive intent and ask for clarification rather than jumping to the worst possible interpretation you can invent and then stridently declaring that you’d take legal action on the basis of the fictional “sadistic” thing you’ve chosen to believe I was making people do.
Why so aggressive? What's the problem with the pen? The idea the OP proposes is to use it as a spring board to get a discussion going, I'm sure a pen, a keyboard or a interpretive dance would all work for what they're proposing
wow. Does ADA lawyer thing work with whiteboard interviews?
I'm only half-sarcastic, I hate my handwriting and this makes whiteboard interviews double hard for me.
Probably not, because the ADA criterion are actually reasonable, and long handwritten essays handily fail it.
Basically, don't test for something which isn't a job criterion and excludes people based on ability. An absurd example would be asking programmers to lift a 20kg sack of flour onto their shoulder and carry it across a room: this is a legal and normal thing to ask if the job involves lifting things, do it for a desk job and you'll get sued eventually.
Whiteboarding is an actual part of the job at many companies, I just let them know that words will probably be illegible and muddle through it.
No software job calls for handwritten pages of text, it clearly discriminates on ability and it shouldn't be a part of hiring. Simple as.
I wanted to add a smart remark saying that once upon a time, in better times long gone, a long time before leetcode was invented, handwriting itself was used as a criterion to hire.
Then the first hit on google was a paper from 2018 about how to hire based on handwriting.
They had this massive programmer test. It was something like 40 pages long, hardcopy, double-sided. With answers needing to be written on the test generally in essay format; no computer allowed (and so no real coding involved; it was all about concepts and not about syntax). It covered everything; math, physics, logic, sound, calculus, hardware, etc. The thing was so long that we estimated it’d take a really strong applicant about four hours to complete the whole thing if they somehow were experts on all the topics simultaneously.
We informed the applicants about the test’s length upfront; that it was designed to be a 4 hour test, and that they were only going to get to spend one hour and that we absolutely didn’t expect them to know everything (or even most things) on it. That the point was for them to pick the questions they were most interested in and to only work on those ones.
And then we never formally scored the test.
Instead, we basically used it as a conversation-starter for the interview which immediately followed the test, using it as a springboard to talk about their understanding and skills and why they had selected to do one section over another, or about particular questions which they’d provided interesting (or problematic) answers to.
The “here are a whole lot of questions, answer whichever ones you want, you can’t answer them all and we’re not scoring you on how many you get right” approach always felt kind of ‘fair’ to me in a way that pure interviews or coding questions never did, for me.
(caveat: I wasn’t hired using that test; I had been hired as a junior long before, based on my portfolio and a short interview, and the folks who did get hired via taking the test may not have liked it as much as I did. But I loved that it didn’t hang everything on a single gimmick code question the way I’d seen from a whole lot of other companies)