Agile Zone is brought to you in partnership with:

I am the API Evangelist. Not in the sense that I’m evangelizing a single API to you--In the sense that APIs are important for everyone to be aware of. I’m paying attention to not just the technical, but the business and politics of the web API movement. I share my insights by blogging on the business of APIs at apievangelist.com, politics of APIs at apivoice.com and you can find more information about me at kinlane.com. Kin is a DZone MVB and is not an employee of DZone and has posted 95 posts at DZone. You can read more from them at their website. View Full User Profile

Sorry Google, Your Programming Test Is Not A Valid Measurement Of My Skills

08.22.2014
| 34153 views |
  • submit to reddit

I’ve been talking with a very nice recruiter over at Google over the last couple weeks, and she has been so kind in keeping me updated about opportunities for evangelism at Google. This is the 3rd round of talks I've had with Google while being the API Evangelist, talks that historically go nowhere because of their programming test, which is a super silly aspect of their HR process.

I was straight up with the Google recruiter a couple of weeks ago when she first emailed me, and again when we talked on the phone last week—I do not take programming tests to open up doors for employment conversations, sorry.  ;-(  It is a waste of my time, and yours, and it doesn’t measure shit. I understand that you have to qualify large number of folks, at your very algorithmic-centric company, but when it comes to measuring what I do, a programming test isn’t a thing.

If programming a tic-tac-toe game on a live screen share is what you need to open up a conversation with professionals around evangelizing your platform, you need to look elsewhere. Nowhere in my role as the API Evangelist do I have to code under time pressure with someone else watching, sorry. I would even say, having hacker skills, trumps programming skills in a public facing evangelism role, and speed, quality of code go out the window. This is about making connections through hacker storytelling, something that doesn't always pencil out to producing the best code and is more about helping people understand what is possible using a platform, in the most meaningful way—requiring more focus on the person and their problems, not the code or algorithm.

I’ve managed to have man very meaningful conversations with other tech giants like Intel, IBM, large institutions like UC Berkeley, BYU, and establish fruitful relationships with partners like 3Scale and API Spark, and across federal agencies like Department of Education, Energy, NASA, and the White House around APIs--all without taking programming tests. I talk to startups, SMBs, SMEs, organizations, institutions, and government agencies all the time, and I never have to code under pressure in front of an audience.

I’m not under the illusion that I will change your hiring practices, Google—you are a very smart, and successful company. All I’m saying is you are probably filtering out some pretty talented folks who are extremely passionate and knowledgeable in what they do, and connected in their space, and when you won’t engage in meaningful conversations without a programming test, your missing out.

I actually prefer working with organizations from the outside in. I think it better reflects the essence of API Evangelism. The companies who have trouble working with outside entities that don't have traditional HR processes are probably not going to lead when it comes developing an API driven ecosystem.

If your company doesn't have the time to research me, and understand what I bring to the API space, and what my skills are, we probably aren't a fit. Everything about me is available online at API Evangelist, Kin Lane, Twitter, and Github--you just have to look. If you are only looking at resumes, and making people take tests, you will probably get what you are looking for!

Published at DZone with permission of Kin Lane, author and DZone MVB. (source)

(Note: Opinions expressed in this article and its replies are the opinions of their respective authors and not those of DZone, Inc.)

Comments

Sigmund Lundgren replied on Sat, 2014/08/23 - 5:27am

Totally agree, went through the same process myself. Pointless and doesn't represent coding is actually done. Write tests, run, evaluate and code on paper? Pretty hard to do TDD on "paper."

Nick Brown replied on Sat, 2014/08/23 - 11:01pm

I take it you haven't been on the other side of the interview table much...

There are two reasons to give candidates coding problems.  Neither is to see how good of a coder you are.  

The first is to weed out the fake candidates who don't know the first thing about programming.  These make up a frighteningly large percentage of applicants.  This usually involves just getting them to write something brain dead simple.  Now I know what you are thinking.  "I have a long and respected career as a software developer, proving that I know how to code is beneath me!"  I know plenty of people who have had "long and respected" careers but still can't program.  They just lean on others to do the work, and then take credit after the fact.  Its ugly, but it happens all the time.

The second reason is to evaluate how the candidate approaches the problem.  Are they able to break down a problem into simpler sub problems?  Are they able to ask questions that will help clarify ambiguities?  Are they able to identify edge cases?  These are all questions which will you much more about the candidate than how well they write for loops.  And you need to watch the person in action to tell this.

Many companies, Google included, admit they probably turn down many qualified applicants.  If you have an interview process strong enough to weed out the majority of applicants that don't belong in the position, you probably have to turn down many qualified applicants.  But there is real danger in hiring someone completely incompetent; they can wipe out all the efforts of your qualified developers.

Srinivas Murthy replied on Mon, 2014/08/25 - 12:03pm in response to: Nick Brown

 

 I guess you have been on the "other side" for way too long. There was a time in life when all I wanted to do was work at Google, even before the stock hit those obscene levels of valuation. I breathed and lived the days with Google, applied, applied and applied,  wrote love letters eulogizing me and Google for eternity, got myself prepared for "algorithms", coding tests and what not. It took a couple of years to get past the first phone screen and eventually I did make it to the campus. I thought I did well but I am guessing Larry thought I would be too old amongst the 25 year old folks  there.

At that point the recruiter had given me a coding test, which I wrote in the evening, after I went home from my work which paid me my bread and butter.  It went fine, I didn't cheat, I did well enough to earn the next four rounds of phone interviews and then eventually the campus interview.

I have interviewed with  other companies, which had a similar model, they would email you the coding problem and then you get back to them in a few hours, or whatever.

Now, the model has changed. Every $%^&#$  wants you to join on a whiteboard application and code live. Not only does that put logistical problems in your way, of having to spare a day from work, I guess I am not the type that would walk into a local Starbucks and start taking coding tests and interviews, but the sheer fact that one is working from home and has a day job, phone meetings and conversations going on with co-workers, makes it that much more harder to focus on the problem.  Folks who are interviewing actively and need to take many of these tests, are screwed.  I am not even arguing about the point of these online exercises, its individual opinion, but I am arguing the need to give some leeway for the hapless candidates.

I guess I have grown up and no longer have the compulsive obsession about going to work for Google. FAQ Google  :-)

Nick Brown replied on Mon, 2014/08/25 - 12:41pm in response to: Srinivas Murthy

I fully agree the initial interviewing should be something that can be done quickly and not take up too much time.  Companies shouldn't expect applicants to invest that much time in them unless there is a serious interest from their side.  Otherwise they do loose out on a lot off currently employed applicants who don't have time to spend several hours on each potential employer out there.

Lukas Eder replied on Wed, 2014/08/27 - 1:33am

Don't regret this. Google HR has their ways. You have yours. You don't fit. They don't fit either. End of the story.

Be glad it went this way, chances you'd be miserable in that company are high.

If it were any other company on this planet, you wouldn't have gone through the hassle of writing a blog post about it. It wouldn't even be interesting or surprising at all.

Lukas Eder replied on Wed, 2014/08/27 - 1:42am in response to: Nick Brown

There are two reasons to give candidates coding problems.  Neither is to see how good of a coder you are.  

Well done Nick, our favourite, passionate, and experienced HR guy. But you know, we're really looking for a janitor ;-)

(have you even read the article?)

Jose Maria Arranz replied on Wed, 2014/08/27 - 3:06pm

I understand this type of evaluation based on coding when you only have a beautiful paper (C.V.), but please, don't ask this kind of evaluation to people with millions of chars of public source code out there, please don't ask stupid (or too complex under pressure) algorithms to people like Lukas Eder :)   

You will show your recruiting methods are similar to robotic processing (everybody are the same), and this is not a good start. 




Lukas Eder replied on Wed, 2014/08/27 - 4:39pm in response to: Jose Maria Arranz

José, who says I wanted to work at Google? ;-)

Jose Maria Arranz replied on Thu, 2014/08/28 - 2:08am in response to: Lukas Eder

Then you will have to do this kind of tests, but in my opinion in some cases and for some people is a very bad start. 

Greg Brown replied on Thu, 2014/08/28 - 8:06am

I went through the same process myself a few years back and had a similar reaction. I don't have any issue with taking a coding test or providing code examples...that's a pretty reasonable thing to ask of a prospective developer. What I didn't like about Google's process was the emphasis on algorithms. Every single interviewer I met with asked me to implement some sort of algorithm. Not once was I asked any sort of higher-level questions, about other skills or career goals.

The ability to write efficient algorithms is important, for sure. But it's definitely not the only criteria for assessing a candidate.

Ryan McDonough replied on Sat, 2014/08/30 - 8:06am in response to: Nick Brown

While I agree with the points you've made there, there's a big difference between giving some a problem to solve and code and let them do it and it's another to stay in the room asking them questions in real-time while they attempt to solve it. Why add the pressure?

One of my earlier employers got this right: give the candidate a problem to solve and give them an hour alone to solve it. Using the Google to look up API docs was okay. (because what idiot doesn't have every API stored in memory, right?). The next hour was a conversation about how the candidate came to their solution. Contrast this to someone who is standing in the room critiquing every line of code or decision you made as you make it. That's not how people actually work in real-life. If that's how a company actually operates, it's probably not the job for you.

And don't get me started with folks who do coding tests on a white board and have some expectation that it's going to be syntactically correct.

Ryan McDonough replied on Sat, 2014/08/30 - 8:15am in response to: Greg Brown

Agreed. But I also have wonder if algorithms are as important for say a web developer? Not that I am suggesting that a developer can get a pass on knowing the impact of different data structure choices, but does someone that operates in higher-level programming languages really need to know how to implement a Linked list or implement a bubble sort if they are operating at a high-level abstraction? Yes, knowing the reasons why one would use a linked list vs. something else is important, but knowing how to implement one is arguably less so. 

It all depends on what the role you are hiring for actually requires. I've watched decent candidates get grilled by someone who is an expert low-level, distributed storage algorithms on things like implementing a high performance B+tree for a job that would ultimately be doing some serious web development in JavaScript. 

Vine Rustia replied on Sun, 2014/08/31 - 9:43am in response to: Ryan McDonough

One of the good reason of emphasis on algorithms is I believe on how you analyze problems. This not just about knowledge of the programming language. You can anyway learn that. But a good sense of analysis is more valuable. Thats why ground breaking frameworks such as AngularJS came up, providing not just solution but best solution for a problem is important for Tech Giant such aa Google. Though not all software companies need to be like google. it depends on their visions and needs.

On the other hand, coding while somrone is watching? Are they measuring your stress management?

Kin Lane replied on Sun, 2014/08/31 - 10:08am

The funniest part of this story, is I've never applied for a Google job, and I've never had a desire to work for Google-still don't Just doesn't interest me. I've very successful doing API evangelism career in general.

The second part is that I never asked to be post here. DZone keeps taking my articles from my personal blog, even though I've told them my business writing has moved somewhere else to apievangelist.com. 

Glad you guys enjoyed having a conversation around my story!

Dave Glasser replied on Sun, 2014/08/31 - 11:40am

 I'm surprised you ever got to first base with Google, considering the ridiculously uninformed statements you make on your website. For example:

APIs are launched primarily to give partners that are outside the company firewall, access to internal data and resources.


Uh, I think you misspelled "web services" here. Would you argue that JDBC is not an API? Yet it has nothing to with partners or company firewalls.

And it just goes on and on...

Ryan McDonough replied on Sun, 2014/08/31 - 2:31pm in response to: Vine Rustia

My problem with asking someone to write something that does a bubble sort, or something similar, is that you typically get back code that's been memorized from a text book to varying degrees. You rarely get insight into how someone solves a problem but more how well they recite a code problem. I'm more of a fan of giving someone an actual business problem and see what they come up. If anything, it helps you identify the range of the candidate rather than hyper-focusing on a specific algorithm. 

Kin Lane replied on Sun, 2014/08/31 - 4:34pm in response to: Dave Glasser

Hey Dave 

Then you should comment on that post where u read that.

Have a discussion about why APIs do not get deployed to help get access outside firewall, and make sure u do it with a real profile.

Especially if you are criticizing someone who has made a career out of something.

Oh wait you won't cause your a troll. I understand your need to tear down. ;-(

Kin Lane (real person with profile he will stand  behind, even if I'm wrong)

@kinlane


Dave Glasser replied on Sun, 2014/08/31 - 5:24pm in response to: Kin Lane

 Do you consider JDBC an API, or don't you?

Dave Glasser replied on Sun, 2014/08/31 - 5:29pm in response to: Kin Lane

 Oh, and what makes you think my profile isn't real?

Vine Rustia replied on Sun, 2014/08/31 - 6:13pm in response to: Ryan McDonough

 Well in your scenario, you are right, those kind of coding questions aren't effective. I think the issue that comes to play, which algorithmic coding tests are good? In my experience, I encountered technical interviews that are a bit exciting, I was asked to provide an algorithm that allows one point to find its way out of a maze with the shortest path and some other rules are given. It was challenging to me, i had keep on asking questions to clarify things up, and finally came up a good solution and explained my answer. I just miss the efficiency part of algorithm, i just thought about it when i got home. But I passed ang got employed. Well I just felt that it was also fulfilling in my part as a  fresh graduate to had such challenges, it also stimulates creative thinking and analysis. They didn't just look up on my grades, lots of people out there had better grades than mine. So its the analysis skills and not programming language or other things.



Jeremy Pulcifer replied on Fri, 2014/09/05 - 10:19am in response to: Vine Rustia

Bullshit.

Algorithms are not problem-solving questions, they are _have_you_done_this_before_ questions. Guys with recently-obtained Masters in Comp Sci love these things, because they _have_ done it before.

The best show-me interview I've had is one where the interviewer gave me a laptop, a problem to solve for which I had little background on, and asked me to work through it. Using my existing API knowledge, google, and some effective back-n-forth discussion, I was able to write an effective solution. The interviewer was looking for collaboration, problem solving, and working under stress. All of which apply to the actual job.

The next interviewer that asks me to solve a riddle or implement a recursive algorithm on a white board is gonna see a very aggressive eye-roll.

Vine Rustia replied on Fri, 2014/09/05 - 11:28pm in response to: Jeremy Pulcifer

Just what I had said before it depends on the companies need.  There are just certain levels of complications as you abstract the solution. For example, creating trees and linked list requires certain algorithms in java language that involves sorting and such. But a hospital billing system required higher level of solutions. The thing is, it depends on what the companies need and problems to solve. If you are to enter google, you will really encounter those problems cause they dont just solve real business problems they also work on low level problems such as creating angular, dart, etc. Do not generalize things.

Jeremy Pulcifer replied on Sat, 2014/09/06 - 8:08pm in response to: Vine Rustia

Alright, I'll agree that there are algorithm-heavy jobs out there. I'm speaking from experience, though, that for many application/website/API jobs, puzzles and algorithms make up a large percentage of the deciding factors in selecting talent.

As an aside, I just went to an interview this past Wednesday, where I was asked to answer 2 riddles (one about how many golf balls fit in a truck, and the other about how to determine what lights were controlled by what switch), and an algorithm problem (fibonacci generator). I successfully solved the latter using a linear algorithm; the interviewer immediately insisted I convert it to recursive. I explained that recursion in this case would perform incredibly poorly (N^2). He disagreed, and I was then shrugged out of the interview.

Vine Rustia replied on Sat, 2014/09/06 - 10:10pm in response to: Jeremy Pulcifer

 I agree. I think there is no disagreement between us. I get ur point now. They should not even tell u what algorithm to use. Its ur job to provide which is appropriate and not theirs nor ur future project manager. The important thing is you have solve the problem with the most appropriate solution.

Steven Goldsmith replied on Mon, 2014/09/15 - 2:20pm

I've been on both sides of the fence, but mostly I've done Sr. Developer, Architect, etc. vetting (maybe 100+ by now). Asking for someone to white board or take a programming test at Sr level is more than likely to be taken as an insult. There's no need to do this to filter out "real" programmers and it's something I consider armature hour from an interviewing perspective. I interview based on principals and fundamentals that transcend programming languages. This is stuff you should know off the top of your head so you can converse intelligently about it. Try asking a Hibernate expert about object-relational impedance mismatch. Chances are only real experts will understand this and have exposure to it. Ask about anti-patterns like the God object, gold plating, arrow, etc. They will be equipped to answer the same lame pattern questions, but knowing anti-patterns can be more important in my opinion since they are so common. Have they heard of the OWASP top ten? If not, then they probably are not as concerned with security as someone that does.

Any ways, you get the point. Tests are for those not comfortable (or versed) conversing about principals and fundamentals. I can filter you out (or in) in 5 or 10 minutes without ever seeing one line of code.

venkat murthy replied on Sat, 2014/09/20 - 10:09pm

I agree with the point on there are things to look far and beyond algorithms. what i do expect in a normal developer is about his well-grounding on basic idioms, principles and fundamental patterns that he can apply. IMO nothing can be as effective as a single page of code that he writes about, which can open up for an engaging discussion. It basically brings about the real developer within. For instance; choice of his interfaces, classes. Once an interface going beyond a single responsibility; does he think/apply  segregation of interface, How well does he apply basic patterns? etc. His comments/thinking on why a certain way certain things are done, taking examples from standard popular libraries/framework.


Klaus Schroiff replied on Sun, 2014/09/21 - 8:58pm in response to: Steven Goldsmith

I am sorry to say but asking somebody about something like the "OWASP Top Ten" is just ridiculous unless explicitly requested in the job ad.  Beating somebody with very specific domain knowledge is a piece of cake. There are a zillion security standards and recommendations out there and you usually stumbled just across a small subset across you career path. That true across all technologies.

In an interview you are usually requested to ask your own questions. If a "specialist" came really up with such questions before I will enjoy to do it just vice versa just to prove the also attending boss that this suddenly stuttering guy isn't quite the smart-ass that he think he is. That's also a piece of cake actually. This may destroy the chances of getting the job but then you won't really like to join such a team anyway.



Steven Goldsmith replied on Mon, 2014/09/22 - 9:10am in response to: Klaus Schroiff

It's only ridiculous if you are not a web developer (which I am not, but I still know most of the OWASP top 10) . This isn't very specific knowledge as you are alluding to. If you cannot not tell me how to protect against SQL injection or prevent XSS attacks (I don't expect you to know all ten), then I wouldn't recommend you as a web developer. Again I'm asking basic stuff, not how does two-way SSL work or name some specific configuration parameter for an Apache reverse proxy. I ask about OOP principals, data structures, enterprise patterns, Linux basics, etc., but if you do not understand how to bake in basic security in this day and age then I know you are not well rounded. These are not trick questions for a Sr. level developer.

Maybe you have an ax to grind, I don't know, but of course you can destroy someone by asking questions you know they don't know. This would be something quite immature and it doesn't benefit any one.


venkat murthy replied on Mon, 2014/09/22 - 10:45am

 All,

Going through all the comments on here.. However just wondering; if there are good templates on what to ask in Coding round for different levels of experience in developers. (say <5 yrs , <10 yrs). I understand there are several ds/algo problems around we can ask; however just trying to understand what would be a good mix of asking on DS/Algo, Coding Practice, Patterns and Idioms.

The reason why i am saying about a mix of questions here is: some of my experiences have been that few of the folks were able to clear coding round (over phone screen) with good to medium complexity algo/ds problems ;however they lacked something fundamental in Java coding practices, patterns etc.

Many times i wonder; for mid level experience folks, (rather than breathing on their neck) a problem should be given with time duration(say 2-3 days) and ask them to come up with solution. The solution typically expected is a brief document on approach, a maven package with unit tests etc.  This perhaps will show the depth to which the solution can be assessed such as Quality, Patterns, best practices, thought process in the design etc etc. Once you find that to your expectation; the same can be asked/grilled with different use cases etc in PI round.

Any thoughts.


Steven Goldsmith replied on Mon, 2014/09/22 - 11:29am in response to: venkat murthy

Venkat, if you are asking for an example project (Maven build, unit tests, etc.) then you can always ask if they have any open source projects on GitHub, etc. This is a good indicator that their passion extends outside work and you can glean some ideas about their code quality and sophistication. Typically, I do a phone interview covering general principals and concepts. Then during the in person interview a group approach is used (I may not even be part of that) and that's the time to see if there's a personality fit along with others asking technical questions. Also, if you put something on your resume I may ask you details about it if I'm familiar with it. If you have no clue, then I will suspect that your resume may have other fluff.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.