John Foobar is a 26 year old semi-technical analyst living in Austin who's sick and tired of Yelp. His favorite Mexican restaurant "Mi pato esta en fuego" has only received 2.4 stars! John wants to find food that he specifically will like, not just restaurants that are rated well. As luck may have it, John happens to have access to a massive amount of perfect data. So far, so good.
John does some Google searches and finds that a "k-nearest-neighbors algorithm" might be what he's looking for. He happens to have scored a free SQL Server license from an old barfly in exchange for listening to his tales about how wonderful life was when the mafia controlled New York, so naturally John searches for ways to implement kNN algorithms in SQL.
John finds out that kNN can be implemented efficiently using a spatial index, but that only works for two dimensions. That doesn't work because John really wants to judge places by location, price, and the ridiculousness of their dessert menu. So he spends a few hours trying to hack things together and finally posts an extremely specific question on StackOverflow about extending spatial indexes to 3 dimensions. He gets several answers that are technically correct but don't quite solve his problem and finally gives up, resigned to eat at popular restaurants for the rest of his days.
But if John had asked about the general problem of implementing kNN algorithms in SQL, he would have found that spatial indexes aren't the best method.
If he had asked about implementing kNN algorithms in general, he would have found that SQL isn't the best language for it.
If he had asked about creating recommendations based on like/dislike data, he would have found that there are better algorithms than kNN, and that there are several existing GUIs/APIs for these algorithms.
And if he had asked about the general problem of finding good food, well, his friends might have told him that Yelp already has a "personal concierge" mode and he just has to click a different setting.
Ha ha. I've been in that meeting -- the one where you have to explain that the requested feature would violate the laws of physics. I've found that it's best to keep asking some variant of "What do you really want?" in order to find out the underlying goal. This usually requires a couple of follow-up meetings though, because the guys doing the requesting don't always know what the real goal is.
Be careful with pricing though. They want something cheap, that won't work. They need something more expensive that will work. Persuading them may be frustrating, may take time, may risk you losing the job to someone less scrupulous than you who puts in a lower quote.
Are you going to be paid for the consultation involved in persuading them to use the better solution?
Compare two other industries: sign making and electronic sub contract engineering.
If you ask a sign maker to make a sign with a spelling error you'll get a sign with a spelling error. It's up to the customer to know what they want, and it keeps a bright line of expected behaviour from the sign maker.
Similar with sub contract electronic engineering. They build stuff to your supplied documents. If your documents are wrong they'll build the device wrong. If they really cannot build the device they'll get in touch. They are charging you for a service and that's what they deliver. They're not charging you for design work, and so they don't do that.
I solve this problem two ways: firstly by being very expensive, charging several hundreds of dollars an hour; secondly by only working on a time and materials basis. This won't work for everyone.
Fixed priced projects only work in a few cases where you know exactly what you are building. They are a lose-lose situation otherwise, with the client usually being the loser.
However, most clients will only agree to pay for building what they need after you've first built what the want.
Very few businesses are interested in getting paid once if they can get paid twice, so this pattern is perpetuated to the mutual benefit of everyone except the developers forced to spend 50% of their time building something they know won't work, and the other 50% rewriting it...
Both stories share a similar problem: a designer (or engineer, it doesn't really matter for this story) is brought in because their expertise is needed. Then, rather than trusting that experience, the thing is micro-managed to the point where it is ridiculous (or, as The Oatmeal says, you are just a cursor in a graphics program, controlled via email). I feel that with design this happens more, because it's more tempting to believe that their own user experience is representative of anything more than themselves (the first thing a proper designer knows NOT to do) and it's easier to get into bike shed discussions (http://bikeshed.com).
I'm bookmarking this, because I might need it in the future.
However, I think the message should be not be (as many engineers may think) that non-engineers are unable to grasp basic technical facts, but rather that engineers are lousy communicators.
Success in business depends on effective communication, especially between people working in disparate specialist fields. Being able to adjust how you explain a subject to the knowledge level of your audience is, unfortunately, a skill that is not commonplace amongst (software) engineers.
I'm a Presales Engineer, and a big part of my responsibility is to translate what engineers are saying into something customers can understand, and vice versa. Over the past 4+ years I've been at this job, I have come to realize that this is a very, very rare skill. I don't think it has much to do with being able to communicate (English isn't even my native language!), but rather being able to think in abstract terms and recognize patterns among different ideas.
One neat trick I learned over the course of my career when talking to non-engineers is to use analogies, especially car-related ones. For example, trying to explain a named user licensing model to someone who has never dealt with software licensing before will be a nightmare. But if you say "it's like having assigned parking spaces, where every parking space is assigned to an individual," people just get it. Whereas an engineer will describe how the model works from a software perspective, and the non-tech person will just bang their head against a wall thinking themselves as stupid.
Very interesting advice re the car analogies. Thank you.
It seems that you are the character in Office Space* that had to madly defend his job to the two Bobs. In my real world development experience, I find that you are right about the rarity of your skill, and am always thankful when someone around me has it.
I don't think you understood that scene from Office Space correctly.
In my opinion it was supposed to be ironic. He is supposed to be communicator but he can't communicate his importance and he is showing very bad communication skills. After all it was taken out of context.
Actually, come to think of it, TFA provides some good ones. If a customer has contradictory requirements, just tell them they're asking for a line that is red and green at the same time.
You are absolutely correct. I spend my day solving problems in software and not converting what's in my head to English that a non-software problem solver will understand. So when I have to do the latter, it takes time.
Funny thing is, this goes both ways-- Mr. MBA also doesn't know how to explain a subject with which he is intimately familiar in a manner friendly to people without his knowledge. Yet, anyone who doesn't grasp his latest master plan must be an idiot.
Stereotypically speaking, I find that most techs tend to have a bit more empathy when it comes to explaining things than the business people have.
If you are in charge of widget production, you should absolutely understand the basics of widget production. Don't blame the widget-specialists you have hired to produce widgets if you are not putting forth any significant effort into understanding your own business.
We should have specialists in communication, and pay them a bucketload. They could also learn all those people skills. They'll effectively be managing the projects, so we can call them the project managers.
No, because they are not technical specialists, and they rely on the technical people to tell them what is and is not feasible, and why, and how much effort something requires.
The author is clearly a good communicator, even then he can be frustrated by coworkers who dismiss what he has to say because of the stereotype that engineers are lousy communicators.
I dont even think it's a case of pitching to a given knowledge level, it's a question of understanding what is important to the other person and expressing yourself in terms that makes sense in that context.
The article is written in such a way as to highlight Smith's dismay and frustration due to his colleagues' questions and suggestions. As a reader you are left with no idea why the Redroot does not see that a line cannot be red and green at the same time.
In practice this is exactly the bridge that Smith must cross otherwise the meeting ends in disaster. To do this it's not just a case of speaking in simple terms (though this is often effective), it's a case of figuring out what the other people will consider to be 'simple', in practice this /could/ be something quite complicated - especially to Smith since he is not used to thinking in this way.
You forgot the part where Smith attempts to explain in great detail and with an inevitable reference to Euclid's fifth postulate how 7 perpendicular lines (which are also referred to as "orthogonal" he might add) are only possible in 7-dimensional space, and that in fact, this can be extrapolated to say that there is a one-to-one relationship between the number of spatial dimensions and the maximum number of orthogonal lines which could conceivably be drawn, but that since we are constrained by the 2 dimensions of the screen and there is no simple method for displaying spatial objects of dimension > 3 in 2 dimensions, for all intents and purposes the display of 7 perpendicular lines is not feasible.
John does some Google searches and finds that a "k-nearest-neighbors algorithm" might be what he's looking for. He happens to have scored a free SQL Server license from an old barfly in exchange for listening to his tales about how wonderful life was when the mafia controlled New York, so naturally John searches for ways to implement kNN algorithms in SQL.
John finds out that kNN can be implemented efficiently using a spatial index, but that only works for two dimensions. That doesn't work because John really wants to judge places by location, price, and the ridiculousness of their dessert menu. So he spends a few hours trying to hack things together and finally posts an extremely specific question on StackOverflow about extending spatial indexes to 3 dimensions. He gets several answers that are technically correct but don't quite solve his problem and finally gives up, resigned to eat at popular restaurants for the rest of his days.
But if John had asked about the general problem of implementing kNN algorithms in SQL, he would have found that spatial indexes aren't the best method.
If he had asked about implementing kNN algorithms in general, he would have found that SQL isn't the best language for it.
If he had asked about creating recommendations based on like/dislike data, he would have found that there are better algorithms than kNN, and that there are several existing GUIs/APIs for these algorithms.
And if he had asked about the general problem of finding good food, well, his friends might have told him that Yelp already has a "personal concierge" mode and he just has to click a different setting.
Poor John.