One of the biggest pieces of advice I have for junior devs (and software engineers in general) is actually quite simple: Remove the phrase "it works for me" from your vocabulary.
The rationale is equally simple: I wouldn't take the time out of my day to send a bug report if it "worked for me." The problem could be any number of things: buggy code, incorrect instructions, browser incompatibilities, race conditions. Who knows?
You, as an engineer, should always always always assume that something is broken until proven otherwise. You must assume that there is a bug, until you can prove that there is not. Once you have ruled out a code error, then you can step into assuming the user is wrong. "OK, I did it the right way and was unable to reproduce it. Therefore, the user must have done something incorrectly."
Now you either try a bunch of different ways to accomplish it, or you go back to the user and say "I'm unable to reproduce this. Can you elaborate on the sequence of actions you took to cause this error?"
But what you should never, ever do is just dismiss the customer's complaint as invalid because you were unable to recreate it. If you can't recreate it, then you need to figure out why. Was the bug fixed by something else? Was the user mistaken? Does it only happen on Tuesdays at 3pm?
I'll say it again because it bears repeating: Remove the phrase "it works for me" from your vocabulary. From a reputation standpoint, it's bad news.
The customer isn't always right, and you don't have to blindly believe them, but customers don't report things unless something isn't working for them. Trust that.
If an organization is having developers triage vague bug reports directly from end users, they're already in trouble.
Not only are developers a very expensive resource, they're generally not good at doing what you describe here. Nor are they usually well set up for it as their systems/devices are often tainted with development and debugging tools.
We can't make end users deliver bug reports that are detailed and reproducible, but (whenever possible) developers should only have to worry about reports that have those qualities.
Any organization that's grown beyond a couple founding engineers needs to have a layer of QA or Customer Support that's responsible for everything that you describe. It's a layer that not only pays for itself but also keeps both users and developers happy.
I think you're taking my post a little bit literally. The "customer" doesn't always have to be the end-either. If a QA member comes to me and says they found a bug, I also should not just go back to them and say "It works for me" and then not proceed any further. (Although, in defense of the literal interpretation of "customer", for small startups, devs often double as customer support for engineering-related issues.)
There's a real difference between a Customer and a QA member, though. You can't demand leave the responsibility to make a good report on a customer because they're external to your organization. But you can do exactly that with the QA member because it's precisely their job to identify and describe defects.
You can indeed say "it works for me" to the QA member, and it's their responsibility to identify the environment or series of steps that reliably produce a failure. From there on out, you can no longer say "it works for me" because it doesn't: you finally know how to make it fail and now it's your job to figure out why.
And like I said, nascent startups with just a few founding engineers are a necessary exception. But the QA or Customer Support hire should come very early! Too many organizations stall on that, and they waste tons of opportunity and productivity by doing so.
I remember reading about a company (game company I think) that got tired of unreproducible bug reports, and added an automatic RAM test to their software. The number of bugs that could be traced to RAM faults was significant.
I wish I could remember who it was, but my Google-fu fails me today.
"It works for me" is not an answer, but it is the first step in a reasoning process. "It works for me, it clearly doesn't work for the bug reporter, so what's different between their setup and mine?" That then leads to questions about steps to reproduce.
> "It works for me" is not an answer, but it is the first step in a reasoning process.
Exactly. "It works on my machine" has over time become the easiest way to get a death glare out of me. To rehash the old quote, we are not shipping your machine!
"It works for me" is a great first step. More specifically, it works for me on Windows XP, Vista, 7, 8, and 10 TP (since they're all ready in virtual machines) is an even better one. Followed up with the usual questions of What operating system are you using? 32-bit or 64-bit? What's the full install path? What antivirus product and version is running? What software firewall, if any, is running? Any other apps running? New install or upgrade? Etc. I have some of this built into the About box of one of my apps and am adding in the rest shortly.
Quite a few "bugs" are no fault of the developer/publisher. But learning where those issues are and letting your app alert/inform the user what's going on as they install to an invalid location or break their TEMP directory thinking they're protecting their machine or remove IE from Windows using a broken 3rd party utility or use an antivirus product that gets more wrong than right, etc... can help cut down on the bug reports, help the user from making 'mistakes' they shouldn't be allowed to make, and just make the whole process easier.
That is nice in theory, in practice many, many users are stupid (for the lack of a better word) and the bugs they report are exactly how the program is supposed to run, or caused by things that we cannot reasonably deal with, typically client computers that are too underpowered and have too much adware installed.
it isn't that black and white. Many times it's simply a misunderstanding, the developer isn't actually testing the right thing. Saying "works on my machine" is the opening to a dialogue that involves "can you show me what you did?"
Are you implying it's a matter of getting your customers to use their brain?
I freelance and therefore I can be a jackass without direct monetary repercussions, perhaps that's why I don't view it that way.
Because I struggle to think of a lot of situations where I've been unable to replicate an issue a Client has reported where it hasn't turned into a miscommunication/misunderstanding of some sort.
No, not at all! If you've been in the business long enough, you learn that people have different preferred ways of working. Some will use menus, some will use icons. Some will use keyboard shortcuts. Some will use drag-and-drop. If the bug description doesn't specify how they performed an action, and you assume your preferred way while they used another, you can easily be unable to replicate their problem.
I don't remember any specific instance where I had this misunderstanding, but a coworker had it happen and will never forget it.
In the context of web development, one of the biggest offenders (in my experience) is coders who attach a "validate this form/submit it via AJAX" handler to the form submit button's click event, instead of the form's submit event. If I try using Enter to submit the form, their code doesn't run.
I misunderstood you, I thought you were implying customers needed to use their brain, but looking closer you did say keyboard and mouse and not keyboard and chair :)
I personally see it all the time, but I interact directly with my customers and it's always possible I'm terrible at communication. They'll say things like "X doesn't work on page Y", and It'll end up being a problem with a single link they clicked on because the data for it is such that it causes a problem (that sort of thing).
They don't know that's the issue either, you have to work with them on it.
99999 out of 100000 yes. Sorry but most bug reports, even from QA departments are noise at best, nuisance at worst. And as I posted here before: I am close 40, have 15 years experience as a professional in this industry, I have seen it all and yes it gets harder for you to break through to me because I will presume you have no clue what you are doing because so many before you didn't. Sorry if you are the one special snowflake.
This is not to say my software is bug free, of course not! But unless you can describe the steps you made clearly and also have an expectation/happened instead , your bug report is useless. Again, this doesn't mean I expect reproducable bug reports but I need something solid to work with.
Your experience certainly doesn't match my experience; I've worked with brilliant QA people who were able to diagnose problems both when deviating from spec and when performance is a feature that isn't being properly met. And when someone has not been up to snuff, in almost every case I've been able to teach the people you arrogantly dismiss as "noise" and make them better able to help us all create better things through that education.
Your attitude, however, does match my experience: it is why people think software people are assholes. So be more mindful, please, from up there on your high, high horse. For yourself and for us as a profession and a craft.
[chx] > Sorry but most bug reports, even from QA departments are noise at best, nuisance at worst.
[eropple] > Your experience certainly doesn't match my experience; I've worked with brilliant QA people who were able to diagnose problems both when deviating from spec and when performance is a feature that isn't being properly met.
Hey, thanks for this. When I got that really weirdly hostile reply I was really confused as to whether I'd missed something somewhere, but this (and reading mreiland's posting history a bit) helped clarify.
Deciphering bad bug reports is absolutely a waste of your time as a developer, but that doesn't imply that the user is mistaken. In nearly all cases, the user is correct in sensing a bug but incorrect or inarticulate in their description of it.
It's almost always a good idea to understand what's behind the report. It's just that somebody in a different role should be doing most of that work.
Small startups are often just a handful of people, and devs can and do pull double-duty as QA and development, and sometimes customer support for engineering issues! Freelancers also don't have customer support teams. For larger orgs, however, you're right. And also, like I said in another comment, the "customer" doesn't have to be your customer -- your "customer" could be the QA guy who says he found a bug, or your boss, or another engineer. Whoever tells you they found a bug, you should not dismiss their report with "well, it works for me."
If there's one thing I can't stand then it's people spewing fantasy numbers out of their ass without anything to back them up in order to give their argument some conjured up weight. You may well be right that a majority of bug reports are unhelpful, but please, if you need to use numbers, use ones you can back up with something.
Sorry to hear that OP's bug reports weren't taken seriously. Good tech support is a must for mission critical APIs.
At HelloSign we take great pride in our API and our tech support. Rather than outsource support or rely on less technical people, we have our developers directly support the API they built. So when you give our API a try, I encourage you to check in with one of our developers by visiting our public HipChat room (https://www.hipchat.com/gq4BMFKt1) or emailing them directly at apisupport@hellosign.com. And that's available to all customers no matter what pricing tier they’re on (including free).
We’ve had a lot of API customers switch over to us from EchoSign’s API due to reliability issues. I have a lot of respect for EchoSign and their original CEO Jason Lemkin, but I do think the API was an afterthought for them. At HelloSign we think it’s the future and are committed to having the best-in-class API.
I also wanted to touch on your point about wanting to work with "a company with a soul". This is something I also look at when making buying decisions. I’ve found a company's Glassdoor reviews to be an effective barometer of how well a company treats its employees and, by extension, how its employees treat the customers. That’s one (imperfect) way to measure "soul".
Happy to answer any questions anyone on HN might have.
HelloFax has a no-subscription tier that lets you send individual faxes at $0.99 each. If you're like me and send only a few a year, this is a great option.
I've been using HelloFax since shortly after they launched, and it just works (TM). Highly recommended.
One of the best parts of working with Hellosign is their amazing tech support team. They are incredibly responsive (and nice!), and have helped us troubleshoot issues on many occasions. We are on a first name basis with these guys, and would recommend working with them!
> Rather than outsource support or rely on less technical people, we have our developers directly support the API they built.
Then you're not paying your developers enough. A developer commands a much larger salary than tech support. You're either underpaying the developers, overpaying techs and calling them developers, or you're not being completely honest in an effort to pull in more customers.
I'd go with 3, but whichever it is, I don't trust anything you've just said.
Our approach may not be scalable (and may not fit into your worldview) but having our developers do API support for us serves two important purposes:
1. Giving our clients the best possible technical support
2. Making our developers feel the pain (and joy!) of clients using what they built. I can't think of a better way to motivate developers to fix what's sub-optimal, share in the success of a job well done, and come up with new ideas to help our customers.
I think your intuition is correct when you suggest that this approach isn't scalable. It runs counter to the concept of "flow", articulated in Peopleware[1] 30 years ago and confirmed many times since then.
Sharing the pain has benefits, but you're almost certainly paying more in lost productivity on both bugs and features than you're gaining in insight and sensitivity. As a small and stable company, you can probably afford the loss as you learn more about your market and as you prepare for more organizational complexity, but it will eventually inhibit your growth and burn out many of your developers.
That's a fair point. The one constant in a startup is change. It may be that in the future we'll have developers only doing front-line support for beta features of our API.
Regarding flow, we don't have all of the developers doing support all of the time. We have one developer doing support at a time and rotate each week. This protects the rest of the team from having their flow I interrupted.
More than that, the links he gave talked about the entire company doing support, not just the developers. In other words, he misunderstood the point being made.
Putting the responsibility for tech support on your development team is how you increase your turnover rate.
Personally, I don't believe he's really doing that, I think it's just marketing.
Not recommended. Adobe Echosign allows NO WAY (at least no easy way) for regular users to export signed documents in a single batch. I've been a paying customer for 6 years. When I asked their tech support where this (surely obvious) feature was, they said that I must have an Enterprise account to export a batch of my own documents. For an account like mine with hundreds of signed documents in it, that would take hours. It took 27 (!!) messages with their support technicians to get a batch of my own documents. Sadly, it seems with the acquisition by Adobe, any hustle, humility, or innovation on the part of Echosign is out. Best of luck to Hellosign to make up for the slack.
What is your complaint about? They have a feature available but costs money which you just don't want to pay?
Counting on them to just "be nice" and do it for you anyway isn't really a valid option, nor something to base your objection on. If you don't like the service and aren't happy with the price then move on to someone else.
It's not a "computer" - it's a document signing/management service, and you could get your data out just fine. The person wants more functionality, which is actually available as a feature, but it costs money. So, pay the money and you get access to the feature.
offtopic, but "plodding craftsmanship" is a more attractive job descriptor than any number of "ninja", "rock star", "changing the planet" and other such overinflated and overused terms. reading between the lines, it means that the company values both doing things right and taking their time to make sure they've done it right, which is the exact thing that both drew me to and keeps me happy in my current job.
> ... the solution is to reinvent Adobe: to change the system from within. If Customer Success could talk directly to Product Engineering, then the bugs would get fixed, and the volume of complaints would go down, making everyone’s lives easier.
That process is enormously beneficial. I run an open source project, and spend a good chunk of my day answering low-level tech support questions. These are often "where is the ANY key". But a high percentage of them are people with real issues. I listen, fix the issues, and give feedback.
The result is enormous customer satisfaction. They get responses (and fixes) in 15 minutes. Their previous commercial solution would be 6 months, if they were lucky.
Shortening the feedback loop is a well-known engineering principle. But apparently not known enough in software companies. For systems with negative feedback, low latency is very positive.
More companies should have engineers deal with customer issues.
This is a perfect example of why the open source programming model (licensing concerns aside) is superior to closed source when it comes to any mission critical program. You cannot let your entire business rely on a single company's buggy software.
As a user, having to file a form using EchoSign, I would to cringe. It requires flash and has some really bad usability. An example of that is requiring at least two non-standardized date formats. UX is important for forms because humans make errors when information is not clear.
You should try AssureSign [1]. I know they have "programmers somewhere inside, who care about the product enough to read and respond to bug reports" because I'm one of them :) Support tickets go to everyone so the person most qualified to answer can do so directly without navigating support tiers first. This can sometimes be a drag on my personal productivity but I know customers (especially technically minded API integrators) appreciate it.
The rationale is equally simple: I wouldn't take the time out of my day to send a bug report if it "worked for me." The problem could be any number of things: buggy code, incorrect instructions, browser incompatibilities, race conditions. Who knows?
You, as an engineer, should always always always assume that something is broken until proven otherwise. You must assume that there is a bug, until you can prove that there is not. Once you have ruled out a code error, then you can step into assuming the user is wrong. "OK, I did it the right way and was unable to reproduce it. Therefore, the user must have done something incorrectly."
Now you either try a bunch of different ways to accomplish it, or you go back to the user and say "I'm unable to reproduce this. Can you elaborate on the sequence of actions you took to cause this error?"
But what you should never, ever do is just dismiss the customer's complaint as invalid because you were unable to recreate it. If you can't recreate it, then you need to figure out why. Was the bug fixed by something else? Was the user mistaken? Does it only happen on Tuesdays at 3pm?
I'll say it again because it bears repeating: Remove the phrase "it works for me" from your vocabulary. From a reputation standpoint, it's bad news.
The customer isn't always right, and you don't have to blindly believe them, but customers don't report things unless something isn't working for them. Trust that.