He's not only filing the bug, but also attempting to reproduce it. In the vein of doing things that don't scale, this idea is a clear winner.
It does seem like a valuable service: The project would get a verified/reproduced bug report from a known entity, and the reporter would get their issue addressed just by sending an email.
Under/over on how long before he can't offer this for free?
I like this. Less so for his actual offer, but more because he is calling attention to how hard it is to do this for people who don't want to deal with a "bug tracking system... optimized for people who spend all day every day working with it".
It would be nice if there was a single, easy to use site for mere mortals to enter bug reports (and others can comment on them, vote on them, process them, etc).....one sort of like Github or StackOverflow, where we all sort of agree to use that site.
> It would be nice if there was a single, easy to use site for mere mortals to enter bug reports (and others can comment on them, vote on them, process them, etc).....one sort of like Github or StackOverflow, where we all sort of agree to use that site.
Where they exist, they promptly get overwhelmed by all kinds of stupid shit. Then the devs have to filter through insane amounts of crap and sundry sorts of spam for the few things that are actually useful bug reports. Google code had this going on for a while, if memory serves.
With this in mind, the effort required to file a bug is not so much a bug as it is a useful filter. Yes, it cuts down on the actual bugs filed, but it cuts down far more on all the crap you don't want.
In my experience the "stupid shit" and "crap you don't want" is often useful feedback. At the very least, it tells you which parts of your app are most confusing.
For example, if you get lots of complaints about something you consider obvious, maybe you should reconsider what's obvious. Often minor changes like improved docs or more googleable error messages can improve the experience for many users. And you'll see if changes are effective as soon as a specific sort of "crap" stops showing up in your bug reporting system.
Except when tons of emails come in that can't even coherently describe the part of the program that doesn't work or the approximate process they are attempting.
You can always ignore these incoherent emails. But if you put up some kind of filter before people can contact you, it won't be your decision which feedback you ignore.
And, if you really try, you can often extract valuable feedback even from what seems to be incoherent babble at first.
You can, but sorting through them has costs measured in time. Filtering through dozens of drug report and smug reports for a useful bug report is not instantaneous.
When your entire project is just you in your spare time or maybe a handful of volunteers, that filter is the difference between support being helpful and support being a useless hellhole. And believe me, when the vast majority of support requests are useless crap, you lose motivation to care about those people with truly impressive speed.
> It would be nice if there was a single, easy to use site for mere mortals to enter bug reports (and others can comment on them, vote on them, process them, etc)
Having worked for Mozilla for 5 years (Zack also is a former Mozillian), I actually don't think that the root problem is the ease of use of the bug tracker.
Although bugzilla sucks, we still managed to get a ton of bug reports from external folks. But getting the bug report is only the first step. Once you have it, you need to, at a minimum:
a) Figure out what part of the software is causing this bug. For complex software like a browser, figuring this out often requires deep technical knowledge of the system, so is expensive time.
b) Figure out if the bug is already known. This is again often nontrivial because one bug can manifest in ways that may look superficially dissimilar, so requires an expensive person's time.
c) Collect additional information from the reporter. Most bug reports from muggles don't have enough information to let you reproduce the issue without additional back-and-forth. Since the bug reporter probably doesn't live in bugzilla like Mozilla folks do, they're usually slow to respond, if they respond at all. I've had bugs I worked on a week only to have the original reporter disappear. It's understandable; all this communication takes a lot of work.
d) Reproduce the bug. If you can't reproduce, go to (c).
e) If it's a regression, figure out when the bug was introduced.
f) Figure out if the bug is fixed in alpha/beta/nightly builds.
This is all before you actually try to write any code.
Every step here is more difficult and risky when the reporter isn't a known entity, so as a result, if Zack or I file a bug against Firefox, all things being equal we're much more likely to be paid attention to than if a random person files a bug. It's just a basic cost-benefit analysis on the part of people who already have too much to do.
I'd say that 80% of bugs I fixed while I worked on Firefox were identified by Mozillians (either employees or regular contributors). Which isn't to say that the remaining 20% weren't important -- they were, which is why I fixed them! But those 20% were a lot harder and riskier to fix, and that's not something I can imagine the best bug tracking software in the world ameliorating.
> I'd say that 80% of bugs I fixed while I worked on Firefox were identified by Mozillians
How much (if any) of that is due to bugs being somewhat more likely in prerelease builds, and Mozillians being much more likely to be dogfooding those?
It would be nice if there was a single, easy to use site for mere mortals to enter bug reports
Many bug trackers suck in many ways, but the intractable problem is that reporting and tracking down bugs is hard. Each bug takes a lot of work to reproduce (sometimes days of work), and the farther you are from the person who saw the behaviour, the harder it gets to reliably reproduce (their combination of hardware, software and human behaviour triggered the bug after all). Bug trackers are an attempt to transfer some of that cost to the reporting party, and to make reproducing the bug easier. The solution you are proposing just transfers it all back to the developer, though I like the idea of crowd-sourcing bug finding, in many ways this is what github does already.
In addition to that bugs are intermingled with features in the user's mind - what they consider a bug (why doesn't it do this), would often be considered a feature request by developers. What they consider a bug might even be considered a feature by the developers or other users.
I think attacking those problems would move the art of bug reporting further on, rather than making life easy for end-users - reporting a bug is already very easy to do on github say, it's just not easy to do it properly so that your report has a chance of being listened to.
I think a much better way would be to just post a public email address for reporting bugs. That's what I'm doing for my (closed source) apps, and it works extremely well.
Even people with limited computer skills know how to use email. This ensures you get the maximum number of bug reports.
When you personally reply to emails (no automatic responses), people are extremely helpful. When I can't immediately reproduce a bug that a customer reported, I ask them a series of questions, and they almost always respond quickly. If you send clear instructions, you will get useful information even from people who have no clue what they are doing.
If you force people to fill out a lot of unnecessary info beforehand, you'll get less bug reports. If you send them automatic replies, they wont respond.
That works for small projects, but doesn't scale brilliantly. At some point, many projects need to handle more bug reports than one person can realistically handle in their spare time, and totally unstructured freeform text slows things down on top of that.
If it becomes too much work for a single person then you just find a second person. When that becomes too much you can find an entire team of people, all contacted by a single email address. This is how customer support is done.
Everybody is more satisfied this way, users get quick feedback that their problem is being looked at, developers get enough information put into the bug tracker to be useful in fixing the issue and the number of useless bugs to wade through goes down immensely.
Most projects and companies don't have the funds to sponsor that level customer support. The modern Internet economy works so well because marginal cost (and price) is tiny. Customer support takes that a way.
Firefox has a feedback feature (in the Help menu). Mozilla watches the flood of feedback for keywords and correlations to identify bug regressions. And, in Mozilla fashion, the anonymous feedback and ugly truth is open for all to see:
Isn’t it usually considered your distribution’s responsibility to handle that? They may have done god-knows-what to the software, so the bug might not even appear in the upstream release.
nice. however a more scalable approach would be a metabug site where people could post "it doesn't work" freeform text, and volunteers could accept a bug and resolve it when they had refiled it with the correct project.
I agree that JIRA is probably not what you want as your user-facing bug tracker, but Bugzilla seems significantly less complicated, and the github issue tracker is easier still (though I think it is over-simplified for a user-facing bug tracker).
Someone above mentioned wanting free form bug trackers, but this runs into the problem of bad bug reports. You want bug reports that you can reproduce, or at least some artifacts such as crash dumps or stack traces that give you something to analyze. Short of that you're left guessing about what the problem is. Hence, bug trackers tend to ask people for more information up front so that there is less back and forth.
Normally, I think of ways to increase involvement with OSS communities.
He's helping people, and probably learning in the process, which is great. But it would be even better if he could add a touch of "help people help themselves"; otherwise I'm concerned that the OSS process will feel a little more sterile.
I think providing a lightweight action->result cycle is just as or more likely to engage people as a process that requires a deeper involvement before there is any chance of a result.
Takes <2GB ram and doesnt cost a thing when starting up on SSD, so why not? Better than bookmarks ;-)
On a serious note its the morning rush, goes down to ~30 by noon. I dont let stuff control my timing (like TV used to), instead I open everything in the background and read/watch at my own pace through the day (between coding/soldering).
It does seem like a valuable service: The project would get a verified/reproduced bug report from a known entity, and the reporter would get their issue addressed just by sending an email.
Under/over on how long before he can't offer this for free?