Great post. Thanks for the praise of New Relic as well.
I'd love to get your feedback on what use cases you had where we were not sufficient. Given that we've been around for 3 years and this is the first release of your profiler, it's not fair to compare features, but I'd still love to know what metrics tipped the scale toward building this.
Three huge features we offer that it seems like you're currently missing out on are automatic profiling of slow requests (vs. manual profiling), automatic real user monitoring to provide insight into the end-user experience, and automatic instrumentation (vs. stitching profiling code into your app).
We totally agree that it would be nice to see web frameworks embrace the concept of performance metrics and provide hooks that tools can use. Rails is way ahead of the curve here. Starting with version 3, Rails ships with plenty of hooks across various aspects of the models, views and controllers.
The big advantage we have is "production ubiquity". As a "trusted user" navigates through the site - in production - she can see the time it took to render the page ... on the page ... this constantly updates with all the ajax calls, which is an innovation. At any point the user can click on the time and break it down into little bits, without navigating away. You can see it in-action at http://data.stackexchange.com just log on.
Monitoring end user experience is a VERY cool feature. It ties in to what I like to call "birds-eye" profiling. This is not at all the problem we are trying to solve with our profiler. In our context we get that same information from the chrome dev tools and firebug. When it comes to "birds-eye" profiling, you guys are innovating and leading. I can not think of any competing products that have a more complete feature set.
Automatic profiling in almost any platform is real tricky business in production. It gets even trickier in statically compiled apps. In an ideal world I would write code like the following:
def bla;
fast_method;
fast_method;
slow_method;
fast_method;
end
My "ideal" production profiler would automatically "know" to instrument "slow_method" and forget about the fast ones, this would all work automatically in production. Technically this can be achieved in a .Net app, but is very tricky. You would have to spin up a new appdomain instrument say 3% random calls using ICorProfiler and route a portion of the traffic to it, then you would have to cycle and do another 3% and so on. At the end of the process you could use something like mono cecil to dynamically recompile the app with all the "interesting" spots profiled and re-route all the traffic to it. The trouble is, the set up is way too elaborate. In a Ruby app this would be way simpler to do and in fact somebody could build something that does something similar ... today.
The trouble with ICorProfiler on a site the scale of StackOverflow, is that it introduces stability issues which are unacceptable, particularly when CPU is way high. We can not afford to use is for the constant "production" profiling.
Now, there is a bunch of profiling you can stitch up more simply like, time in controller, time in view and time in db. But, the .Net framework is way behind on providing the hooks there, sadly.
All of this said, I must admit I have not tried NewRelic in the last 6 months, and would be happy to give it another shot and send you through some of my ideas, you are doing real important work that is making the internet a better place.
Out of interest why did you go with google code rather than github? Google code doesn't have any good mechanism for submitting patches and contributing to the code like github does.
"Step" is a static-call to an extension method. This way allows your code to keep working fine (unchanged) for users that aren't profiling (i.e. 99.99% of the users).
I meant more the "it's ok if this is null" part. Does that comment not mean what I'm thinking it means? Because invoking a method on a null object will throw an exception.
In the case of an extension method, it's actually just syntactic sugar the compiler provides. An extension method is actually a static method underneath, this allows null checking to be centralized, and cleaner everywhere a step is used. The method looks like this: "public static IDisposable Step(this MiniProfiler profiler, string name, ProfileLevel level = ProfileLevel.Info) { return profiler == null ? null : profiler.StepImpl(name, level); }"
Nice. I've used extension methods from time to time but not in this way. Could prevent you from writing if foo != null a lot if you have an extension method by doing: public static bool Exists(this Foo foo) { return foo != null; }
So then you can say if(foo.Exists()) { foo.bar(); }
Or make Exists return an instance of Foo (foo if not null, a new Foo if not) and go with foo.Exists().bar();
