Hacker News new | past | comments | ask | show | jobs | submit login
A better way to profile multithreaded programs (jvns.ca)
80 points by jvns on Nov 1, 2015 | hide | past | favorite | 10 comments



As someone who has written an APM profiler for multi-threaded, async .NET web applications I can attest to how hard it is. There are actually a few ways to do async type methods in .NET and supporting all of them was even more fun.

Begin/End older style method delegates

async/await with normal tasks

async/await with TaskCompletionSource (aka fake tasks)

and of course completely custom ways in 3rd party libraries

Crazy hard profiling magic, FTW!

FYI, I wrote the .NET profiler for Stackify (http://stackify.com) as part of our APM solution


Author tries to optimize for execution time (finish program fast). But most users want to decrease total CPU resource consumption, because most tasks runs in parallel on server. In second case traditional hotspot profiling works just fine.


That can definitely be true, but note lots of things are latency-, not throughput-bound. In particular, microservice-based systems are likely to hit 99% latency of some service on (many/most/all) requests; I'd expect that a similar trick would be helpful there.


Actually, Coz lets you profile for latency and throughput. That's one of its main points.


Nice review of an interesting paper.

Aside: How did you design the ribbon? I looked at the triangles on each end of the nav bar, and it looks like you've done that by styling the ::before and ::after section with a top and bottom margin respectively, which because it's zero content makes a triangle. How on Earth did you come up with that? Is that a standard technique?


Not really standard, but also not unheard of. Google "css only shapes" for examples.

Top hit is https://css-tricks.com/examples/ShapesOfCSS/, which has several examples of before: and after: use to create, for example, a five-pointed star.


I'm getting into concurrency from living a calm life where blocking the main thread was a thing.

What are HNers experiences with different models for concurrency and inter-process communication? To me, something like go and clojures channels (actually CSP) combined with green threads seems like a really neat solution.


Or Erlang/Elixir, which I found simpler than Go (though that may be due to my experience with Erlang, and my tending to think in Erlang ways, and it not quite matching Go's). Certainly, the guaranteed immutability, better pre-emptive scheduling, supervision, easy distribution of processes across multiple nodes, and better process isolation meant I felt my code was more reliable than with Go.

EDIT: Since you didn't mention Erlang, just to give you a point of reference, Erlang message passing is directly between lightweight processes, rather than an intermediate channel like in Go. I've heard people reference it as files vs file descriptors, but for myself, I think of it architecturally. Go creates an abstraction for the message type, that is, create a channel for ~this~ kind of message, and pub/sub to it as needed, and you shouldn't have to care about who is subscribed to determine what channel to publish on (it's enough to say 'it's this kind of message, so it goes on this channel'). Erlang, on the other hand (and Elixir) send messages directly to different processes, no additional abstraction. As such, you end up thinking about who needs to communicate with who, and how best to do that. So by default, it's like having a channel with exactly one known listener. Of course, you can easily build a channel like behavior, by creating a process whose sole job is to collect messages, and determine how to federate them out to other subscribed processes. I prefer the Erlang way because oftentimes the level of abstraction Go gives is unnecessary (and sometimes even a source of bugs, where I did a poor job of defining my messages, and not thinking about who was listening to the channel meant that a process got a message that it should not have, because of another, similarly typed message that it should have; I needed two channels, but created only one), but when it's useful I can always easily build it in.



Coz co-author here. Feel free to ask us any questions.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: