> When I started this project in January, gdb failed on every program I tried it on. delve didn’t work on OS X, and print-statement-debugging was too slow and limited. What's a developer to do? Make my own debugger, of course.
It seems like he tried to use delve and it wasn't portable at the time. Sure he could have worked on making delve portable instead, but he probably learned a lot more poking around the internals himself.
This looks really great. When I saw how Go does coverage testing using the same approach I felt that the code generation approach would work for a debugger. I'm glad someone did it - and from the few moments of playing with it it seems to be working great. I'm adding this to my toolkit for sure.
This demonstrates the power of Go generators. We're seeing what are typically language features such as generics and debugging being implemented before runtime using code generation. The result is more functionality without an overall increase in runtime size or decrease in runtime performance.
>
This demonstrates the power of Go generators. We're seeing what are typically language features such as generics and debugging being implemented before runtime using code generation. The result is more functionality without an increase in runtime size or a decrease in performance.
Monomorphization is a bog-standard implementation technique for generics. It's not at all unusual.
I was hoping for some article or wikipedia entry that would tie your comment with the parent's comment, and with the original article. Monomorphization is, to my understanding, taking a polymorphic function and instantiating it to the specific type requested. This is done at compile time. Now, the tool in this article interleaves debugging code with original source code. So the connection is... both monomorphization of generics/templates and this tool operate by emitting source code prior to compilation?
The original comment was about how both debugging and generics are being implemented with code generation before runtime in Go and described this as "unusual." The response was that this precise technique is how generics are implemented in many languages already; in fact, CFront (the original C++ compiler) was entirely implemented as code generation on top of C. The response specifically discussed generics, not debugging, so I'm not sure why you thought it was talking about debugging.
"that's a necessary cost that comes with the debugging capability"
I'm doing some research that shows this isn't the case. You can have a system where enabling the debugger has zero impact on runtime performance, via dynamic deoptimization.
They're not the same thing. This is about code generators (https://blog.golang.org/generate) while in Python context you're most likely talking about iteration.
What do you do for inlining + code movement, where functions become interleaved (as do lines)?
In particular, after inlining, how are you guaranteeing statements from a given function don't get moved before the inlined enterfunction call (and similarly with lines)
Or do you not expect it to ever be able to report right answers for optimized programs?
(which is a valid way to live life, of course, but ...)
Since it's modifying the source before compiling it, I expect that the compiler will conclude that most optimizations can't be applied when they cross breakpoint boundaries.
So, when debugging is turned on, the code would return the right answers, but it wouldn't have the same performance.
"Since it's modifying the source before compiling it, I expect that the compiler will conclude that most optimizations can't be applied when they cross breakpoint boundaries."
While true, this depends on the compiler knowing this is a magical breakpoint barrier it can't move things across.
The compiler has no idea this is a magical barrier unless something has told it it's a magical barrier.
Looking at godebug library, i don't see this being the case, it looks like it translates into an atomic store and an atomic load to some variables, and then a function call, which the compiler is definitely not going to see as a "nothing can move across" barrier.
(Also, having the debugging library alter the semantics of the program is 100% guaranteed to lead to bugs that are not visible when using the library, etc)
I think you're right that it's going to introduce bugs in concurrent code. For example, it's legal to send a pointer through a channel as a way of transferring ownership and never access the object again. If the debugger rewrites the code so that "never accesses it again" is no longer true, it's created a data race.
On the other hand, godebug generates straightforward single-threaded code that creates pointers to locals in a shadow data structure and accesses them later. There's no reason it shouldn't work if you're not using goroutines.
In particular, a previous call to godebug.Declare("x", &x) will add a pointer to what was previously a local variable to a data structure. This effectively moves all locals to a heap representation of the goroutine's stack, to be accessed later. It's going to kill performance, but it's legal to do.
"It's going to kill performance, but it's legal to do."
Sure, but it's going to cause the optimizer to do different things than it would have to that variable. As I said, this essentially changes what the compiler is allowed to do, and will expose or hide bugs (usually hide if it hurts the optimizer) :)
One only has to look at the bugzilla's of gcc and llvm to discover all sorts of fun things that barriers hide/expose.
"Also, having the debugging library alter the semantics of the program is 100% guaranteed to lead to bugs that are not visible when using the library, etc"
Can you give an example of the kind of bug you expect to see?
Sure, i'll stick to bugs invisible with the library, and visible without it:
If you've inserted compiler barriers it can't move code across, the compiler will no longer perform the same optimizations with and without your debug library.
Those optimizations often make bugs visible, because variables no longer have the value you expect them to at the time you expect it, etc.
Look at the behavior this has with and without the barrier.
It's buggy without it. With it, it's fine.
This is exactly equivalent to:
Without the debugging library, the code is clearly buggy and doesn't work in user-visible ways.
With the debugging library, if i insert a breakpoint where the current asm barrier is, it now behaves correctly 100% of the time, even though it's broken.
You set a breakpoint on the call to bar and examine the value of x. You would expect it to be 2, but what if the compiler had decided to move the allocation of x = 2 to after the call to bar? There's no reason why it shouldn't. You'd then see x = 1, which would confuse you.
But it's not really material, because bar doesn't touch x (and if it did, this problem wouldn't exist to begin with).
In other words, it might be a bit strange and cause a slight detour in your quest to discover the cause of a bug, but it wouldn't actually change any behavior or cause any issues.
Your argument is essentially: The barriers you insert, even if they block optimization, will never block optimization in a way that changes behavior. This is demonstrably false, since you are, among other things, taking the address of a variable, which means escape analysis won't do things to it, etc.
You can argue "The behavior it changes doesn't matter". As i've shown,
1. it does in a threaded environment (like, you know, go)
2. It depends on whether your code is buggy or not.
IT's certainly true that it never, on it's own, causes bugs.
But as i've shown, it can make bugs appear to come or go.
If you don't think that will ever happen, i don't know what to tell you, other than "It has happened in literally every compiler that has ever had barriers like this".
Without any evidence why Go should be different here, i don't see why go will be different here.
Thanks! gopherjs is a really great project. This was my first time using it and I was impressed.
I've considered putting it up as a permanent playground like http://play.golang.org, where you can debug little Go snippets on the web. Is that something that you would find useful?
I agree with you about gopherjs being an awesome project. And I would love to see a playground-like debugger that you could run directly in your browser for quick debugging!
Hi, not relevant to the topic. I visited your site, specifically "Homepage > Show all categories > Crypto" (http://golangtoolbox.com/categories/crypto) and was met with:
While I totally applaud this effort, how does it not lie? It inserts code into the actual source claiming it is on a line in the original file, but it is in a different line in the compiled version. That seems like it could be a problem at some point, and despite the serious utility in this, I'm not 100% convinced it is the right tool for the job.
Author here. I agree that this could cause surprises. Are there specific cases you have in mind?
Two cases I have thought of are stack traces and logging that inspects the stack. In the former case, if you get a stack trace while debugging it will not mean much because the lines do not match those of your code. In the latter case, logging statements may print the wrong things if they depend on being called a certain number of stack frames below user code. glog does this, for example[1]. I don't have solutions to either case yet, but I have some ideas of how to start. I think the utility the tool provides is well worth those two issues, and I'm hopeful that both can be fixed.
A neat trick in some languages is to modify the source so that the line numbers don't change, for example by using semicolons instead of newlines and making sure a newline never appears in inserted instrumentation code. But I'm not sure if that's possible in Go.
> Another is that you may inadvertently corrupt or change the behavior of the code in an unintentional way if you modify it incorrectly
I'm concerned about this, too. One of my next high priorities for godebug is to download many open source projects and search for behavior differences caused by the debugger. I would also like to generate new programs and test them as well.
> But, you get the easy 80% with this for sure. I'm just prodding you to work on the other 20%.
I will. I'm excited about the project and want to make it as useful as I can.
I tried using this facility a few months ago based on this very short documentation but couldn't get it working, nor could I search out any further doco on it, so because it wasn't really important I gave up. Can I ask... have any of you ever successfully used this?
At one point even I implemented the V8 debugger protocol so you could use the Web Inspector, and an HTTP proxy that would automatically instrument JavaScript files (using synchronous XMLHttpRequests to block execution...), but I don't think I ever posted that code :(
Author here. Yes, only one goroutine is paused at a time. I'm planning to add commands to show and jump between goroutines. Still sketching those out, though. Do you think it would be helpful to have multiple goroutines paused by the debugger at once?
I'd like to be able to completely pause all goroutines and slowly analyze the state of each one. You can probably just add some mutex or waitgroup to your line() method and just lock it when a goroutine pauses, no?
Yeah, that should be relatively easy to do. We would just skip the "am I the goroutine under the debugger" check here [1] and add some kind of channel communication here [2]. The hardest thing is probably just the UI. How should that functionality work? I'm imagining something like this:
>>> pause all
All goroutines paused
>>> show goroutines
1: foo.go:16
2: foo.go:24
3. bar.go:10 [current]
>>> next
-> // some code from goroutine 3
>>> goroutine 2
Now tracing goroutine 2. Current location:
/*
Code listing from goroutine 2's current location
*/
>>> next
-> // some code from goroutine 2
In a well behaved program, all theads should probably be synchronized by mutexes/channels etc, so pausing one thread, should in theory pause all related ones.
LiteIDE was probably using gdb as a backend. gdb is not a very reliable debugger as of the last few Go releases. Here's a demo of a common failure mode of gdb on Go: occasionally running the entire program when you try to take a single step https://youtu.be/qo4RAPxCTa0?t=351
This failure mode is a large part of why I decided to write godebug.
