Hacker News new | past | comments | ask | show | jobs | submit login
Visual programming should start in the debugger (interjectedfuture.com)
126 points by iamwil 4 months ago | hide | past | favorite | 55 comments



I sympathize a lot with this post. Today, modern IDE's still don't provide automatic ways to inspect program execution and nicely visualize it. Instead, they rely on having you place all the breakpoints, step-step-step and restart everything again if you did not inspect something in time. I don't want to insert print statements that end up clogging the console completely. This workflow also quickly breaks with a lot of logging.

I'm quite a bit into developing more tooling around that, and it's really hard - developers particularly want tools that completely integrate with existing IDE's. Debugger infrastructure is usually not built to be very exstensible or accessible. For my extension for Visual Studio https://d-0.dev/ I had to put serious effort into making sure it all still works correctly with VS breakpoints which is extremely difficult when the debugger expects all of the code to be in the same place all the time.


I see three issues with the nice ideas presented:

1. Incentives.

I’ve made my contributions to debuggers (Firebug mostly), and devs are not willing to pay for it (we had donations, but I’ve seen people create projects in the space) and so tool makers (browser makers in this case) eat the cost and see it as a cost center. Because of this, the debuggers are at the abstraction layer of the tool itself. The tools change and the cost of keeping the debugger in sync (usually well after the feature is out there) is already high enough.

I put hooks into Firebug to make it better for higher level debugging of frameworks. Later I made a paid extension [*]. It paid for a new computer, and spawned many framework specific extensions over time. I pushed Chrome's DevTools team to add some of the same hooks. Sadly, a decade later, many of those framework level extensions don't use what is available. Again, it is a cost center for the framework development, so just enough gets done, no more.

2. Tools are Siloed.

You may have OpenTelementry, etc., but that data is siloed off somewhere. The IDE doesn't have it. But the IDE has extensions, so maybe someone like Microsoft can bridge the gap and put up a sidebar that is a heatmap of the time that that code takes on the CPU.

3. Levels of Abstraction.

When you talk about state and data modeling, not only is it siloed, but the levels of abstraction are plentiful. It is somewhat possible, but I refer you back to 1--Incentives.

[*] https://vimeo.com/showcase/2541003/video/75271334


The self-intro to wasm project I want to get up to is a service-workers that is a OTel collector/store-and-forward proxy.

It is kind of wild, otel has these collection protocols but there's afaik not any enduring access protocols. It's all over the wire protocols, nothing is defined for reading an archive of traces. Otel avoids specifying any common APIs for the storage layers, seemingly.


Otel is sponsored by cloud providers, they want you to land data in their systems to be queried and visualized by their tools.

There is no incentive to provide a common access layer because that is where all the players in the telemetry market see their profits.


I'm still shocked how limited our tools are. People in the early 2000s could animate complex 3d scenes in real time with physics and soft body dynamics. But tracing a few variables feels like an herculean effort.

Maybe it's time to flip the game on its head. Bring reactive, lispmachines, responsive UIs.. I don't know.


Those tools exist. Time travel debugging has existed in product-ready forms since the early 2000s let alone the advancements since then.

The problem is that game development companies are willing to invest money in buying quality tools to make their developers productive, where as traditional trillion dollar software companies prefer to let their developers wallow in filth.

Getting a company to invest 5,000$/yr into making their breathing revenue machines that cost 200,000$/yr and produce 1,000,000$/yr in revenue is like pulling teeth in this industry. Companies with breathing electrical engineer revenue machines do not bat an eye at 50,000$/yr in making them productive. Think about how much cool and effective tooling you could get with that kind of budget. But if you are not willing to spend even one coffee worth on your tooling, then one coffee of productivity is all you get to have.


The hard part of game development might be more centralized around complex, big programs that simulate game logic and push tons of complex and mathematically sophisticated data to GPU‘s.

On the contrary the complexity web development, distributed systems emerges from many communicating processes, that execute in different environments.

A sophisticated debugger might help the former category more than the latter. There’s no shortage of monitoring products that act on the distributed systems level.


It's still strange to see this contrast between game dev doing magic for small salaries versus big corps paying huge lumps of money for CRUD++ (sorry for being exaggerating here).

It's also cultural.. when everybody in the room thinks a REST api is state of the art, you'll have a hard time selling them more R&D.


A game dev I know went to work for Google on Android. What he found was a team full of people who had given up on gdb back in uni, had worked for Google their entire career, printf debugged all day every day, took forever to fix anything, and everyone was fine with that because that’s all they’d ever known.

It was a shock because in gamedev debuggers are a Big Deal. The vast majority of AAA gamedev happens in Visual Studio’s debugger or some alternative that is at least as capable.


Interesting. And scary.. and depressing (about Google). Gamedev has always had a special relationship with ergonomics (dev ux, dx) I assume (due to the interactive nature of their product)... Reminds me of Naughty Dog developing a whole lisp DSL to be able to try different game designs on the fly on PS1.

That said gdb and similar debuggers are still too limited IMO, there's so much we could add on top. Well maybe I'm not aware of improved variants.


> Naughty Dog developing a whole lisp DSL to be able to try different game designs on the fly on PS1

The DSL was called Game Oriented Assembly Lisp.

> It supports a long term compiling listener session which gives the compiler knowledge about the state of the compiled and thus running program, including the symbol table. This, in addition to dynamic linking, allows a function to be edited, recompiled, uploaded, and inserted into a running game without having to restart.

https://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp

Found more info on that here:

https://lisp-lang.org/success/graphics/

> Naughty Dog Software used a Common Lisp DSL to write the Jak and Dexter series of games for the Play Station.

> Naughty Dog co-founder Andy Gavin, says the unique capabilities of Lisp enabled fast development and execution of character and object control – something that was needed to fully realize the numerous 3D creatures and devices which interact with the player in real-time (60 frames per second).

> “Lisp was just the best solution for this job,” comments Gavin. “With leading edge game systems like ours, you have to deal with complicated behaviors and real-time action. Languages like C are very poor with temporal constructs. C is just very awkward for a project like this. Lisp, on the other hand, is ideal.”

> As Gavin explains, “With Lisp, one can rapidly develop meta constructs for behaviors and combine them in new ways. In addition, Lisp allows the redefinition of the language to easily add new constructs; particularly those needed to deal with time-based behaviors and layering of actions. Contrary to popular belief, there is nothing inherently slow about Lisp. It is easy to construct a simple dialect which is just as efficient as C, but retains the dynamic and consistent qualities that make Lisp a much more effective expression of one’s programming intentions.”


Gavin also made a long blog 12 part series about that period

https://all-things-andy-gavin.com/2011/02/02/making-crash-ba...

I wonder how he works these days.


Ooh, I'm enjoying the article series.

> But the craziest thing I did was create a new programming language – with Lisp syntax – for coding all of the gameplay. It had all sorts of built in state machine support (very useful with game objects), powerful macros, dynamic loading etc. It was also highly irregular and idiosyncratic, and in true Naughty Dog fashion “powerful but complicated.”

As for how he works these days, I see in the About page:

> Briefly in 2008, and then full time from the second half of 2009 on, Andy has been concentrating on novel writing.

https://all-things-andy-gavin.com/writing/


The GDB feature set has been largely unchanged since the 80s and even then mostly just parroted the design of other CLI debuggers from the 70s. Of course it is limited, you are talking about technology that was not even cutting edge nearly half a century ago.

To be fair, the core components of a low level debug agent are still largely the same, so it is not like the technology is bad per se. It is just that it only constitutes table stakes, a bare minimum foundation, these days. Even looking at 20 year old technology like time travel debugging that allows you to time travel execution back and forth gives a far better idea of what people are missing by using primitive tools.


Same, see my post above. Many of the printf-debugging people I've run into also came from Google.


I'm a bit shocked that Google is so full of them. That said the FP link seems weak, in theory (sic) functional programming will make you think about domains beforehand so the debugging game changes. But if it's mild FP over brittle ecmascript 5 libs... then it's another story.


I think you are expecting too much.

I've worked with expensive software before (embedded stuff, few K$ per seat) and it it was horrible - missing features slow, crashes a lot, horrible SDK designs. Yes, maybe it did generate some magical code (I have no way to check this) but the dev UX was terrible.

I am working with clang & command line build system now and this is so much better. If someone offered me an expensive IDE I'l do all I can to make sure I don't have to work with it.


For reasons I don't fully understand most younger programmers today don't know anything about debugging or debuggers.

The syndrome seems to track with FP and async and possibly with "teaching" programming. FP and async typically don't work well with today's debuggers. Back in the day nobody was taught to program -- they figured it out by doing. When you are taught by someone who themselves never "did" something about the practice gets lost, imho.

\end{curmudgeon}


> Back in the day nobody was taught to program

Do you mean the 1940s and 1950s? Because by the 1960s, Comp Sci was taught as a discipline and by the 1970s and 1980s anybody who wanted to work in the industry was expected to get a BS degree as a minimum (PhD if you were serious).

Sure, there were still people learning on their own, but they were either smart enough to improve significantly, or amateurs who would always produce spaghetti code which the pros (read: Comp Sci grads) ended up rewriting.


Windows, Linux, and Mac desktop OS are so limited still. I'd like a little scripting language built-in to the OS that allows you to do stuff like draw to the screen. No, Python + turtle graphics doesn't count :)

There are simply far too many hurdles to do basic computing things. I think Microsoft could do some minor additions to powershell and maybe get something like this.

I wish some billionaire would put up some money and fund a modern lisp machine or something like that. I know it'll probably never work, but I can dream.


> a little scripting language built-in to the OS

MacOS has something like this: https://en.wikipedia.org/wiki/AppleScript#Basic_concepts You can send around events and macro/script GUI stuff.


Not bad. I'd like a little more, but that's the idea.


Do you mean something like Gnome's Javascript ? https://gjs.guide/


A little bit of that, but from a graphics perspective, this seems to be more about UI and nothing that can just draw to the screen. For example, how easy is it to throw together Tetris? Rebol can do that as well as a dozen other things with a page of code or less. Too bad it was basically last maintained like 20 years ago.


I too miss Display PostScript.


Elon needs to sponsor Jonathan Blow to build a Windows competitor


I'm not the biggest fan of Elon although SpaceX is pretty cool and successful.

It would be cool if he did something like that though. Just a clean and simple OS and not the corporate hellscape that is Windows or the alternatives of Mac and Linux. I'd buy a license or the hardware itself if he sold it and develop for something like that.


What kind of user-space programs would you run on this OS?


I honestly think the desktop and web need a redo. This will never happen, but this is my naive dream:

I want to open my laptop to an operating system with no advertisements or 1000 programs running in the background. It should be pretty simple with basic application support like a text editor, office suite,: database...etc. It would have a super powerful scripting language built-in with DSLs for graphics, UI, mathematics/science, OS, sound, hardware...etc. It would also have some good hardware peripherals you could control. Basically an ideal work and hobby computer that has some centralized/integrated way of accessing all the tools you need with minimal fuss. It just makes me wonder why in some respects there are still many things that the Xerox Alto or Commodore 64 can do that I can't easily do in Windows.

Need to write a simple videogame? That should just involve opening the shell and typing in some high level primitives instead of downloading some crazy big software and libraries

If(Foo){ Draw-Circle -Vertex [100,25] -Radius [2] }

Want to do some linear algebra work?

Invert-Matrix -FileInput 'example.csv' -Sparse 'True'

Maybe you want to read some data from some kind of hardware device that has a sensor (e.g. measuring air quality) and send it to a chart?

ForeverLoop(Get-Data -Port [0]) | LineChart

The commands above are obviously made up, but that is kind of what I'm thinking about. With modern OS, I have to download, install, and configure many tools to typically do these things and it is so messy and cumbersome. I often can't share with anyone else either without them installing all the same apps (e.g .maybe they have to download anaconda Python and postgres).

The Web is also just way too complex now. Ideally there would be a much simpler way to build beautiful and simple sites similarly to the hypothetical apps above.


I've always said so. The essence of visual programming is watching code and runtime values juxtaposed, not connecting code primitives with fancy connectors.

Modern IDEs have lots of visual features, specially tailored to support production languages. Inspection panels, tooltips with variable values and function definitions...

Surely we could add more spatial visualization tools to them. But the step up from printf to a debugging inspector panel was on its own a great advance in building a mental model of the program state and making sense of its behaviour, which is what visual programming is all about.

For programmers interested in advanced visual tools tailored to software creation, see Bret Victor's essays on notations:

https://worrydream.com/MediaForThinkingTheUnthinkable/

https://worrydream.com/LadderOfAbstraction/


If you search for the keywords “program understanding” and “program visualization” in Google Scholar, you’ll see a lot of research on how to visualize and comprehend systems. Around 2007, one visualization that caught the internet’s attention was CodeCities by M. Lanza, and there is also Moose[1], which is based on Pharo and provides abstractions to query and visualize code bases.

But, for some reason, none of these tools catch the attention of the most popular IDEs. I hypothesize that there are major roadblocks to implementing this in a generic and useful way.

The first one is the variety of languages, frameworks, and build tools. For example, analyzing a TypeScript+React code base is not the same as analyzing a TypeScript+Vue code base, even when both use TypeScript and the TSC API is very easy to use.

The second roadblock is that useful visualizations also depend on the characteristics of your system, and creating them is not easy.

Maybe things will change with the addition of AI to analyze code bases, but so far, all the tools I’ve seen are either very niche or very short-lived.

[1]: https://moosetechnology.org/


Having actually partially designed and maintained a visual programming product with I guess what would be called a time travel debugger (among many other fascinating and strange features) inside of a major technology company, the biggest push back was always from developers themselves. These things sound good, and often they're very useful to those without strong programming backgrounds, but inevitably developers are pretty aggressive about poo-pooing tools like this as lots of things are much harder to implement, like diffing for example. The tool itself was honestly very radical (we were able to do a ton of really cutting edge things around resource scheduling, containerization pre-Docker, real actual shared time travel debugging at massive scale), but in the end, I think the company shitcanned it for something much simpler.


I can't be the only one here who would love to hear more about this.


Agreed. Visual programming focuses on visualising the structure of the _program_, but what matters more is visualising the structure of the _data_ and, even more importantly, what the program is doing to the data (visualising in the time domain).

A major bottleneck for people learning programming is developing a mental model of what is in the memory during the runtime and what it looks like. The most successful introductions to programming start by making drawings or small games since it's more intuitive for beginners to iterate and have immediate feedback.


Screenshot at the end of the article is from this video. He calls it spatial programming and mentions some other tools which did that in the start of the video.

https://www.youtube.com/watch?v=eQgxFuw8f1U


The Cellpond video was quite mindblowing, and lu wilson throws other similarly mind bending ideas on mastodon semi regularly. I loved the live remapping thing here https://mastodon.social/@TodePond@mas.to/112733298578556294


who is he?


The person in the video, presumably. Lu Wilson / Luke Wilson. (They said in the video that either name is fine.)


I developed Call Stacking as a visual debugger.

It captures everything for a given request - each method call, params/argument/return values, class name/method/line of execution.

You can visibly see which method called which, as each method call is nested relative to its parent callee, in a single consolidated timeline.

https://callstacking.com/


Similar post from a few days ago:

"We need visual programming. No, not like that." https://news.ycombinator.com/item?id=40937119


Shoutout to https://whitebox.systems/

It only really works with C-with-Classes style C++. But, the fact that it works at all is pretty neat.


People talk about using LLMs to write code, but is there any work on using RNNS (perhaps LLMs) to explain what a piece of code does? Seems like that might be a good feature for a debugger.


I've tried that — copy paste in code, ask it to explain what it does.

This worked (with the original ChatGPT 3.5) about as well as asking it to write code if you did the documentation first. Not great, not terrible, misses more precisely when you want the most help.

I should try again with 4o.


i've found gpt-4 to be mediocre at writing code but superb at explaining it


Yeah. Debugging is getting _way_ less attention than it should. I know some people who _boast_ that they don't use a debugger. To me it feels like a doctor saying: "We don't use new-fangled X-Rays here, you should be able to find problems with just a stethoscope".

And good debugging support doesn't need to be some fancy "Minority Report" style 4D animated UI. Just give us good type rendering support, an easy ability to chain breakpoints and set conditional breakpoints, "watch variable" support. Some other things: "where did this value came from?", "watch for value use".

Go is an especially bad example. Its debugger doesn't even allow custom type rendering, so if you have something like a custom generic map container, you can't inspect it at the debug time.


For me, not using a debugger is kind of like not using valgrind. If you stop operating on memory directly, you forget you ever needed valgrind. Likewise, if you write functions instead of methods, then you can just take the problematic input, and evaluate your function on it, and inspect the output - without needing to get the whole program into the right state for stepping through.

I kind of gave up on step-through right around the same time I started writing multithreaded code as well.


Debuggers allow to "look around" the running code. This is really helpful in large systems, where you don't necessarily even know all the components.

It's fine when you're working in a nice segregated part of the system, where everything is well-defined. But it's not always the case.

Also, single-stepping is a nice mental tool, kinda like rubber-ducking. You single-step and explain to yourself what is going on.


Well now you have us curious: What programming language treats methods as something other than what is ultimately a function with an additional implied parameter? Whether a parameter is implicit or explicit doesn't really change anything with respect to the state of the program. You need to construct the state either way.


Indeed, in JS you can call the method with the object explicitly (which allows you to call one objects method with a different object—for better or worse, though usually some meta-programming).


Paper and pen. Sure might not work on LARGE projects, but when I’m reverse engineering something or trying to debug, I always start with a control flow diagram… works a treat!

But in the past 6 months I’ve started to use graphviz and other tools to quickly document code paths for others


I had a similar idea. I was working with some complex recursive functions and it was impossible to figure out what was going on. Trying to parse through 1000 console logs was nearly impossible so I had the idea of rendering the function call stack and attaching the logs to where in the stack it was called from and rendering it as an interactive graph.

Its mostly just a demo but I think something like this a standard tool would be really helpful. https://github.com/ando818/Svelte-Logger


I know this is after the fact but I've gotten much of the same functionality by indenting each line in the log to a level commensurate with how far down in the call stack it's happening. Just being able to see the indentation tells me most of what I need to know.

On a related topic, what would really help debugging in most languages is more consistent functionality for built-in introspection, which allows devs to write sophisticated analysis tools. Just being able to access things like the call stack, argument count, etc. is super important, but unfortunately every language has its quirks about what can actually be accessed, and in what contexts (function vs. method), etc.


The debugger? You mean console.log("here") ?


There was a time before console.log existed.

In fact, later when it existed in FB if you called it in production code it would cause an exception for your users (console didn’t exist unless you had Firebug installed).


The article is missing all industrial visual programming.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: