Insightful – commenting to keep a reference as I spent the last four years engineering around this very aspect.
I'm trying to wrap my head around the difference between execution models. On one hand we have real-time node systems like Unreal Blueprint, Modo or DaVinci Resolve, which depend heavily on state nodes, i.e. nodes whose sole purpose is to hold and serve information (a mesh, a transformation, an RGB matrix). These nodes act as complementary inputs to nodes in the main execution flow.
On the other hand we have Node Red and the likes, whose nodes rely on a single input. At first glance this approch might make sense from a UX perspective – simply connect your components and deploy. However with time I've come to see this as a major limitation to the extent that you're often better off writing code. After building a number of flows you start to notice certain patterns... Since nodes are uncapable of fetching data from other nodes ("state nodes" above), it's up to you as a developer to serve the node with the exact data structure it needs. The GUI gives you no hint as to the structure itself, so you're left reading the node's documentation: {payload, action, topic, subject} – different for every node.
At the core, this approach is the FBP version of polish notation in the sense that you first have to fetch all the operands before sending them to the operation. So what you're left with is a flow where every core node is surrounded by data shuffling nodes: "put this into 'action', put that into 'body', take 'payload' and store it away because the next node will overwrite it". As an effect of this "serial execution model", flows often stretch wide from the left to the right where two thirds of nodes are pure data transformations.
Somewhere around here the USP of FBP gets literally lost in the flow – the USP being to offload working memory. During building/debugging/revisiting flows, I often know right away what piece is broken. Nevertheless I spend a lot of time double-clicking on function, change, and switch nodes just to find that piece of code. The irony of it all is that I wanted to use FBP to _visualise_ logic. Instead the same paradigm hides it away.
So what you've accomplished is transforming code that goes from up to down into boxes that go left to right.
The thinking behind it all is that Node Red shouldn't make assumptions about the incoming data, however someone has to make those assumptions, and that someone is you, the developer.
It would be trivial to create a node that takes an arbitrary input (a sample message), analyses the message structure and exposes the appropriate output pins, however as long as Node Red persists with single input ports and the "message-based execution" it entails, it will take tolls on working memory and lag behind the Unreal model which I too consider state of the art.
Yeah, this is a really tricky space to delve into because the people who are most familiar with it are working on mathematical models and have very strong opinions about the "kind" of graph you're making, or what you're actually doing with the underlying data. Which is all definitely important for a node library maker to know, but the whole purpose of the library is that an implementing dev shouldn't have to know any of that stuff. So you really have to dig through a lot of irrelevant stuff about graph theory and whatnot to get to any relevant programming knowledge on the topic.
What's galling to me is that nodes map 1-1 to function calls. I can't even think of a scenario where a program that can be written as functions and objects couldn't be represented as a node graph. I can certainly shudder at the horrific types of graphs most programs would make, but they can all be mapped out. And, on the flip side, every single program I've written in visual scripting could easily be written as functions and objects. So, with that in mind, it's frustrating to go looking for information about node graphs and getting a bunch of muck about why they're 'not good', and why they don't handle certain things well, and why they aren't suited for x or y, when - in my head - I'm literally just asking for a well-formed API, and I'm getting a bunch of pushback about how APIs aren't really a good way to program. -_-
But because it's so hard to get good comparative information on visual scripting, I've also wrestled with what makes for good abstractions vs what is too broad. And that's what I think the Node Red stuff is: just too broad. Abstracting away everything breaks the value of the abstraction. So it's always about finding a balance, which is why I appreciate Epic's maintenance of Blueprints. A good API abstracts concepts you don't need to understand behind concepts you already understand. "GetDistance()" is a great abstraction because I don't care about the vector math underneath. But "Login()" is a terrible abstraction because a library shouldn't have the intimate details of my login process (meaning that it should be abstracted into more modular parts that I can more easily inject my business logic/keys into). So a node, or a library of nodes, or a node library system - they're all only as good as their abstractions. And Node Red is just way too abstracted to be "good". And, of course, the irony there is that if it were only a little more flexible, it could even be abstracted, itself, into something more use-able! If you could write a node that DID accept two inputs, you could abstract away all of the setting and swapping and trading. But since even that node could only ever be kick started by a single node, you would have to come up with some kind of hacky scheme to make any kind of utility actually function. So it's the one type of abstraction that they don't allow (their node model) which I find to be fundamental to successful visual scripting languages.
Honestly, though, I think there are some fundamental designs that visual scripting languages just need to implement, but I have a very hard time describing those designs with rigor. I would naively say that they need nodes, edges, a variable deck, a node discovery utility, and a good variable property manager. But that's just because that's what I'm used to. I think once someone really serious sits down to formalize these things, we'll have a much richer environment. I just hope they do it soon!
I'm trying to wrap my head around the difference between execution models. On one hand we have real-time node systems like Unreal Blueprint, Modo or DaVinci Resolve, which depend heavily on state nodes, i.e. nodes whose sole purpose is to hold and serve information (a mesh, a transformation, an RGB matrix). These nodes act as complementary inputs to nodes in the main execution flow.
On the other hand we have Node Red and the likes, whose nodes rely on a single input. At first glance this approch might make sense from a UX perspective – simply connect your components and deploy. However with time I've come to see this as a major limitation to the extent that you're often better off writing code. After building a number of flows you start to notice certain patterns... Since nodes are uncapable of fetching data from other nodes ("state nodes" above), it's up to you as a developer to serve the node with the exact data structure it needs. The GUI gives you no hint as to the structure itself, so you're left reading the node's documentation: {payload, action, topic, subject} – different for every node.
At the core, this approach is the FBP version of polish notation in the sense that you first have to fetch all the operands before sending them to the operation. So what you're left with is a flow where every core node is surrounded by data shuffling nodes: "put this into 'action', put that into 'body', take 'payload' and store it away because the next node will overwrite it". As an effect of this "serial execution model", flows often stretch wide from the left to the right where two thirds of nodes are pure data transformations.
Somewhere around here the USP of FBP gets literally lost in the flow – the USP being to offload working memory. During building/debugging/revisiting flows, I often know right away what piece is broken. Nevertheless I spend a lot of time double-clicking on function, change, and switch nodes just to find that piece of code. The irony of it all is that I wanted to use FBP to _visualise_ logic. Instead the same paradigm hides it away.
So what you've accomplished is transforming code that goes from up to down into boxes that go left to right.
The thinking behind it all is that Node Red shouldn't make assumptions about the incoming data, however someone has to make those assumptions, and that someone is you, the developer.
It would be trivial to create a node that takes an arbitrary input (a sample message), analyses the message structure and exposes the appropriate output pins, however as long as Node Red persists with single input ports and the "message-based execution" it entails, it will take tolls on working memory and lag behind the Unreal model which I too consider state of the art.