Hacker News new | past | comments | ask | show | jobs | submit login

In general I'm curious how people handle managing diffs in the workflow over time. I've found when working with Microsoft SSIS, that I end up preferring something in code where changes are obvious.



I'm not sure what the preferred approach is, but since it seems to have a REST API with full query and update ability on the flow graph, it seems like you should be able to do a workflow where you make an update on a dev server (updates are immediately live), run a process that captures that via the REST API, serializes it, and pushes the serialization to version control, and have another process that takes a serialized version and pushes it to other servers (prod, etc.) via the REST API.

Why visual programming tools don't (at least in documentation) address version control is an interesting question, given how important that is in programming generally, and the fact that those tools are often focused on enterprise audiences.


I worked on a product once that provided users an interactive graph. The users could apply layout algorithms AND modify the grqph by moving nodes and edges around. From time to time, nodes and edges would be added and removed by the system.

So there's a distinction there between the graph, and layout of the graph ie x,y coordinates of nodes, and routing of edges. What would version control track? This is a non trivial problem imo. Edit; specifically, storing a graph in some canonical form so that trivial changes would not create massive 'false positives' in the diff. For example changing an edge indicatin an entire sub graph was added or removed. The problem is similar to xml tree diffing.

This is why I think versioning in these tools is pretty much non existent, imo of course.


commercial version of Talend does, but it's SVN at it's core, and code would still be a hell of a lot better to deal with.


Not sure about how NiFi handles this (<- an interesting question). But, I believe this is one of the strengths of AirFlow's "workflow as code" model. Thus, the workflow can use git for version control.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: