See, history can give you a more inert syntax. And maybe a new way of thinking about how to make this thing... I would love to have a more robust version to do C-style goto cleanups.
Since he mentions this being for a work thing, how I've handled a similar situation (long running shell thingies) is:
Break up the script into steps, and place them in order in a dot-d style folder, like 1.foo.sh, 2.bar.sh, 3.baz.sh. Have a separate _context.sh which supplies the upfront variables or whatever are needed for each step. Ensure that each step can run on its own with just the context sourced. Then have a run.sh in the folder above that sources the context and the steps in order.
Now, that said, typically these kinds of things are dataflow or even buildsystem-adjacent, so I've also experimented with doing them in CMake (intermediate products managed with add_custom_target) or as Nix derivations (intermediate products are input-addressed in the Nix store), but I'd love to be aware of other tools that are better for expressing this kind of thing.
I'd write run.sh like this: Put the names of the files from that dot-d style folder into some ".cc" file (if it doesn't exist yet); then loop over the lines in the ".cc" file, executing each line if it doesn't start with '#' and then prepending '#' with sed to the just executed line. After the loop, delete the ".cc" file (maybe with a diagnostic messages if all lines in it were #-prefixed). Maybe throw in some traps for reliable SIGINT-handling as well.
1) Use bash PS4 to show the line number currently executing (perhaps append start of line with #).
PS4='# ${LINENO}: ' bash -x script
2) Log to a script transaction file.
unset PS4 before a function, reset after function call to restart at function call instead of restarting within a function.
trap signal error & dump variables to state file (via env or set command)
3) check for 'transaction file' on restart of script, if exists,
a) restore saved state file settings;
b) pick up where to start via sdiff of original script and transaction file (aka first line not starting with #
sdiff -v ^'# script transaction_file | sh -
I would just use Make, or write a checkpoint file for each step, and wrap them all in if statements so they don't run if their checkpoint is there.
Or, if I were doing it from scratch, I would not have a bash script that takes days to run, that sounds like some Real Programming, and I'd do it in Python, where it's easier to manage complexity, in case there was some more fine grained caching that could be done, maybe even between runs.
Maybe I need to make another coffee or something but I really don't understand at all why they wanted Goto for this.
I feel like there has got to be a better common denominator if you're willing to stick with just Linux.
Yesterday I was trying to track down why a bash script that worked on pretty much every platform I'd tried and across a bunch of different bash versions was not working on a new operating system. Turns out the script was relying on "non-standard" behavior in tr(1). Definitely not the first thing you think of when you see bash complaining about undefined variables.
A bit further back I had similar fun with FreeBSD's installer now that they've ripped perl5 out of the base system. The installer is, of course, an unholy mix of C and sh.
Well, there is Perl, which is what actually was used to be for "can we do this in something that is not shell but also portable?" kinds of scripts for quite some time. But then again, it's Perl.
Why is this a thing still? Who is out there saying "We want you to develop an app but we won't let you use Python"?
Granted, I really wish there was a python-lts that didn't break stdlib stuff every few years, but it seems like everything remotely modern breaks compatibility constantly...
Not many people are saying that, but there's a few cases it could make sense. Like if you want to setup something on an embedded machine with no/restricted internet, then bash scripts will be easier.
Some people also like not having to do extra installs to run something simple.
And some scripts are simple enough that it's just easier to use bash.
But I agree, these cases are pretty obscure and most of the time the benefits of using a real language is worth the investment.
He writes "prepare to cringe" and he is not wrong. As far as I understand, this technique implements GOTO by reading the source code (again) into memory, filtering out everything before the target label and evaluating the remaining source code. I think this doesn't preserve state, variables etc. so not really a GOTO. But interesting technique.
The more frightening part (at least for me) is the reddit thread that points out this is how windows batch files implement goto, and shows how to see it happen.
Great heavens! I assumed at first that your "dlopen" was a separate executable, but you are implying you allow calling arbitrary C functions within the memory space of the bash process itself!?
You can also create callbacks (i.e. function pointers to bash code) that you can pass to qsort() or bsearch()... or pthread_create? That last one was a joke, I mean, you probably could but I don't know what would happen - I don't think bash is reentrant :)
Wow, I thought the original article was awful tho good, then I saw the lseek goto which seems much worse, then I see this dlopen for bash and I find it's the worse. But impossibly cool.
Did you have a usecase for this or it was just fun?
Which deletes the label itself, so you don't need the subsequent `grep -v ':$'`, but then you also aren't allowed to put any statements on the same line as the label.
BAT files writing other BAT files was often the only way to get scripting workflows done on primordial Windows versions, but chaos typically ensued if you attempted to rewrite the file you were presently running, as it appears that cmd didn't do anything fancy like read the whole file into memory before processing it (presumably due to the draconian memory limits people laboured under back then)
This is going off of memory as I'm not at my computer. Bash does something similar. It will read and execute the script line by line so if it's modified before bash gets to that line then weird things can happen. However, functions need to be read completely so the trick is to create a main function and call it at the end of the script.
> but chaos typically ensued if you attempted to rewrite the file you were presently running, as it appears that cmd didn't do anything fancy like read the whole file into memory before processing it
Neither does bash! That’s why you should always wrap things in a function and have the entry point of the script be the end of the file.
Yup I've done that too which also gives you free parallelization and as a bonus if the rules don't have a natural product you can always touch a sentinel file in each rule so that make can just pick up where it left off.
Hah! But COME FROM is more insidious - no amount of following the chain of calls will reveal that something is lurking elsewhere in the program ready to redirect the control flow
the author mentions in a note that bash was complaining and that they might put the labels in comments to dodge the issue. They might also be able to change the label format to `: label`. `:` just returns true regardless of what parameters you pass it, so it could still look "labelish" without having to use an actual comment.
That's one of the few places where it would be appropriate to store the current execution point somewhere in /var/cache or /var/lock and write the script so that it would look there at launch and dispatch accordingly.
"GOTO is considered harmful". This kind of thinking just makes me want to run off screaming into a paper bag. Language developers are, in many ways, the modern aristocrats telling us what we can't or should not do. I still miss assembly language.
GOTO originally allowed one to jump across functions, into the middle of the loops, etc.
And nobody stops you from writing in assembly! You'll just spend more time on writing (and debugging) your programs but sure, go ahead, it's not like your time on Earth is finite.
It feels pretty broadly accepted that overuse (or even moderate use) of goto becomes a nightmare for readability. Plenty of modern languages still support labels in loops to break to to be able to break out of multiple layers, which is the only main case that I can think of where a goto would be handy.
"Language developers are, in many ways, the modern aristocrats telling us what we can't or should not do."
This is offensive nonsense and breathtaking entitlement. They're providing you free tools to try to help you, usually not even being paid for their work.
Edit to add: I just got copypasta trolled, didn't I ...
I had mostly done some short stuff in bash that does not need much flow controls and loops etc; most programming I will use C instead, which does have goto and is sometimes useful even though usually the other flow controls are better. My own programming language designs do have a goto command because it is sometimes useful; the Free Hero Mesh programming language has both goto and gosub, in addition to the structured flow controls (if/then, begin/while/repeat, etc).
The way is done in that article doesn't seem best way to do; modifying the shell itself to seek the shell script file for it seem to be better in my opinion, or perhaps using the "enable" command to add your own built-in "goto" command (although I don't know if the API supports that). Another message on here mentions an external program messing with the file descriptors, but does bash use a consistent file descriptor number for the shell script file with which this will work?
That's neat but doesn't `case` support fallthrough? So I expect you could just put your script in one big `case` statement and skip to the branch you need.
I didn't think so but it turns out it was added with Bash 4.0 (released Feb 2009). Instead of terminating the case with ";;", you terminate it with ";&" for fall through.
And I did cringe, and then I thought it looked kinda fun. It would literally never have occurred to me in a million years to try to start a shell script half way though - so trapped am I in the paradigm of the familiar.
As for the script that takes several days and often breaks half way through... sounds like what Makefiles are for to me.
after many years, I learned that usually shell scripts are only good for the most basic of uses. here is a similar program in Go that doesn't require any hacks:
package main
import "os"
func main() {
var x int
switch len(os.Args) {
case 1: goto start
default:
switch os.Args[1] {
case "foo": goto foo
case "mid": goto mid
}
}
start:
x = 100
goto foo
mid:
x = 101
println("This is not printed!")
foo:
if x == 0 { x = 10 }
println("x is", x)
}
See manpage at https://etsh.nl/man/_goto.1.html and source code at https://github.com/eunuchs/tsh/blob/master/goto.c.
See, history can give you a more inert syntax. And maybe a new way of thinking about how to make this thing... I would love to have a more robust version to do C-style goto cleanups.