Yes you can. I want to hook this up to an emulator (like qemu or MAME) with an emulated serial port. That is the PTY equivalent. I'm writing this emulator.
So the emulator gets bytes from the emulated virtual machine, and then what? This needs to get fed into a terminal. While it is possible to invoke xterm with the very buggy -S option, there is no way for the emulator to suck the state back out of xterm for coherent snapshots. Simics uses xterm with the -S option. It sucks.
A library interface is required. The whole point of a library interface is to allow session state saving. Using a $SHELL replacement is way off. That has nothing to do with the problem.
So, if you were hacking on an emulator (qemu, MAME, Simics, VirtualBox, etc.) and you were trying to emulate a device with a serial port or even a modem, how would you get snapshots to contain the terminal state? To make this work, you need that state within the emulator. Passing stuff over a PTY is not going to work.
You're only solving half of the problem (and doing it in a needlessly complicated way at that). As I said, capturing the TTY output is the easy part. Storing the running state of the program that's writing to the TTY is your bigger issue. That's why I suggested a custom $SHELL (I guess there's no need to reinvent the command line so you could probably use a multiplexer instead?) that could manage the running state of the executing programs and their PTY.
Storing the running state of the program that's writing to the... serial port or whatever... is a solved issue. Most emulators and virtual machines can do this. For example, VMWare Workstation can do it. Simics can do it. Qemu can do it.
The state of an entire OS, with all running programs, gets saved. It's more complete than when a laptop/notebook does suspend-to-disk. The emulator saves the CPU registers, the RAM, the disks, the parallel port mode, etc.
The trouble is that a connected terminal is not included in that list.
There isn't anything writing to the serial port. The process of writing to the serial port is fast enough that the IO isnt the problem. What you're after is the capture of stuff that has already been outputted on the terminal and that can be captured. It's called a scroll back buffer history. Some terminals can even make that persistent.
You keep trying to reinvent terminals with features that already exist to solve a problem that needs to be fixed in the server itself.
I emulate lots of different things. I'll make up an extra-simple case to illustrate the problem.
The guest OS running in the emulator is just a bootable floppy image for testing. All it does is print a counter to the serial port, once per hour. Just after 3 hours it has written 0, 1, 2, 3. At this point I make a snapshot file named 0123.snap and let the software continue running. More numbers get printed. Eventually the terminal is showing 0, 1, 2, 3, 4, 5, 6. I decide that I wish to go back in time, so I issue a command to load the 0123.snap file into the emulator. The internal state of the emulator warps back to the point at which I took the snapshot. The CPU registers, the RAM, and everything else within the emulator are now as they were earlier. The terminal retains the old state however, because it is a separate program with no awareness of the fact that I just loaded the 0123.snap file into my emulator. The guest OS carries on from the original point of course, since that is what the CPU and RAM state dictates, so a "4" will be printed next. The terminal then shows 0, 1, 2, 3, 4, 5, 6, 4. The numbers are simply wrong. They should be 0, 1, 2, 3, 4.
I can only avoid this fate by saving full terminal state whenever I save the emulator state, and of course restoring it at the same time too. There is no reasonable way to extract terminal state from a separate terminal program. (attacking it with ptrace is not reasonable) I thus conclude that the terminal must be built into the emulator.
> I can only avoid this fate by saving full terminal state whenever I save the emulator state, and of course restoring it at the same time too. There is no reasonable way to extract terminal state from a separate terminal program. (attacking it with ptrace is not reasonable) I thus conclude that the terminal must be built into the emulator.
It's called a "terminal multiplexer" and that is exactly what I suggested with the tmux solution right from the bloody start. I've lost track of how many times I've said this needed to be solved on the server yet you repeatedly pushed back on both points when they were made, insisting it was a terminal emulator issue on the client side.
I'm normally not a BSD user. I'm more familiar with "screen" than "tmux". To me, a "terminal multiplexer" is a physical piece of hardware. It has numerous slow serial lines to which one may attach terminals, modems, or consoles. It time-multiplexes them to/from a single high-speed link, typically by interleaving the streams of data bit by bit.
I don't see tmux being available as a library that I could link into my emulator. If it can be a library, then yes my emulator could implement the tmux protocol. This turns out to be almost exactly what I was proposing, with the terminal state implemented by a library within the emulator. I'd just be missing the user-friendly aspect of automatically popping up the terminal windows when my emulator starts, but perhaps that could be arranged by having the emulator start xterm running tmux.
This is comparable to using vnc protocol for VGA. I have this implemented, but nobody likes to use it. Everybody prefers the built-in video window to show the guest's VGA.
If you meant to not link in the tmux code though, that won't work.
Instead of insisting that there is a terminal emulator issue on the client side, I insist that there not be a distinct client. The code that tracks terminal state (cursor position, current character attributes, etc.) pretty much needs to live in the emulator. That is the only reasonable way to ensure that the state can be saved and restored by the emulator. Actually displaying this state could be done by connecting a client (annoying) or just by popping up a window.
Another nice thing about having the terminal windows built into the emulator is that they all go away when the emulator does, even if that is a crash.
This whole conversation is weird, and I'm really not sure what is missing from my explanation. In case you missed it, I am an author of an emulator. I write the emulation, and my emulator supports guest software that needs terminals.
The serial port is an emulated 16550 chip. It is much more than a PTY. It has clock divider registers, an interrupt cause register, a transmit holding register, etc.
Scroll back buffer history won't let me untype something. If I take a snapshot, then a character is output to the terminal, and then I load that snapshot, the character needs to disappear from the screen. Of course, "character" could be something more destructive, like a scrolling operation or the clearing of a line.
I'm the author of several serial interfaces for 8 bit micros plus a bunch of Linux terminal solutions. If you've taken a snapshot of the server and restore that then it should resume where you left it. All the terminal emulator then needs to do is restore a scroll back session. Thus when you press backspace (or whatever) on the terminal emulator it would transmit that character to the server and the server side software will understand that a character has been deleted.
If you disable local echo and have the server side software handle that (like how the SSH protocol works) then it becomes even easier because you then also manage the terminal state (to some degree) via the server as well.
I've written quite a bit of software around this kind of area and I really doesn't sound like a problem that hasn't already largely been solved. I mean if you're feeling really lazy you could just build a terminal server that supports detachable sessions (eg tmux or screen) and that would get around the scroll back issues without you having to write a single line of code.
Like I said, I've done a lot of work with serial interfaces much more archaic than what you've described and have been able to get them working without having to write too much magic. So I suspect you've over thought your problem because it still seems really straightforward to solve from this latest description. Like I said before, terminals literally are just flows of text (ironically there's even less complexity there since you're dealing with serial interfaces rather than PTYs) so all you actually need to do is ensure your terminals output is persistent - which is actually a surprisingly easy job.
Just capture the output from the serial port in a buffer inside the emulator, and use that to save snapshots.
You can then also forward any output to a file/terminal emulator/whatever is convenient, but relying on the terminal emulator to save part of your emulator's state seems weird to me, unless you are going to ship the two bundled together.
Terminal state is far more than just text on the screen. For example, an escape code could change the color. After that, there could be megabytes of output created over many hours. Loading a snapshot should not require replaying all that data.
The point of a library is that the two are more than just bundled together. They are linked together, making one executable. The terminal window is a part of the emulator.
Your arguements don't really make any sense with regards to the technology you're discusing. Eg why would we replay past TTY output? Also escape are inlined so they would also be captured via the parents method. In fact contrary to your point, TTYs effectively are just text on a screen.
The only thing that wouldn't be inlined via control characters nor escape sequences would be the PTY operating mode (which are defined via kernal syscalls). Im talking about stuff like local echo, flow control, how CR/LF characters are handled, etc. However that's solved with a multiplexer (screen or tmux type thing) or a custom shell, thus you could save that running state and reset it when needed (shells actually already need to do this to some extent so they can pass control of the TTY to forked processes. But we'd obviously need to take things a step further and store any state changes made by forked processes as well). However now we are back to my original solution you dismissed.
I'll go through what happens without replaying. The emulator runs, and the guest OS inside it boots up. For this example, suppose it is DOS running a multi-user BBS with 2 modems. This is not a UNIX clone; there is no /dev and there is no shell.
The emulator needs to pop up 2 terminal windows so that we can log in to the BBS. We do that, and a colorful menu is displayed on each of them. At this point, we save a snapshot and shut down the host computer. Later, we start the host computer, then start the emulator, and then try to resume the snapshot. Each terminal window remains blank, which is not the correct state. The guest OS (a DOS BBS) is in the state where it is showing a colorful menu, but we can't see it at all. Our terminals, being separate from the emulator and without an ability to save/restore snapshots, are inconsistent with the emulated guest OS.
In the above example, the guest OS (a DOS BBS) never did any kernel syscalls related to PTY operating mode. It directly acted on serial port hardware. There is no PTY, and adding one (where?) doesn't help.
Fundamentally, the emulator needs to be able to save/restore all terminal state. That even includes stuff like incomplete escape sequences, such as when just the first byte of the escape sequence has been passed to the terminal.
Right, that's completely different to the thing originally described. It's also not possible without running a bespoke BBS server that can handle stateless connections. Again what you're trying to solve at the client side should really be solved at the server side.
I don't know why you even mentioned PTYs (now I understand your requirements). A PTY is just a detachable interface for local terminals, but since you're connecting via serial anyway there's no need for a PTY. There's also no need for a special emulated serial interface like you suggested because the DOS VM would have one already baked into it, just use that and have the BBS listen on a couple of COM ports.
Regarding storing the previous session, there are already terminal emulators out there that will restore the scroll back session from a previous instance, so you might find your idlogical terminal already exists.
It still seems like a daft problem you're trying to solve though. Putting aside you're suspending a server for no apparent reason (why would you even want to do this?), you're effectively running a forum for one local user. But I'm sure you also have your own reasons for these anti-patterns as well.
It's not different. A bespoke BBS server is not an option; the whole point is to run an existing piece of software for which there is no source code available. Suggesting a bespoke BBS server is like telling a person running Super Mario All-Stars in a Super Nintendo emulator that they should upgrade to a PC game.
I have implemented (wrote C code for) what you refer to as "a special emulated serial interface" and thus "the DOS VM would have one already baked into it". That is my code and it works fine, and I do indeed "have the BBS listen on a couple of COM ports".
I mentioned PTYs because I don't want them. Most terminal emulators will only accept data from a PTY, with themselves on the master side and a shell on the slave side. This alone makes these terminal emulators essentially worthless; to use them I'd really need to have my emulator act as a telnet server and then ask the user to telnet into the server! Perhaps a really screwy $SHELL could perform that automatically, but it still doesn't solve the replay/snapshot issue.
An xterm is very special because xterm supports the -S option. With this, my emulator can fork off a process and then exec an xterm with a file descriptor that I can feed data into. This works OK until I want to load a snapshot.
There may be "terminal emulators out there that will restore the scroll back session from a previous instance", but this gets unusable. My emulator supports multiple named snapshots. Using a terminal program with distinct snapshot functionality would require that the user save/restore snapshots both in my emulator and in each of the terminal programs. I have a case (other than a DOS BBS) where my emulator has dozens of serial ports, so this would place a heavy burden on the user.
Suspending a server is good for debugging it. I can go back to before a crash. I can load a snapshot to bypass slow and annoying start-up.
With the greatest of respect, the way you casually drop technical terms out of context suggests to me that you really don't understand how terminals work in the slightest. (That and the new revelation that you're restoring your BBS snapshots in random orders yet expecting your terminal to be smart enough to guess where to resume). This is why I think we are having such a high degree of difficulty agreeing on your problem.
The reason I commented about hacking the BBS software is because that's the only way to solve the weird requirements you're asking. You can complain about my suggestion all you like but it's your unusual requirements that are at fault The fault here, not my solution.
Maybe you need to stop and think for a moment that perhaps what you're asking is silly rather than just assuming that everyone else on the internet is stupid.
The closest work around you're going to get (and is buildable with your experience) would be to build a terminal server (like a minimal Linux distribution running on a VM or Raspberry Pi) which you can SSH onto from Linux or Windows (eg via PuTTY) and which has a serial interface to your BBS software (be that virtualized or physical). Install 'tmux' or 'screen' onto the terminal server and when you SSH onto the terminal server reattach to your screen session. That screen session will be connected to your BBS and you'll then have your detachable but persistent terminal sessions.
Edit:
If your terminal server is a VM and the BBS software is a DOSbox instance running on the terminal server then you also only need to snapshot one host and have persistent scroll back PLUS the ability to restore from random snapshots. The only thing you would lose is your TCP/IP handshake to the xterm / PuTTY session but they're detachable anyway via tmux / screen so you'd just have to SSH back into the host and reattach to your multiplexer (screen / tmux).
I have written termcap and terminfo entries, and I am a vttest contributer. I admit that I haven't yet written a terminal. I've owned a few physical ones: VT100, VT220, VT320, VT510, and some awful Televideo thing.
I do restore snapshots in random orders, and I really want my terminal to know exactly where to resume. I don't expect the terminal to guess. I expect it to accept a blob of serialized state data, and of course I also need it to produce that sort of data on demand. I want that data in the snapshot files that my emulator creates.
I could indeed put the emulator, along with separate terminals (xterm) and the OS they require, within an emulator. This works, but adds the overhead of a whole extra OS. It gets confusing to use, takes up lots of memory, runs slowly, and takes up extra space on the screen.
Assuming all that is true, in that case you must be aware that the unusual requirements you're placing makes this completely impossible. I just don't understand how you can claim to understand how terminals work yet still expect the kind of behaviour you're expecting and making the kind of comments you've been making.
Honestly, just stick this stuff behind a terminal multiplexer and move on with your life. You're trying to over-engineer a solution to a problem you're inventing.
So the emulator gets bytes from the emulated virtual machine, and then what? This needs to get fed into a terminal. While it is possible to invoke xterm with the very buggy -S option, there is no way for the emulator to suck the state back out of xterm for coherent snapshots. Simics uses xterm with the -S option. It sucks.
A library interface is required. The whole point of a library interface is to allow session state saving. Using a $SHELL replacement is way off. That has nothing to do with the problem.
So, if you were hacking on an emulator (qemu, MAME, Simics, VirtualBox, etc.) and you were trying to emulate a device with a serial port or even a modem, how would you get snapshots to contain the terminal state? To make this work, you need that state within the emulator. Passing stuff over a PTY is not going to work.