Read the SSH man pages every so often, even if you think you know how to use it. There's a lot of features in there. Don't miss the "AUTHORIZED_KEYS FILE FORMAT" in sshd's man page for the uber-cool "command='command'" options for authorized keys (restricts a given key to just be able to run a certain command, very useful). See also SSH's port forwarding, -D, learn how to use ssh-agent, and "man ssh_config".
They're not really related concepts. A jail is allows you to run arbitrary things in a restricted environment while ssh's command feature just forces a particular command to run no matter what's passed.
If your authorized_keys line for a key has command="blah" on it and you go:
ssh server ls
the server will, rather than running ls as it normally would, run blah and pass the command you specified in as an environment variable. Then blah can parse it and do whatever it wants with it.
You could use command to establish a jail or virtual environment of some sort in which to run the program specified if you wanted to.
The command= simply changes "what" gets executed when the user logs in.
A jail (ChrootDirectory in ssh) changes "where" the user gets when logs in.
The most creative thing I've do with command= was a "select" menu in bash (with some actions in the shudoers).
Other interesting tool I've discovered recently is rrsync. I'm doing the backups of my systems isolated with this. It's distributed with the rsync sources, you put it like:
I used to use rsnapshot with command= set to a custom script that would let a few things through (rsync being one of them), by instrospecting `$SSH_ORIGINAL_COMMAND`.
A true BSD chroot jail restricts the user's mobility around the file system, effectively limiting them to running only what binaries have been explicitly placed within that jail. It means, among other things, that the fork hacks that command= is vulnerable to (e.g., vim running other programs) don't work because non-permitted programs basically don't exist as far as the user can see. The downside to jails is that, without automation, they're tedious to maintain, and disk overhead is not insignificant as you have to have a duplicate set of system binaries for each user. In practice, chroot jails end up being very similar to OpenVZ “virtualization” on Linux.
> The downside to jails is that, without automation, they're tedious to maintain, and disk overhead is not insignificant as you have to have a duplicate set of system binaries for each user
On FreeBSD it's common to use nullfs to reduce this - you make one jail skeleton hierarchy and then null-mount it for individual jails so they all share the same underlying files (and cache). zfs clones can be used similarly. I'm sure other OS's have similar capabilities.
They work in totally different spheres. The command directive can assign different commands per ssh key. Think git@example.com for all of your users, a la Github.
Jails limit the capabilities of users of the OS, instead of users of the network.
Or use mosh[1] on top of SSH and stop worrying about that stuff.
It works much better over high-latency links (mobile). It is not bothered by saturated links, tolerates IP changes and losing the underlying connection like when you suspend your laptop and take it elsewhere.
I now have mosh connect to several servers in tabs when I run gnome-terminal the first time, and only disconnect on reboot. I also run a mosh-capable Irssi Connectbot fork on the phone[2].
It's a massive improvement, fixing many of the little annoyances of ssh.
The server I connect to most often is behind NAT and I'm usually also behind NAT, and mosh doesn't support that config, alas. I'd use it in a heartbeat if I could, though.
> Afterwards data is sent AES-encrypted, which relatively simple.
It's simple to encrypt using AES, but that doesn't mean it is simple to encrypt in a secure way (i'm looking at you ECB mode). There are too many ways to accidentally mess up.
It uses ssh to set up the connection, so that setup is fine. The actual traffic is encrypted using AES-128-OCB, which is a great mode of operation.
I haven't digged too deeply into their security, but from what the main page says, it seems pretty trustworthy. Hopefully they're using the appropriate authenticated data, with packet numbers and so forth, to prevent silly things like replay and reordering attacks.
It might have faulty implementation for how it glues security features together, or encrypts the traffic data. I think the parent comment is talking about this kind of "audit" (i.e., not theoretically, but in action and by perusing the source code).
Yes, I addressed the "glue" thing with the note about authenticating the correct data (notably things like IVs and packet numbers).
But given that it uses ssh for the initial setup, I doubt there are any problems to be found with the initial key exchange, which is one of the trickiest parts of a "secure channel" protocol. And OCB, as an AEAD mode, is pretty hard to screw up, so long as (again) you make sure you're authenticating the proper data and generating IVs/nonces appropriately. Plus, they're using the (optimized) reference implementation for OCB, so that's probably pretty solid.
Now, I definitely agree that this isn't a proper security audit. But at a cursory glance, the general impression I get is that it's probably okay. Honestly, they've really removed most of the protocol complexity by using ssh to do all of the setup work.
I love mosh, but do note it's a lot slower than plain ssh. I run everything through a 266x188 tmux session, and mosh badly struggles to keep up even with light use. Resizing panes is juddery, vim is visibly laggy, and if anything spams a lot of output to a term (e.g. find /) it basically grinds to a halt. ssh is butter smooth in comparison.
No mouse support in mosh either.
(gnome-terminal/vte (so also Terminator, Xfce Terminal and others) is similarly slow, but as with mosh you don't really notice with smaller terms. I switched to urxvt).
The default escape character ~ does not work if the tilde key in your keyboard layout is a dead key [1], like it is in many European layouts. It can be changed via the EscapeChar config option or the -e command line parameter. It seems, though, that not just any old character is accepted - I tried to use §, which, in the Finnish layout, is in the same physical position as ~ is in the US version, but ssh complains about "bad escape character".
EDIT - I suppose it must be an ASCII character, which is not an entirely unreasonable requirement.
It is useful, yes. Here's another thing I picked up last week - how do you reboot a remote linux box that's somehow lost its root drive but you still have a shell open (because you left ssh running on another machine)?
echo 1 > /proc/sys/kernel/sysrq
echo b > /proc/sysrq-trigger
The machine in question has a zfs pool that it would be nice to sync, and a MySQL database running off one of the filesystems on that pool. So yes, all of this would be lovely.
But will it work?
You're sitting on an ssh session that's just about hanging on. There's no other way into the system (it's remote). As soon as you issue e (or definitely i) the shell dies, the ssh session dies and your reisub job gets killed, no?
I tried this:
echo e >/proc/sysrq-trigger; echo b >/proc/sysrq-trigger;
but what I described above happened, and the box did not reboot.
(--edit-- deleted comment above had a perfectly reasonable explanation of a safe shutdown using the "Magic SysRq" stuff, but it applied to a machine you're at locally rather than the b0rked remote box I described)
$ killall mysqld
$ while pgrep mysqld; do sleep 5; done
# waiting for mysql to shut down
$ echo s >/proc/sysrq-trigger
$ watch dmesg
# wait for "sync complete" msg
$ echo b >/proc/sysrq-trigger
# or if you can force-reboot remotely
$ echo o >/proc/sysrq-trigger
# /proc will no longer be mounted after this
$ killall mysqld
/usr/bin/killall - file not found
$ while pgrep mysqld; do sleep 5; done
/usr/bin/pgrep - file not found
$ echo s >/proc/sysrq-trigger
$ watch dmesg
/usr/bin/watch - file not found
$ echo b >/proc/sysrq-trigger
Unfortunately in this situation only the bash builtins seem to work. I even had an old system drive mounted under /altroot, but it wouldn't execute any of the commands in /altroot/bin or /altroot/usr/bin either (can't find
something it needs to execute stuff. ld? libc? unknown).
So really only the 'b' seems to help. (edit: 's' is probably useful, even if you can't kill everything)
Of course the long term solution is to stop using a USB stick on an unreliable/badly supported USB3 add-on card for the root drive :)
$ pgrep() { for proc in /proc/[0-9]*; do read CMDLINE < $proc/cmdline; if [[ ${CMDLINE/*$1*/x} == x ]]; then echo ${proc#/proc/}; return 0; fi; done; return 1; }
$ kill $(pgrep mysqld)
$ while pgrep mysqld; do sleep 5; done
$ echo s >/proc/sysrq-trigger
$ while sleep 2; do dmesg | tail; done
$ echo b >/proc/sysrq-trigger
I admire your tenacity, but I think sleep, dmesg and tail are all out as well!
Though all could b mitigated by leaving it for a minute or two at the relevant point, and manually checking for the mysql process every so often. I'll keep this on file for next time it happens (though I'm hoping to replace the dodgy parts at the weekend).
If /lib/ld.so is hosed and you cannot run any dynamically linked program, then you might be able to start programs in /altroot by running something similar to...
If you frequently hit Ctrl-S by accident, and don't consider this a feature, you can turn off XON/XOFF flow control using
$ stty -ixon -ixoff
then Ctrl-S/Ctrl-Q will be normal characters without side-effects in your terminal, you'll probably just see (depending on shell, terminal, ...) ^S on your screen when you hit it.
And besides, the older I get (41) the more I can use frequent reminders of the things I used to know. It's disturbing the number of times I've searched the web for answers, and discovered an answer I wrote myself. And I hadn't known this could be easily turned off with 'stty' as suggested in another answer. Or maybe I did once know that?
i'm turning 30 this year, have been dealing with this issue for 15 years, and still mini-panic for about 2 seconds whenever i hit ctrl-s by accident. it's good to read about.
Being a windows developer who dabbles in linux I accidentally smack ctrl+s a dozen times a day when vimming. I've only just recently got used to to recovery mechanism:
Ctrl+q to recover the terminal, a stiff drink to recover my nerves.
From a comment above, I think you'll be able to put
stty -ixon -ixoff
in your .profile (or .bashrc or whatever) and ctrl+s/q won't be a problem anymore.
WELL HOW ESL&^H^HLSE ARE YOU GOING TO READ A 25+ LINE FILE OTHER THAN MASHING CTRLQ BEFORE THE LINES SCROLL OFF THE SCREEN? BY READING THRE AFTERGLOW? I SUPPOSE YOU`VE ASL^H^HSLO GOT A DIRECT CONNECT MODEMS WHERE YOU CAN`T EVEN HEAR WHAT`S GOING ON.
+++ATH...
Please don't let HN become a substitute for RTFM. This should be known by all SSH users who have skimmed the man page. Fair enough as a blog post but for this trivia to get 46 points so far is deeply depressing.
Maybe I should write a blog post about the use of CTRL-Z in the shell and post that here, should get me Kilo-karma points if this is anything to go by.
This is like not knowing the return or exit function, and is also used for telnet or serial console connections. If knowledge is so shallow on a particular tool, it should not count as being part of the toolset. There's nothing more basic or essential to SSH than connecting and disconnecting. And it's right there on the man page, not exactly a hidden feature. This is more beginner level knowledge, maybe novice if being very very generous, but nowhere near expert level.
Connecting: type ssh and the hostname. Disconnecting: type exit or ctrl-d or whatever on the remote end. That's enough for many people.
For a group of people as large as HN, there are bound to be many people who are experts in one thing, and marginal with others, so articles like this are likely to appeal to them, and be useful.
I would agree if it were for some more obscure yet useful feature, but this is one of the basics, that's clearly documented. This should not be useful for anyone who's ever read the man page, and anyone who's used ssh should have read the man page at least once. That so many people apparently haven't, in what's supposed to be highly technical community, is a very disappointing sign of intellectual indolence.
At more than 6 million words, that's approximately 10 War and Peace's. And of course in terms of documentation, that's only man pages, which doesn't cover all the stuff that's in other formats like info or html.
You cannot be an expert in everything: I knew about that command for closing ssh, but frankly it is not something I use because it's easier to just close the rxvt and be done with it. I have zillions of other things to occupy my brain.
I'd also argue that in terms of ssh, since this function is so easily accomplished in other ways, this is really just handy triva. Much more useful to know about are all the tunneling things, as they are not necessarily obvious, and can be extremely useful.
I'm hardly advocating reading every single man page; just the man pages for things you actually use. And 10x War and Peace's is hardly an extreme amount of reading to become a knowledgeable systems administrator. That would probably be akin to two semesters of 5 courses each to become a systems administrator; I think most community college or vocational school programs for systems administration require 2 years, and wouldn't teach you nearly as much.
In any event, I'm not taking the position that casual *nix users should be reading every single man page. Just for the essentials like ssh, cp/mv, ls, man itself, etc. You do not need to be an expert to read a few man pages on utilities you often use. Everyone should RTFM, at least for the relevant bits, no matter what level you're at. Anyone who found the initial post useful would also find reading the man page for ssh useful. Thus, they should have read it. That's not at all suggesting that they need to become an expert.
The ~. key sequence is used in pretty much any console type application, not just ssh, so it's quite useful to know. If you do any work with a serial console i.e. configuring network devices, pdus, even some servers, etc., then it's pretty useful for ending a session as there's no TCP connection to close and take you back to the local console on your terminal emulator. If you're working on the actual local console and not in a full blown GUI on a system without virtual consoles, there may be no other way to close the serial console session on some platforms (notably man different OS'es running on Sparc64's as well as not on Solaris at all until recently).
Regardless, it's generally less effort to type ~. than to close and reopen a window/tab in your terminal emulator of choice. The time spent to read the man page would inevitably be made up by a few clicks saved here and there throughout a long career. It would have taken less time than to make multiple posts in this thread.
I've been using SSH for fifteen years. I have never read the man page because I have never needed to--I have scanned it for specific topics but I have never been quite so teeth-pullingly. This is new and useful information to me and so I upvoted it. On the other hand, I am certain that many, many things I consider trivial would be out of your realm of knowledge and relative understanding of what is and is not trivial should lead you to act better than you are acting.
At the end of the day, RTFM is much more helpful than spoon feeding someone an answer. Maybe it comes off as being an asshole, but a little shame can be a good motivator for a neophyte to learn how to teach themselves, and they will be better off for it in the long run. "Teach a man to fish" and all that.
Do you want a shovel for that giant hole you're digging?
"A little shame is a good thing" says to everybody around you that you're more interested in feeling good about yourself than helping people. And it's a joke to boot. I've been doing this for around fifteen years, I've written multiple telnet server applications and a MUD client, and I didn't know about this feature in telnet/rlogin/ssh.
Or maybe because I was once a neophyte that was shamed into RTFM, which had a very positive effect on my life, I'm advocating other people RTFM also? Which would hopefully help them as much as it did me? This is generally how things work for online communities, be it an IRC channel, a mailing list, or web forums. Why shouldn't it be that way on HN, especially as RTFM is as much part of hacker culture as anything.
Meanwhile, you're resorting to making baseless assumptions about a total stranger on the Internet, and bringing your own credentials into this rather than focusing on the argument at hand. Who's digging the giant hole? Who's more interested in feeling good about himself?
I really couldn't care less what your personal story is, or what you've imagined up about mine. Let's stick to why someone should or should not RTFM, okay?
If someone had posted a question asking how to end an unresponsive ssh session, perhaps RTFM would be an appropriate response. In this case someone is sharing knowledge that could be useful to many people. Just because something is written in a man page, does that mean it should be locked away to only those intrepid enough to read the man pages of every tool they use? Community curated knowledge is useful.
As well, I'm not sure if you're intending to, but you're coming off as an elitist asshole. Not knowing an esoteric bit of what is essentially ssh trivia does not make someone a neophyte. For anyone who entered the field in the past 15-20 years, hitting the (x) close window/tab button is likely more second nature than 'enter ~ .' Please don't be like the guy who hangs out in IRC help channels just to tell people wanting help to RTFM.
I will concede that nobody posted the question, but I do think RTFM is still appropriate as a surprisingly large number of people here apparently have not done so. Because it's written in a man page, it's certainly not locked away. And reading it is hardly going to extreme lengths. Man pages are precisely there to be an easy way to look something up, and are written and maintained by the community, so they certainly qualify as community curated knowledge. It's also a much more efficient way of spreading information than a blog post. I'm not sure where you get the notion that man pages are for a select few and not for everyone. They're also all on the web so it's easy to search their contents as well.
I would take the position that anyone who hasn't read the man page for something is a bit of a neophyte, or a novice at best. Unless perhaps they've read the source. How else are you going to learn something, without reading the documentation? Online tutorials and blog posts make for a poor substitute at best, and being less direct often introduce inaccuracies and other misinformation. I think it's fair to say that someone whose entire knowledge of a subject is based on incomplete information is not an expert. Not to say that man pages are perfect; they can be outdated and inaccurate themselves. But perhaps if more people read them, more people would submit revisions to make the documentation better. If suggesting that people read the proper documentation makes me an elitist asshole, so be it.
I don't quite agree with your 15-20 years timeline; GNOME and KDE were only founded about 15 years ago, and it was a few years before they matured to the point where decent windowing support was commonplace, so it wouldn't have been a given that you would have a full GUI environment at that time. Opening and closing xterm in twm was enough of a hassle to warrant using screen to accomplish windowing instead, which is much more esoteric than ~. Using ssh from an actual console without a GUI was still pretty common at the time, at the very least as part of setting up a box to the point where it had a GUI. But that is all beside the point. ~. takes less time than closing a window/tab and opening a new one, be it through a mouse or a keyboard sequence. And it doesn't nuke your scrollback history. This is just as true today as it was 15-20 years ago. If it's still useful enough to vote up a post, it should be useful enough to read about in a man page.
More people should be like the guys who hang out in IRC help channels. If you think they're there just to tell people to RTFM, you are very much mistaken. They are generally genuinely helpful people. However, they're not inclined to be as helpful to people who come waltzing in demanding everything they want to know handed to them on a silver platter, when they could have looked it up themselves with minimal effort. I don't think it's wrong to point out that there's no place for that sense of entitlement. However when someone asks a question who's clearly done their research but is still stuck on an issue, you'll find the same people who normally respond with RTFM will do everything they can to help. I don't think it's too much to ask to take enough consideration of their time not to waste it with something that can be answered in the first page of Google results. I would posit that instead of these people being labelled elitist assholes, a lot of people should be labelled entitled assholes instead.
Being a shaming douche doesn't help people--it hurts them. Some react in an in-the-end-positive method because they are hurt by people like you--they develop scar tissue to inure themselves against your consciously-adopted sociopathy. Most say "fuck it, I won't deal with these people", throw their hands up, and leave. It's toxic behavior and it is unacceptable because they are more desirable people to have in a community than people who glorify the shaming of people who don't know obscurities.
So expecting someone else to take 10 minutes of their time to explain something to you, when you could have just spent 5 minutes of your own makes the other guy a douche? Really?
Anyone emotionally fragile enough to turn away over someone telling them to try to find answers on their own before pestering others probably isn't cut out to be part of the community in the first place, at least not a technical one, seeing as how they're too lazy to do it, and selfish enough to think that their own time is more valuable than that of the person helping them.
If people followed your advice, aside from the more experienced people shooting themselves in the face, they'd waste all their time answering questions for those too self-entitled to put in any efforts of their own where their expertise isn't even utilized, while the other neophytes who have a genuine interest and a willingness to put in their own efforts wouldn't have a resource to turn to.
Maybe it's toxic behavior to you, but it's human nature and the modus operandi because the set of technical experts is small to begin with, the set of technical experts who have enough patience to answer beginners' questions on a regular basis is much smaller still, and the set of technical experts willing to endlessly put up with the self-entitled and intellectually indolent is as rare as unicorns.
Be more realistic and open-minded to other perspectives.
> probably isn't cut out to be part of the community in the first place
Thank you for so wonderfully illustrating the problem you wish to perpetuate.
"Be more realistic?" I am being realistic. I'm telling you that being an asshole is bad. That is quite realistic. But it means having to not get your jollies. So have fun abusing beginners, as you think is your right. I hope never to encounter you in a project I care about.
It has nothing to do with abusing beginners, and everything about helping people who are actually willing to learn versus those that just want someone to do the work done for them, because the reality is there isn't enough of experts' time available to do both. You've more or less completely ignored what I've said, and are just sticking to your preconceived notions about some stereotype you've ascribed to me. It's rather hypocritical that you regard yourself highly enough as a moral authority to tell strangers on the Internet to "Be a better person", while calling them an "elitist asshole".
I'm quite happy to never encounter someone who can't put together an intellectually honest argument and instead resorts to straw man arguments, appeals to authority, and ad hominem attacks. You can label me an asshole all you want, but I'll let your own words speak as to what type of person you are.
Not everyone on HN is a terminal expert, and not sure why you would assume such. It got a lot of points because people found this interesting and helpful.
Also, you should write a blog post about CTRL-Z, if people find the info valuable, it'll get the upvotes.
Apparently it's rsh, not telnet. The telnet manual does mention it though, since you can put telnet in a rsh-like mode where ~ does work. (May depend on telnet version.)
man 1 telnet on FreeBSD or OpenBSD contains:
"When in rlogin mode, a line of the form ~. disconnects from the remote
host; ~ is the telnet escape character. Similarly, the line ~^Z suspends
the telnet session. The line ~^] escapes to the normal telnet escape
prompt."
although on the Debian man page, it appears slightly differently as:
"-r Emulate rlogin(1). In this mode, the default escape character is a tilde. Also, the interpretation of the escape character is changed: an escape character followed by a dot
causes telnet to disconnect from the remote host. A ^Z instead of a dot suspends telnet, and a ^] (the default telnet escape character) generates a normal telnet prompt. These
codes are accepted only at the beginning of a line.
"
telnet is not installed by default on CentOS, but has the same man page as on Debian.
If it becomes acceptable to use HN for gamified man pages instead of expecting people to do their homework, well, there are more than enough man pages in and out of print to fill HN for years. We don't have downvotes on stories, so the only flow control is flagging and complaining about stuff that isn't timely and isn't useful for most of the audience.
That is ridiculous strawman argument. How would such stories end up in a prominent position on HN unless enough people from the community likes it.
In any case that was not what my comment was about. I was, and still am, more than surprised by the poster's puerile lament that someone is getting a few odd karma that the poster thought was undeserved. I assumed that this place is somewhat removed from such "Mama ! he took my toy" behavior.
If you happen to be a few sessions deep, ~~ will send ~ to the next session along. A casual ~~~~~~~~~~~. or so later and everything is wonderful again!
Though you have to be nested really deeply to do ~~~~~~~~~~~. - unlike some other escaping schemes, it only requires one extra tilde for every layer of nesting rather than doubling each time.
I can never decide how I want to set this. Most sites recommend 180 seconds or so, but with the default ServerAliveCountMax set to 3, this is 9 minutes before a dead terminal is actually disconnected.
I've started to set it really really low personally - like, 5 seconds, so the connection drops after 15 seconds. I'm tempted to go down to 1, but I have a lot of long running sessions and I start to worry about the traffic counts.
But, after all this there's still a problem: none of it seems to work with any of the connection mux'ing options - once the background session dies, I still have to manually kill it to get anything working again.
While I've frequently used this to kill connections, my favorite thing I've done with it is to list existing and dynamically add new forwarding ports through SSH.
Hitting '.' at a prompt used to be a common idiom for exiting a program. I first saw it when I was a kid working on an HP-3000 system where the system programming language was BASIC and all programs followed this convention. Don't know where it came from originally, but you can still see it in places like rsh/SSH etc.
On a side note, does anyone know what ~B actually does? Does it send a SIGINT to the remote terminal? What does ssh mean by the phrase "send a BREAK to a remote system"?
I've tried to use it without success to kill a runaway listing of megabytes of scrolling text, but frantically hitting ctrl-C seems to work much better.
In the beginning, there was RS232, where each character was a fixed number of bits, optionally with a stop bit and a parity bit. If the sender transmitted too many 0 bits to represent a valid character, that protocol failure was called a 'break', and some receiving equipment would detect such a failure and do something about it, like reset itself to a known-good state. Thus, a 'break' was occasionally a useful thing to send, so sending equipment would often have a special keystroke to cause a 'break' condition.
The Unix 'tty' subsystem was basically designed to support simple serial terminals, and so it had a bunch of behaviour designed to interoperate with the pre-existing 'break' conventions. If a Unix system's serial port received a break, Unix would (optionally) send any processes running via that connection a SIGINT, to represent the 'reset to known-good state' behaviour (this is controlled by the stty command's 'brkint' flag). Also, if the user's terminal didn't provide a specific 'send a break signal' command, Unix could be configured to send a break signal when it received some particular character (^C by default; this is controlled by stty's 'intr' setting).
Of course, nobody uses physical RS232 terminals anymore, but for compatibility reasons the Unix tty API lives on, and the "psuedotty" implementation used for things like terminal emulators maintains compatibility. ssh is basically a tool for exporting the tty API over the network, and so for compatibility it too must have a way to transmit the information "pretend a break condition has occurred on the RS232 connection we're pretending to use."
To summarise: yes, unless you've messed with the stty command, ~B will probably result in a SIGINT. Ctrl-C is probably more reliable since you can hit it much faster than you can type <Enter>~B, but ~B is still useful if your terminal is in 'raw' mode, where ^C is not converted to SIGINT (for example, if you're running an app that wants to bind ^C to some other function).
