Email or username:

Password:

Forgot your password?
Top-level
njvack

@b0rk haha yeah I have been doing this a lot longer than 15 years and this is the first time I have really _thought_ about how terminals work

one thing I don't get is: If ^W isn't handled by readline (or friends), what *is* doing it? How do backspace and word-backspace work?

How... does character entry work at all??

19 comments
Julia Evans

@njvack honestly I didn't dig into that in the post largely because I don't understand it well either, I think it's the "unix terminal driver", but like what is that?? how does it work? it's a weak point for me and I'm hoping to understand it at some point

wfk

@b0rk @njvack I think "man termios" may be a good starting point. This documents the ioctl interface of the line discipline of the tty driver, which is what the tty utility uses to do its magic. It is mostly based on the POSIX standard, with some Linux extensions. Key point is that the driver can operate in different modes. Older commands run in cooked mode where the tty driver does very basic command line editing. Bash, pico, vim, etc use raw mode and handle everything themselves.

pelavarre

@b0rk @njvack

⌃W and ⌃U and ⌃Q and ⌃S and so on hide away deep inside: stty all

for example

$ stty -a
...
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>; eol2 = <undef>;
swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R; werase = ^W; lnext = ^V;
...
$

macOS

% stty -a
...
cchars: discard = ^O; dsusp = ^Y; eof = ^D; eol = <undef>;
eol2 = <undef>; erase = ^?; intr = ^C; kill = ^U; lnext = ^V;
min = 1; quit = ^\; reprint = ^R; start = ^Q; status = ^T;
stop = ^S; susp = ^Z; time = 0; werase = ^W;
%

part of how you can test for this is to try inside of: cat -

the ⌃W and ⌃U will work there, even when Arrow Keys etc don't work there

<=

macOS lets you spell out 'stty -a' as 'stty all', but Linux doesn't

Linux lets you abbreviate 'stty -a' as 'stty --all', but macOS doesn't

'stty -a' reports the 'stty ixon' vs 'stty -ixon' toggle in a different output line apart from these mentions of ^Q ^S, but you often need to add 'stty -ixon' to make the ⌃S forward-search-history properly undo
the ^R reverse-search-history mentioned
by Bash bind -p |grep C-[rs]
or the ^R history-incremental-search-backward
by Zsh bindkey |grep '\^[RS]'

i can't remember how often Zsh needs this workaround = maybe less often than Bash

@b0rk @njvack

⌃W and ⌃U and ⌃Q and ⌃S and so on hide away deep inside: stty all

for example

$ stty -a
...
intr = ^C; quit = ^\; erase = ^?; kill = ^U; eof = ^D; eol = <undef>; eol2 = <undef>;
swtch = <undef>; start = ^Q; stop = ^S; susp = ^Z; rprnt = ^R; werase = ^W; lnext = ^V;
...
$

macOS

% stty -a
...
cchars: discard = ^O; dsusp = ^Y; eof = ^D; eol = <undef>;
eol2 = <undef>; erase = ^?; intr = ^C; kill = ^U; lnext = ^V;
min = 1; quit = ^\; reprint = ^R; start = ^Q; status = ^T;
stop = ^S; susp...

pelavarre

⌃W and ⌃U are not precisely ^W and ^U, because "^" is pure Ascii and more classic, but ⌃ is the Unicode Up Arrowhead U+2303 that's part of ⌃ ⌥ ⇧ ⌘ ← ↑ → ↓ ⎋ ⏎ ⇥ ⇤

Gomijacogeo

@b0rk @njvack It helps to remember that a tty used to just be an RS-232 serial line. A character would come in, an interrupt would get generated, and the tty handler would read the char from a register and put it in a buffer for the attached program to eventually read(). That's raw mode. Cooked is when the driver kept a line-buffer, and would interpret chars as they came over the wire - printable? echo and place in buffer; ^J?, send buffer on to consumer to read(), ^U, kill buffer and start over

Gomijacogeo

@b0rk @njvack ^H? send back '^H ^H' to erase char on terminal, and move the head of the buffer back one. Other chars like ^C, ^Z, ^\, etc would instead send a signal to the process group attached to that tty and the signal handler would be invoked or the process killed.

njvack

@gomijacogeo @b0rk I think the thing I need to keep remembering is that at some point, ... something (I'm not 100% sure what — the tty handler? readline?) is telling _my terminal emulator_ to do something with the characters

Gomijacogeo

@njvack @b0rk Yeah, it was a lot simpler 40+ years ago where each component was a separate, tangible box connected by wires. Made it easy to keep track of what was done where. A modern terminal emulator, on one side talks to the window system and sees keypress events and does advanced font rendering. On the other side, it talks to a pty which is an abstraction of that ancient RS-232 line and is mostly a serialized byte stream that the kernel then treats as input and output.

Gomijacogeo

@njvack @b0rk If the tty is in cooked mode, those simple tasks - echoing, ^H, ^W, etc are done by the tty driver. In raw or cbreak (like raw, but still looks for things like ^C ^Z ^\ etc), then it is the consuming program (eg readline, vim, etc) that reads chars and then decides what to send back to the tty and then to the emulator to be displayed (usually lots of escape codes to handle positioning along with the one or two chars to actually display).

Gomijacogeo

@b0rk @njvack Then X11 came along and we needed a tty-like abstraction to attach these newfangled terminal windows to processes, because all the I/O and job control was wired to talk to a tty-shaped device. And some new side channels were added (e.g. SIGWINCH). So there are a ton of ioctls() to talk to the driver and control/interrogate exactly what kind of tty it really is.

Gomijacogeo

@b0rk @njvack Oh yeah, you also needed ptys for telnet/rlogin/ssh-like services. And the tty driver was already hellishly complex even before GUIs entered the picture. Tons of respect to anyone who unpeels the onion in the modern era - so many moving parts.

Gomijacogeo

@b0rk @njvack One other thing that was an A-ha! moment for me back in the day was learning that stty uses an ioctl() to determine the tty behind stdin, so you can redirect from another tty (assuming you have permission) to see its settings. So, assuming /dev/ttys000 is a different window, I can 'stty -a < /dev/ttys000' and see how the flags change between cat, bash, and vim for example.

Mark Dominus

@njvack @b0rk In Unix there are parts of the kernel, called "device drivers", that handle device I/O. In this case it's the "terminal driver". The terminal driver gets control when a process issues a `read()` or `write()` call on a file descriptor that has been opened to a device. Also, when the hardware I/O bus signals an I/O interrupt, the kernel transfers control to the appropriate driver the handle the interrupt.

When you type a character on a terminal, there is an I/O interrupt and the kernel asks the terminal driver to handle it. Normally, the driver just copies the character into a per-device buffer, waiting for the next read() call from the user process that has the terminal open. If the character was a control-W, though, the driver instead erases characters out of that buffer back to the last whitespace. Later, when the user process does `read()` to ask the driver for the contents of the buffer, the erased characters will be gone as if they had never been typed.

The driver may also send some delete characters back to the terminal to cause it to backspace and delete the erased word. (Normally, the terminal is in "no echo" mode which means it doesn't display what you type on it, instead it only displays whatever the terminal driver sends back.) If the driver knows you're on a non-backspacing terminal it may send something else to try to indicate that the word was deleted.

This is all in what's called "cooked" mode. Unix terminal devices also have a "raw" mode where the driver just copies stuff into the buffer with no processing. Tools like readline put the terminal into raw mode.

@njvack @b0rk In Unix there are parts of the kernel, called "device drivers", that handle device I/O. In this case it's the "terminal driver". The terminal driver gets control when a process issues a `read()` or `write()` call on a file descriptor that has been opened to a device. Also, when the hardware I/O bus signals an I/O interrupt, the kernel transfers control to the appropriate driver the handle the interrupt.

kepstin

@njvack @b0rk I think this dates back to the days of slow remote serial terminals, or even teletypes? If you put some really basic line editing into the terminal itself and only send the entered text when pressing "enter", the experience feels a lot more responsive - and every program doesn't need to independently write its own implementation of backspace.

But yeah, very interesting legacy thing at this point, even tho many tools still rely on the behaviour.

I think in Linux it's implemented in the kernel's pty layer rather than separately in each terminal app?

@njvack @b0rk I think this dates back to the days of slow remote serial terminals, or even teletypes? If you put some really basic line editing into the terminal itself and only send the entered text when pressing "enter", the experience feels a lot more responsive - and every program doesn't need to independently write its own implementation of backspace.

kepstin

@njvack @b0rk I only just realized why the mode where the terminal does line editing is called "cooked" mode. It's in contrast to the mode where apps get all the key presses one by one without preprocessing - "raw" mode.

njvack

@kepstin @b0rk I do know that a lot of this dates back to teletypes and the like. "/dev/tty0" is referring to a teletype, and a pty is a pseudo-teletype, and "vt100" was a "DEC Video Terminal 100" system.

I suspect you could, today, hook a modern linux system up to a teletype over a serial line and have it work correctly with relatively little effort

Then you wouldn't have a "terminal emulator program" — you would have a literal terminal

hmm

kepstin

@njvack @b0rk oh, I agree that would probably work! I don't know anyone with a teletype, but I do know that one of my friends uses a pre-ANSI (not vt100 compatible) monochrome IBM terminal connected to their Linux box with only minor issues. Issues are mostly apps that don't use terminfo or termcap and assume an ANSI terminal is in use - they often print escape codes the terminal doesn't understand. Using "screen" as a translator is the easiest workaround.

njvack

@kepstin @b0rk hmmmmm looks like you can get a DECWriter for as little as $600 plus shipping

I could spend a lot of money and learn to use `ed`

I cannot think of a worse idea, and yet

Go Up