Carriage returns and line feeds in TTY

It is often said that line breaks are <LF> on Linux and <CR><LF> on Windows, and even on Linux, TTY has additional handling of line breaks.

In the last century, the physical terminals that TTY emulates, the prototype typewriters, really had carriage return and line feed, so in TTY output, even though <LF> should have been used as a line feed, it was output as <CR><LF> in order to emulate the logic of a typewriter. The behavior of carriage returns and line feeds can be briefly summarized as follows.

For <CR>, you can use the Enter key on the keyboard, or Ctrl + M, but for <LF>, you can only use Ctrl + J, since this key is no longer on the keyboard.

Linux Kernal’s Terminal Driver works in canonical (also called cooked) mode, which converts <CR> to <LF> on input and <LF> to <CR><LF> on output.

Feel the phenomenon

First create an interactive program interactive.sh which simply asks who is here and says hello:

#!/usr/bin/env bash
# with name interactive.sh

echo "Who's here?"
echo -n "> "
read name

echo "Oh hello, $name"

Then create an expect script, interactive.exp, to interact with the interactive program you just created instead of manually. It will start the interactive program and enter “Normal User\n” when the interactive program outputs “Who’s here”, and end when the interactive program exits.

#!/usr/bin/expect -f

# exp_internal 1        # Uncomment this for debug output
spawn "./interactive.sh"

expect {
    "Who's here?\r" {               # Must end with "\r" or "\r\n" or neither
        exp_send "Normal User\n" ;  # Can end with either "\r" or "\n"
        exp_continue;
    }
    eof
}

In the expect script, when expecting “Who’s here?” output, if you want to match line breaks, you need to use \r or \r\n to match some or all of the line breaks. If you enable debug output in interactive.exp, you will be able to see a line like the one below, which tells you that the actual text being matched is Who's here?\r\n>, and naturally, you can only use \r or \r\n to match text with a \r\n line break.

expect: does "Who's here?\r\n> " (spawn_id exp4) match glob pattern "Who's here?\r"? yes

TTY conversion of line breaks in interactive.exp

One counterintuitive aspect when matching the output of “interactive.sh” is that \n cannot be used to match newline characters. This is speculated to be because “expect” virtualizes a terminal but does not handle newline characters properly, and combined with TTY converting \n to \r\n during output (the newline character output by “interactive.sh” is \n, without further detailed verification), “expect” ends up receiving \r\n, so \r\n must be used for matching in the script.

When providing input to “interactive.sh,” newline characters can be entered using either \r or"\n freely. This is because TTY converts \r to \n during input, ensuring that the program receives \n regardless. Additionally, the default delimiter for the Bash “read” command is $IFS, which includes whitespace, tab, and newline (\t\n). This allows the “read” command to correctly recognize \n as the termination character, enabling the reading of the “Normal User” string.