Terminal should provide an emulated Meta key

[simonsouth]

Currently Terminal provides no mechanism for emulating a Meta key, which makes it difficult to use certain console-based UNIX applications under Haiku. Terminal should be enhanced to support this.

I'll be uploading a patch that does this by allowing the left Option key to be used as a Meta key, but since I expect this may need a bit of context I'm also adding this ticket to provide some background.

What's a "Meta key"?

Certain early workstation keyboards included a key labeled "Meta" that functioned as an extra modifier key alongside Shift and Control. Its purpose was to give users an easy way of entering special characters: When held down it caused the eighth bit to be set on all characters read from the keyboard, allowing access to ​the "extended" portion of the ASCII character set (character codes above 127).

Since this functionality did not generalize well---not all serial connections support eight-bit data and not all character encodings use eight-bit characters---applications began also recognizing the Escape character as a signal that the next character should be interpreted as having been modified by the Meta key. Terminal emulators often provide a means of switching between these two behaviours, either setting the high-order bit or prefixing characters with Escape.

See ​the Wikipedia article on the subject for more.

What's the Meta key used for?

Although the Meta key is absent from modern keyboards, two major pieces of UNIX software still rely on it being present: GNU Emacs and the GNU readline library, which is used by bash and a wide variety of other software to read input from the terminal. (Python's interactive shell uses readline, for instance.) These applications use the Meta key as a "command" key and provide additional editing features when it's available.

How do other systems handle this?

Terminal emulators on Windows and Linux (on PC) normally interpret the Alt keys as Meta keys. This creates a conflict as the Alt keys are also used to access application menus on these platforms. Consequently terminal emulators often provide a means of disabling menu shortcuts, as do the emulators included with GNOME and Xfce.

macOS' Terminal app doesn't emulate the Meta key by default, but does provide a setting that makes the Option keys function as Meta keys.

What does this patch do?

This patch enhances our Terminal app to function like macOS': It adds a configuration option to the "Settings..." dialog that, when enabled, causes the left Option key (only) to function as a Meta key. The right Option key retains its normal function, and can be used to enter special characters at the keyboard---ironically the Meta key's original purpose.

With a PC keyboard plugged in, enabling this setting and un-swapping the assignment of the Command and Option keys using the Keymap preflet (oddly, these key assignments are reversed by default) places the Command key on the "Windows" key, the Meta key on the left Alt key and the Option key on the right Alt key, replicating very closely the layout described in ​the bash reference manual:

On keyboards with two keys labeled ALT (usually to either side of the space bar), the ALT on the left side is generally set to work as a Meta key. The ALT key on the right may also be configured to work as a Meta key or may be configured as some other modifier, such as a Compose key for typing accented characters.

This layout is also functionally equivalent to the default on Linux, with the added benefit of removing the conflict between the Meta key and the menu bar---one less hurdle for developers migrating from Linux to Haiku.

How can I test this?

With the patch applied, Terminal rebuilt and the option enabled in its "Settings..." dialog, you should find the left Option key (which may be the Windows key, if you're using a PC keyboard and haven't changed the key assignments) gives you new ways of editing the command line in bash:

• M-f ("Meta-f") and M-b move forwards and backwards between words.
• M-d deletes the word at the cursor.
• M-l, M-u and M-c change a word's letter case.

Terminal also now recognizes standard, xterm-compatible Escape sequences that control the Meta key's behaviour. By default, the "interpret Meta" (set eighth bit) and "Meta sends Escape" behaviours are both enabled, with Meta-sends-Escape taking precedence. To disable Meta-sends-Escape, at a bash prompt enter

echo -e '\E[?1036l'

To disable interpret-Meta as well, enter

tput rmm  # "reset Meta mode"

(Happily, that Escape sequence has a mnemonic.) With both behaviours disabled, the Meta key functions as a regular Option key.

To re-enable both behaviours, enter

tput smm  # "set Meta mode"
echo -e '\E[?1036h'

Note there are two wrinkles with using interpret-Meta:

• In Haiku's default encoding of UTF-8 the eighth bit has a special meaning: It signals the start or continuation of a multi-byte character. Consequently, Terminal cannot be arbitrarily setting the eighth bit of characters that might be printed to the console.
• For the same reason, bash itself by default does not recognize interpret-Meta mode when the currently active encoding supports multibyte characters.

The patch handles the first issue by re-encoding the output when both interpret-Meta mode and UTF-8 encoding are in use, following ​the logic implemented by xterm (which Terminal claims to emulate; see "echo $TERM").

The second issue is handled by selecting a different character encoding in Terminal and then re-enabling interpret-Meta support in bash.

To test interpret-Meta mode, then,

• Change Terminal's character encoding to ISO-8859-1 (or similar) via its "Settings" menu.
• At the bash prompt, enter

bind 'set convert-meta on'
tput smm
echo -e '\E[?1036l'

Meta-key commands should function as they did before, but now Terminal is setting the eighth bit on characters entered with the Meta key down. To restore the default, Escape-prefixing behaviour:

echo -e '\E[?1036h'
tput rmm  # Optional; Meta-sends-Escape takes precedence regardless
bind 'set convert-meta off'

and then set Terminal's character encoding back to UTF-8.

What's left to do?

It would be nice to give the user more flexibility in reassigning the Option keys, perhaps along the lines of ​how macOS' iTerm2 does it. Providing a nice, point-and-click means of toggling the Meta-sends-Escape behavior might be of some value as well.

These things should probably be done in the context of redesigning Terminals' settings UI, which has become a bit disorganized.

[simonsouth]

The patch is here, for review: ​https://review.haiku-os.org/c/haiku/+/1727

[waddlesplash]

What is the reason for reading the keymap file itself, instead of using get_key_map() and the key_map struct?

[simonsouth]

The keymap file itself isn't read, it's simply monitored for changes (using ​watch_node()). Once TermWindow receives notice the file has changed it reloads the keymap in the conventional manner, using get_key_map().

[waddlesplash]

Right, now I see that. I guess this really is the only way to handle things.

[simonsouth]

I've updated the patch on Gerrit so that TermView::DefaultState::KeyDown() (in TermViewStates.cpp) no longer queries the keymap in the (typical) case the Option key is used on its own. In this case the code can simply use the raw character provided in the message.

[simonsouth]

Jérôme's ​comment on Gerrit has me thinking this might be better implemented as a "virtual Meta key" ​input filter. This would have a straightforward way of listening for keymap changes and might even define a "B_META_KEY" modifier flag applications could easily check for. Not only would this be more elegant (Jérôme's right; the keymap-file monitoring is a hack) it could potentially be reused in other applications as well, such as the Haiku-native port of GNU Emacs I hope will one day exist.

I'm reading up on this now to see what's feasible.

[simonsouth]

The input-filter mechanism isn't nearly as flexible as I imagined and won't be of use here.

What I have done is follow Jérôme's suggestion and create a patch that makes the Input Server notify applications when a new key map has been loaded, using a newly defined message, B_KEY_MAP_LOADED:

​https://review.haiku-os.org/c/haiku/+/1741

(Note B_KEY_MAP_CHANGED is ​already taken.) The patch uses BRoster::Broadcast() to send the message to every application that's running. This appears safe and seems to work fine, but I'm open to feedback if there's a better approach.

I've also updated my patch for Terminal to have TermApp simply listen for B_KEY_MAP_LOADED and forward the message to its TermWindow when it arrives. This allows the node monitor to be dispensed with altogether, and feels like a much more elegant solution. Those changes can be seen here:

​https://review.haiku-os.org/c/haiku/+/1727

[waddlesplash]

Fix merged in hrev53461.

[nielx]

Assign tickets with status=closed and resolution=fixed within the R1/beta2 development window to the R1/beta2 Milestone