Bug in terminal when using curses app over ssh

[noisetonepause]

I'm filing this as a bug in Terminal, though I think this is related to termios.h.

If you look at the screenshots, you'll see the curses app silc-client running over an SSH connection from my friend's FreeBSD machine. Silc-client is based on irssi, so shouuld be familiar. It's basically a irc-like chat application. The line below the blue line is an input field. I type some input and press enter. As can be seen from the second screenshot, all the text is then moved up a line, including my text from the input box (which you'd expect to be erased). The line is written to the where'd you expect it to go on the first screenshot (ie., first empty line).

Incoming text from the server and other chatters works fine. Cycling through the 'windows' in the client redraws the text as you'd expect. The only problem is when inputting text.

The screenshots are from BeOS but the exact same thing happens in Haiku. An xterm running under BeOS/X11 does not have this issue, though.

I've compared the termios.h headers from Haiku, BeOS and Free+OpenBSD, and BeOS defines VKILL and VEOL opposite. What I think is happening is that silc-client, which is linked against BSD's libcurses, is sending what's defined as VKILL on BSD to the Haiku terminal app, which is interpreting it as VEOL. That seems consistent with the issue, at least.

[noisetonepause]

1st screenshot

[noisetonepause]

2nd screenshot

[noisetonepause]

I tried swapping the definitions of VKILL and VEOL in headers/posix/termios.h and recompiling, but that it's the same... still digging, would appreciate pointers!

[bonefish]

Unfortunately Trac doesn't let me see the attachments. *sigh*

The V* macros are just indices into termios::c_cc. I doubt that the termios structure or the indices are ever sent over the ssh connection. Or if they are, I'm pretty sure the OpenSSH authors have considered that those indices can differ (they are also different in Linux). I wouldn't expect the problem there.

It is more likely that certain xterm features are used that Haiku's Terminal doesn't support yet. Or, of course, there might still be bugs in the supported features.

[noisetonepause]

Comment(by bonefish):

Unfortunately Trac doesn't let me see the attachments. *sigh*

The screenshots are also at nisbeth.dk/niklas/silc.png and nisbeth.dk/niklas/borkd.png.

If anyone wants to experience it first hand, that can probably be arranged.

The V* macros are just indices into termios::c_cc. I doubt

that the termios structure or the indices are ever sent over the ssh connection. Or if they are, I'm pretty sure the OpenSSH authors have considered that those indices can differ (they are also different in Linux). I wouldn't expect the problem there.

No, you really wouldn't, but the symptoms fit perfectly (a newline instead of a kill). I've no real idea about exactly how curses apps interact with the terminal, but I don't think SSH comes into play? Isn't SSH just a transport?

I'm trying to dig up as much documentation on this stuff as I can. Happily enough, that "Porting UNIX Applications" book which I bought thinking I'd never use it for anything provided me a few good starting points...

It is more likely that certain xterm features are used that Haiku's Terminal doesn't support yet. Or, of course, there might still be

bugs in the supported features.

I was afraid you'd say that. I'm hoping there's some way around debugging it that doesn't involve going learning the gritty details of all the involved software, but I can't really think of anything. More digging, I suppose.

[noisetonepause]

Now this I wasn't expecting:

When I run Haiku in QEMU and telnet to the host OS (FreeBSD), silc-client works flawlessly.

Odd. Very odd?

[jackburton]

Could depend on the keymap ? Qemu by default emulates an US keymap.

[noisetonepause]

Replying to jackburton:

Could depend on the keymap ? Qemu by default emulates an US keymap.

Ah, you misunderstand - it would appear that SSH is the problem, as it works over Telnet but now SSH.

[bonefish]

In case this issue still exists, please compare the environment variables of the telnet and the ssh sessions. It is possible that ssh forwards less or more variables from the local host.

[pepol]

The problem was checked during GCI 2011. I haven't noticed such behavior, using irssi over ssh on hrev43238, that runs on VirtualBox 4.1.4 on my Debian system.

[siarzhuk]

Replying to bonefish:

In case this issue still exists, please compare the environment variables of the telnet and the ssh sessions. It is possible that ssh forwards less or more variables from the local host.

Environment is identical besides of SSH-specific variables. But I found a bit more interesting comparing

stty -a

output in the ssh and telnet sessions.

SSH:

[zzbsd]/home/zharik % stty -a                                                                                                 [22:07]
speed 9600 baud; 39 rows; 134 columns;
lflags: icanon isig -iexten echo echoe -echok -echoke echonl -echoctl
        -echoprt -altwerase -noflsh -tostop -flusho -pendin -nokerninfo
        -extproc
iflags: -istrip icrnl -inlcr -igncr -ixon -ixoff -ixany imaxbel -ignbrk
        brkint -inpck -ignpar -parmrk
oflags: opost onlcr -ocrnl tab0 -onocr -onlret
cflags: cread cs8 -parenb -parodd hupcl -clocal -cstopb -crtscts -dsrflow
        -dtrflow -mdmbuf
cchars: discard = ^O; dsusp = ^Y; eof = ^D; eol = ^@; eol2 = ^@;
        erase = ^?; erase2 = ^H; intr = ^C; kill = ^U; lnext = ^V;
        min = 1; quit = ^\; reprint = ^R; start = ^S; status = ^T;
        stop = ^Q; susp = ^Z; time = 0; werase = ^W;

Telnet:

[zzbsd]/home/zharik % stty -a                                                                                                 [22:06]
speed 9600 baud; 39 rows; 134 columns;
lflags: icanon isig iexten echo echoe -echok echoke -echonl echoctl
        -echoprt -altwerase -noflsh -tostop -flusho -pendin -nokerninfo
        -extproc
iflags: -istrip icrnl -inlcr -igncr ixon -ixoff ixany imaxbel -ignbrk
        brkint -inpck -ignpar -parmrk
oflags: opost onlcr -ocrnl tab0 -onocr -onlret
cflags: cread cs8 -parenb -parodd hupcl -clocal -cstopb -crtscts -dsrflow
        -dtrflow -mdmbuf
cchars: discard = ^O; dsusp = ^Y; eof = ^D; eol = <undef>;
        eol2 = <undef>; erase = ^?; erase2 = ^H; intr = ^C; kill = ^U;
        lnext = ^V; min = 1; quit = ^\; reprint = ^R; start = ^Q;
        status = ^T; stop = ^S; susp = ^Z; time = 0; werase = ^W;

Note the difference in echonl local flag. It is set for ssh session but disabled for telnet one. Obviously I have issued "stty -echonl" command in ssh session and the issue is not observed in this session anymore. Looks like it was the source of this problem.

The question, before I dig into it deeply some time in the future, has anybody quick ideas how to fix it permanently?

PS:Note that lot of flags fior telnet and ssh are different and even undefined like eol/eol2 for telnet sessions. Is it OK at all?

[Kev]

First apologies if this is a separate bug, but I wanted to chime in because I think I'm experiencing something related, although much of the discussion here is over my head.

If I open Terminal in hrev46102 and ssh into a CentOS box, it seems to work OK most of the time, but if I launch GNU nano from ssh, then strange things happen. Whenever I hold down the right arrow key, for instance, it will wipe out a random third of characters as the cursor moves along, and occasionally jump a whole tab full at once. However, it hasn't actually sent these space characters over the wire, because if I save the file and reload, it looks normal again. This makes nano over ssh pretty tough to use practically.

I was about to file a new bug report, but I found out nano uses curses, so...here I am. I can add screenshots or if you think this is different I'll file a new one after all.

[siarzhuk]

Replying to Kev:

If I open Terminal in hrev46102 and ssh into a CentOS box, it seems to work OK most of the time, but if I launch GNU nano from ssh, then strange things happen. Whenever I hold down the right arrow key, for instance, it will wipe out a random third of characters as the cursor moves along, and occasionally jump a whole tab full at once. However, it hasn't actually sent these space characters over the wire, because if I save the file and reload, it looks normal again. This makes nano over ssh pretty tough to use practically.

Does it occure at working with nano on hosts with other OS-es? What terminal info the command infocmp xterm returns on your CentOS box?

PS: You can acquire dump of programs output by using Ctrl-Alt-C shortcut. It is like toggle switch - press it once to activate capturing, reproduce the problem in your ssh session in the Terminal than press the shortcut again to stop capturing. Capture logs will be stored under /var/log/ directory.

[Kev]

terminal capture of arrow key making spaces

[Kev]

I don't have any other OSes to try, but infocmp xterm says:

$ infocmp xterm
#       Reconstructed via infocmp from file: /usr/share/terminfo/x/xterm
xterm|xterm terminal emulator (X Window System),
        am, bce, km, mc5i, mir, msgr, npc, xenl,
        colors#8, cols#80, it#8, lines#24, pairs#64,
        acsc=``aaffggiijjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~,
        bel=^G, blink=\E[5m, bold=\E[1m, cbt=\E[Z, civis=\E[?25l,
        clear=\E[H\E[2J, cnorm=\E[?12l\E[?25h, cr=^M,
        csr=\E[%i%p1%d;%p2%dr, cub=\E[%p1%dD, cub1=^H,
        cud=\E[%p1%dB, cud1=^J, cuf=\E[%p1%dC, cuf1=\E[C,
        cup=\E[%i%p1%d;%p2%dH, cuu=\E[%p1%dA, cuu1=\E[A,
        cvvis=\E[?12;25h, dch=\E[%p1%dP, dch1=\E[P, dl=\E[%p1%dM,
        dl1=\E[M, ech=\E[%p1%dX, ed=\E[J, el=\E[K, el1=\E[1K,
        flash=\E[?5h$<100/>\E[?5l, home=\E[H, hpa=\E[%i%p1%dG,
        ht=^I, hts=\EH, ich=\E[%p1%d@, il=\E[%p1%dL, il1=\E[L,
        ind=^J, indn=\E[%p1%dS, invis=\E[8m,
        is2=\E[!p\E[?3;4l\E[4l\E>, kDC=\E[3;2~, kEND=\E[1;2F,
        kHOM=\E[1;2H, kIC=\E[2;2~, kLFT=\E[1;2D, kNXT=\E[6;2~,
        kPRV=\E[5;2~, kRIT=\E[1;2C, kb2=\EOE, kbs=\177, kcbt=\E[Z,
        kcub1=\EOD, kcud1=\EOB, kcuf1=\EOC, kcuu1=\EOA,
        kdch1=\E[3~, kend=\EOF, kent=\EOM, kf1=\EOP, kf10=\E[21~,
        kf11=\E[23~, kf12=\E[24~, kf13=\E[1;2P, kf14=\E[1;2Q,
        kf15=\E[1;2R, kf16=\E[1;2S, kf17=\E[15;2~, kf18=\E[17;2~,
        kf19=\E[18;2~, kf2=\EOQ, kf20=\E[19;2~, kf21=\E[20;2~,
        kf22=\E[21;2~, kf23=\E[23;2~, kf24=\E[24;2~,
        kf25=\E[1;5P, kf26=\E[1;5Q, kf27=\E[1;5R, kf28=\E[1;5S,
        kf29=\E[15;5~, kf3=\EOR, kf30=\E[17;5~, kf31=\E[18;5~,
        kf32=\E[19;5~, kf33=\E[20;5~, kf34=\E[21;5~,
        kf35=\E[23;5~, kf36=\E[24;5~, kf37=\E[1;6P, kf38=\E[1;6Q,
        kf39=\E[1;6R, kf4=\EOS, kf40=\E[1;6S, kf41=\E[15;6~,
        kf42=\E[17;6~, kf43=\E[18;6~, kf44=\E[19;6~,
        kf45=\E[20;6~, kf46=\E[21;6~, kf47=\E[23;6~,
        kf48=\E[24;6~, kf49=\E[1;3P, kf5=\E[15~, kf50=\E[1;3Q,
        kf51=\E[1;3R, kf52=\E[1;3S, kf53=\E[15;3~, kf54=\E[17;3~,
        kf55=\E[18;3~, kf56=\E[19;3~, kf57=\E[20;3~,
        kf58=\E[21;3~, kf59=\E[23;3~, kf6=\E[17~, kf60=\E[24;3~,
        kf61=\E[1;4P, kf62=\E[1;4Q, kf63=\E[1;4R, kf7=\E[18~,
        kf8=\E[19~, kf9=\E[20~, khome=\EOH, kich1=\E[2~,
        kind=\E[1;2B, kmous=\E[M, knp=\E[6~, kpp=\E[5~,
        kri=\E[1;2A, mc0=\E[i, mc4=\E[4i, mc5=\E[5i, meml=\El,
        memu=\Em, op=\E[39;49m, rc=\E8, rev=\E[7m, ri=\EM,
        rin=\E[%p1%dT, rmacs=\E(B, rmam=\E[?7l, rmcup=\E[?1049l,
        rmir=\E[4l, rmkx=\E[?1l\E>, rmm=\E[?1034l, rmso=\E[27m,
        rmul=\E[24m, rs1=\Ec, rs2=\E[!p\E[?3;4l\E[4l\E>, sc=\E7,
        setab=\E[4%p1%dm, setaf=\E[3%p1%dm,
        setb=\E[4%?%p1%{1}%=%t4%e%p1%{3}%=%t6%e%p1%{4}%=%t1%e%p1%{6}%=%t3%e%p1%d%;m,
        setf=\E[3%?%p1%{1}%=%t4%e%p1%{3}%=%t6%e%p1%{4}%=%t1%e%p1%{6}%=%t3%e%p1%d%;m,
        sgr=%?%p9%t\E(0%e\E(B%;\E[0%?%p6%t;1%;%?%p2%t;4%;%?%p1%p3%|%t;7%;%?%p4%t;5%;%?%p7%t;8%;m,
        sgr0=\E(B\E[m, smacs=\E(0, smam=\E[?7h, smcup=\E[?1049h,
        smir=\E[4h, smkx=\E[?1h\E=, smm=\E[?1034h, smso=\E[7m,
        smul=\E[4m, tbc=\E[3g, u6=\E[%i%d;%dR, u7=\E[6n,
        u8=\E[?1;2c, u9=\E[c, vpa=\E[%i%p1%dd,

Attached is a log from modifying smb.conf using nano. Note the spaces in the names between the commas. All I was doing was holding down the right arrow for most of it.

[siarzhuk]

Thanks for the info.

The infocmp output is almost identical with Haiku's current xterm. So it is ok with it, I think.

In the capture log are raw data received by Terminal from the program - so this is data "with spaces" were send by remote programm - nano. I have checked remote nano on my freebsd system as described: 1) I have opened smb.conf and 2) press the right arrow key and hold it pressed for a long time. No visual problems were observed in my case. So I have to suspect nano at your CentOS as the troublemaker.

May be you did something else to get the problem in your case? Please provide me with exact reproduce how-to sequence, than. Thank you.

PS: What remote nano tries to do with the Terminal:

// position cursor at columns from 4 to 15
[4d[5d[6d[7d[8d[9d[10d[11d[12d[13d[14d[15d

// move cursor up three times
[A[A[A

// output the line
         dc 

// position cursor at column 38
[38d

// erase to right until the end of line
[K

// position cursor absolute to row 12, column 12
[12;12H

// print all this as single line
rpc     ndpoint servers = epmap er, wks vc, rpc cho, sa r, netl gon, ls rpc, sp olss, d suapi,  ssetup, unixinf , brows r, even

[Kev]

Replying to siarzhuk:

Thanks for the info.

The infocmp output is almost identical with Haiku's current xterm. So it is ok with it, I think.

In the capture log are raw data received by Terminal from the program - so this is data "with spaces" were send by remote programm - nano. I have checked remote nano on my freebsd system as described: 1) I have opened smb.conf and 2) press the right arrow key and hold it pressed for a long time. No visual problems were observed in my case. So I have to suspect nano at your CentOS as the troublemaker.

I'm not sure how that's possible. I have no problems doing the exact same thing with another CentOS box as the client, or PuTTY on Windows as the client, or MobaXTerm for Windows as the client.

May be you did something else to get the problem in your case? Please provide me with exact reproduce how-to sequence, than. Thank you.

Sure:

1. ssh ..., enter
2. type password, enter
3. sudo nano /etc/samba/smb.conf, enter
4. type password, enter
5. hold down right arrow key: makes a mess on every line:

# Globa parame ers [global [...security edit...]

server ole = a tive di ectory omain c ntrolle server ervices = rpc, bt, wre l, ldap cldap, kdc, dr pl, win ind, nt _signd, kcc, dn update, s3fs, dns

Although that's something I didn't notice before, it seems to be hitting the 8th or 9th character from the beginning of each line.

[Kev]

So I have to suspect nano at your CentOS as the troublemaker.

I'm not sure how that's possible. I have no problems doing the exact same thing with another CentOS box as the client, or PuTTY on Windows as the client, or MobaXTerm for Windows as the client.

I just tried vi, which I don't normally use, and it doesn't exhibit this behaviour. So maybe to be more precise, it's not nano at my CentOS, and it's not Terminal, but the combination of the two. Change either of them for something else and it works. The question is, is it one or the other or both that might need changing in some way.

[Kev]

I made a test file with 0123456789 repeated 5 times per line. I then saved, opened, and ran the cursor along it for a few lines, and got:

0123456 8901234 6789012 4567890 2345678 0123456 89 0123456 8901234 6789012 4567890 2345678 0123456 89 0123456 8901234 6789012 4567890 2345678 0123456 89 0123456 8901234 6789012 4567890 2345678 0123456 89 0123456 8901234 6789012 4567890 2345678 0123456 89 01234567890123456789012345678901234567890123456789 01234567890123456789012345678901234567890123456789 01234567890123456789012345678901234567890123456789 01234567890123456789012345678901234567890123456789

...so something resets "it" at the beginning of each line, and it deletes every 8th character after a reset.

[diver]

#10031 sounds related.

[siarzhuk]

A pair of tickets that are left since I was fond of Terminal improvements. I'm not sure I can finalize them in the nearest future.

[waddlesplash]

Is this still reproducible?

[jessicah]

Terminal has had a number of improvements over the years, and various examples listed are not reproducible.