SSH reverse tunneling broken (seemingly not a haikuport issue)

[ttcoder]

We use SSH for reverse-tunneling to stations

That stopped working some time ago, ca. the time of the libbind/libresolv switch over or so

First filed a ticket against the OpenSSH port as it was updated around the same time, ​https://github.com/haikuports/haikuports/issues/288 but it turns out that even if reverting to an older version of OpenSSH, the regression is still there; so looks like it's the hrev change which triggers the bug.

This pretty much means we have to keep using older (491xx) hrevs for now since we've grown quite dependant on that feature. Those work ok for us so no biggie... at this point

[ttcoder]

See haikuports ticket for symptoms ..etc.

Also of note, I'm trying to find a workaround by tweaking the ssh config files. It's possible to look around after hrev49619, but didn't find an actual workaround yet. Oddly, specifying ClearAllForwardings=yes shuts up the connect_to localhost: unknown host (No address associated with hostname) error, but gives me a "connection refused" error instead..

[pulkomandy]

A more precise revision range (last known working / first known broken) would help, even if they are quite far apart. Also taking an strace of openssh may give some hints as to what's going on. Please attach the output of:

strace ssh

with the ssh command line options that trigger the problem (which I can also use to try to reproduce the issue).

[kallisti5]

This looks like a duplicate of #12260... I need to confirm #12260 is still an issue though.

[ttcoder]

Indeed kallisti5, and the work-around described over there works great, thanks!

(for reference:

~/config> cat /system/settings/network/hosts
127.0.0.1       localhost

)

With this work-around in place, the priority I set seems heavy-handed now, can be set back to 'normal' I guess

I might still work on getting some strace data to you pulkomandy; if there is a part of haiku that still wants to access the deprecated 'hosts' file there is probably interest in finding it and correcting it?

Indeed the strace indicates that ssh does access the hosts file, though it does not readily indicate if that access comes from a userland function (like gethostbyname() ..etc) in a library or the ssh tool or something else:

[ 16167] _kern_getpeername(0x3, 0x72dabb08, [0x20]) = 0x00000000 No error (2 us)
[ 16167] _kern_getpeername(0x3, 0x72dabaf4, [0x20]) = 0x00000000 No error (1 us)
[ 16167] _kern_write(0x3, 0xffffffffffffffff, 0x181e7e18, 0x38) = 0x00000038 (17 us)
[ 16167] _kern_getsockopt(0x3, 0x6, 0x1, 0x72dabb80, [0x4]) = 0x00000000 No error (2 us)
[ 16167] _kern_setsockopt(0x3, 0x6, 0x1, 0x72dabb80, 0x4) = 0x00000000 No error (1 us)
[ 16167] _kern_getsockname(0x3, 0x72dabad8, [0x20]) = 0x00000000 No error (1 us)
[ 16167] _kern_setsockopt(0x3, 0x0, 0x3, 0x72dabb94, 0x4) = 0x00000000 No error (1 us)
[ 16167] _kern_sigaction(0x1, (nil), 0x72daba60) = 0x00000000 No error (1 us)
[ 16167] _kern_sigaction(0x1, 0x72daba74, (nil)) = 0x00000000 No error (1 us)
(..)
[ 16167] _kern_select(0x4, [3 100 101 102 103 106 107 108 ...], [68 69 71 73 75 76 77 79 ...], (nil), 0xffffffffffffffff, (nil)) = 0x1 (9607827 us)
[ 16167] _kern_read(0x3, 0xffffffffffffffff, 0x72da9a98, 0x2000) = 0x00000058 (10 us)
[ 16167] _kern_create_sem(0x0, "pthread rwlock") = 0x001bf28b (3 us)
[ 16167] _kern_read_stat(0xffffffff, "/etc/nsswitch.conf", true, 0x72dab6f0, 0x58) = 0x80006003 No such file or directory (35 us)
[ 16167] _kern_open(0xffffffff, "/boot/system/settings/network/hosts", 0x0, 0x0) = 0x00000004 (19 us)

[ttcoder]

Yay, an even better work-around: I simply pass 127.0.0.1 instead of localhost as an ssh arg, no need for a hosts file then!

For what it's worth / in case of interest:

Before that (when still referring to "localhost") I ran strace with and without a hosts file to try and spot a difference, but the "without" is truncated, probably because of output buffering; in C++ I would use fflush(stdout) to solve that problem but in bash dunno..

Spotting differences between my st_WithHosts and st_With127001 files was more fruitful: they are identical, except for this very block, precisely 7 lines that are "inserted" in st_WithHosts:

[ 19334] _kern_create_sem(0x0, "pthread rwlock") = 0x00204a82 (4 us)
[ 19334] _kern_read_stat(0xffffffff, "/etc/nsswitch.conf", true, 0x72c4b930, 0x58) = 0x80006003 No such file or directory (35 us)
[ 19334] _kern_open(0xffffffff, "/boot/system/settings/network/hosts", 0x0, 0x0) = 0x00000004 (19 us)
[ 19334] _kern_read_stat(0x4, (nil), false, 0x72c496e0, 0x58) = 0x00000000 No error (4 us)
[ 19334] _kern_read(0x4, 0xffffffffffffffff, 0x182113b8, 0x2000) = 0x00000014 (9 us)
[ 19334] _kern_read(0x4, 0xffffffffffffffff, 0x182113b8, 0x2000) = 0x00000000 (2 us)
[ 19334] _kern_close(0x4) = 0x00000000 No error (5 us)

EDIT: also got to check whether RemoteDesktop (e.g. ​here) is affected as well.

[kallisti5]

should be resolved in hrev49626. Please test.

Thanks!

[ttcoder]

Please close --

I can indeed connect in all cases now (with the initial line based on localhost and the one based on 127.0.0.1). Did my testing by lifting the contents of the new hosts file from the 49626 change-log rather than installing a fresh nightly but should not matter. Thanks all.

[diver]

Thanks for the update.