[WebPositive] https redirects seems to be broken

[diver]

hrev51365 x86_64

WebPositive ​http://ya.ru doesn't work while ​https://ya.ru works same with google.ru - http doesn't work However both ​http://google.com and ​https://google.com don't load at all.

[pulkomandy]

It would be interesting to build libbnetapi in debug mode and use that to get some logs for network events. However I fear it would make the issue disapear.

[diver]

Indeed, building libnetapi in debug mode by adding in build/jam/UserBuildConfig

HAIKU_BUILD_FEATURE_SSL = 1 ;
SetConfigVar DEBUG : HAIKU_TOP src kits network libnetapi : 1 : global ;

and running:

LD_PRELOAD=/data/haiku/generated/objects/haiku/x86_64/debug_1/kits/network/libnetapi/ssl/libbnetapi.so WebPositive http://ya.ru

makes this problem go away...

[pulkomandy]

The problem also happens (but less frequently) on x86. It is timing/scheduling related.

The code for redirects in Web+ is quite messy, and does not use the libnetapi handling because the browser needs to be aware of redirections to update the "current" URL. When a redirect happens, the WebKit code will immediately delete the HTTP request and create a new one for the new URL. However, all messages queued by the original request (as BMessages to the WebKit main app) will still be processed. The code tries to detect and ignore these, but it doesn't work reliably.

My guess is that it interprets the "end of request" from the previous request and ends the second one because of it. When debug is enabled, the logging changes the timing and the problem doesn't happen.

[korli]

Replying to PulkoMandy:

The problem also happens (but less frequently) on x86. It is timing/scheduling related.

If so, then it's a duplicate of #13642?

[pulkomandy]

Probably, but it's hard to tell if it's exactly the same problem without any logs.

[diver]

I *think* Web+ still worked for me when 13642 was created. It might be the same issue in the end. What kind of logs are needed? Do I need to open a page with debug build of libnetapi that doesn't open otherwise and attach stdout log?

[korli]

I built libbnetapi with DEBUG=1 (in SubDirC++Flags) and launched WebPositive on ​http://www.free.fr, a pop up with "The URL cannot be shown" is displayed.

    HTTP: Resolving http://www.free.fr/
    HTTP: Hostname resolved to: 212.27.48.10:80
    HTTP: Connection to www.free.fr on port 80.
    HTTP: Connection opened, sending request.
--> HTTP: Host: www.free.fr
--> HTTP: Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
--> HTTP: Accept-Encoding: gzip
--> HTTP: Connection: close
--> HTTP: User-Agent: Mozilla/5.0 (Macintosh; Intel Haiku R1 x86) AppleWebKit/602.1.26 (KHTML, like Gecko) WebPositive/1.2 Version/9.2 Safari/602.1.26
    HTTP: Request sent.
    HTTP: Status line received: Code 302 (Moved Temporarily)
<-- HTTP: Server: nginx
<-- HTTP: Date: Thu, 26 Oct 2017 15:09:22 GMT
<-- HTTP: Content-Type: text/html
<-- HTTP: Content-Length: 154
<-- HTTP: Connection: close
<-- HTTP: Location: http://www.free.fr/freebox/index.html

Dunno if that helps.

[pulkomandy]

Please try ​https://github.com/haiku/webkit/commit/163fb721b25cbab00eb6387f2d052bf54dacdc4e .

[korli]

Hey Adrien, thanks, I'm away for a few days, won't be able to test until next week.

[pulkomandy]

There is another problem left, apparently a memory corruption. I'm trying to understand what happens, eventually it seems I get a BHttpHeader where the fName Bstring ends up pointing to freed memory for its fPrivateData. But I don't see a scenario which would lead to that. It happens betweenTrim and CapitalizeEachWord in the SetName method. At that point the BString is accessed only from one thread and it was just set to a new data buffer initialized with the name.

Possibly a bug in BString or we are misusing the API here.

[korli]

Hi Adrien! The patch is not enough, the problem stays the same.

It seems the relevant code showing the popup was introduced in ​https://github.com/haiku/webkit/commit/95db3d3d08d3e64072caca4ef38c3c54152e697d. Which means that a valid base URL is required on a redirect. A URL object only becomes valid when parsing a URL string.

I changed the used URL constructor in KURLHaiku.cpp and the behavior described in this bug is then fixed. This is basically a revert of the first part of ​https://github.com/haiku/webkit/commit/4d26e350bbbf2502d00657a1d8c73e5e81018bb5

   parse(String.fromUTF8(url.UrlString()));

[pulkomandy]

Sounds right, this constructor would lead to a partially uninitialized object (m_protocolIsInHttpFamily would not be set either, for example). No way to avoid re-parsing the URL from a string here, it seems. And I'm not sure the URL string would properly be allocated and generated, which may explain other problems as well.

[korli]

Patched in 1.6.2-3. Updated in hrev51515 for x86_64.

[diver]

Thanks, korli! Everything seems to be working fine now :)

[korli]

This should already be fixed, or not?

[pulkomandy]

Yes, changes were integrated in HaikuWebKit 1.6.3.