WebPositive unusable on slow internet connections

[bbjimmy]

WebPositive is a fast browser on a fast connection, But on a slow connection it chokes. Open a link in a new tab and the page/tab that was already completely loaded is unusable. One cannot scroll up or down until the new tab is loaded. The habit of the user is to open new tabs to let the link load in the background while reading the current page.

This is not acceptable. Both BeZillaBoowser and Arora on Haiku as well as IE and Firefox on Windows will allow the loaded tab to be browsed while there are new tabs loading.

[anevilyak]

That's most likely a limitation of the webkit libcurl backend Web+ currently uses for the networking side of things ; in order to be fixed properly, a new backend would need to be written, which is a bit less trivial unfortunately. It may be possible to leverage the Chromium backend in some way though.

[bbjimmy]

Isn't Arora using the same backend?

[anevilyak]

Replying to bbjimmy:

Isn't Arora using the same backend?

Arora's using the Qt backend as far as I'm aware.

[mmadia]

Hi. Our WebKit port (and thus WebPositive) was updated in July. Could you retest this in a new nightly image and report back? ​http://www.haiku-files.org/haiku/development Thanks!

[bbjimmy]

Using hrevr1alpha4-4458 on a faster connection, WebPositive is now worse than before. It hogs the cpu and locks up the app server while dificult pages are loading, and one cannot scroll on preveously loaded pages until all page loads are complete. This is more of a threading and priority problem than a webkit problem. If I lower the priority of Webpositive's main thread to normal, it plays well with other apps and no longer locks up the system. If I change WebPositve to allow multiple launch and launch all new pages in a new browser, I can use the loaded pages while other ones ( instances of WebPositive ) are loading pages.

[ttcoder]

@bbjimmy are you on a single-core computer ? If yes then the app_server lockup sounds like #8007 , #7882 ; though the inefficient networking back-end can be laid at the feet of W+ indeed :-j ...

[aldeck]

Since other factors might come into play, can you test again with the old WebPositive r-586 on your current Haiku revision? Available here ​http://haiku-files.org/files/optional-packages/

[bbjimmy]

Replying to aldeck:

Since other factors might come into play, can you test again with the old WebPositive r-586 on your current Haiku revision? Available here ​http://haiku-files.org/files/optional-packages/

The issue is the same with the older Web+ version.

[bbjimmy]

Replying to ttcoder:

@bbjimmy are you on a single-core computer ? If yes then the app_server lockup sounds like #8007 , #7882 ; though the inefficient networking back-end can be laid at the feet of W+ indeed :-j ...

Yes, I have a single core cpu, and these do indeed look like my issue. #8007 with many heavy web pages loaded. and #7882 would help playing well with other apps.

The page load, though, I beleive should be an alpha 4 blocker. Unless, we want to give the impression that Hailu is dog slow to everyone that will try it based on a new alpha release.

[bbjimmy]

hrevr1alpha4-44631 on 2 processors intel Atom 1.6ghz

load ​http://haiku-os.org in WebPositive: 51 seconds BeZilla: 11 seconds

Same internet connection. This is a real problem.

[anevilyak]

Making the ticket description so vague as to be practically useless isn't helpful.

[bbjimmy]

Since I reported the issue, it has gotten worse, The internet connection no longer needs to be slow to make WebPositive unbarably slow. Hence the change in the description. Web+ is now just plane unusable.

[bbjimmy]

Replying to anevilyak:

Making the ticket description so vague as to be practically useless isn't helpful.

Stating that it is only on slow internet connections is now misleading.

[anevilyak]

Replying to bbjimmy:

Stating that it is only on slow internet connections is now misleading.

1) That doesn't discount the fact that "Webpositive unusable" is completely useless as a ticket summary since it's so vague that it could equally apply to many completely unrelated situations.

2) Whatever's going on for you isn't a universal problem either way. I see no such issue with Web+ over here across many different sites and my connection's nothing special for instance.

[bbjimmy]

2) Whatever's going on for you isn't a universal problem either way. I see no such issue with Web+ over here across many different sites and my connection's nothing special for instance.

Whatever is going on for me is on multiple computers with multiple internet connections, and has been completely unbarable since the recent work on WebKit. Pretending it is not happening is not going to help HAIKU, quite the contraty, it will dammage HAIKU if we let R1A4 happen without resolving this issue. This seems to be the white elephant in the room that everyone is ignoring. It is time to stop ignoring it.

[tangobravo]

Nobody is doubting your particular symptoms. Equally there is no reason to doubt that anevilyak is not seeing the same thing on his setup, so it seems a reasonable conclusion that this bug does not affect all hardware. Getting angry is not the way to reach a solution.

I believe Web+ doesn't do any local caching, so will result in more network requests for each page load. I wonder if there is an issue with the network drivers in use on the machines you have tested. Could you provide more details of the hardware on which you have observed the slow page loads? Some network drivers suffered a performance regression after the latest FreeBSD driver update; the plan is to patch them manually for the alpha4 release so if you have others affected by the same issue it would be good to know about them before the release. See #8454

I agree that Web+ should be better at handling background tab loading, but it sounds like that will require a rewrite of the network backend which is a significant undertaking. Although we want R1A4 to be as good as possible, we have to accept it will not be perfect (otherwise it would be R1!) - it is still an alpha release, and people testing should assume that performance and stability are still all to be improved.

[anevilyak]

Thinking about it more, it's possible this is the same issue as (or related to) #6052 ; does removing Cookies.curl make a difference?

[bbjimmy]

Replying to anevilyak:

Thinking about it more, it's possible this is the same issue as (or related to) #6052 ; does removing Cookies.curl make a difference?

NO, I get the same result after removing /boot/home/config/settings/WebPositive/Cookies.curl , and the issue is present on a fresh install.

[anevilyak]

Replying to bbjimmy:

NO, I get the same result after removing /boot/home/config/settings/WebPositive/Cookies.curl , and the issue is present on a fresh install.

Then we're at a bit of an impasse unfortunately. Webkit's source code alone is in the same size range as all of Haiku's combined, so unless one of us who's familiar with it can actually reproduce and analyze this problem, it's near impossible to really make a good guess as to where the problem actually lies. As I said, I see no such problems over here, so I really can't even begin to guess where to look as far as why it's behaving so badly for you.

[bbjimmy]

Replying to tangobravo:

Nobody is doubting your particular symptoms. Equally there is no reason to doubt that anevilyak is not seeing the same thing on his setup, so it seems a reasonable conclusion that this bug does not affect all hardware. Getting angry is not the way to reach a solution.

I wasn't getting angry, just pointing out that the change in description I made was to improve understanding, not the opposite as was stated by anevilyak. His condecending tone suggested that this is just a figment of my immagination, and not a "real" problem.

I believe Web+ doesn't do any local caching, so will result in more network requests for each page load. I wonder if there is an issue with the network drivers in use on the machines you have tested. Could you provide more details of the hardware on which you have observed the slow page loads? Some network drivers suffered a performance regression after the latest FreeBSD driver update; the plan is to patch them manually for the alpha4 release so if you have others affected by the same issue it would be good to know about them before the release. See #8454

My hardware: on my desktop: 1 processor Inter Pentium 4 2.80 GHz 1528MiB

Realtek RTL-8139/8139C/8139C+

on my netbook: 2 processors Intel Atom 1.60GHz 1015MiB Atheros wifi AR242x/AR542x Wireless Network Adapter

Atheros network adapter

AR8121/AR8113/AR8114 Gigabit or Fast Ethernet

I agree that Web+ should be better at handling background tab loading, but it sounds like that will require a rewrite of the network backend which is a significant undertaking. Although we want R1A4 to be as good as possible, we have to accept it will not be perfect (otherwise it would be R1!) - it is still an alpha release, and people testing should assume that performance and stability are still all to be improved.

Indeed, but this is a large regression from the performance of R1A3.

[anevilyak]

Replying to bbjimmy:

I wasn't getting angry, just pointing out that the change in description I made was to improve understanding, not the opposite as was stated by anevilyak. His condecending tone suggested that this is just a figment of my immagination, and not a "real" problem.

I said no such thing, please don't put words in my mouth. My point was simply that you seemed to believe this to be a universal Web+ issue, which it decidedly is not, and that your new description was so vague as to be useless, which it was. Trac exists first and foremost as a resource for us developers to help us organize the problems we need to fix. If I go looking for tickets related to network performance and connectivity issues in Web+ to investigate, then "Webpositive unusable" isn't even going to register on my radar as being related to that since it's so vague that it could just as well describe UI issues, crashes or any number of other things that have nothing whatsoever to do with networking. Hence, while the old description may not be 100% accurate, it at least yields more of a hint as to the direction / origin of the bug than the new, hence why I put it back.

[tangobravo]

According to tqh (comment 37 of #8454), atheros and realtek are believed to be OK. You still might want to try ping and wget to see if you're getting acceptable network performance generally. You might also want to try grabbing the appropriate driver binary from the nightly for hrev43722 which was just before the latest FreeBSD driver update to see if that helps at all.

[bbjimmy]

Replying to tangobravo:

According to tqh (comment 37 of #8454), atheros and realtek are believed to be OK. You still might want to try ping and wget to see if you're getting acceptable network performance generally. You might also want to try grabbing the appropriate driver binary from the nightly for hrev43722 which was just before the latest FreeBSD driver update to see if that helps at all.

I compared the ping times with a Linux install on my desktop. Both Haiku and ubuntu are about the same. I installed the rtl8139 driver from hrev43722 and am still experiancing the same issue with extremely slow page loads in WebPositive.

I think this can be eliminated as the cause.

[anevilyak]

Another possibility is that your ISP is routing http traffic through a caching/filtering proxy, and that the resulting header rewriting is interacting badly with libcurl. Such a setup would certainly explain why it affects both systems, but only for Web+ and not other traffic, and why others aren't seeing such behavior. Assuming wifi is working on the netbook, a simple test to verify that might be to take it to a coffee shop or something else that has free wireless and see if similar issues still show up there.

[bbjimmy]

Replying to anevilyak:

Another possibility is that your ISP is routing http traffic through a caching/filtering proxy, and that the resulting header rewriting is interacting badly with libcurl. Such a setup would certainly explain why it affects both systems, but only for Web+ and not other traffic, and why others aren't seeing such behavior. Assuming wifi is working on the netbook, a simple test to verify that might be to take it to a coffee shop or something else that has free wireless and see if similar issues still show up there.

This may indeed be an issue at home, as I connect via sattelite internet access, but I have the same issue with the laptop at work, a cable connection, at a friends house ... a different cable provider, and at my remote work location, a third cable provider. I think we can rule this out as well.

[umccullough]

Preferably, we'll see some benchmarks comparing different versions of Web+ against each other on the same machine, same connection. (a stable connection - *NOT* satellite)

It should be done with clean installations and be repeatable from a fresh install. It would preferably be done via some kind of web-based browser test suite so that it can be recreated on a whim by others.

If this ticket finally boils down to "Web+ unusable on satellite", then I'll believe it - the latency involved and the funky caching that satellite modems utilize could probably cause untold numbers of issues for Haiku and Web+ at this stage.

Until we get some hard numbers and something that others can reproduce, I suspect this ticket will go nowhere - as it's too subjective and vague at this point.

[bbjimmy]

Replying to umccullough:

Preferably, we'll see some benchmarks comparing different versions of Web+ against each other on the same machine, same connection. (a stable connection - *NOT* satellite)

Older versions of WebPositive will not run on recent nightlies, and recent versions will not run on r1a3. This would require compiling WebPositive on an older version of HAIKU to compare then side by side.

It should be done with clean installations and be repeatable from a fresh install. It would preferably be done via some kind of web-based browser test suite so that it can be recreated on a whim by others.

If this ticket finally boils down to "Web+ unusable on satellite", then I'll believe it - the latency involved and the funky caching that satellite modems utilize could probably cause untold numbers of issues for Haiku and Web+ at this stage.

I have the same issue with the laptop at work, a cable connection, at a friends house ... a different cable provider, and at my remote work location, a third cable provider as well as a DSL connection. None of these are due to the Sattelite connection I use at home, and on an older version of WebPositive, in an earlier version of HAIKU, I do not experiance these extreemly slow page loads even on the Sattelite internet connection. I believe that the Internet connection can be ruled ouit.

Until we get some hard numbers and something that others can reproduce, I suspect this ticket will go nowhere - as it's too subjective and vague at this point.

[luroh]

bbjimmy: reading your comment:10, is there any chance you can try out jua's patch in #8007? Or, if you're not comfortable building Haiku yourself, what's your preferred flavour (anyboot, iso, raw)? I could give it a shot tomorrow.

[bbjimmy]

Until we get some hard numbers and something that others can reproduce, I suspect this ticket will go nowhere - as it's too subjective and vague at this point.

These numbers might be interesting:

​http://www.speed-battle.com/statistics_e.php

r1alpha3 (revision 42211) Calculate 6.32 Store 13.47 Render 33.03 OVERALL SCORE 52.82

Haiku hrevr1a4-44580 Calculate 23.81 Store117.03 Render 3.88 OVERALL SCORE 144.72

Although the newer browser scores higher on most tests, it is the rendering that looks to be the problem. The older version renders much faster. By the way, the page loads in about 15 seconds on r1a3, and about 5 minutes on r1a4-44580

[bbjimmy]

Replying to luroh:

bbjimmy: reading your comment:10, is there any chance you can try out jua's patch in #8007? Or, if you're not comfortable building Haiku yourself, what's your preferred flavour (anyboot, iso, raw)? I could give it a shot tomorrow.

I used to build HAIKU myself, but do not have anything set-up to do it at the moment. If you could provide a raw image I can test it out.

[umccullough]

Replying to bbjimmy:

Although the newer browser scores higher on most tests, it is the rendering that looks to be the problem. The older version renders much faster. By the way, the page loads in about 15 seconds on r1a3, and about 5 minutes on r1a4-44580

I think you can rule out the "Render" test there - on my 64bit Windows 7 machine, with a dual-core AthlonX2, running Chrome, the "Render" spits out 4.49 using that same test (while the overall score is 379.68). Webkit (and thus Chrome), are known to have pretty bad rendering time, since there's no hardware acceleration. Why it was that fast before, I couldn't say.

If you're of the opinion that it's not network related, you should probably test out some of the common suites such as Sunspider, Kraken, V8 Bench, etc.

​http://en.wikipedia.org/wiki/Browser_speed_test

I suspect these all test CPU-bound algorithms, however...

Are you not experiencing high CPU when it's slow? If not, then I'm guessing it's some sort of I/O issue - either network, or disk - or a locking problem.

[phoudoin]

Replying to bbjimmy:

Could you run the same Web+ tests but under bin/debug/profile scrutiny, so we could see where the time is actually spent during page loading and rendering ?

Check profile -h for options details.

The summary output will be good, or you can even enable the recorder and attach one here for futher analysis.

Both profile tool and DebugAnalyzer app are in Haiku source tree, but they're not part of default Haiku image. Either build them from source or add them in your custom image via UserBuildConfig: add these two lines:

AddFilesToHaikuImage system bin : profile ; 
AddFilesToHaikuImage system apps : DebugAnalyzer ;

[luroh]

bbjimmy: Here are two rather bare bones gcc4 images for your testing pleasure, one ​without any patch and one ​with jua's patch 5. In addition to WebPositive, they both contain the DebugAnalyzer and the scheduling_recorder, should you need them.

[pulkomandy]

Hi, A new version of webKit with another network backend was merged. Could you try if this improves the situation for you ?

[bbjimmy]

Using hrev46537 this issue seems to have been resolved. Thanks for the great work Pupkomandy!

[pulkomandy]

Ok, closing.

There's still some work to do in this area, as well as improving the network performance even more.