System hangs on high RAM usage

[bbjimmy]

hrev56686 x68_64 Memory 3583 Mib swap space 64.92 GiB

Using Web/Epiphany ( due to known memory/process leak) and many ShowImage instances I used more than 3583 Mib of ram. the swap spaced kept me at 99% usage ( acording to ActivityMonitor, VirtialMemory showed obytes used ) and 1% cache but the system hard crashed when the swap reached about 700Mib.

The swap space should be used earlier and attempt top keep cache closer to 10% amd used memory closer to 90% to keep the system responsive and avoid a hard reset due to an out of memory dvdnt.

[bbjimmy]

out of memory event

[waddlesplash]

The hangs on out of memory may be related to swap space usage; it may be trying to transfer things into swap but then hangs somehow.

That is the real problem which needs to be fixed. I think pulkomandy also mentioned encountering it, perhaps he managed to collect some information from KDL?

[bbjimmy]

Out of memory is a symptom, the issue is improper utilzation of swap space.

[nephele]

I think it's a bit fast to blame swap here, this behaviour should not happen even if there is no swap at all.

[bbjimmy]

Replying to nephele:

I think it's a bit fast to blame swap here, this behaviour should not happen even if there is no swap at all.

True, but if the swap was working properly, I would not have reached the out of memory situation.

[waddlesplash]

I managed to reproduce a hang on high RAM usage (with some effort) where the cursor would still move, but basically nothing else would work. Dropping to KDL, it appears the app_server event loop is trying to acquire some rwlock, however it's a userland rwlock so determining who owns it or last released it is not really possible. Furthermore the backtrace into app_server is useless, it doesn't contain function names.

Though, there's another app_server thread (one for Tracker's main thread) which is waiting for a BLocker semaphore named "event dispatcher". So perhaps that's related, but I don't know a whole lot about this part of app_server internals.

Do these sound like the symptoms you were experiencing, or have I run across something else?

[bbjimmy]

sounds right to me.

[waddlesplash]

If this was a user_mutex deadlock, it's possible hrev57550 fixed the problem. Any chance you've seen it recently?

[waddlesplash]

bbjimmy: Did you get around to testing this again, perhaps with a recent nightly or beta5, especially after last month's fixes?