Opened 5 years ago

Closed 5 years ago

Last modified 5 years ago

#15495 closed bug (fixed)

app_server runs out of memory while running webkit test suite

Reported by: pulkomandy Owned by: waddlesplash
Priority: normal Milestone: R1/beta2
Component: System/libroot.so Version: R1/Development
Keywords: Cc:
Blocked By: Blocking:
Platform: All

Description

Since the switch to rpmalloc, it is not possible to run the webkit testsuite through. App_server will eventually run out of memory and crash. Attached is an area_waste report (with the tool modified t dump just one team, since that's more useful on 32bit systems where virtual address space runs out first)

Attachments (1)

app_server_waste.txt (46.1 KB ) - added by pulkomandy 5 years ago.

Download all attachments as: .zip

Change History (11)

by pulkomandy, 5 years ago

Attachment: app_server_waste.txt added

comment:1 by pulkomandy, 5 years ago

Fixed for now by reverting to hoard2, and confirmed the webkit testsuite runs through without problems in that setup. So we need to fix rpmalloc or switch to yet another allocator.

comment:2 by tqh, 5 years ago

The rpmalloc dev is asking for more feedback from us, so I think that should be a first step: https://github.com/mjansson/rpmalloc/issues/117

comment:3 by waddlesplash, 5 years ago

We can do that, but the ticket itself was about wasting 100kB for every super-aligned allocation, which is still an issue.

comment:4 by tqh, 5 years ago

Do we have a ticket for the real issue?

comment:5 by waddlesplash, 5 years ago

That is the ticket for that issue...?

The "real issue", in looking into this more, is one that I'm not sure is really solvable under rpmalloc's design. As mjansson noted in that ticket, rpmalloc kind of assumes that virtual address space use != physical memory reservation, which of course Haiku does so assume, and with good reason. It also seems to be designed for "high-performance", i.e. 64-bit only where lots of virtual address space waste is fine, and Haiku needs to support 32-bit for a while more (and on non-x86, eventually..)

musl, it seems, is in the process of writing a new allocator, which may be a better fit for us: https://github.com/richfelker/mallocng-draft

Original mailing list post: https://www.openwall.com/lists/musl/2019/11/28/1

More comments on the design: https://www.openwall.com/lists/musl/2019/11/29/2 -- note "- compatibility with 32-bit address space", and "- elimination of heap size explosion in multithreaded apps", which are problems that most modern allocators, rpmalloc included, seem to have.

comment:6 by tqh, 5 years ago

Ouch, I guess we should compare malloc pro's and con's.

comment:7 by waddlesplash, 5 years ago

Blocking: 15359 added

comment:8 by waddlesplash, 5 years ago

Blocking: 15359 removed

comment:9 by waddlesplash, 5 years ago

Resolution: fixed
Status: assignedclosed

rpmalloc is now just gone, so this is "fixed".

comment:10 by nielx, 5 years ago

Milestone: UnscheduledR1/beta2

Assign tickets with status=closed and resolution=fixed within the R1/beta2 development window to the R1/beta2 Milestone

Note: See TracTickets for help on using tickets.