"Cached memory" continually increases

[waddlesplash]

Diver noticed this. On nightly builds, the "cached memory" metric (and actual memory usage) continually goes up while the system is on, and does not seem to go down. This appears to be a relatively recent regression.

[X512]

Occurs for me too. It starts swapping when become full.

[waddlesplash]

Maybe this is related to or just the same as #17208.

[diver]

#17208 is definitely not my case.

[waddlesplash]

Qt Creator seems to trigger this incredibly reliably.

Diver, did you have any luck determining when this started?

[diver]

Nope.

[AGMS]

While you're at it, see if virtual address space ever gets released. On 32 bit Haiku, this is actually a finite resource!

[waddlesplash]

So! On one test run on x86_64, ActivityMonitor claims "cached memory" is 2.3GB, but the "cached pages" queue has only 187450 pages (~767MB) in it according to the "page_queue" KDL command.

It appears the real culprit is the block cache: it has 1.6GB used.

[waddlesplash]

Not seeing any recent changes to the block cache or to BFS that have any relevance at all: not a good sign. I guess the thing to look at next would be the GCC 11 upgrade, then.

[waddlesplash]

Now this is interesting: I dropped to KDL and dumped statistics on the block cache:

kdebug> block_cache 0xffffffff8238c780
BLOCK CACHE: 0xffffffff8238c780
 fd:           0
 max_blocks:   16777216
 block_size:   2048
 next_transaction_id: 5
 buffer_cache: 0xffffffff8239ea00
 busy_reading: 0, no waiters
 busy_writing: 0, no waiters
 298 blocks total, 0 dirty, 0 discarded, 0 referenced, 0 busy, 298 in unused.

However, vmstat has a very different claim about how much memory is in use:

block cache memory:     41418752

That's over 20,000 blocks at 2048 bytes per block!

Compiling BFS or the block_cache at -O0 seems to make no difference. Something else seems to be going on...

[waddlesplash]

kdebug> block_cache 0xffffffff8238c780
BLOCK CACHE: 0xffffffff8238c780
 fd:           0
 max_blocks:   16777216
 block_size:   2048
 next_transaction_id: 20
 buffer_cache: 0xffffffff8239ea00
 busy_reading: 0, no waiters
 busy_writing: 0, no waiters
 1444 blocks total, 1 dirty, 0 discarded, 0 referenced, 0 busy, 1443 in unused.
kdebug> slabs | grep cache
           address                   name  objsize    align    usage  empty  usedobj    total    flags
0xffffffff8238fc48          cached blocks      104        8  1572864      0     1477    15120 20000000
0xffffffff8238fa88    cache notifications       72        8     4096      0        2       56        0
0xffffffff8239ea00    block cache buffers     2048        8 27787264      0     2899    13568 20000000

Somehow, there are quite a lot more "used cached blocks" than the block_cache actually thinks there are, and even more cached buffers than that.

[waddlesplash]

Ah, at least part of the extra usage is because of the allocated comparison blocks, which are enabled in KDEBUG_LEVEL_2. The leak in the above output is thus "only" 11 blocks. The prior example has a much bigger leak, then.

[waddlesplash]

This may be helped by hrev55862. We'll see how much memory I see leaked on longer uptimes.

[waddlesplash]

As far as I can tell, this is now fixed. Cached memory still grows large, but when you actually try to use lots of memory, it rapidly decreases as it should.

[diver]

Not fixed for me unfortunately.

[korli]

Shall we reopen or some other tickets already exist?

[waddlesplash]

Probably a new ticket should be made.