Opened 11 years ago

Closed 10 years ago

#3054 closed bug (fixed)

Tracker doesn't load all poses on folders with thousands of files.

Reported by: aldeck Owned by: aldeck
Priority: normal Milestone: R1
Component: Applications/Tracker Version: R1/pre-alpha1
Keywords: Cc:
Blocked By: Blocking:
Has a Patch: no Platform: All

Description

Tracker doesn't load all poses on folders with thousands of files. This problem was first mentioned in #2760's comments, but now it's more clear it's a different bug.

Investigating on #3011 i found some new infos:

I added some debug output to AddPosesTask and apparently GetNextDirents returns 0 although there are still more items in the dir. http://dev.haiku-os.org/browser/haiku/trunk/src/kits/tracker/PoseView.cpp#L1263 . It fails after a few thousand files but it's quite variable.

Interestingly, when i tried commenting "model->CloseNode()" here http://dev.haiku-os.org/browser/haiku/trunk/src/kits/tracker/PoseView.cpp#L1742 for #3011's investigation, i noticed the problem was worse. That is GetNextDirents would fail after a few hundred files.

Closing model nodes "model->CloseNode()" on each model in PoseView::AddPosesTask loop and reopening it in CreatePoses, made the problem disappear, although hitting performance a bit.

There must be something fishy with models not being all closed by seeing the old debug ifdefs in the related methods. Anyone has an idea why GetNextDirents would fail though?

setting tracker component, might be a kernel/fs bug.

Change History (7)

comment:1 Changed 11 years ago by axeld

What error does it return?

comment:2 Changed 11 years ago by aldeck

It just returns 0 as if it had visited all entries.

comment:3 Changed 11 years ago by aldeck

When creating lots of poses, something is going wrong with the way BNodes are opened/closed, further node opening fails with B_NO_MORE_FDS. I'll have a look at that.

Would that explain GetNextDirents giving up though?

comment:4 Changed 11 years ago by axeld

There is a natural upper limit of the open file descriptors for every app - if Tracker hits that, it just means it keeps too many files open at once. I remember that has been a problem for file panels under BeOS as well (missing files).

Are you sure that GetNextDirents() give up early? Ie. it should always return all files, it's just that Tracker won't be able to create poses for them all if it hits the file descriptor limit.

At least there is nothing that should get GetNextDirents() to bail out early once the directory is up and running.

After all, a "ls" in the Terminal is using the same kernel call that powers it.

comment:5 in reply to:  4 Changed 11 years ago by aldeck

Replying to axeld:

There is a natural upper limit of the open file descriptors for every app

Yes that's the problem, is it still limited to 256 like in beos? Couldn't find the limit in the sources.

Are you sure that GetNextDirents() give up early? Ie. it should always return all files

I'm pretty sure about that, unless i totally messed up my debug output. Ex: it would stop returning new entries (returning 0) at 8000 entries when there are 12000 in the directory. That's under the stress conditions i mentioned, after lots of NO_MORE_FDS errors. Note also that the file opening is done in a different thread than the (deffective) closing.

Fixing Tracker shouldn't be too hard. However i don't want to give up on that GetNextDirents mystery, i'll probably make a test app for that...

comment:6 Changed 10 years ago by aldeck

Owner: changed from axeld to aldeck
Status: newassigned

Closing the node in the AddPoses thread and reopening it on reception works well. I added a small snooze in the AddPoses thread between two sent messages, it's a lot more fluid, the window thread (receiver) was stalling a lot. Doing some cleaning and testing, and i'm ready to commit.

comment:7 Changed 10 years ago by aldeck

Resolution: fixed
Status: assignedclosed

fixed in hrev28872

Note: See TracTickets for help on using tickets.