BFS index issue: Deskbar shelf items don't display after install/reboot.

[mr.Noisy]

Happens with clean hrev36930. Deleting Deskbar_shelf in /boot/home/config/settings/ does not solves the problem.

[humdinger]

You mean Deskbar_scurity_code?

FWIW it works here with mail_daemon, ProcessController and Media volume control. Doesn't though when adding something via the "desklink" command. Those are gone at the next boot-up; there's a ticket on that already.

[anevilyak]

Can you be a bit more specific? Deskbar's shelf file is more or less irrelevant since it doesn't use that to determine what replicants to reload on startup (replicants that are designated as living in the Deskbar are found by query). Also, many Deskbar replicants are intended to only be present for as long as their app is around, and those aren't supposed to be persisted at all.

[mr.Noisy]

I mean an applications like "NetworkStatus", "ProcessController" or "Media Volume Control" which can "live" in Deskbar. These applications live in the Deskbar until the next boot.

[anevilyak]

That's a problem specific to those which is already tracked in #5367. Closing as duplicate.

[humdinger]

Are you sure? #5367 concerns things added via desklink. "NetworkStatus", "ProcessController" or "Media Volume Control" use their own ways to add their Replicant and usually survive a reboot. At least here it does, but not for mr.Noisy.

[anevilyak]

Whoops, missed ProcessController in that list..I think volume control is desklink-based though. Out of curiosity, what does query -a be:deskbar_item_status="*" at a Terminal return, Mr. Noisy?

[stippi]

All the above named deskbar replicants survive a reboot here.

[mr.Noisy]

The command

query -a be:deskbar_item_status="*"

returns nothing.

[anevilyak]

How about: cd /boot ; lsindex ?

[mr.Noisy]

BEOS:APP_SIG BEOS:LOCALE_LANGUAGE _trk/qrylastchance _trk/recentQuery be:deskbar_item_status last_modified name size

[mr.Noisy]

Sorry, forgot the "BEOS:LOCALE_SIGNATURE"

[Disreali]

hrev36970 now reloads NetworkStatus in my testing. ProcessController and MediaVolumeControl still do not get reloaded on reboot.

[Disreali]

I just did some testing. With hrev36804 after install ProcessController, NetworkStatus and Volume Control all reload after each reboot. After installing hrev36805, only NetworkStatus reloads. Therefore, changeset hrev36805 would be the culprit. No idea why it was changed, but it seems to be a regression, and I for one, would like it reverted.

[anevilyak]

Replying to Disreali:

Therefore, changeset hrev36805 would be the culprit.

All that hrev36805 did was make it such that the items are installed to Deskbar on first boot rather than every single time. Previously it was forcing an install on every boot, which only served to hide the bug listed in this ticket. The bug is still there whether that changeset is reverted or not, and needs to be found/fixed rather than hidden via workarounds. However, this isn't so simple since it obviously isn't happening to everyone.

[Disreali]

Then, I have clearly mis-understood this ticket. My apologizes.

[Disreali]

With hrev37183, ProcessController, NetworkStatus, and VolumeControl now persist after several reboots on my system. Not certain whick changeset fixed things, nor if this is solved for others.

[augiedoggie]

Just ran into this problem after installing hrev37184 in virtualbox. I booted the image file, initialized a new partition, and then installed to the new partition. This was done all on the same boot. Running the following command seems to have fixed it, although I performed some other steps before that didn't work. So it could be a combination of steps.

reindex -r -v be:deskbar_item_status /boot/system

[anevilyak]

(In #6259) Duplicate of #6085.

[Disreali]

Looks like ticket:6508 is a duplicate.

[umccullough]

I get this problem often - here's how I usually run into it:

1) Build nightly-anyboot image 2) dd nightly-anyboot image to USB stick 3) boot USB stick, wait for mime database and ssh keys to be created 4) use DriveSetup to initialize a partition on the HD (use defaults, enable query support) 5) use Installer to install from USB stick to HD partition 6) reboot to HD partition

At this point, sometimes everything is fine, but often times I will end up with no replicants in the deskbar. Adding them works, but they are gone again after reboot.

At first, I was only able to reproduce this when the anyboot image was GCC4 or GCC4Hybrid - but now it also happens with GCC2Hybrid.

In the past, I have messed around with recreating indexes and fixed the problem - but it's generally pretty annoying.

Note: Building directly to a partition has never produced this problem for me.

[umccullough]

Just happened to me again, and per augiedoggie, the following fixes it:

reindex -r -v be:deskbar_item_status /boot/system

So, it seems perhaps something in the DriveSetup/Installer process is leaving this index broken.

[luroh]

I can repeat it on two machines here, using both raw and anyboot images (nightly, hrev40448, gcc4). The reindex command above indeed fixes the problem.

Fwiw, I can not repeat the problem in VMware, using the corresponding nightly iso image.

[umccullough]

Lately, I'm having all sorts of index/query issues on recent builds that I install from a USB stick using the previous mentioned method in comment:21

I'm raising this to blocker because this produces a horrible experience upon installation for new users - using one of the more common methods of install that they're likely to encounter.

[umccullough]

Also changing this to BFS issue, as it requires a reindex to fix.

[stargatefan]

snip

[anevilyak]

Replying to stargatefan:

I have seen similar issue with SDL librarys. I will try the reindexing fix to see if the problem is related and report back.

The chances of that being related are about zero, as neither SDL itself, nor loading/locating of libraries in general makes use of BFS queries, which are the only thing making use of the mentioned indexes.

[umccullough]

I think perhaps I'm starting to understand the underlying cause for this issue after speaking with Rene more about it. Here's what I suspect is going on:

• Boot to USB stick in "Live" mode
• First boot processing occurs, adding default replicants to deskbar (writing the attributes necessary for this)
• Installing to fresh partition copies these attributes, but does not create the index
• Rebooting to freshly installed partition causes Deskbar to create the missing index, but apparently this does not reindex the existing attributes (this seems like a major issue)
• Reindexing fixes the problem by forcing the index to update with the existing replicants that were installed with their attributes intact from the USB stick.

Does that make sense? What's the right solution here?

I must admit, I'm starting to los a bit of respect for BFS in understanding these limitations and weaknesses :(

[stargatefan]

Replying to anevilyak:

Replying to stargatefan:

I have seen similar issue with SDL librarys. I will try the reindexing fix to see if the problem is related and report back.

The chances of that being related are about zero, as neither SDL itself, nor loading/locating of libraries in general makes use of BFS queries, which are the only thing making use of the mentioned indexes.

snip

[umccullough]

Replying to stargatefan:

anyways what happens is IF I compile a alpha/nightly image with the SDL librarys and I DD that image to a usb drive, the SDL librarys work fine. However if I install from the same USB stick to a real drive, it looses the symlinks or library links.

I don't know if it is entirely related but I though it might be worth a mention becuase both problems seem to be similar in nature, data is being lost during instilation.

Pretty sure symlink issues are completely unrelated - although that does sound like a potential issue (or enhancement needed) in the Installer.

Probably best if you open another ticket for that one.

My other question is why does the cache never seem to deplete/count down/unload ? IE if you use activity monitor and you select CACHE it may idle fine at 192mb, but when you do a disk copy,file copy,install the cache stays full at say 4000mb and never seems to depopulate. Also sync does not seem to help here. I suspect the issue might actually be in the file system code and that its not clearing the cache and ensuring that all data is transferred to a drive, this can be easily noted by trying to unmount a drive after a file copy. I have noticed this issue is not present on a live CD running a installer but ONLY when you install from the desktop. All problems are persistent and consistent in that behavior.

This is also completely unrelated to index/query problems with the deskbar.

Please open new tickets for these!

[stargatefan]

This is also completely unrelated to index/query problems with the deskbar.

Please open new tickets for these!

snip

[anevilyak]

Replying to stargatefan:

I don't think I would go so far to suggest they might not. If you run a from a A usb install there is no problem with queries. The problems typically only arise "that I have noticed" when you install from a running desktop enviroment. If you install from a live CD you get the same issues IF you install from the desktop but not if you just boot to the installer. and install.

Please stop spamming unrelated tickets with your pet theories when you clearly know absolutely nothing about how the OS/kernel works. The cache will never shrink unless forced to by memory pressure (i.e. an app requesting more mem than is available without freeing some cache pages). This is by design, and is completely unrelated to if/when they're written to disk. They're kept in memory in order to increase performance, since random free memory that's unused isn't accomplishing anything anyways, so any use of it to decrease disk access is a good thing. You'll notice large amounts of cache usage on every other major OS as well.

[stargatefan]

Replying to anevilyak:

Replying to stargatefan:

I don't think I would go so far to suggest they might not. If you run a from a A usb install there is no problem with queries. The problems typically only arise "that I have noticed" when you install from a running desktop enviroment. If you install from a live CD you get the same issues IF you install from the desktop but not if you just boot to the installer. and install.

Please stop spamming unrelated tickets with your pet theories when you clearly know absolutely nothing about how the OS/kernel works. The cache will never shrink unless forced to by memory pressure (i.e. an app requesting more mem than is available without freeing some cache pages). This is by design, and is completely unrelated to if/when they're written to disk. They're kept in memory in order to increase performance, since random free memory that's unused isn't accomplishing anything anyways, so any use of it to decrease disk access is a good thing. You'll notice large amounts of cache usage on every other major OS as well.

snip

[umccullough]

I added an enhancement request #7320 that might be a suitable resolution to this bug - unless there are better ways to resolve it.

[umccullough]

Replying to stargatefan:

this might be the exact problem people are observing with metadata as well as other file system copy problems during install when running from the desktop.

It's not the same - pretty sure I've narrowed down the cause of this bug now.

It would really make more sense to open a new bug for the behavior you're experiencing.

[stargatefan]

Replying to umccullough:

Replying to stargatefan:

this might be the exact problem people are observing with metadata as well as other file system copy problems during install when running from the desktop.

It's not the same - pretty sure I've narrowed down the cause of this bug now.

It would really make more sense to open a new bug for the behavior you're experiencing.

snip

[umccullough]

With the changes made in ticket #7320, the problems mentioned in this ticket are alleviated for me.

Would that be considered the proper/suitable fix for this issue?

Note: If using something other than Installer to copy a live system to a freshly initialized partition, the behavior here may still be experienced, requiring a reindex of the proper attribute to resolve it.

[stippi]

If you have the chance, could you check what happens when you boot into an ISO based Live-CD and install from the Desktop? The problem probably persists. If so, one could simply create all relevant indices in InstallerInitScript, though this solution is a bit fragile, since things could be broken again when someone thinks of a new important index to create.

Also, if one wants the ability to copy an installation to another drive via Tracker, one would have to add detection to Tracker, that the user copies the system folder to the root of another volume, and then ask the user if the indices from the source volume shall be mirrored on the target volume, or else the target volume will be subtly broken as a new boot volume.

[umccullough]

Failed on live iso cd - results added to #7320

[stippi]

ISO Live CD issue fixed, and Installer will now always create all needed (as of now) indices. I'll leave the ticket open until it's decided what to do with Tracker and how we want to support using Tracker to mirror an installation. A possibility is outlined in comment:39.

[axeld]

I would find a Tracker automatism misplaced and a bit useless as well. The problem existed on BeOS as well, and it is not restricted at all to copying a Haiku installation -- if you copy your media files to a new partition, you also have to make sure manually, that the respective indices are created.

Therefore, I would suggest two solutions: 1) have DriveSetup be able to copy the indices from a source volume, or 2) have FileTypes maintain the indices in the MIME database, and DriveSetup automatically create all of them on new volumes.

[stippi]

You are right with the media files problem. All of the following will help a bit to solve the problem:

• The Tracker Find panel desperately needs this check box to include only index attributes. When the check box is enabled (not the default), all the attributes you can select for the given file type will be disabled, if they don't have an index on the search volumes.
• The meaning of the BFS Query flag to search also for non-indexed attributes should be reversed and searching also non-indexed attributes should be the default. The result will be that the system will only be slow in the worst case, not totally broken and irritating anymore. Query clients want to find files, not be concerned about the implementation details and why finding may not work. If some client indeed wants to prefer only indexed results over potentially no results at all, it can specify the (reversed meaning) flag.
• FileTypes needs to have controls added to make the user aware of indices in the first place, and to be able to control them. In the attributes editor, there could be a pop-up menu where the user can put a checkmark on every currently mounted volume, to have the attribute indexed there. Also, a button which gives acces to the re-index tool, probably also per volume, would be helpful.

All changes combined will a) fix the system to find the expected files, unless some app specifically only wants to find indexed files, and b) it will make the user aware enough of indices and what they are good for, so that it will hopefully be very infrequent to have slow queries.

[tangobravo]

I think the nicest solution is for find to search for all attributes, indexed or not, but have some UI feedback if some of the attributes used in the query are not indexed.

I'm a bit anti-popups, I think the solution in web-browsers where a question appears at the top of the browsing window (would you like to save the password, etc) without actually interrupting the user is a good way to go. Something like: -- This query is using the "media:title" attribute, which is not currently indexed on the "/boot/" disk. Adding an index will speed up future queries like this. [Add Index] [X] --

[axeld]

Replying to stippi:

• The Tracker Find panel desperately needs this check box to include only index attributes. When the check box is enabled (not the default), all the attributes you can select for the given file type will be disabled, if they don't have an index on the search volumes.

That would be a rather poor solution, though, as a query usually only needs a single indexed attribute to perform well - everything else doesn't matter.

• The meaning of the BFS Query flag to search also for non-indexed attributes should be reversed and searching also non-indexed attributes should be the default. The result will be that the system will only be slow in the worst case, not totally broken and irritating anymore.

I disagree; this could be the default for Tracker, but not for general queries. It's usually better to let something fail (easily noticeable, and fixable), than slowing down operations down to several minutes in the worst case without giving any clue what's happening. That would be Windows all over :-)

Besides, I wouldn't mind removing misplaced query uses such as what the Deskbar is doing.

[khallebal]

the "original" problem wich is (after installing from a usb key the items on the deskbar dissepear) seems to have been fixed,i didn't come across this problem since rev:40894 up to rev:40978.so i guess it's fixed?

[stippi]

Yes, one could consider it fixed. However, installing via Tracker (by simply copying system folder onto another volume) will have the same problem again. So I don't know if it would be good to close the ticket already. Also, the Deskbar is doing something strange in the first place, I have no idea what this security code thing is about. It appears to be completely besides the point. Does it solve any actual problem? All good reasons to leave the ticket open until everything is cleaned up.

[anevilyak]

Replying to stippi:

Does it solve any actual problem?

It does actually. Marco Nelissen added that, in order to prevent Deskbar from trying to load replicants from other installations, i.e. if you have multiple BeOS/Haiku installs on separate partitions, and they're all mounted. The security code is theoretically unique per install, and was intended to ensure that only the particular install that marked that replicant as 'Live in Deskbar' will try to load it.

[stippi]

Huh? Why isn't the shelf simply archived like Tracker does it?

[anevilyak]

Don't know, Deskbar's shelf has always worked by doing a live query on the be:deskbar_item_status attribute, that was true in R5 and no one's ever rewritten it. Archiving the shelf could be problematic though since it can also contain ephemeral items like a status indicator installed by, say, an IM app, which is only supposed to persist as long as the app is running, whereas the Tracker shelf only contains things the user explicitly put there.

[axeld]

For items where the "install-into-deskbar" attribute is set, it could be arguably done via archiving, however, that's not how it works. Instead, the Deskbar will load those apps as add-on, and call instantiate_deskbar_entry() on them - there is no archiving involved in this case.

Therefore, to prevent downloaded software automatically being launched and added to the Deskbar, the security code has been added.

Since we break compatibility with the switch to GCC4 anyway, I guess we could clean up that as well at that point, but I see no way to get rid of it before :-)

[axeld]

(In #5367) Nope, #6085 is actually a different problem.