No attributes of MIME "application/*" in Tracker's "Attributes" menu

[Giova84]

As the attached screenshot shows, In Tracker, attributes view for People files is not available. However, is possible use advanced queries to find attributes of People files. Instead, the following screenshot show how attributes should be available for "Person" files: ​http://www.haiku-os.org/files/attributes-1.png

P.S: I have manually created People folder under /home, shouldn't this folder exist by default in Haiku?

hrev46179

[humdinger]

This is a crippling bug for working with attributes in Tracker. I changed the summary to reflect that this is a general problem. (Does anyone know a workaround? I'd need the URL attribute column...)

[anevilyak]

This doesn't appear to be a general problem, so the change in description isn't correct. i.e. for audio/video files and several others, I still see the additional attributes listed over here. It seems to relate to specific file types, but how isn't yet clear.

[anevilyak]

This may relate to some of the changes that came about as a result of package management. When Tracker is enumerating the set of mimetypes in a folder in order to add their respective attributes to the menu, it breaks each one down into its supertype first, in order to add a top level menu for that (i.e. application, audio, video).

However, one of the steps involved in that is checking if the type in question is actually installed, which involves reading the META:TYPE attribute on the corresponding folder/file. For mime_db/audio and mime_db/video, this is present, but for application it is missing (I should note, at least over here it's missing on both the system folder and its corresponding twin in ~/config/settings/mime_db/).

As a consequence, these mime types wind up getting skipped, and their corresponding attributes don't get added. As of now, I'm uncertain as to who is responsible for populating/writing that attribute though.

[anevilyak]

As a further note, application seems to be the only one affected, the other six supertypes in /system/data at least all have the full set of meta attributes.

[humdinger]

Here's a work-around provided by friend anevilyak on IRC:

AnEvilYak: as a temporary workaround what you can do is: check if ~/config/settings/mime_db/application exists, if it doesn't, create it. Then, assuming you're in Terminal at ~/config/settings/mime_db , addattr -t string META:TYPE application application

Hope that work-around doesn't delay the actual fix of the bug. :)

[anevilyak]

After a bit more digging, the problem appears to be as follows, as best I can tell: For all the other top level mime types, we explicitly install an entry which defines attributes that all subtypes have in common (i.e. track and artist for audio). This in turn results in the supertype folder getting the full complement of attributes, including META:TYPE. However, we do no such thing for application/, as the types there don't really have anything in common. As such, that supertype folder is most likely simply created indirectly as a result of creating the first application/ subtype we come along. DatabaseLocation::_CreateType() (storage/mime/DatabaseLocation.cpp) does this by simply creating the directory and nothing more, and as such, the application supertype never winds up being registered as installed.

Assuming this analysis is correct, this presents two possible resolutions: 1) Add a dummy application supertype entry during the system package build simply to house the meta attribute, or 2) when DatabaseLocation is forced to create the supertype folder on a subtype's behalf, it always at least writes the type attribute for the respective supertype folder. Thoughts?

[bonefish]

We do already add an application super type to the system package. The problem is that other packages that include an application MIME sub type may will include the directory as well and will shadow the system package one, if theirs is newer.

Not sure what the best solution is. I see two possible approaches: 1. Change how the super types are stored in the MIME DB, i.e. use a separate file (e.g. ".super") instead of attaching the attributes to the directory itself. 2. Solve the directory shadowing issue at the packagefs level.

For the latter there are different options: a) For a directory the attributes of all the merged directories from the respective packages could be merged, similar to how the contents of the directories are merged. b) Instead of choosing the newest directory as a representative for the merged directory node, we could choose the one with the most attributes. c) We could introduce a special marker attribute to override the time criterion and tag the super type directories in the system package accordingly.

[anevilyak]

Replying to bonefish:

We do already add an application super type to the system package. The problem is that other packages that include an application MIME sub type will include the directory as well and will shadow the system package one, if theirs is newer.

That was my first thought as well, since that folder was initially coming from the Web+ package according to its sys:package attr, but removing packages until it came from the system package still shows the same issue, so if we are adding it, something's going wrong there in addition.

[bonefish]

Checking with "package list -a" shows that the directory the system package has the relevant attributes. I haven't tried removing the other packages yet. Please note that due to the missing node monitoring FS changes in the MIME DB won't take effect live.

[anevilyak]

I will have to look with "package list" on my end as well then, even after a reboot that folder is still missing those attributes here. Will let you know what I discover.

[anevilyak]

Replying to bonefish:

Checking with "package list -a" shows that the directory the system package has the relevant attributes. I haven't tried removing the other packages yet. Please note that due to the missing node monitoring FS changes in the MIME DB won't take effect live.

Problem found, and it confirms your theory: The system package containing the secondary architecture libs also contains /system/data/mime_db/application and a few subordinate entries. However, its copy of application has only the BEOS:TYPE attribute and nothing more. Since that one gets built after the base system package, its copy winds up winning, as does the one in Web+'s case, which exhibits the same problem.

Of the proposed solutions, while 1) is appealing from the standpoint of simplicity, I suspect that will simply defer the issue to a similar clash elsewhere down the road. As such, I'd personally lean towards 2), and of the proposed solutions there, I think I would probably go with the special tagging, as quantity of attributes seems a bit more of a special case to this particular instance, and couldn't necesssarily be counted on generically.

Proposal a) would certainly be another possibility, though I suspect that one would be more complex from an implementation standpoint, and possibly more costly on the performance side. Thoughts?

[bonefish]

I think 2a) would be the most elegant and consistent solution, but indeed it would be the most complex to implement and the one with the most overhead.

The reasoning for 2b) is that directories usually don't have attributes (except BEOS:TYPE), so that for special cases where attributes are associated with a directory, there would be only one package providing those anyway. The advantage of this approach is that no external assistance (i.e. explicit marking) would be needed.

Which is the main downside of 2c). Since the marker is actually packaging meta information, using a BFS attribute would be rather inelegant. More appropriately it should be a package attribute, which raises the question of how to specify it. That aside, in packagefs itself this would be the simplest change.

Approach 1) would indeed just defer the problem. An icon or the attributes for name localization are other cases I can think of.

So, I guess 2c) is the approach to go for.

[axeld]

What to do if there are two special markers? Newer one wins?

Anyway, I would prefer 2a), but that's not surprising, isn't it? :-)

[bonefish]

Replying to axeld:

What to do if there are two special markers? Newer one wins?

Sure. I don't think this will really happen in practice, though. The only cases I can think of are when one package provides a directory structure with extra attributes on some directories and other packages provide additional entries within the structure.

Anyway, I would prefer 2a), but that's not surprising, isn't it? :-)

Yeah, it would be natural to merge all aspects of directories, i.e. not only entries, but also attributes. It's just relatively expensive to do that. I don't think it is reasonable to do the attribute merging live -- for some directories many or even all packages will take part, and the management of indices needs to be taken into account as well -- so that needs to happen at package activation time and an additional layer of objects is needed for the (directory) attributes.

Since I can't think of any situation where we actually need the attribute merging, I don't think it's worth the overhead.

[jessicah]

Perhaps the most elegant solution would be to have the MIME DB physically stored on the BFS, rather than inside packagefs. You'd get the merging for free, and I'm sure it wouldn't be that hard to update the package daemon to update the real MIME DB as required?

However, looking at some of the other comments above, I've modified the _CreateTypeNode to always set the BEOS & META type attributes for supertypes, since these follow a standard convention anyway. And now People attributes are available in Tracker!

I don't know whether that's considered elegant or sufficient enough... but that's up to you guys to decide :)

[bonefish]

Replying to jessicah:

Perhaps the most elegant solution would be to have the MIME DB physically stored on the BFS, rather than inside packagefs. You'd get the merging for free, and I'm sure it wouldn't be that hard to update the package daemon to update the real MIME DB as required?

It might be relatively complicated. E.g. if two packages provide a MIME type, when uninstalling one package the package daemon would have to scan all remaining packages whether they contain any file that would be removed by uninstalling the package.

Generally I'd like to keep as many files within packagefs as possible. The more files we move to BFS the less useful having a packagefs becomes.

However, looking at some of the other comments above, I've modified the _CreateTypeNode to always set the BEOS & META type attributes for supertypes, since these follow a standard convention anyway. And now People attributes are available in Tracker!

Besides the missing error checking (and the superfluous blank line), this doesn't solve the general shadowing problem, though. These two attributes would be present after rebuilding all packages, but there are other attributes that would still be missing (cf. src/data/mime_db/application.super). Adding those as well would certainly not be an elegant solution.

I'm still favoring the suggested solution 2c) as it is relatively simple to implement and should suffice for all similar situations I can think of.

[humdinger]

(In #11433) For a work-around, see comment 5 of #10071.

[axeld]

While merging (2a) is too complex, I'm a bit unsure about 2c). Besides that, how about a 2d) that should work well enough at least for the problem at hand, and should even be simpler to implement: for directories, always let the first package win? The folders introduced by the system package should know best.

[waddlesplash]

"first package" is not always defined here I think, because maybe zlib or libpng or one of the other dependents of the Haiku package will be activated first, right? So maybe the the solution is to let the Haiku package always "win" in the mimetypes directories.

[waddlesplash]

Otherwise, indeed, 2c seems the best solution.

[waddlesplash]

So, I've attempted to implement axeld's 2(d) from comment:23; that is, "system_package"s should "win" when conflicts occur. However, as I believe was also noted above, the secondary-arch packages also are marked as "system_package"s, and these include broken attributes directories also.

Continuing with a "pure" 2(d) is still possible: we can make the "haiku" package take precedence over others, at least when it comes to directory attributes; but this means adding a hard-coded package name into packagefs, which I think is okay but is perhaps not the cleanest.

Or we could go with 2(b), which is for directory attributes, use the package with the most attributes. I think that's the best solution here, then.

[pulkomandy]

Content of secondary arch package:

data                                     0  2018-07-19 21:12:23  drwxr-xr-x
  mime_db                                0  2018-07-19 21:12:23  drwxr-xr-x
    application                          0  2018-07-19 21:12:24  drwxr-x---
      x-vnd.haiku-libbe                  0  2018-07-19 21:12:23  -rw-r--r--
      x-vnd.haiku-libmail                0  2018-07-19 21:12:23  -rw-r--r--
      x-vnd.haiku-libtracker             0  2018-07-19 21:12:23  -rw-r--r--
      x-vnd.moon-flanger.media_addon         0  2018-07-19 21:12:23  -rw-r--r--
      x-vnd.equalizer.media_addon         0  2018-07-19 21:12:24  -rw-r--r--
      x-vnd.haiku-hmulti_audio.media_addon         0  2018-07-19 21:12:24  -rw-r--r--
      x-vnd.haiku-mixer.media_addon         0  2018-07-19 21:12:24  -rw-r--r--
      x-vnd.vst_host.media_addon         0  2018-07-19 21:12:24  -rw-r--r--

I think this is a packaging error. These files are already in the primary arch package and should not be re-included here. Can we just remove them and be done?

[waddlesplash]

Fixed by using the "more attributes > less attributes" for directories solution in hrev52172.