Incorrect collation in Tracker and Deskbar

[rq]

Both Tracker and Deskbar (not sure about other applications) seem to sort objects (Tracker: files/folders; Deskbar: applications) incorrectly. In particular, the collation algorithm seems to ignore locale settings and sort everything that is non-ascii (e.g. extended Latin characters) after all ASCII entries. This has been verified by me using Lithuanian locale and by Humdinger using German locale.

Filing this under Tracker per Stippi's suggestion (quoting: "The code for this is in Tracker (libtracker to be precise). Deskbar shares it. Filing under Tracker would be right for practical purposes.").

[jscipione]

Tracker and Deskbar do natural sorting so that numbered files are sorted in a human readable way. Unfortunately, this is not using the Locale kit right now so it doesn't take the sorting rules of your locale into account. There is an option to do natural sorting while taking the sorting rules for your locale in ICU based on the B_COLLATE_NUMERIC_COLLATION flag, but, the Locale Kit first needs to be updated to support this. Once the Locale Kit supports natural sorting getting Deskbar and Tracker to use it is trivial.

[axeld]

I don't see why it can't be done as is. All that is needed is to use a strcasecmp() that does support collations -- the natural sort algorithm wouldn't need to be replaced at all (of course, once we have that kind of functionality in the locale kit, it should).

[jscipione]

Replying to axeld:

I don't see why it can't be done as is. All that is needed is to use a strcasecmp() that does support collations

BCollator->Compare() is a strcasecmp() that supports localized collations.

the natural sort algorithm wouldn't need to be replaced at all (of course, once we have that kind of functionality in the locale kit, it should).

Yes it does, you can either perform a natural ASCII sort or a localized compare.

See ​http://cgit.haiku-os.org/haiku/tree/src/apps/deskbar/WindowMenuItem.cpp#n149 for how comparisons of window titles is done in Deskbar now.

Unless we want to resort the entire list we need to use a single comparison method.

The way to fix this is to extend BCollator to support localized natural compares and then alter Tracker and Deskbar to use this algorithm. All the hard work is already done by ICU, it simply needs to be exposed by the Locale Kit.

Also, there are locking issues when using BCollator that must be taken into account. I tried replacing the existing NaturalCompare() with BCollator->Compare() without locking and it crashes Deskbar. I believe protecting the Compare() method with locks will fix the crashing.

[bonefish]

Replying to axeld:

I don't see why it can't be done as is. All that is needed is to use a strcasecmp() that does support collations -- the natural sort algorithm wouldn't need to be replaced at all (of course, once we have that kind of functionality in the locale kit, it should).

I don't think that works for right-to-left scripts. There might also be oddities w.r.t. numbers/digits.

BTW, to anyone attacking this issue, please note that the package kit and packagefs are using NaturalCompare() as well (neither is built by default ATM). For both the current solution is fine, since only package names are compared and those will remain restricted to ASCII. In case of packagefs (being a kernel module) the locale kit will not be available.

[pulkomandy]

Fixed in hrev51188.