FirstBootPrompt: can't display Chinese and Japanese text.

[mt]

After hrev51020, FirstBootPrompt can't display Chinese and Japanese information and button text (please see attached screenshot). I think we need to change fallback font order.

[axeld]

The problem is that the font seems to pretend to contain glyphs it actually hasn't. Unfortunately, this is also true in the other direction, which is why the font ended up being in second place (the other problems weren't known beforehand).

I guess we'll need to properly support fontconfig in order to fix this one for real.

[perelandra]

We can either revert hrev51020 to remove Noto Sans Symbols as fallbock or modify the order to put Symbols last. I have decided not to use symbols in Repositories (for now) until a better fallback system is implemented since even when using some fonts like Bitstream the symbols still do not show up with Noto Sans Symbols as fallback.

[perelandra]

Moved the Noto Sans Symbols font to last spot in fallback list in hrev51034

[pulkomandy]

Maybe I missed something when implementing the multi-font fallback and I don't use the right way to check if the font actually has a glyph or not? Well, using fontconfig is the best way to go anyway.

[tsundoku]

I just updated my (Japanese) system to hrev51034 and I still have this problem. No Japanese UI strings are displayed anywhere. Web content in WebPositive displays it, but using different fonts from before the update. Sorry for lack of more details; I can't read any of my menus or dialogs right now.

[pulkomandy]

Check your font settings (in preferences>appearance), make sure the font selected there is actually installed (either VL-Gothic if you want to continue using that, or Noto Sans CJK if you want the new default one).

[axeld]

The point of the font fallback is that one don't have to select the right font there, but using a CJK font there should definitely help.

[tsundoku]

I have all of the relevant Noto fonts and VL Gothic installed. Manually setting the display fonts to Noto Sans CJK (ignoring the fact that it is hardcoded as a fallback) does not fix the problem, either.

The only way I have found so far to resolve this issue is to rebuild Haiku with the changes to GlyphLayoutEngine.h from hrev50997 reverted.

[pulkomandy]

Then I probably broke something. But I did test that rendering japanese characters was working ok, at least in FontDemo, so why would it be different for you?

[perelandra]

It looks like at least part of the problem is fallback is not working for the bold font. See here where I have booted the hrev50998 ISO and selected "Boot to Desktop", the first screenshot has the regular Noto Sans as the bold font but bold Japanese characters are not displayed. When changing the bold font to CJK they are displayed: ​http://brianh.ftml.net/photos/haiku/CJK_bold1.png ​http://brianh.ftml.net/photos/haiku/CJK_bold2.png

Also there is no option to choose Noto CJK for the mono font. And when I boot into a hrev51034 ISO I see the exact same behavior.

[tsundoku]

Replying to perelandra:

It looks like at least part of the problem is fallback is not working for the bold font. See here where I have booted the hrev50998 ISO and selected "Boot to Desktop", the first screenshot has the regular Noto Sans as the bold font but bold Japanese characters are not displayed. When changing the bold font to CJK they are displayed:

​http://brianh.ftml.net/photos/haiku/CJK_bold1.png ​http://brianh.ftml.net/photos/haiku/CJK_bold2.png

Also there is no option to choose Noto CJK for the mono font.

And when I boot into a hrev51034 ISO I see the exact same behavior.

In hrev51034 I don't get any CJK text at all, in any style, but I will try booting from a clean system soon to confirm again. Something else worth noting is that the "Light" and "Regular" styles for Noto Sans CJK appear to be switched. I noticed this when previewing fonts on my system earlier today, and it is also observable in your screenshot with the Light text inside the window that should presumably be Regular.

[pulkomandy]

There is a strange thing about the Noto CJK: the latin characters used there for some reason aren't from Noto Sans, but from some other font, which is indeed lighter by default.

I suspect something strange is happening because normally, for missing characters, there should be an "empty square" shown, and currently there isn't. This means the characters were found somewhere, but are empty. It could be a bug in the way I implemented the multiple font fallback, or it could be a problem further down, in the way we detect wether a font contains a glyph or not.

As for monospace, Noto Sans CJK can't be used there, because it is not a monospace font. We should check BFont.IsFullAndHalfFixed(), but that function isn't implemented so IIRC there is a check for the width of some characters only to filter out obviously not monospace fonts.

[tsundoku]

Replying to pulkomandy:

There is a strange thing about the Noto CJK: the latin characters used there for some reason aren't from Noto Sans, but from some other font, which is indeed lighter by default.

It's not just the Latin characters, but the CJK characters, too.

[perelandra]

I just installed hrev50134 in Japanese to a new partition and I can see all the charactes displayed properly. Noto Sans CJK was set as the bold font.

​http://brianh.ftml.net/photos/haiku/hrev50134CJK-clean.png

When changing the bold font to Noto Sans Bold again the bold characters are missing as I saw when booting to live desktop.

I suspect there is something wrong with your install. It may be a good idea to do a complete backup of everything in your boot directory to another BeFS partition. The you can try to run "pkgman full-sync" in Terminal (then do a restart) and see if syncing all the installed packages resolves the issue. If that doesn't do it, a clean install may be in order. After a clean install you can just copy everything from your backup of your home directory back into /boot/home.

[perelandra]

Replying to pulkomandy:

As for monospace, Noto Sans CJK can't be used there, because it is not a monospace font. We should check BFont.IsFullAndHalfFixed(), but that function isn't implemented so IIRC there is a check for the width of some characters only to filter out obviously not monospace fonts.

There is a Noto Sans Mono CJK JP font, so I would think that would be an option to select for the Fixed font. It is an option for selecting in the other Plain, Bold and Menu fonts in the Appearance preflet.

[pulkomandy]

As I said in comment 15, we cheat a bit when detecting monospace fonts for lack of the proper IsFullAndHalfFixed() method implementation. I guess Noto Mono CJK does not have the glyphs we are checking for, or they are not monospace enough for our simple check.

The solution here is to implement and use IsFullAndHalfFixed to detect monospace fonts properly.

[tsundoku]

Replying to perelandra:

I just installed hrev50134 in Japanese to a new partition and I can see all the charactes displayed properly. Noto Sans CJK was set as the bold font.

I can see that it works now from a clean install, although I would not quite call it finished. The font that displays by default is rather ugly and much thinner than the Roman Noto text. I mentioned this earlier, but if I manually change the font to Noto Sans CJK JP Light, I get a thicker font that looks more reasonable, but if I change it to Noto Sans CJK JP Regular, I get the thin mismatched font that is the current default, so something appears to be mixed up here.

I have already solved this problem for myself by doing a custom build with my own preferred fallback fonts (not Noto), but I am concerned about the appearance of the default Japanese environment.

[pulkomandy]

You should not select Noto Sans CJK JP as your font. This is because for whatever reason, they have decided to include Latin glyphs in it, which are NOT the Noto ones.

Source: ​https://www.google.com/get/noto/help/guidelines/

It is recommended to retain "Noto Sans" in the list. Other Noto fonts usually do not cover Latin letters, digits or punctuation.

Put "Noto Sans" before "Noto Sans CJK". Currently, the Latin characters in the CJK fonts are from Adobe’s Source Sans Pro

So, you should leave it with the default settings (use Noto Sans) and let the fallback system fill in the Japanese characters. And yes, we really need to make this better, but that means using FontConfig for a smartest font selection.

However, I did not specify a font weight in the fallback list, so it is possible that the "light" variant is picked (if it comes first in the font, possibly). We should see if it's possible to force the use of the "Regular" variant in the fallback system when the initial font is also "regular" or "book" (as DejaVu words it).

[Giova84]

As i reported on ticket:13404 I found a similar issue: I'm no longer able to see Ascii characters, by using another font: "Inconsolata". On older nightlies there are no issues. As stated by Humdinger on my ticket:

"Looks like another case of non-working fallback to the Noto Symbols font. The character " ⚡ " (miscellaneous symbol 0x26A1) is visible in Pe when DejaVu Mono or Noto Symbols is set, but not when using Noto Mono or Regular."

[pulkomandy]

Hi, Can we get an update on this please?

I think the issue with font wheight has been fixed now, and "Regular" will not use a super-thin variant anymore.

However, it seems bold text isn't working currently (the "Language" list item title and the window title). Is that all there is missing or are there other issues with the japanese or chinese text?

[mt]

Hi, FirstBootPrompt can display Japanese text except window title (bold font), so I rewrite window title to English at rdev hrev51517.

I think we could add to release note that people use Non-ASCII text need to change fonts via Appearance preference.

[mt]

FirstBootPrompt at hrev51612

[pulkomandy]

I would prefer that you leave the text translated and I'll try to fix the problem on Haiku side instead

[diver]

I noticed that "Language" title above list view is missing for Japanese, Chinese and Korean languages:

[diver]

Ah, it's also in bold.

[pulkomandy]

I started investigating this but had to leave the coding sprint. For some reason, the font fallback is not working for Noto Sans Bold and we end up with no glyphs. I still don't know why, however.

[waddlesplash]

Remaining issues fixed by hrev53928.

[nielx]

Assign tickets with status=closed and resolution=fixed within the R1/beta2 development window to the R1/beta2 Milestone