Error converting string from 'UTF-8' to native encoding during download from svn

[taos]

Using hrev41122 (gcc2hybrid).

When downloading Haiku's source code from svn I get the following error:

svn: Can't convert string from 'UTF-8' to native encoding:
svn: /SVN/trunk/src/data/keymaps/French (B?\195?\169po).keymap
svn: Can't convert string from 'UTF-8' to native encoding:
svn: /SVN/trunk/src/data/keymaps/French (B?\195?\169po).keymap

When updating the error is:

svn: Error converting entry in directory '/SVN/trunk/src/data/keymaps' to UTF-8
svn: Can't convert string from native encoding to 'UTF-8':
svn: French (B?\195?\169po)p-0.keymap
svn: Error converting entry in directory '/SVN/trunk/src/data/keymaps' to UTF-8
svn: Can't convert string from native encoding to 'UTF-8':
svn: French (B?\195?\169po)p-0.keymap

Used to work fine with hrev41094.

Last change to haiku/trunk/src/data/keymaps/French (Bépo).keymap was some 4 months ago ([39571]).

[luroh]

Yes, I noticed it as well this weekend on a gcc4 installation that I cross-compiled from Ubuntu 10.10 (32-bit). A nightly vmware build worked fine, however. I'll see if I can narrow down the regression range.

IIRC, to work around the problem, type: export LC_CTYPE="en_US.UTF-8"

[luroh]

Problem introduced with hrev41106. No remedy in hrev41136.
It turns out the nightly I tried this weekend was 41103, which is the last good one.

[scottmc]

can you try hrev41136 and see if that version has any effect?

[scottmc]

This is probably related to this issue: ​http://ports.haiku-files.org/ticket/422 And most likely related to apr/apr-util builds?

[luroh]

scottmc: having pondered things a bit more, I'm actually inclined to believe that your latest packages of apr/apr-util are actually more correct than the 2010-11-12 ones, for two reasons:

1. Using the new packages, Tracker now displays 'Bépo' correctly (see attached screenshots) after deleting the src/data/keymaps folder and svn up'ing.
2. Haiku now behaves exactly like my Linux installation, it needs export LC_CTYPE="en_US.UTF-8", else it throws the "svn: Can't convert string from 'UTF-8'..." error.

[bonefish]

There was a ​thread on the HaikuPorts mailing list regarding the issue. The first mail doesn't quite hit it, but the outcome is that the current behavior -- "UTF-8" must be specified via environment variable -- is entirely expected and that for a login shell some script (/etc/profile?) should set the locale environment variable(s).

[scottmc]

So we need add LC_CTYPE="en_US.UTF-8" to /etc/profile, or perhaps /boot/system/boot/SetupEnvironment?

The readd of the new apr package has broken this again, but executing export LC_CTYPE="en_US.UTF-8" on the commandline then let's subversion do a full checkout of haiku's source code. So it seems we just need to add it to a script. /boot/system/boot/SetupEnvironment seems like a logical place, can it be overridden in case a user wants a different setting?

If no objections then I'll add it.

[scottmc]

applied in hrev41385.

[bonefish]

Replying to scottmc:

So we need add LC_CTYPE="en_US.UTF-8" to /etc/profile, or perhaps /boot/system/boot/SetupEnvironment?

No. We need a command line tool that fetches the current locale settings (via the locale kit) and returns a corresponding POSIX locale string. In /etc/profile the locale environment variable(s) should be set using this command. SetupEnvironment would not be the best place as it is executed only on boot-up, so it cannot work for multi-user and locale settings changes wouldn't take effect until reboot.

The readd of the new apr package has broken this again, but executing export LC_CTYPE="en_US.UTF-8" on the commandline then let's subversion do a full checkout of haiku's source code. So it seems we just need to add it to a script. /boot/system/boot/SetupEnvironment seems like a logical place, can it be overridden in case a user wants a different setting?

If no objections then I'll add it.

I guess it doesn't harm as a temporary work-around.

[mmadia]

Created #7495, to clarify the desired solution.

[leavengood]

Taking ownership as I also took ownership on the blocking ticket #7495.

[leavengood]

I have fixed the blocking ticket so I'm marking this as fixed.