Opened 10 years ago

Closed 10 years ago

Last modified 10 years ago

#4113 closed bug (fixed)

mbtowc() is unusable

Reported by: schily Owned by: zooey
Priority: normal Milestone: R1
Component: - General Version: R1/pre-alpha1
Keywords: Cc: planche2k@…, black.belt.jimmy@…
Blocked By: Blocking: #4116
Has a Patch: no Platform: All

Description

mbtowc() returns 0 for ASCII char input. This is only allowed in case that the input string points to a null character, but it happens for any ASCII char.

There seem to be other bugs in mwtomb() and in wcwidth(), but I did not yet inverstigate in them.

Attachments (1)

testmb.c (951 bytes) - added by zooey 10 years ago.
glibc test file that invokes mbtowc()

Download all attachments as: .zip

Change History (24)

comment:1 Changed 10 years ago by zooey

Owner: changed from axeld to zooey
Status: newassigned

comment:2 Changed 10 years ago by zooey

I have tried, but failed to reproduce any error. The testmb.c test program of glibc invokes mbtowc() and that seems to work just fine.

Could you please post an invocation of mbtowc() that fails for you?

comment:3 Changed 10 years ago by schily

Where is this program?

Could you tell me why this test program works for you?

The program that does work is the command line editor from bsh

See ftp://ftp.berlios.de/pub/schily/schily-dist-pre2.tar.bz2

the code is in the file bsh/inputc.c and this code was written according to the POSIX standard and works on any other OS I was able to test until now: Solaris, Linux, HP-UX, Mac OS X......

The behavior seen with the mbtowc() from Haiku (returning 0 although the string input points to a valid 7 bit ASCII character) is not permitted.

Changed 10 years ago by zooey

Attachment: testmb.c added

glibc test file that invokes mbtowc()

comment:4 Changed 10 years ago by zooey

I have attached testmb.c to this ticket.

Which revision have you used when you noticed the problem? I am asking because we have changed our wchar_t type in hrev31443 (from short to int). If you haven't updated beyond that, please do so (and make sure to update your buildtools, too).

comment:5 Changed 10 years ago by schily

Thank you for forwarding the test program!

The test program is not correct:

1) calling mbtowc(%wc, &c, MB_CUR_MAX) is illegal as there is no valid string

in the second parameter.

2) The test program does not set up a locale. This does never apply to

correct software that uses wide chars. You need to first call something like:

if (setlocale(LC_ALL, "") == NULL)

error("Bad locale in inital environment.\n");

comment:6 in reply to:  5 Changed 10 years ago by zooey

Replying to schily:

Thank you for forwarding the test program!

The test program is not correct:

1) calling mbtowc(%wc, &c, MB_CUR_MAX) is illegal as there is no valid string

in the second parameter.

POSIX doesn't say anything about that parameter being a valid c-string, but I adjusted it anyway ...

2) The test program does not set up a locale. This does never apply to

correct software that uses wide chars. You need to first call something like:

if (setlocale(LC_ALL, "") == NULL)

error("Bad locale in inital environment.\n");

... adding that doesn't change anything either: testmb.c runs through without failure for me.

So,

  • which revision of haiku are you using?
  • which revision of haiku's buildtools are you using?
  • does testmb.c work for you?

comment:7 Changed 10 years ago by schily

I just tested the program test,b.c and as expected, it works on Solaris and it fails on Haiku:

mbstowcs FAILED! wcstombs FAILED! mblen 1 FAILED! conversion to wide failed, result: 0 Test FAILED!

BTW: am using the latest version which is hrev31644

comment:8 Changed 10 years ago by anevilyak

Did you update your build tools before building that version? It sounds like you're using a cross compiler toolchain that hasn't been updated with the wchar_t changes.

comment:9 Changed 10 years ago by schily

Do you like to tell me that you deliver a non-working GCC together with Haiku?

Anyway, the autoconf results tell me that sizeof(L'a') is 4 as well as sizeof(wchar_t). In addition, the specific problem cannot be a rsult of such size problems as the function mbtowc() cannot know about this and the incorrect return value is a result from interpreting the input string that is an array of 8 bit chars.

As mentioned before, I asume that the internal tables used by glibc are set up incorrectly.

Another problem is that Haiku does not set any if the LC_* or LANG environment variables and an application thus would in a POSIX compliant anvironment asume the "C" locale which could be either "ISO-646" or "ISO-8859-1" coding.

As the problem is easy to reproduce even with your small test program, it should be easy to fix....

comment:10 in reply to:  9 Changed 10 years ago by anevilyak

Replying to schily:

Do you like to tell me that you deliver a non-working GCC together with Haiku?

This isn't about the version that comes a development package in Haiku. It's about the one used to build the system itself. As you haven't specified where your copy of Haiku comes from, I assume you're cross compiling it yourself. If so, you would have needed to svn update your copy of the buildtools/trunk subdir and rerun ./configure --build-cross-tools(-gcc4), as otherwise the libs would be out of sync with the headers and/or compiler builtins.

Anyway, the autoconf results tell me that sizeof(L'a') is 4 as well as sizeof(wchar_t). In addition, the specific problem

Which is irrelevant since within Haiku it's getting that from the compiler optional package which has nothing to do with which one you used to build the system.

As the problem is easy to reproduce even with your small test program, it should be easy to fix....

Except we can't reproduce it, otherwise these questions wouldn't be getting asked.

comment:11 Changed 10 years ago by schily

Do you like to tell me that you cannot reproduce the problem using:

haiku-pre-alpha-hrev31644-raw.zip

For me it is reproducable with two different installations.

comment:12 Changed 10 years ago by andreasf

Cc: planche2k@… added

comment:13 Changed 10 years ago by bga

Could it be that the build tools in the build factory were not updated after the change to wchar? schilly, any chance you could try compiling Haiku yourself to test this? I can't reproduce this bug either on my instalarion (hrev31711).

comment:14 Changed 10 years ago by schily

I did compile and run testmb.c on an installation that is based on the HD image from:

haiku-pre-alpha-hrev31644-raw.zip

If you do the same, you must get the same error as I get.

If it does not fail for you, please tell me what specific things you do in order to prevent the failure.

comment:15 in reply to:  14 Changed 10 years ago by anevilyak

Replying to schily:

I did compile and run testmb.c on an installation that is based on the HD image from:

haiku-pre-alpha-hrev31644-raw.zip

That would be the missing piece of the puzzle, and indicates that the build factory was never updated with the newer toolchain. Most of us are using Haiku installations that were built directly on our own systems with all the needed updates. Have sent an email to the maintainer of that build factory. Until he gets around to updating that, the only way you'll work around this is by building Haiku yourself.

In any case, on a correctly built system:

~> uname -a
Haiku leviathan 1 r31713 Jul 23 2009 07:36:06 BePC Haiku
~> gcc --version
gcc (GCC) 4.3.3
Copyright (C) 2008 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

~> ./testmb 
wide value: 0x0078
Test succeeded.

comment:16 Changed 10 years ago by umccullough

Has anyone notified Phil yet? AFAIK, he doesn't follow the bug list.

comment:17 Changed 10 years ago by bonefish

Blocking: 4116 added

(In #4116) I'd guess that this is a dup of #4113 and will disappear as soon as the build factory uses the correct build tools.

comment:18 in reply to:  16 Changed 10 years ago by anevilyak

Replying to umccullough:

Has anyone notified Phil yet? AFAIK, he doesn't follow the bug list.

Yes, I notified Phil, I was under the impression he was working on updating them, but he never gave me confirmation that he was finished with that yet. Will see what's going on (or you can if you see him before me).

comment:19 Changed 10 years ago by bbjimmy

Cc: black.belt.jimmy@… added

comment:21 Changed 10 years ago by anevilyak

Resolution: fixed
Status: assignedclosed

Confirmed fixed.

comment:22 Changed 10 years ago by schily

OK, it now works with the gcc4 version

comment:23 Changed 10 years ago by bbjimmy

fixes the gensyscalls issue using gcc2.

Note: See TracTickets for help on using tickets.