Integrate changes from the japanese community

[jackburton]

The Japanese community (the people at JPBE.net) made some changes to our libtextencoding.so to support their language more correctly. We asked them for the changes and we got them (already some time ago, actually). We should find a way to integrate the patches without breaking anything. I'm including here everything I got from Momoziro.

[jackburton]

Patch for unzip

[jackburton]

patch for libtextencoding

[jackburton]

I also got this link from Momoziro:

​http://www2d.biglobe.ne.jp/~msyk/software/libiconv-patch.html

[jackburton]

And here's the explanation from Momoziro:

The revision part of the unzip only erased "OEM_INTERN((string))" in "Ext_ASCII_TO_Native" macro of trunk/src/bin/unzip/unzpriv.h. The file name broke when the file name (Shift_JIS encoded multi-byte Japanese language characters) was mis-recognized to iso8859-1. I did the change that invalidate the character-code conversion. I think that this is not a good method.

[pulkomandy]

Hello, I noticed that we are using a pretty old version of unzip and there are plans to fix the issue upstream : ​http://www.info-zip.org/board/board.pl?m-1248086794/s-45/

Our options for that are :

• Wait for unzip6.1 to be out, hopefully with a patch autodetecting the encoding using libiconv
• Upgrade to unzip 5.2, which has a -O option for forcing the encoding (we use 5.0)
• Backport one of the two changes above to our 5.0 version.

As for the other patch, I can commit it, but I'm not sure what it does. Can someone explain me ?

[pulkomandy]

Ok, unzip 6.1 beta have the -O/-I flags, with support based on iconv. I updated the beos makefile to include it. Attaching the patch here, I'll try to get it upstreamed.

It's still a beta version, but I think we could switch to it...

[pulkomandy]

Patch to Unzip 6.1beta for Haiku support

[pulkomandy]

The changes to unzip have been upstreamed, and haikuporter now has a recipe for the current beta of unzip 6.10.

I still don't understand the changes to libiconv and textencoding, so I'm not merging those until a better explanation is given.

[pulkomandy]

For the libtextencoding patch: what this does is replacing the "Shift_JIS" with "cp932". The IANA does not have an encoding named "cp932": ​http://www.iana.org/assignments/character-sets/character-sets.xhtml

The cp932 encoding comes from Windows, where they extended Shift_JIS with some extra characters in the available space. This is similar to cp1252 for ISO-8859-15.

The extension is known to the IANA as "Windows-31J". I think it is not a good idea to treat that the same as Shift_JIS. Microsoft calls it "Shift_JIS" internally so it adds to the confusion. We could add it as a separate encoding, however, as we do for cp1252.

[pulkomandy]

The unzip issues are now solved and as I said above I think the libtextencoding fix is not right. Until there is a more detailed bugreport explaining exactly what the problem is, let's close this one.