Opened 12 years ago

Closed 18 months ago

#1735 closed enhancement (fixed)

Integrate changes from the japanese community

Reported by: jackburton Owned by: pulkomandy
Priority: normal Milestone: R1
Component: Kits/libtextencoding.so Version: R1/pre-alpha1
Keywords: Cc: shinta
Blocked By: Blocking:
Has a Patch: yes Platform: All

Description

The Japanese community (the people at JPBE.net) made some changes to our libtextencoding.so to support their language more correctly. We asked them for the changes and we got them (already some time ago, actually). We should find a way to integrate the patches without breaking anything. I'm including here everything I got from Momoziro.

Attachments (3)

unzpriv.h.patch (734 bytes ) - added by jackburton 12 years ago.
Patch for unzip
character_sets.cpp.patch (656 bytes ) - added by jackburton 12 years ago.
patch for libtextencoding
unzip61_haiku.diff (7.4 KB ) - added by pulkomandy 8 years ago.
Patch to Unzip 6.1beta for Haiku support

Download all attachments as: .zip

Change History (14)

by jackburton, 12 years ago

Attachment: unzpriv.h.patch added

Patch for unzip

by jackburton, 12 years ago

Attachment: character_sets.cpp.patch added

patch for libtextencoding

comment:2 by jackburton, 12 years ago

And here's the explanation from Momoziro:

The revision part of the unzip only erased "OEM_INTERN((string))" in "Ext_ASCII_TO_Native" macro of trunk/src/bin/unzip/unzpriv.h. The file name broke when the file name (Shift_JIS encoded multi-byte Japanese language characters) was mis-recognized to iso8859-1. I did the change that invalidate the character-code conversion. I think that this is not a good method.

comment:3 by shinta, 11 years ago

Cc: shinta added

comment:4 by mmadia, 10 years ago

Component: - GeneralKits/libtextencoding.so

comment:5 by mmadia, 9 years ago

Has a Patch: set

comment:6 by pulkomandy, 9 years ago

Owner: changed from axeld to pulkomandy
Status: newassigned

comment:7 by pulkomandy, 9 years ago

Hello, I noticed that we are using a pretty old version of unzip and there are plans to fix the issue upstream : http://www.info-zip.org/board/board.pl?m-1248086794/s-45/

Our options for that are :

  • Wait for unzip6.1 to be out, hopefully with a patch autodetecting the encoding using libiconv
  • Upgrade to unzip 5.2, which has a -O option for forcing the encoding (we use 5.0)
  • Backport one of the two changes above to our 5.0 version.

As for the other patch, I can commit it, but I'm not sure what it does. Can someone explain me ?

comment:8 by pulkomandy, 8 years ago

Status: assignedin-progress

Ok, unzip 6.1 beta have the -O/-I flags, with support based on iconv. I updated the beos makefile to include it. Attaching the patch here, I'll try to get it upstreamed.

It's still a beta version, but I think we could switch to it...

by pulkomandy, 8 years ago

Attachment: unzip61_haiku.diff added

Patch to Unzip 6.1beta for Haiku support

comment:9 by pulkomandy, 5 years ago

The changes to unzip have been upstreamed, and haikuporter now has a recipe for the current beta of unzip 6.10.

I still don't understand the changes to libiconv and textencoding, so I'm not merging those until a better explanation is given.

comment:10 by pulkomandy, 3 years ago

For the libtextencoding patch: what this does is replacing the "Shift_JIS" with "cp932". The IANA does not have an encoding named "cp932": http://www.iana.org/assignments/character-sets/character-sets.xhtml

The cp932 encoding comes from Windows, where they extended Shift_JIS with some extra characters in the available space. This is similar to cp1252 for ISO-8859-15.

The extension is known to the IANA as "Windows-31J". I think it is not a good idea to treat that the same as Shift_JIS. Microsoft calls it "Shift_JIS" internally so it adds to the confusion. We could add it as a separate encoding, however, as we do for cp1252.

comment:11 by pulkomandy, 18 months ago

Resolution: fixed
Status: in-progressclosed

The unzip issues are now solved and as I said above I think the libtextencoding fix is not right. Until there is a more detailed bugreport explaining exactly what the problem is, let's close this one.

Note: See TracTickets for help on using tickets.