Opened 12 years ago
Last modified 7 years ago
#9496 new bug
JPEG Translator not read large files.
Reported by: | Premislaus | Owned by: | nobody |
---|---|---|---|
Priority: | normal | Milestone: | R1 |
Component: | Add-Ons/Translators/JPEG | Version: | R1/Development |
Keywords: | Cc: | ||
Blocked By: | Blocking: | ||
Platform: | x86 |
Description
http://www.edtruthan.com/mars/Sol177-MAHLI-MSL-Self-Portrait-22000x17000px.zip - I have a problem with this file! JPEG Translator is not able to read it.
Attachments (4)
Change History (12)
by , 12 years ago
Attachment: | ShowImage.png added |
---|
by , 12 years ago
Attachment: | WonderBrush.png added |
---|
comment:2 by , 11 years ago
Another example: http://www.nasa.gov/images/content/618486main_earth_full.jpg
by , 11 years ago
Attachment: | WebPositive-848-debug-28-12-2013-03-43-08.report.txt added |
---|
WebPositive trying to open large jpeg
comment:3 by , 11 years ago
To my tremendous surprise, ShowImage opened the file linked above without issue (once I downloaded it via wget).
comment:4 by , 7 years ago
For me this is not an error in the JPEG translator. The translator only allocates memory for 2 scan lines. The output buffer is allocated by the client application, e.g. Showimage. This can be tested with the command line tool translate:
translate --verbose image.jpg output.bin bits
The image is translated into a B_TRANSLATOR_BITMAP without issue. For the example linked above (Sol177...) this results in a ~1.4GB file, which is too large to be handled by a simple memory buffer used by Showimage:
From ImageCache.cpp:
// Translate image data and create a new ShowImage window BBitmapStream outstream; status = roster->Identify(&file, &ioExtension, &info, 0, NULL, B_TRANSLATOR_BITMAP); if (status == B_OK) { status = roster->Translate(&file, &info, &ioExtension, &outstream, B_TRANSLATOR_BITMAP); } if (status != B_OK) return status; BBitmap* bitmap; if (outstream.DetachBitmap(&bitmap) != B_OK) return B_ERROR; . . .
Possible solution would be to change Showimage and use some kind of memory mapped file, but there seems no support in BFile and/or BBitmapStream/BBitmap for this(?).
comment:5 by , 7 years ago
Indeed, BBitmapStream itself does not hande this because its goal is to create a BBitmap, which is an in-memory structure.
An on-disk buffer does not seem like the best thing to do, however (do we really want to store gigabytes of decoded data on disk for this?)
I think the right strategy would be to have the translator accept extra parameters in the BMessage* ioExtension
passed to it, so that ShowImage (or other apps) could request a bitmap with a reduced size, or only a small part of the complete image (if zoomed in).
comment:6 by , 7 years ago
I think this needs some more thinking.
Scenario 1 "partial request"
The application would request the translated buffer from a translator. Since the translator is (most likely) deleted after the operation, it would request a new translation everytime the user scrolls up/down/left/right and the end of the image is reached. For me this sounds strange, since it most probably will result in sluggish behaviour of the application. If you use the example here, translate needs 5-7 seconds on my i7 here.
Scenario 2 "reduced size"
For viewing okay, but what about editing? I mean you want to edit the original and not just some kind of "low quality copy". Editing the original picture would still be impossible
Scenario 3 "memory mapped file"
Application would not create a BBitmapStream, but would create a temporary BBitmap on disk. This temp file would be memory mapped and be used to create an in-memory BBitmap or BBitmapStream so that the application can decide what to show and how to handle. Comming from Objective-C, I would create a NSData object from the file with memory mapped flag set and feed this into whatever consumer I have...
Questions:
- How does x64 Haiku behave? Showimage should have no problems allocating enough memory
- How is this handled on other 32-Bit OS?
- Any other ideas?
comment:7 by , 7 years ago
Scenario 1: translate takes a long time because it decodes the whole image. It should be faster to decode just one part of the picture (in any case, much faster than a memory mapped file that goes into the hard disk). This relies on the translator being able to decode just one part of the picture, however, which may not be possible on all platforms.
The "display" tool from ImageMagick can do things like this, so maybe we should look at their code.
For editing, the matter is different. The editor would have its own storage, and if large files should be supported, then it would indeed use an on-disk backing store (probably cut in small tiles to allow fast scrolling). This is up to the editor to do, but it will still need a way to populate the tiles from the translators, so it would decode piece-wise.
comment:8 by , 7 years ago
Looking at this http://www.imagemagick.org/Usage/files/#massive gives also some ideas, but I still doubt that all that should be handled by a translator. Cropping could be done, though (even if it is rather painful for JPG).
BTW #7740 gives some infos about x64 and large Jpegs, too.
The size of the picture after decompression is
22000 * 17000 * 4
which is more than 1.4 GB. On x86 process heap starts at 384 MB. Lower addresses are partially occupied by elf images and addresses near the top of address space available for process are occupied by stacks. Any attempt to allocate anything big (including temporary buffers for decompression) will fail.