package create fails with file too large

[jessicah]

Trying to create a package from a directory of size 3.7G fails with package create.

~> du -s -h texlive_x86
3.7G    texlive_x86
~> package create -C texlive_x86 texlive.hpkg
Failed to write data: File too large

Which appears to come from ​https://github.com/haiku/haiku/blob/master/src/kits/package/hpkg/PackageFileHeapWriter.cpp#L584

[bonefish]

Alas, I won't have the time to debug this anytime soon. Could you get an strace, so we can at least see whether the error comes from a syscall and whether the size argument is sensible?

If the error is indeed reported by PackageFileHeapWriter::_WriteDataUncompressed(), it's probably a bug in the PackageFileHeapWriter as the method is only called for writing individual chunks -- which are <= 64 KiB -- and the chunk size table -- which requires roughly 1/4096th of the total size of the uncompressed data, i.e. the critical data size limit is about 8 TiB.

[jessicah]

The interesting line from strace is the following:

[ 10765] _kern_write(0x2, 0xffffffffffffffff, 0x7282fb08, 0x25) = 0x00000025 (8 us)

[jessicah]

The first time I ran it, however, it actually succeeded. Second and third times it failed, both with slightly different file sizes.

This was the basic outline of my bash session of success/failure:

~> pamrel strace -T -s package create -C texlive_x86 tl.hpkg
# succeeded, although I didn't end up getting a URL printed
# or copied to the clipboard...
~> package create -C texlive_x86 tl2.hpkg
# failed
~> strace -o trace.log package create -C texlive_x86 tl3.hpkg
# failed
~> ls -l
total 4374994
drwxr-xr-x 1 jessica root       2048 Jun  4 12:42 texlive_x86
drwxr-xr-x 1 jessica root       2048 Jun  4 12:41 texlive_x86_devel
-rw-r--r-- 1 jessica root 2135392416 Jun  4 13:01 tl.hpkg
-rw-r--r-- 1 jessica root 1186638081 Jun  4 13:25 tl2.hpkg
-rw-r--r-- 1 jessica root 1135415397 Jun  4 13:44 tl3.hpkg
-rw-r--r-- 1 jessica root       2746 Jun  4 13:50 trace-tail.log
-rw-r--r-- 1 jessica root   20262374 Jun  4 13:44 trace.log

where "pamrel" is defined in my bash profile as:

function pamrel {
        ( echo $PWD\> $@ ; $@ ) |& tee /dev/tty | curl -s -F 'content=<-' http://pamrel.lu | tee /dev/tty | clipboard -i
}

[bonefish]

Replying to jessicah:

The interesting line from strace is the following:

[ 10765] _kern_write(0x2, 0xffffffffffffffff, 0x7282fb08, 0x25) = 0x00000025 (8 us)

No, that's actually the error message to stderr (37 bytes written successfully).

[jmairboeck]

I finally got texlive 2021 to finish the install stage (see ​https://github.com/jmairboeck/haikuports/tree/texlive/app-text/texlive, I haven't created a Pull Request yet). I wanted to test this and commented out the line that removes the documentation. I now got an actual crash of the package tool (run via haikuporter).

[jmairboeck]

package debug report

[X512]

Similar problem: #15806.