Concerns about ISO 9660 implementation

[scdbackup]

On occasion of ticket #8460 i noticed two potential problems of the ISO 9660 filesystem read-only implementation:

It looks like data files of 4 GiB or larger are not supported. Even if it is not intended to augment Haiku for reading large data files, it should at least be tested whether it tolerates such files gracefully.

It has not been challenged whether the block-to-byte address computations of Directory Entry addresses are safe against byte address of 4 GiB or larger. Only the address computations for data file content have been fixed by #8450.

I am not a user of Haiku, but a producer of ISO 9660 filesystems. In that role i would be glad to help with better readability of ISO 9660.

---

Large data files:

struct iso9660_inode in ​http://cgit.haiku-os.org/haiku/tree/src/add-ons/kernel/file_systems/iso9660/iso9660.h only has provisions for memorizing a single data extent:

        uint32          startLBN[2];                    // Logical block # of start of file/directory data
        uint32          dataLen[2];                             // Length of file section in bytes

ECMA-119 (the public specification of ISO 9660) says:

6.5.1 Relation to File Sections

"Each file shall consist of one or more File Sections. Each File Section of a file shall be identified by a record in the same directory. The sequence of the File Sections of a file shall be identified by the order of the corresponding records in the directory. A File Section may be part of more than one file and may occur more than once in the same file. A File Section may be identified by more than one record in the same or a different directory."

So if struct iso9660_inode is indeed the 1:1 representation of a filesystem inode, then it should be in a 1:n relation to ISO 9660 Directory Entries. (The array size [2] is not a 1:2 relation but rather mirroring the fact that ISO 9660 Directory Entries bear address and length in both byte orderings.)

More challenging will be to collect all ISO 9660 Directory Entries which bear identical names in the same directory.

But as said, at least it should be made sure that two ISO 9660 Directory Entries with the same name do not confuse Haiku beyond the scope of that single data file.

Proposal for producing a challenging ISO image:

Create a file with at least 4 GiB of size. Plus some small files with names alphabetically lower and higher than the big one's ("aaa", "zzz").

Put them into an ISO 9660 image e.g. by my program

xorriso -outdev my_image.iso -add aaa file_of_4gb zzz --

or use mkisofs

mkisofs -o my_image.iso -R -iso-level 3 aaa file_of_4gb zzz

(xorriso can operate directly on DVD burners on systems with Linux kernel, FreeBSD, or Solaris. Like -outdev /dev/sr0 , -outdev /dev/cd0, ...)

Mount the image and check whether all three files behave nice. E.g. whether file_of_4gb does not appear twice in a directory listing, whether at least the small files show correct content, ...

---

Directory entries in high block numbers:

Normally the Directory Entries are stored in a few hundred blocks shortly after block 16. So there occur no large byte addresses. But in the case of multi-session, the new directory tree gets written after the data blocks of the previous session.

So if the first session is 4 GiB or larger, then the address computations for ISO 9660 Directory can produce byte addresses which may get spoiled by 32-bit bottlenecks like the one of #8460.

Proposal for producing a challenging ISO image:

Produce an ISO image which contains two data files of exactly 2 GiB each:

xorriso -outdev my_image.iso -add file1 file2 --

(Note that other than mkisofs -o, this will refuse to overwrite an existing image.)

Add a session to the image by adding a small marker file "this_is_session_2"

touch this_is_session_2
xorriso -dev my_image.iso -add this_is_session_2 --

(Note that the second run uses option -dev rather than -outdev. By this it loads the tree of the first session and adds the new file to it.)

The resulting file "my_image.iso" will bear two sessions. The PVD at block 16 will direct the reader to a root directory above 4 GiB. The image will still fit on a single-layer DVD.