ISO9660 KDLs when it is not the first session on a CD.

[pulkomandy]

The CD is "Shangri-La limited edition" by Wraygun. There is the usual CDDA track (which works fine) followed by a "VIDEOCLIP" track in ISO9660 format. That one is mounted fine, but trying to open it in Tracker goes to KDL (failed to read block).

[pulkomandy]

I can reproduce this with all mixed CDDA/ISO9660 CDs I have:

• Mika - Life in Cartoon Motion
• Tribute to Arno
• Daft Punk - Tron Legacy soundtrack
• Datarock - Datarock Datarock

These disks all have the same format: 2 sessions, the first one with the audio tracks, and the second one with the data tracks.

I wanted to test with CDs that are the other way around (data track first, then audio tracks), as was common in video games 20 years ago (I tried "Rayman Forever"). But in that case the ISO9660 track isn't visible at all in Haiku and I can't try mounting it. Playing the cdda part seems to work fine. It seems the iso9660 FS detects the data track properly but returns 0.6, then the CDDA detects the audio track and returns 0.8. There is a single session in this case, so we don't expose both. Maybe we should identify the data on a track-by-track basis instead of session-by-session (but that would lead to each CDDA track being mountable separately, probably?)

[pulkomandy]

Ok, and a more interesting test:

dd if=/dev/disk/0/1/0/1 of=datatrack
mountvolume datatrack

Trying to access the mounted volume leads to a very similar crash, with the same block number.

The data track is 23.06MB big. The panic message is:

could not read block 211161: bytesRead: 0, error: No error

Assuming 2048 bytes per block, there would be only 11806 blocks in the data track. So 211161 is massively out of range. It seems to be an offset of 412MB, which could well be the absolute position on the CD. So it seems we assume ISO9660 uses offsets from the start of the session (partition), whereas it uses offsets from the start of the disc. And we can only notice this when the session is not the first one on the disc.

[pulkomandy]

Ok, after reading the ISO 9660 spec I now understand the problem. The data session is declaring its size as 412MB, which is the size of the whole disc. Then it puts all the data in the last 23MB of this. CDs are made that way because it allows for multi-session recording: a later ISO9660 session can reuse data from a previous one (within the same track), by using a sector number that's outside its own session.

So we will hit the same problem with any multi-session disc, I think. Our trick to expose sessions as partitions doesn't really work.

[axeld]

That's very interesting, indeed. The ISO 9660 file system has a special support for multi-session disks that will then enable that absolute positioning. Maybe there is something broken with this, or it should not kick in in that case.

[pulkomandy]

Mh, I can't see this in the iso9660 filesystem. The only thing I see is we get the partition information (​http://cgit.haiku-os.org/haiku/tree/src/add-ons/kernel/file_systems/iso9660/iso9660.cpp#n614) and use the whole size of the CD rather than using the declared size of the volume. However, I don't see any other special handling there, to offset block numbers or anything like that.

The sector numbers stored on disc are "absolute" (that is, relative to the start of the disc, not the start of the track or session). I saw some special code attempting to handle various things in the session code. It seems to be merging or splitting sessions in several case in order to get things working as expected, and export one partition for each data track, and a single partition for all CDDA tracks.

In the "session" code we have support for mixed data/audio in a single session: ​http://cgit.haiku-os.org/haiku/tree/src/add-ons/kernel/partitioning_systems/session/Disc.cpp#n707 This is for "mixed mode CDs": ​http://en.wikipedia.org/wiki/Mixed_Mode_CD I could not get this to work (the data track is not visible at all on those CDs)

My CDs which triggers the KDL follow the "Blue Book" format instead: ​http://en.wikipedia.org/wiki/Blue_Book_(CD_standard) Unfortunately no specs for this are freely available. We don't seem to have any explicit support for them, or multisession, for that matter.

[bonefish]

Replying to pulkomandy:

So we will hit the same problem with any multi-session disc, I think. Our trick to expose sessions as partitions doesn't really work.

I guess the sessions need to be exposed as partitions that cover the whole range that can be addressed. If that means partitions may intersect (or even have identical ranges) that should generally be OK. There probably are sanity checks for intersections in some places, so we may need to introduce a partition flag to indicate that intersections are fine.

[pulkomandy]

I created a separate issue for mixed CDs where the data partition isn't visible: https://dev.haiku-os.org/ticket/11467

Back on topic here, I'm not sure making the session look like a partition which starts from the start of the disk would work. In that case, I think the ISO9660 filesystem will not find the table of content, which is somewhere towards the end of the disc, and not at the start.

Something that may work is substracting the session start sector ID from all sector numbers in the ISO9660 filesystem. This way, things would be in line with what's on the disc, without having to overlapp the sessions. However, on multisession discs where a later session references data from a previous one, we would access those using a negative offset from the last session (with the session partitionning system translating that back to a positive offset from the start of the disc). Would there be a problem with allowing accessing things before the partition start by using negative offsets? I can't think of any problems (seek has a signed parameter for the offset), but maybe I'm missing something here.

[axeld]

Replying to pulkomandy:

Mh, I can't see this in the iso9660 filesystem. The only thing I see is we get the partition information (​http://cgit.haiku-os.org/haiku/tree/src/add-ons/kernel/file_systems/iso9660/iso9660.cpp#n614) and use the whole size of the CD rather than using the declared size of the volume. However, I don't see any other special handling there, to offset block numbers or anything like that.

Because it is not needed. As you noticed, the block numbers on the CD are absolute, so we can use them as-is iff we work on the whole disk, rather than the session/partition/track.

[pulkomandy]

Yes, but if we give the whole disc to ISO9660, it will not be able to find the table of contents (which is at a fixed offset from the start of the data track, not the start of the disc).

I understand this is how it works usually, when there is a single session, the ISO9660 filesystem works directly with the disc (the session partitionning system is not even used in that case), and everything is fine.

When there are multiple sessions/tracks, and the ISO9660 track is the first one (as seen on video games with audio tracks, or the BeOS CDs for example), everything is also fine. The session partitionning system is used, but the first session has an offset of 0 and everything still works.

In the case of these Blue Book CDs, the ISO9660 filesystem starts in the last track on the disc, and in a separate session. So, the ISO9660 filesystem should look for his header at the start of that session, but then use disc-relative numbers as soon as it tries to access something else. So I see two possible solutions:

• Have the session partitionning make all sessions overlapped and starting from the first sector of the disc, to make ISO9660 happy, and then somehow inform it of where is the ISO9660 header.
• Have the partitionning system as it is now (so sessions match the physical layout of the disc), have ISO9660 convert the on-disc asolute numbers to session-start-relative (possibly negative), then the session partitionning system convert back to absolute numbers. This menas more involved changes to the ISO9660 and makes it more complex, but somehow it feels more "natural" to me than overlapping the partitions.

In the first case, I think we would also break any other filesystem used, for example the BFS track on the BeOS CDs. In the second case these would continue to work, using only positive offsets from the session start (as they do now).

[bonefish]

Replying to pulkomandy:

So I see two possible solutions:

• Have the session partitionning make all sessions overlapped and starting from the first sector of the disc, to make ISO9660 happy, and then somehow inform it of where is the ISO9660 header.

That's fairly simply. The partitioning system can use the ​partition_data::parameters field to store additional information (in driver settings format).

• Have the partitionning system as it is now (so sessions match the physical layout of the disc), have ISO9660 convert the on-disc asolute numbers to session-start-relative (possibly negative), then the session partitionning system convert back to absolute numbers. This menas more involved changes to the ISO9660 and makes it more complex, but somehow it feels more "natural" to me than overlapping the partitions.

Seeking to or reading from negative offsets is not possible. That wouldn't be a big problem, though, as the ISO9660 module could determine that it is mounting a session and instead work with the underlying disk device. I can't say that find this particularly "natural". IMO partitions should cover the accessed data range.

In the first case, I think we would also break any other filesystem used, for example the BFS track on the BeOS CDs. In the second case these would continue to work, using only positive offsets from the session start (as they do now).

But those aren't even multi-session -- and it wouldn't make any sense either -- or do I miss something?

[pulkomandy]

(In #7899) Duplicate of #11455 which has an analysis of the problem in the ticket comments.