KDL when opening folder on external exfat volume

[izaki]

Running hrev52866 x86_64 on three different systems

Description: Mounting a usb external exfat volume works, but opening a particular folder called "Amazon Drive" triggers the KDL attached to this ticket.

The folder behaves normally on Linux, FreeBSD, Windows 10, and macOS. The disk is free of defects.

Fun fact: xhci on every machine... Throwing it there just in case.

[izaki]

Consistently getting this KDL on every machine

[rrobgill]

What size is the exfat partition?

[izaki]

The partition is ~2TB. GNU Parted output follows.

Model: LaCie Rugged Mini USB3 (scsi)
Disk /dev/sdb: 2000GB
Sector size (logical/physical): 512B/4096B
Partition Table: gpt
Disk Flags: 

Number  Start   End     Size    File system  Name                  Flags
 1      20.5kB  210MB   210MB   fat32        EFI System Partition  boot, esp
 2      211MB   2000GB  2000GB 

[waddlesplash]

We don't properly support USB devices of size over 1.5TB, I think? So that may be the problem here.

[izaki]

That's an interesting note. So far I've tried opening every folder, opening random files, etc... Except for this folder that triggers KDL, everything else works. It would make sense that the folder is beyond those 1.5T (will check that)

what is the reason for this limit?

[pulkomandy]

The limit would be something like 512B (sector size) * 2³², so 2TiB.

But, if you hit this, we fail to read the HDD capacity and it shows as a 0 byte drive and you can't use it. So it seems your disk is just under that limit.

The limit is due to the way USB (SCSI, actually) commands for mass storage evolved. They increased the number of bits for the sector number several times to allow for larger disks, and we did not implement the new commands yet.

However, the block numbers in the panic are both in the 2TB range, but one of them shows as negative which makes me think we have just a signed vs unsigned error somewhere.

[rrobgill]

However, the block numbers in the panic are both in the 2TB range, but one of them shows as negative which makes me think we have just a signed vs unsigned error somewhere.

Exfat's Directoryiterator.h has fOffset as signed, compared to unsigned in other filesystems with similar usage. But it also explicitly assigns negative values to fOffset.

[korli]

Please check with hrev52917 or newer.

[izaki]

Panic on hrev52931 via ls

[izaki]

Tested on hrev52931, two different devices. Both produce errors.

I can get the same panic as before via tracker, I also reproduced running ls from inside the 'Amazon Drive' folder on a terminal (KDL shot attached).

[korli]

Please check with hrev52934 or newer.

[izaki]

Tested on hrev52934, got the same KDLs.

[korli]

I checked with a 2TB sparse drive that the filesystem is correctly mounted, but unfortunately this doesn't help in your case when data are at the end of the partition.

I added nonetheless checks in hrev52939, to try to at least avoid the panic.

[izaki]

Works on hrev52939!

Thanks!

[korli]

Could you check that the directory file list is the same size as on Linux? (for instance with 'ls | wc -l')

[izaki]

On Linux,

$ du -ah --apparent-size Amazon\ Drive/
103K  Amazon Drive/Thumbs.db
30K Amazon Drive/1421894120724.jpg
191 Amazon Drive/desktop.ini
261K  Amazon Drive/

On Haiku,

> du -ah --apparent-size Amazon\ Drive/
103K    Amazon Drive/Thumbs.db
30K     Amazon Drive/1421894120724.jpg
191     Amazon Drive/desktop.ini
du: 'Amazon Drive/': Bad data
261K    Amazon Drive/

Another interesting note is that I got a KDL while running du -ah on the top level folder of the drive. I am going to attempt to reproduce so I can capture it.

Also, all timestamps on the files are off on Haiku. I just noticed it.

> ls -lah
total 3.1M
dr-xr-xr-x 1 user root    0 Nov 23  1843  .
-r--r--r-- 1 user root 4.0K Feb  2  1882  ._0008
drwxrwxrwx 1 user root    0 Feb 28 22:01  ..
-r--r--r-- 1 user root 3.7K Jun 27  1882  .bashrc
dr-xr-xr-x 1 user root 128K Jul  4  1882  .fonts
-r--r--r-- 1 user root  869 Aug 12  1882  .profile
dr-xr-xr-x 1 user root 128K Jul  5  1882  .ssh
-r--r--r-- 1 user root 1.4K Jul 11  1882  .taskrc
-r--r--r-- 1 user root 2.8K Jul 16  1882  .tmux.conf
dr-xr-xr-x 1 user root 128K Jan  7  1883  .Trash-1000
dr-xr-xr-x 1 user root 128K Feb  2  1882  0001
dr-xr-xr-x 1 user root 128K Aug 29  1881 'Amazon Drive'
dr-xr-xr-x 1 user root 128K Oct  1  1882  0002
dr-xr-xr-x 1 user root 128K Jan 18  1882  0003
dr-xr-xr-x 1 user root 128K Jan 18  1882  Documents
dr-xr-xr-x 1 user root 128K Feb  2  1882  dotfiles
dr-xr-xr-x 1 user root 128K Sep 27  1882  Downloads
dr-xr-xr-x 1 user root 128K Jan 18  1882  Favorites
dr-xr-xr-x 1 user root 128K Jan 28  1882  0004
dr-xr-xr-x 1 user root 256K Sep 27  1882  Music
dr-xr-xr-x 1 user root 128K Sep 27  1882  Notes
dr-xr-xr-x 1 user root 128K Dec 29  1881  0005
dr-xr-xr-x 1 user root 128K Sep 27  1882  Pictures
dr-xr-xr-x 1 user root 128K Nov 24  1843  Portable
dr-xr-xr-x 1 user root 128K Jan 19  1882  Projects
dr-xr-xr-x 1 user root 128K Feb  2  1882  Reading
dr-xr-xr-x 1 user root 128K Sep 27  1882  Sandbox
dr-xr-xr-x 1 user root 128K Oct 11  1881 'System Volume Information'
dr-xr-xr-x 1 user root 128K Oct 11  1881  Videos
dr-xr-xr-x 1 user root 128K Jul 16  1882  0009