ioctl FIONREAD should set errno to ENOTTY for non supported file descriptors

[korli]

see ​http://www.daemon-systems.org/man/ioctl.2.html

[ENOTTY]           The specified request does not apply to the kind of
                        object that the descriptor d references.

This affects GNU Classpath. see ​http://cvs.savannah.gnu.org/viewvc/classpath/native/jni/java-nio/gnu_java_nio_VMChannel.c?revision=1.22&root=classpath&view=markup method Java_gnu_java_nio_VMChannel_available

  if (ioctl (fd, FIONREAD, &avail) == -1)
    {
#if defined(ENOTTY) && defined(HAVE_FSTAT)
      if (errno == ENOTTY)
        {

[korli]

I would add a check for FIONREAD op in src/system/kernel/fs/fd.cpp:fd_ioctl() For FDTYPE_FILE, FDTYPE_DIR, FDTYPE_ATTR_DIR, FDTYPE_ATTR values of descriptor->type, it would return ENOTTY. There is no B_* error mapped to ENOTTY at the moment.

I'm wondering why it returns B_NOT_SUPPORTED (EOPNOTSUPP) instead of ENOTTY which is recommended by the ​spec.

[korli]

common_ioctl() returns B_NOT_SUPPORTED, fifo_ioctl() EINVAL, rootfs_ioctl() B_BAD_VALUE, bfs_ioctl() B_BAD_VALUE. EINVAL could be converted to ENOTTY in upper levels in case the op is FIONREAD, I don't know.

[korli]

Fixed in hrev40686. ENOTTY is only a POSIX error code, an Haiku error code would be preferred for the kernel.

[korli]

Ingo mentioned on the commits list:

This is not correct. FDTYPE_FILE means a vnode based file descriptor. Whether
the particular ioctls are supported utterly depends on the vnode. E.g. FIFOs,
while the implementation doesn't ATM, actually could support those. In case
of devices -- and this includes the TTY driver -- the call is forwarded to
the device's control() hook.

The other listed FD types are also vnode based and apparently the
common_ioctl() hook is called for them as well. Which is seriously broken,
since they pass the wrong cookie to the vnode's hook. I'd use a new (no-op)
hook for them and perhaps put the FIONBIO/FIONREAD check there.

[korli]

Tested OK with hrev40787.