Identify Missing/Incorrect POSIX Functionality

[bonefish]

Haiku is missing quite a bit of POSIX functionality (mandatory and optional), some is not implemented correctly, others is exposed in the wrong header. To identify those issues:

• Make a complete POSIX header list.
• Iterate through the headers an compare them with the specification (​http://www.opengroup.org/onlinepubs/9699919799/).
• Create tickets for all issues. Or even better: fix them.

[phoudoin]

Review made the 7th december 2010 on headers/posix/sys/* by a GCI student

[scottmc]

Another GCI student was tasked with reviewing headers A-H, here's the results from that research task, another task will be opened to implement these:

arpa/inet.h is fine.

---

assert.h is fine.

---

ctype.h None of the locale functions, ..._l(int, locale_t), are declared.

---

dirent.h The dir parameter of the dirfd function shouldn't be const.

---

dlfcn.h is fine.

---

errno.h is fine.

---

fcntl.h Flags for fcntl() are missing: F_GETOWN and F_SETOWN. File creation flag for open() is missing: O_TTY_INIT. File access mode flags for open() are missing: O_EXEC and O_SEARCH. The following optional functionality is missing: int posix_fadvise(int, off_t, off_t, int); int posix_fallocate(int, off_t, off_t); POSIX_FADV_DONTNEED POSIX_FADV_NOREUSE POSIX_FADV_NORMAL POSIX_FADV_RANDOM POSIX_FADV_SEQUENTIAL POSIX_FADV_WILLNEED

---

fenv.h is fine.

---

fnmatch.h is fine.

---

glob.h is fine.

---

grp.h is fine.

---

The following headers are missing: aio.h, complex.h, cpio.h, float.h, fmtmsg.h, ftw.h.

---

[korli]

On dirent.h, Axel might give some insights since he added the const in hrev26469.

[axeld]

I'm not entirely sure if I remember correctly, but it could be that dirfd() was not part of the POSIX specs back then, or maybe I just weren't aware that it is. In any case, having it accept only "const" should not cause any problems, but restrict us in the implementation of the function. Since it doesn't really matter, feel free to change it to what POSIX says.

[scottmc]

results from a research task by GCI student Hamish Morrison to check headers I-P

[korli]

Replying to axeld:

I'm not entirely sure if I remember correctly, but it could be that dirfd() was not part of the POSIX specs back then, or maybe I just weren't aware that it is. In any case, having it accept only "const" should not cause any problems, but restrict us in the implementation of the function. Since it doesn't really matter, feel free to change it to what POSIX says.

Changed in hrev40137.

[korli]

the part on ctype.h has a dependency on definition and implementation of locale_t in locale.h.

[kallisti5]

attaching work by Mohammed Nafees in gci 2011-2012

[kallisti5]

In fcntl.h:

Added definitions - F_GETOWN , F_SETOWN , O_EXEC , O_SEARCH , O_TTY_INIT

int posix_fadvise(int, off_t, off_t, int); int posix_fallocate(int, off_t, off_t);

POSIX_FADV_DONTNEED POSIX_FADV_NOREUSE POSIX_FADV_NORMAL POSIX_FADV_RANDOM POSIX_FADV_SEQUENTIAL POSIX_FADV_WILLNEED

In fcntl.cpp:

Implemented posix_fadvise & posix_fallocate. I don't know the code so just for initialization, added error returning code (as suggested by mmu_man in IRC)

In ctype.h:

Declared the locale functions.

In ctype.cpp:

Implemented the locale functions from refering the header file.

[kallisti5]

nafcool gci 2011-2012 work

[zooey]

Replying to kallisti5: [ ... ]

In ctype.h:

Declared the locale functions.

In ctype.cpp:

Implemented the locale functions from refering the header file.

While the function declarations in ctype.h look correct, all the ..._l() macro definitions and their implementation in ctype.cpp are lacking the locale parameter, which makes them useless.

[richienyhus]

Lastest info

[bonefish]

As written in the ticket description an actual list -- ideally with comments what is missing/lacking -- would be nice. I find bits and pieces scattered over ticket comments and attachments not that helpful. What I'm thinking of is a wiki page (or rather a structured set of pages), listing all the missing/lacking stuff. With comments adding details where necessary, links to tickets for the issues, and mention of POSIX test suite failures where applicable.

To detail on the process I had in mind:

1. Create a wiki page with a list of all POSIX headers. Mark each header "unchecked".
2. Pick a header marked "not checked" and check whether Haiku implements all of the contained functionality. That involves the following steps:
   • Check whether Haiku has the header and whether all functionality that should be declared there is declared.
   • Check whether declared functions and variables are actually defined in libroot.
   • If something is missing, create a wiki page for the header listing all the functionality missing. In the overview page replace the "unchecked" marker by "incomplete" linking to the page for the header. If nothing is missing, just replace the marker with "complete".
3. Repeat 2. until all headers have been checked.

If a header is very large, 2. can be broken down: A subpage for the header could be created, listing the progress of the checking -- the list of what has been found lacking so far plus e.g. just the point in the header up to which things have been checked or a complete list of yet unchecked functionality.

This process is rather simple and it will result in a very nicely structured view of Haiku's POSIX support status. When bug reports for functionality that is there but buggy come in, we could even track those in the wiki pages, giving us an even better picture of Haiku's POSIX compliance.

[luroh]

Move POSIX compatibility related tickets out of R1 milestone (FutureHaiku/Features).

[korli]

posix_fadvise and its definitions are added in hrev53154.

[cocobean]

DATE: 08/23/2019

Complete list of headers per POSIX.1-2017, 2018 edition (as of Haiku hrev53395):

<aio.h> - NOT IMPLEMENTED
<arpa/inet.h> - PASSED
<assert.h> - PASSED
<complex.h> - PASSED
<cpio.h> - PASSED (see/stat.h)
<ctype.h> - PASSED
<dirent.h> - PASSED
<dlfcn.h> - PASSED
<errno.h> - PASSED
<fcntl.h>  - work needed (posix_fallocate)
<fenv.h> - PASSED
<float.h> - NOT IMPLEMENTED
<fmtmsg.h> - NOT IMPLEMENTED
<fnmatch.h> - PASSED
<ftw.h> - PASSED
<glob.h> - PASSED
<grp.h> - PASSED
<iconv.h> - PASSED (GNU)
<inttypes.h> - PASSED
<iso646.h> - PASSED (GCC)
<langinfo.h> - PASSED (nl_langinfo_l)
<libgen.h> - PASSED
<limits.h> - work needed (PAGE_SIZE,PTHREAD_THREADS_MAX,RTSIG_MAX,SEM_*,PTHREAD_DESTRUCTOR_ITERATIONS,...)
<locale.h>  - work needed (LC*MASK,locale_t(duplocale.freelocale,duplocale))
<math.h> - PASSED
<monetary.h> - PASSED (strfmon_l)
<mqueue.h> - NOT IMPLEMENTED
<ndbm.h> - NOT IMPLEMENTED
<net/if.h> - PASSED
<netdb.h> - PASSED
<netinet/in.h> - PASSED
<netinet/tcp.h> - PASSED
<nl_types.h> - PASSED
<poll.h> - PASSED
<pthread.h>  - work needed
<pwd.h> - PASSED
<regex.h> - PASSED
<sched.h>  - work needed (sched_param*)
<search.h> - PASSED
<semaphore.h> - PASSED
<setjmp.h>  - work needed
<signal.h>  - work needed
<spawn.h> - PASSED (posix_spawnattr_getsched* not implemented)
<stdarg.h> - PASSED (GNU)
<stdbool.h> - NOT IMPLEMENTED
<stddef.h> - NOT IMPLEMENTED
<stdint.h> - UNCHECKED
<stdio.h>  - work needed
<stdlib.h> - PASSED
<string.h> - work needed (stpncpy, strcoll_l not implemented)
<strings.h> - PASSED (locale_t, strncasecmp_l, strcasecmp_l not implemented) 
<stropts.h> - NOT IMPLEMENTED
<sys/ipc.h> - PASSED
<sys/mman.h>  - work needed (POSIX_TYPED_MEM_*, MCL_*)
<sys/msg.h> - PASSED
<sys/resource.h> - PASSED
<sys/select.h> - PASSED
<sys/sem.h> - PASSED
<sys/shm.h> - NOT IMPLEMENTED
<sys/socket.h>  - work needed
<sys/stat.h> - PASSED
<sys/statvfs.h> - PASSED
<sys/time.h> - PASSED
<sys/times.h> - PASSED
<sys/types.h> - PASSED
<sys/uio.h> - PASSED
<sys/un.h> - PASSED
<sys/utsname.h> - PASSED
<sys/wait.h> - PASSED
<syslog.h> - PASSED
<tar.h> - PASSED
<termios.h> - work needed (pid_t tcgetsid(int);)
<tgmath.h> - NOT IMPLEMENTED
<time.h> - PASSED (*asctime_r(const struct tm *restrict, char *restrict))
<trace.h> - NOT IMPLEMENTED
<ulimit.h> - NOT IMPLEMENTED
<unistd.h>  - work needed (_CS_*,_PC_TIMESTAMP_RES*,_SC_SHELL,_SC_SPAWN,_SC_SPIN_LOCKS,... )
<utime.h> - PASSED (obsolete as of POSIX 1.2017)
<utmpx.h> - NOT IMPLEMENTED
<wchar.h> - PASSED
<wctype.h> - PASSED
<wordexp.h> - NOT IMPLEMENTED

Open POSIX Test Suite 1.5.2 results (hrev53393):

posix-tests-hrev53393-THR:conformance/interfaces/pthread_cancel/5-1: execution: FAILED 
posix-tests-hrev53393-THR:conformance/interfaces/pthread_detach/4-1: execution: FAILED 
posix-tests-hrev53393-THR:conformance/interfaces/pthread_detach/4-2: execution: FAILED 
posix-tests-hrev53393-THR:conformance/interfaces/pthread_kill/6-1: execution: FAILED 

[Tunas]

Haiku hrev53610 - POSIX compliance report

The following tests failed:

conformance/definitions/errno_h/4-1: execution: FAILED 
conformance/interfaces/clock_getres/6-2: execution: FAILED 
conformance/interfaces/clock_gettime/8-2: execution: FAILED 
conformance/interfaces/clock_settime/17-2: execution: FAILED 
conformance/interfaces/clock_settime/4-1: execution: FAILED 
conformance/interfaces/clock_settime/7-1: execution: FAILED 
conformance/interfaces/difftime/1-1: execution: FAILED 
conformance/interfaces/fork/12-1: execution: FAILED 
conformance/interfaces/clock_settime/8-1: execution: FAILED 
conformance/interfaces/fsync/7-1: execution: FAILED 
conformance/interfaces/mmap/11-4: execution: FAILED 
conformance/interfaces/mmap/11-5: execution: FAILED 
conformance/interfaces/mmap/23-1: execution: FAILED 
conformance/interfaces/mmap/31-1: execution: FAILED 
conformance/interfaces/mmap/14-1: execution: FAILED 
conformance/interfaces/pthread_attr_setscope/4-1: execution: FAILED 
conformance/interfaces/pthread_attr_setstack/2-1: execution: FAILED 
conformance/interfaces/pthread_cancel/5-1: execution: FAILED 
conformance/interfaces/pthread_detach/4-1: execution: FAILED 
conformance/interfaces/pthread_detach/4-2: execution: FAILED 
conformance/interfaces/pthread_mutex_getprioceiling/1-1: execution: FAILED 
conformance/interfaces/pthread_mutexattr_getprioceiling/1-1: execution: FAILED 
conformance/interfaces/pthread_mutexattr_setprioceiling/1-1: execution: FAILED 
conformance/interfaces/pthread_mutexattr_setprotocol/3-1: execution: FAILED 
conformance/interfaces/pthread_rwlock_rdlock/2-3: execution: FAILED 
conformance/interfaces/pthread_rwlock_rdlock/4-1: execution: FAILED 
conformance/interfaces/pthread_rwlock_timedrdlock/6-1: execution: FAILED 
conformance/interfaces/pthread_rwlock_timedwrlock/2-1: execution: FAILED 
conformance/interfaces/pthread_rwlock_timedwrlock/6-1: execution: FAILED 
conformance/interfaces/pthread_rwlock_wrlock/2-1: execution: FAILED 
conformance/interfaces/pthread_setschedparam/1-1: execution: FAILED 
conformance/interfaces/sched_get_priority_max/1-2: execution: FAILED 
conformance/interfaces/sched_get_priority_min/1-2: execution: FAILED 
conformance/interfaces/sem_init/7-1: execution: FAILED 
conformance/interfaces/sem_timedwait/6-1: execution: FAILED 
conformance/interfaces/sem_timedwait/6-2: execution: FAILED 
conformance/interfaces/sem_unlink/3-1: execution: FAILED 
conformance/interfaces/shm_open/37-1: execution: FAILED 
conformance/interfaces/shm_open/38-1: execution: FAILED 
conformance/interfaces/signal/6-1: execution: FAILED 
conformance/interfaces/signal/7-1: execution: FAILED 
conformance/interfaces/sigprocmask/17-1: execution: FAILED 
conformance/interfaces/sigprocmask/17-2: execution: FAILED 
conformance/interfaces/sigprocmask/17-3: execution: FAILED 
conformance/interfaces/sigprocmask/17-4: execution: FAILED 
conformance/interfaces/sigqueue/12-1: execution: FAILED 
conformance/interfaces/sigqueue/3-1: execution: FAILED 
conformance/interfaces/sigset/8-1: execution: FAILED 
conformance/interfaces/sigwait/3-1: execution: FAILED 
conformance/interfaces/timer_create/10-1: execution: FAILED 
conformance/interfaces/timer_create/11-1: execution: FAILED 

The following tests hung and/or were interrupted due to a crash:

conformance/interfaces/fork/11-1: execution: HUNG 
conformance/interfaces/mmap/6-1: execution: HUNG 
conformance/interfaces/mmap/6-2: execution: HUNG 
conformance/interfaces/mmap/6-3: execution: HUNG 
conformance/interfaces/pthread_rwlock_wrlock/3-1: execution: HUNG
conformance/interfaces/pthread_mutex_lock/3-1: execution: INTERRUPTED 
conformance/interfaces/shm_open/5-1: execution: INTERRUPTED 
conformance/interfaces/timer_getoverrun/speculative/6-1: execution: INTERRUPTED 
conformance/interfaces/timer_gettime/speculative/6-1: execution: INTERRUPTED 
conformance/interfaces/pthread_attr_setdetachstate/2-1: execution: INTERRUPTED 
conformance/interfaces/timer_settime/speculative/12-1: execution: INTERRUPTED 
conformance/interfaces/pthread_key_create/speculative/5-1: execution: INTERRUPTED 
conformance/interfaces/pthread_detach/1-1: execution: INTERRUPTED 
conformance/interfaces/mmap/11-2: execution: INTERRUPTED 
conformance/interfaces/pthread_join/speculative/6-1: execution: INTERRUPTED 
conformance/interfaces/timer_delete/speculative/5-1: execution: INTERRUPTED

[cocobean]

I compared the tests on Haiku with Linux and FreeBSD - and the Haiku code is comparable. Needs further review - but issues seem minor.

NOTE:

1. SCHED_FIFO is defined/exists in Haiku, but failed for sched_get_priority_min(SCHED_FIFO) and sched_get_priority_max(SCHED_FIFO). The other SCHED_* functions passed. These are considered optional functions for POSIX compliance. Removed from testing.

2. All POSIX pthread and semaphores defined, but fail some tests. Removed non-mandatory interface failures.

3. MMAP - See posix/sys/mman.h and #8514.

4. SHM - See Gerrit ticket 77 and #2657.

hrev54722 x86 (UPDATED 2020/11/18):

3 POSIX tests FAIL:

conformance/interfaces/pthread_cancel/5-1.test:execution:FAIL 
conformance/interfaces/pthread_detach/4-1.test:execution:FAIL 
conformance/interfaces/pthread_detach/4-2.test:execution:FAIL 

HAIKU is now mostly UNIX 03 conformant.

[cocobean]

Haiku hrev56921 x64 - POSIX UNIX 03 compliance report

MMAP memory protection testing:

conformance/interfaces/mmap/11-2: execution: INTERRUPTED 
conformance/interfaces/mmap/6-1: execution: UNRESOLVED 
conformance/interfaces/mmap/6-2: execution: UNRESOLVED 
conformance/interfaces/mmap/6-3: execution: UNRESOLVED 

NOTE: These tests passed on Linux.