Curious behaviour with calloc(-1, -1)
|Reported by:||ohnx56||Owned by:||nobody|
I noticed some curious behaviour with calloc(-1, -1) on hrev51615.
Since calloc uses unsigned ints, -1 = 4294967295.
4294967295 * 4294967295 overflows when multiplying to obtain 1.
Haiku does not check the overflow after multiplying here: https://github.com/haiku/haiku/blob/17b2a3cfcbc4fb1eb25d7eeb61e8fac997d7d835/src/system/libroot/posix/malloc/wrapper.cpp#L330
This results in Haiku assuming that the user wants 1 byte of memory allocated, which is technically incorrect...
A way to check for overflow would be to divide the result by one of the inputs (n or elem size) and ensure it equals the other input. From my understanding, glibc uses this: https://github.com/lattera/glibc/blob/master/malloc/malloc.c#L3170
This issue is notable because there are other ways for a 32-bit unsigned integer multiplication to overflow, possibly resulting in incorrect values being returned from Haiku.
Bottom line is this: When I call calloc() with impossibly large amounts of memory requirements, I expect a return value of NULL, and not a pointer :)
Change History (11)
comment:3 by , 3 years ago
|Component:||- General → System/libroot.so|
|Platform:||x86-64 → All|