Outsource packageable third-party software

[bonefish]

Most (ported) third-party software should be removed from the Haiku source repository. HaikuPorts recipes should be created and the software should be provided as a package (or multiple packages). For libraries it may be necessary to provide them as build features (cf. build/jam/BuildFeatures for reference (e.g. icu or openssl)); in rare cases the build system may need to build (parts of) them from the sources (cf. build/jam/BuildFeatures for reference (e.g. zlib or libsolv)). Some packages will need to be built during the bootstrap process.

Software	Version	Build feature	Bootstrap	Status
agg	2.4 (warning: in-tree version has changes)	yes (source?)	yes	broken recipe
bash	4.3.27 (was 4.0.35(1))	-	yes	done
bc	1.07 (was 1.06)	-	-	done
compress	?	-	?	done
coreutils	8.22 (was 8.4)	-	yes	done
diffutils	3.3 (was 2.8.1)	-	yes	done
expat	2.1.0 (was 1.95.8)	yes	?	done
findutils	4.2.33	-	yes	done
fluidsynth	1.0.9	?	?	done
ftp	?	-	-	done
ftpd	?	-	-
gawk	4.1.0	-	yes	done
gdb	6.3	-	-	done
glut	?	?	-
gzip	1.6 (was 1.2.4)	-	?	done
less	451	-	-	done
libcrypt	removed in hrev50426	yes	?	removed
libedit	2015_03_21_3.1-2	yes	TODO	done
libfdt	1.3.0	? (!x86)	?
libgutenprint	5.2.9 (was 5.2.7)	yes	-	done
libiconv	?	yes	?	recipe available
libtelnet	?	?	-
libtiff	3.9.6 (was 3.9.4)	yes	-	done
libudis86	1.7.2 (was ?)	yes (x86, source)	?	recipe available
libutil	?	yes	?
mapm	?	yes	?
ncurses	5.9 (was 5.5)	yes	yes	done
netcat	1.10	-	-	done
patch	2.7.4 (was 2.5.4)	-	?	done
pdflib	5.0.3	yes	-	done
ping	?	-	-
ping6	?	-	-
qrencode	3.3.0	yes	-	done
rcs	5.9.4	-	-	done
sharutils	4.14.1	-	-	done
tcpdump	4.7.4 (was 3.9.5)	-	-	done
telnet	?	-	-
telnetd	?	-	-
traceroute	1.4a12	-	-
unzip	6.0 (was 5.50)	-	?	done
watch	0.2.0	-	-
wget	1.15	-	-	done
x86emu	?	yes (x86*, source)	?
zip	3.0 (was 2.32)	-	?	done

[bonefish]

(In #10265) I suppose the file format is no longer compatible with that of our ancient less version. less should be outsourced to HaikuPorts, which will, as a side effect, avoid this build issue.

[bonefish]

(In #10444) Also cf. #10267 which tracks the status of third-party software to be outsourced.

[pulkomandy]

(In #10572) How would that fix #10265? That was an issue with the version of less on the host system (Fedora, in that case).

Anyway, the first step is getting package built for all 3 supported archs (x86_gcc2, x86, x86_64). I built the first, will upload once vmrepo is back online...

Waiting for other people to do the other archs, before we proceed with removing the in-tree version.

[korli]

findutils added in hrev46925.

[korli]

gawk 4.1.0 added in hrev46929.

[waddlesplash]

Jerome, while you're at it can you look at #10572 and maybe finish it?

[pulkomandy]

I tried doing a clean build of Haiku, and I got this:

gawk: /Donnees/Dev/Haiku/haiku/src/apps/devices/pci-header.awk:169: fatal error: internal error

Seems the new gawk is not compatible with what we used to have?

Moreover, a lot of haikuports packages need either gawk, find, or xargs (starting with libtool). They all lack the needed "requires", and it seems the Haiku package still provides those, so even adding the requires to the recipes doesn't work. I was trying to see if a clean haiku build would solve that (as I couldn't find why it would still provides findutils in the sources), but I hit the other issue with gawk.

[anevilyak]

FYI, the package provides in src/data/package_infos/.../haiku need to be updated to reflect the commands that are no longer present due to outsourcing, otherwise there will be dep resolution problems ; this explains some of the issues I was running into in ports. Will see if I can find time to update those later, assuming no one beats me to it.

[anevilyak]

Updated in hrev46959. Also updated some of the ports recipes accordingly. However, I'm now running into build issues with pkgconfig on gcc4, specifically link failures related to its built-in copy of glib, i.e.:

./glib/glib/.libs/libglib-2.0.a(gcharset.o): In function `get_alias_hash':
/sources/pkg-config-0.27.1/glib/glib/gcharset.c:54: undefined reference to `_g_locale_get_charset_aliases'

Any ideas as to what might be going on there?

[korli]

Replying to pulkomandy:

Seems the new gawk is not compatible with what we used to have?

It seems gawk 'configure' detects multibyte support and tries to use it. I disabled it as a workaround in hrev46965.

[korli]

sharutils 4.14.1 added in hrev47117. No use in bootstrap process.

[korli]

less 451 added in hrev47121.

[korli]

qrencode 3.3.0 added in hrev47126.

[korli]

(In #10444) Fixed indeed.

[korli]

gutenprint added in hrev47217.

[korli]

wget added in hrev47295.

[korli]

tiff added in hrev47300 (still missing secondary architectures packages though).

[mmu_man]

fluidsynth added as external package in hrev47698. Still need to remove the in-tree version and declare the external one as dependency.

[korli]

ncurses should also probably be built during the bootstrap process, as bash depends on it.

[bonefish]

Replying to korli:

ncurses should also probably be built during the bootstrap process, as bash depends on it.

​It already is. The work remaining for ncurses is to declare a build feature, use it where needed, and remove the ncurses sources from the source tree. It becomes less work, if most of the software needing it is outsourced before, but at least Debugger will still require ncurses.

[zooey]

Fix color of 'recipe available' status to something a bit more readable.

[pulkomandy]

ncurses outsourced in hrev48517.

[korli]

rcs outsourced in hrev48663. coreutils outsourced in hrev48782, except for "su".

[waddlesplash]

GDB outsourced in hrev48791.

[korli]

patch outsourced in hrev48811. Should a bootstrap package be added isn't clear yet.

[korli]

unzip outsourced in hrev48833.

[korli]

zip outsourced in hrev48851.

[korli]

netcat outsourced in hrev48856 (command is nc).

[korli]

tcpdump outsourced in hrev49137 (along with libpcap).

[korli]

gzip outsourced in hrev49261.

[korli]

expat outsourced in hrev49270.

[kallisti5]

libedit outsourced in hrev49366

We need to evaluate removing Debugger and bin/net/bc libedit requirements on bootstrap builds. We might be able to get away with not consuming libedit on bootstrap resulting in one less bootstrap package :-)

Otherwise, I need to build out a libedit bootstrap package that depends on ncurses.

[korli]

x86emu upstream seems to be located at ​http://cgit.freedesktop.org/xorg/xserver/tree/hw/xfree86/x86emu. Sadly I see no way to just download the x86emu directory without the rest (git isn't helpful with that), this makes writing a recipe for it pretty useless. I would let this project in the tree. Moreover Gentoo has no recipe for it. Thoughts?

[bonefish]

Why not host a source tarball somewhere? Isn't that what has been done with other ports where the repository wasn't suitable for our needs as well?

[pulkomandy]

Gentoo has a libx86emu: ​http://gpo.zugaina.org/dev-libs/libx86emu. Apparently they get the source as an RPM from OpenSUSE: ​http://download.opensuse.org/source/factory/repo/oss/suse/src/libx86emu-1.5-1.2.src.rpm.mirrorlist

It is also up on github: ​https://github.com/wfeldt/libx86emu

[korli]

diffutils outsourced in hrev49494. Is is really a bootstrap package?

[pulkomandy]

I think it is/was needed in the bootstrap image so haikuporter can apply and extract patches. But maybe haikuporter uses git only for that now?

[korli]

removed libcrypt in hrev50426.

[korli]

switched to fluidlite.

[korli]

pdflib removed in hrev51243.

[korli]

bc removed in hrev51466

[KapiX]

I'll take care of OpenEXR (recipe for gcc2 already committed to HaikuPorts).

[waddlesplash]

compress done in hrev51718.

[Coldfirex]

FTP says recipe available. Where would it be so we can finish those changes?

[pulkomandy]

Actually FTP is done but I forgot to update this. We ship tnftp now.

The recipes are at Haikuports when something is noted "recipe available". The work to do is:

• Delete the sources from Haiku
• Add some definitions in jamfiles to use an hpkg file
• Put the hpkg file in the generated/downloads/ directory manually
• Make any needed adjustments to the code so that Haiku builds and run correctly

You can see various hrev numbers linked above, showing examples of that.