Opened 6 years ago

Last modified 22 months ago

#10267 new enhancement

Outsource packageable third-party software

Reported by: bonefish Owned by: bonefish
Priority: normal Milestone: R1
Component: Build System Version: R1/Development
Keywords: Cc:
Blocked By: #10572 Blocking:
Has a Patch: no Platform: All

Description (last modified by waddlesplash)

Most (ported) third-party software should be removed from the Haiku source repository. HaikuPorts recipes should be created and the software should be provided as a package (or multiple packages). For libraries it may be necessary to provide them as build features (cf. build/jam/BuildFeatures for reference (e.g. icu or openssl)); in rare cases the build system may need to build (parts of) them from the sources (cf. build/jam/BuildFeatures for reference (e.g. zlib or libsolv)). Some packages will need to be built during the bootstrap process.

SoftwareVersionBuild featureBootstrapStatus
agg2.4 (warning: in-tree version has changes)yes (source?)yes broken recipe
bash4.3.27 (was 4.0.35(1))-yes done
bc1.07 (was 1.06)-- done
compress?-? done
coreutils8.22 (was 8.4) -yes done
diffutils3.3 (was 2.8.1)-yes done
expat2.1.0 (was 1.95.8)yes? done
findutils4.2.33-yes done
fluidsynth1.0.9?? done
ftp?-- recipe available
ftpd?--
gawk4.1.0-yes done
gdb6.3-- done
glut??-
gzip1.6 (was 1.2.4)-? done
less451-- done
libcryptremoved in hrev50426yes? removed
libedit2015_03_21_3.1-2yesTODO done
libfdt1.3.0? (!x86)?
libgutenprint5.2.9 (was 5.2.7)yes- done
libiconv?yes? recipe available
libtelnet??-
libtiff3.9.6 (was 3.9.4)yes- done
libudis861.7.2 (was ?)yes (x86, source)? recipe available
libutil?yes?
mapm?yes?
ncurses5.9 (was 5.5)yesyes done
netcat1.10-- done
patch2.7.4 (was 2.5.4)-? done
pdflib5.0.3yes- done
ping?--
ping6?--
qrencode3.3.0yes- done
rcs5.9.4-- done
sharutils4.14.1-- done
tcpdump4.7.4 (was 3.9.5)-- done
telnet?--
telnetd?--
traceroute1.4a12--
unzip6.0 (was 5.50)-? done
watch0.2.0--
wget1.15-- done
x86emu?yes (x86*, source)?
zip3.0 (was 2.32)-? done

Change History (51)

comment:1 by bonefish, 6 years ago

Blocking: 10265 added

(In #10265) I suppose the file format is no longer compatible with that of our ancient less version. less should be outsourced to HaikuPorts, which will, as a side effect, avoid this build issue.

comment:2 by bonefish, 6 years ago

Blocking: 10444 added

(In #10444) Also cf. #10267 which tracks the status of third-party software to be outsourced.

comment:3 by pulkomandy, 6 years ago

Blocked By: 10572 added

(In #10572) How would that fix #10265? That was an issue with the version of less on the host system (Fedora, in that case).

Anyway, the first step is getting package built for all 3 supported archs (x86_gcc2, x86, x86_64). I built the first, will upload once vmrepo is back online...

Waiting for other people to do the other archs, before we proceed with removing the in-tree version.

comment:4 by korli, 6 years ago

Description: modified (diff)

findutils added in hrev46925.

comment:5 by korli, 6 years ago

Description: modified (diff)

gawk 4.1.0 added in hrev46929.

comment:6 by waddlesplash, 6 years ago

Jerome, while you're at it can you look at #10572 and maybe finish it?

comment:7 by pulkomandy, 6 years ago

I tried doing a clean build of Haiku, and I got this:

gawk: /Donnees/Dev/Haiku/haiku/src/apps/devices/pci-header.awk:169: fatal error: internal error

Seems the new gawk is not compatible with what we used to have?

Moreover, a lot of haikuports packages need either gawk, find, or xargs (starting with libtool). They all lack the needed "requires", and it seems the Haiku package still provides those, so even adding the requires to the recipes doesn't work. I was trying to see if a clean haiku build would solve that (as I couldn't find why it would still provides findutils in the sources), but I hit the other issue with gawk.

comment:8 by anevilyak, 6 years ago

FYI, the package provides in src/data/package_infos/.../haiku need to be updated to reflect the commands that are no longer present due to outsourcing, otherwise there will be dep resolution problems ; this explains some of the issues I was running into in ports. Will see if I can find time to update those later, assuming no one beats me to it.

comment:9 by anevilyak, 6 years ago

Updated in hrev46959. Also updated some of the ports recipes accordingly. However, I'm now running into build issues with pkgconfig on gcc4, specifically link failures related to its built-in copy of glib, i.e.:

./glib/glib/.libs/libglib-2.0.a(gcharset.o): In function `get_alias_hash':
/sources/pkg-config-0.27.1/glib/glib/gcharset.c:54: undefined reference to `_g_locale_get_charset_aliases'

Any ideas as to what might be going on there?

in reply to:  7 comment:10 by korli, 6 years ago

Replying to pulkomandy:

Seems the new gawk is not compatible with what we used to have?

It seems gawk 'configure' detects multibyte support and tries to use it. I disabled it as a workaround in hrev46965.

comment:11 by korli, 6 years ago

Description: modified (diff)

sharutils 4.14.1 added in hrev47117. No use in bootstrap process.

comment:12 by korli, 5 years ago

Description: modified (diff)

less 451 added in hrev47121.

comment:13 by korli, 5 years ago

Blocking: 10265 removed

comment:14 by korli, 5 years ago

Description: modified (diff)

qrencode 3.3.0 added in hrev47126.

comment:15 by korli, 5 years ago

Blocking: 10444 removed

(In #10444) Fixed indeed.

comment:16 by korli, 5 years ago

Description: modified (diff)

gutenprint added in hrev47217.

comment:17 by korli, 5 years ago

Description: modified (diff)

wget added in hrev47295.

comment:18 by korli, 5 years ago

Description: modified (diff)

tiff added in hrev47300 (still missing secondary architectures packages though).

comment:19 by mmu_man, 5 years ago

fluidsynth added as external package in hrev47698. Still need to remove the in-tree version and declare the external one as dependency.

comment:20 by korli, 5 years ago

Description: modified (diff)

ncurses should also probably be built during the bootstrap process, as bash depends on it.

in reply to:  20 comment:21 by bonefish, 5 years ago

Replying to korli:

ncurses should also probably be built during the bootstrap process, as bash depends on it.

It already is. The work remaining for ncurses is to declare a build feature, use it where needed, and remove the ncurses sources from the source tree. It becomes less work, if most of the software needing it is outsourced before, but at least Debugger will still require ncurses.

comment:22 by korli, 5 years ago

Description: modified (diff)

comment:23 by korli, 5 years ago

Description: modified (diff)

comment:24 by pulkomandy, 5 years ago

Description: modified (diff)

comment:25 by pulkomandy, 5 years ago

Description: modified (diff)

comment:26 by zooey, 5 years ago

Description: modified (diff)

Fix color of 'recipe available' status to something a bit more readable.

comment:27 by pulkomandy, 5 years ago

Description: modified (diff)

comment:28 by pulkomandy, 5 years ago

Description: modified (diff)

ncurses outsourced in hrev48517.

comment:29 by korli, 5 years ago

Description: modified (diff)

rcs outsourced in hrev48663. coreutils outsourced in hrev48782, except for "su".

comment:30 by waddlesplash, 5 years ago

Description: modified (diff)

GDB outsourced in hrev48791.

comment:31 by korli, 5 years ago

Description: modified (diff)

patch outsourced in hrev48811. Should a bootstrap package be added isn't clear yet.

comment:32 by korli, 5 years ago

Description: modified (diff)

unzip outsourced in hrev48833.

comment:33 by korli, 5 years ago

Description: modified (diff)

zip outsourced in hrev48851.

comment:34 by korli, 5 years ago

Description: modified (diff)

netcat outsourced in hrev48856 (command is nc).

comment:35 by korli, 4 years ago

Description: modified (diff)

tcpdump outsourced in hrev49137 (along with libpcap).

comment:36 by korli, 4 years ago

Description: modified (diff)

comment:37 by korli, 4 years ago

Description: modified (diff)

gzip outsourced in hrev49261.

comment:38 by korli, 4 years ago

Description: modified (diff)

expat outsourced in hrev49270.

comment:39 by kallisti5, 4 years ago

Description: modified (diff)

libedit outsourced in hrev49366

We need to evaluate removing Debugger and bin/net/bc libedit requirements on bootstrap builds. We might be able to get away with not consuming libedit on bootstrap resulting in one less bootstrap package :-)

Otherwise, I need to build out a libedit bootstrap package that depends on ncurses.

comment:40 by korli, 4 years ago

x86emu upstream seems to be located at http://cgit.freedesktop.org/xorg/xserver/tree/hw/xfree86/x86emu. Sadly I see no way to just download the x86emu directory without the rest (git isn't helpful with that), this makes writing a recipe for it pretty useless. I would let this project in the tree. Moreover Gentoo has no recipe for it. Thoughts?

comment:41 by bonefish, 4 years ago

Why not host a source tarball somewhere? Isn't that what has been done with other ports where the repository wasn't suitable for our needs as well?

comment:43 by korli, 4 years ago

Description: modified (diff)

diffutils outsourced in hrev49494. Is is really a bootstrap package?

comment:44 by pulkomandy, 4 years ago

I think it is/was needed in the bootstrap image so haikuporter can apply and extract patches. But maybe haikuporter uses git only for that now?

comment:45 by korli, 3 years ago

Description: modified (diff)

removed libcrypt in hrev50426.

comment:46 by korli, 3 years ago

Description: modified (diff)

switched to fluidlite.

comment:47 by pulkomandy, 3 years ago

Description: modified (diff)

comment:48 by korli, 2 years ago

Description: modified (diff)

pdflib removed in hrev51243.

comment:49 by korli, 2 years ago

Description: modified (diff)

bc removed in hrev51466

comment:50 by KapiX, 22 months ago

I'll take care of OpenEXR (recipe for gcc2 already committed to HaikuPorts).

EDIT: done in hrev51687.

Last edited 22 months ago by KapiX (previous) (diff)

comment:51 by waddlesplash, 22 months ago

Description: modified (diff)

compress done in hrev51718.

Note: See TracTickets for help on using tickets.