Opened 3 years ago
Last modified 3 years ago
#17277 new bug
@bootstrap-* doesn't filter by HaikuPorts requirements
| Reported by: | kallisti5 | Owned by: | nobody |
|---|---|---|---|
| Priority: | normal | Milestone: | Unscheduled |
| Component: | Build System | Version: | R1/Development |
| Keywords: | bootstrap | Cc: | |
| Blocked By: | Blocking: | #17318 | |
| Platform: | All |
Description
While running a boootstrap-* build, i noticed that our build system doesn't filter the _source* package creation by what is required within build/jam/repositories/HaikuPorts/(arch)
What this means is during the bootstrap, the latest version of *every* package is turned into a _source.hpkg to be "put into" the bootstrap image.
to help highlight this, here is llvm :-)
[kallisti5@eris haiku]$ ls .../haiku/generated.riscv64/objects/haiku/riscv64/packaging/repositories/HaikuPorts-sources-build/packages/ -la | grep llvm -rw-r--r-- 1 kallisti5 users 101861588 Sep 22 09:33 llvm12_source-12.0.0-1-source.hpkg -rw-r--r-- 1 kallisti5 users 101832088 Sep 22 09:35 llvm12_source_rigged-12.0.0-1-riscv64.hpkg -rw-r--r-- 1 kallisti5 users 68067275 Sep 22 09:28 llvm7_source-7.0.1-2-source.hpkg -rw-r--r-- 1 kallisti5 users 68078411 Sep 22 09:29 llvm7_source_rigged-7.0.1-2-riscv64.hpkg -rw-r--r-- 1 kallisti5 users 81062508 Sep 22 09:39 llvm9_source-9.0.1-2-source.hpkg -rw-r--r-- 1 kallisti5 users 81061377 Sep 22 09:41 llvm9_source_rigged-9.0.1-2-riscv64.hpkg -rw-r--r-- 1 kallisti5 users 57360450 Sep 22 09:24 llvm_source-5.0.0-4-source.hpkg -rw-r--r-- 1 kallisti5 users 57371741 Sep 22 09:25 llvm_source_rigged-5.0.0-4-riscv64.hpkg
I'm guessing this is a remnant of @bootstrap-* being designed before we had a massive haikuports repository.
Overall, @bootstrap-* should only build source packages (and dependencies) from haikuports for packages listed in build/jam/repositories/HaikuPorts/(arch) as this represents the purpose of @bootstrap (generating build-packages)
Change History (6)
comment:1 by , 3 years ago
comment:2 by , 3 years ago
https://dev.haiku-os.org/ticket/17318 was closed as a duplicate of this one
comment:3 by , 3 years ago
| Blocking: | 17318 added |
|---|
comment:4 by , 3 years ago
Reverting to eee899ad694b4fd65f5ce60365fe685f34ae652a of haikuporter seems to not build all sources, but I will upgrade and compare. It does seem to pull some weird things according to logs, here is what ends up in dir (limited to depth of four):
du -hd4 4.0K ./disk 0 ./tmp 12M ./cross-tools-arm64/lib/gcc/aarch64-unknown-haiku 12M ./cross-tools-arm64/lib/gcc 8.0K ./cross-tools-arm64/lib/bfd-plugins 12M ./cross-tools-arm64/lib 8.4M ./cross-tools-arm64/share/info 3.1M ./cross-tools-arm64/share/man/man1 84K ./cross-tools-arm64/share/man/man7 3.2M ./cross-tools-arm64/share/man 104K ./cross-tools-arm64/share/gcc-8.3.0/python 104K ./cross-tools-arm64/share/gcc-8.3.0 12M ./cross-tools-arm64/share 28M ./cross-tools-arm64/bin 0 ./cross-tools-arm64/aarch64-unknown-haiku/bin 2.0M ./cross-tools-arm64/aarch64-unknown-haiku/lib/ldscripts 40M ./cross-tools-arm64/aarch64-unknown-haiku/lib 11M ./cross-tools-arm64/aarch64-unknown-haiku/include/c++ 11M ./cross-tools-arm64/aarch64-unknown-haiku/include 50M ./cross-tools-arm64/aarch64-unknown-haiku 72M ./cross-tools-arm64/libexec/gcc/aarch64-unknown-haiku 72M ./cross-tools-arm64/libexec/gcc 72M ./cross-tools-arm64/libexec 0 ./cross-tools-arm64/include 171M ./cross-tools-arm64 2.8M ./build 13M ./objects/linux/x86_64/release 188K ./objects/linux/x86_64/common 13M ./objects/linux/x86_64 3.1M ./objects/linux/lib 16M ./objects/linux 19G ./objects/haiku/arm64/packaging 5.2M ./objects/haiku/arm64/common 142M ./objects/haiku/arm64/release 19G ./objects/haiku/arm64 19G ./objects/haiku 1.3M ./objects/common/data/mime_db 4.0K ./objects/common/data/keymaps 4.0K ./objects/common/data/keyboard_layouts 4.0K ./objects/common/data/deskbar 1.3M ./objects/common/data 4.0K ./objects/common/system/libnetwork 4.0K ./objects/common/system 4.0K ./objects/common/build/libsolv 4.0K ./objects/common/build 704K ./objects/common/libs/compat 396K ./objects/common/libs/termcap 1.1M ./objects/common/libs 2.4M ./objects/common 112K ./objects/catalogs/kits/package 364K ./objects/catalogs/kits/textencoding 4.7M ./objects/catalogs/kits/tracker 116K ./objects/catalogs/kits/media 132K ./objects/catalogs/kits/mail 5.9M ./objects/catalogs/kits 128K ./objects/catalogs/servers/debug 228K ./objects/catalogs/servers/keystore 252K ./objects/catalogs/servers/mount 132K ./objects/catalogs/servers/notification 232K ./objects/catalogs/servers/package 252K ./objects/catalogs/servers/registrar 132K ./objects/catalogs/servers/syslog_daemon 1.4M ./objects/catalogs/servers 488K ./objects/catalogs/apps/deskbar 128K ./objects/catalogs/apps/tracker 888K ./objects/catalogs/apps/aboutsystem 2.5M ./objects/catalogs/apps/charactermap 436K ./objects/catalogs/apps/deskcalc 844K ./objects/catalogs/apps/devices 1.2M ./objects/catalogs/apps/diskprobe 248K ./objects/catalogs/apps/diskusage 1.2M ./objects/catalogs/apps/drivesetup 504K ./objects/catalogs/apps/expander 252K ./objects/catalogs/apps/networkstatus 492K ./objects/catalogs/apps/processcontroller 612K ./objects/catalogs/apps/showimage 848K ./objects/catalogs/apps/stylededit 1.3M ./objects/catalogs/apps/terminal 388K ./objects/catalogs/apps/text_search 244K ./objects/catalogs/apps/workspaces 13M ./objects/catalogs/apps 248K ./objects/catalogs/bin/desklink 128K ./objects/catalogs/bin/dstcheck 16K ./objects/catalogs/bin/filepanel 128K ./objects/catalogs/bin/screen_blanker 520K ./objects/catalogs/bin 732K ./objects/catalogs/preferences/appearance 256K ./objects/catalogs/preferences/backgrounds 128K ./objects/catalogs/preferences/deskbar 1.6M ./objects/catalogs/preferences/filetypes 432K ./objects/catalogs/preferences/input 1.4M ./objects/catalogs/preferences/keymap 368K ./objects/catalogs/preferences/locale 472K ./objects/catalogs/preferences/network 344K ./objects/catalogs/preferences/notifications 508K ./objects/catalogs/preferences/screen 484K ./objects/catalogs/preferences/shortcuts 484K ./objects/catalogs/preferences/time 128K ./objects/catalogs/preferences/tracker 248K ./objects/catalogs/preferences/virtualmemory 7.5M ./objects/catalogs/preferences 260K ./objects/catalogs/add-ons/translators 252K ./objects/catalogs/add-ons/tracker 128K ./objects/catalogs/add-ons/input_server 384K ./objects/catalogs/add-ons/disk_systems 1.0M ./objects/catalogs/add-ons 29M ./objects/catalogs 19G ./objects 5.4M ./build_packages/gcc_bootstrap_syslibs_devel-8.3.0_2021_09_21-1-arm64/develop/lib 12M ./build_packages/gcc_bootstrap_syslibs_devel-8.3.0_2021_09_21-1-arm64/develop/headers 17M ./build_packages/gcc_bootstrap_syslibs_devel-8.3.0_2021_09_21-1-arm64/develop 17M ./build_packages/gcc_bootstrap_syslibs_devel-8.3.0_2021_09_21-1-arm64 2.4M ./build_packages/gcc_bootstrap_syslibs-8.3.0_2021_09_21-1-arm64/lib 2.4M ./build_packages/gcc_bootstrap_syslibs-8.3.0_2021_09_21-1-arm64 20K ./build_packages/icu_bootstrap-67.1-1-arm64/data/licenses 28M ./build_packages/icu_bootstrap-67.1-1-arm64/data/icu 28M ./build_packages/icu_bootstrap-67.1-1-arm64/data 16K ./build_packages/icu_bootstrap-67.1-1-arm64/lib/icu 7.7M ./build_packages/icu_bootstrap-67.1-1-arm64/lib 4.8M ./build_packages/icu_bootstrap-67.1-1-arm64/develop/headers 16K ./build_packages/icu_bootstrap-67.1-1-arm64/develop/lib 4.9M ./build_packages/icu_bootstrap-67.1-1-arm64/develop 24K ./build_packages/icu_bootstrap-67.1-1-arm64/bin 8.0K ./build_packages/icu_bootstrap-67.1-1-arm64/documentation/man 8.0K ./build_packages/icu_bootstrap-67.1-1-arm64/documentation 40M ./build_packages/icu_bootstrap-67.1-1-arm64 4.0K ./build_packages/zlib_bootstrap-1.2.11-1-arm64/data/licenses 4.0K ./build_packages/zlib_bootstrap-1.2.11-1-arm64/data 224K ./build_packages/zlib_bootstrap-1.2.11-1-arm64/lib 112K ./build_packages/zlib_bootstrap-1.2.11-1-arm64/develop/headers 128K ./build_packages/zlib_bootstrap-1.2.11-1-arm64/develop/lib 240K ./build_packages/zlib_bootstrap-1.2.11-1-arm64/develop 8.0K ./build_packages/zlib_bootstrap-1.2.11-1-arm64/documentation/man 8.0K ./build_packages/zlib_bootstrap-1.2.11-1-arm64/documentation 480K ./build_packages/zlib_bootstrap-1.2.11-1-arm64 952K ./build_packages/libsolv_bootstrap-0.3.0_haiku_2014_12_22-1-arm64/lib 4.0K ./build_packages/libsolv_bootstrap-0.3.0_haiku_2014_12_22-1-arm64/data/cmake 4.0K ./build_packages/libsolv_bootstrap-0.3.0_haiku_2014_12_22-1-arm64/data 188K ./build_packages/libsolv_bootstrap-0.3.0_haiku_2014_12_22-1-arm64/develop/headers 0 ./build_packages/libsolv_bootstrap-0.3.0_haiku_2014_12_22-1-arm64/develop/lib 188K ./build_packages/libsolv_bootstrap-0.3.0_haiku_2014_12_22-1-arm64/develop 1.2M ./build_packages/libsolv_bootstrap-0.3.0_haiku_2014_12_22-1-arm64 2.3M ./build_packages/libsolv_bootstrap_source-0.3.0_haiku_2014_12_22-1-source/develop/sources 2.3M ./build_packages/libsolv_bootstrap_source-0.3.0_haiku_2014_12_22-1-source/develop 2.3M ./build_packages/libsolv_bootstrap_source-0.3.0_haiku_2014_12_22-1-source 4.0K ./build_packages/zlib_bootstrap_source-1.2.11-1-source/data/licenses 4.0K ./build_packages/zlib_bootstrap_source-1.2.11-1-source/data 3.5M ./build_packages/zlib_bootstrap_source-1.2.11-1-source/develop/sources 3.5M ./build_packages/zlib_bootstrap_source-1.2.11-1-source/develop 3.5M ./build_packages/zlib_bootstrap_source-1.2.11-1-source 8.0K ./build_packages/freetype_bootstrap-2.6.3-1-arm64/data/licenses 8.0K ./build_packages/freetype_bootstrap-2.6.3-1-arm64/data/aclocal 16K ./build_packages/freetype_bootstrap-2.6.3-1-arm64/data 3.5M ./build_packages/freetype_bootstrap-2.6.3-1-arm64/lib 4.0K ./build_packages/freetype_bootstrap-2.6.3-1-arm64/documentation/man 4.0K ./build_packages/freetype_bootstrap-2.6.3-1-arm64/documentation 4.0K ./build_packages/freetype_bootstrap-2.6.3-1-arm64/bin 1.3M ./build_packages/freetype_bootstrap-2.6.3-1-arm64/develop/headers 5.8M ./build_packages/freetype_bootstrap-2.6.3-1-arm64/develop/lib 7.0M ./build_packages/freetype_bootstrap-2.6.3-1-arm64/develop 11M ./build_packages/freetype_bootstrap-2.6.3-1-arm64 1.2M ./build_packages/ncurses6_bootstrap-6.0-1-arm64/documentation/man 1.2M ./build_packages/ncurses6_bootstrap-6.0-1-arm64/documentation 2.4M ./build_packages/ncurses6_bootstrap-6.0-1-arm64/lib 1020K ./build_packages/ncurses6_bootstrap-6.0-1-arm64/bin 16K ./build_packages/ncurses6_bootstrap-6.0-1-arm64/data/tabset 6.7M ./build_packages/ncurses6_bootstrap-6.0-1-arm64/data/terminfo 6.7M ./build_packages/ncurses6_bootstrap-6.0-1-arm64/data 368K ./build_packages/ncurses6_bootstrap-6.0-1-arm64/develop/headers 2.1M ./build_packages/ncurses6_bootstrap-6.0-1-arm64/develop/lib 2.4M ./build_packages/ncurses6_bootstrap-6.0-1-arm64/develop 14M ./build_packages/ncurses6_bootstrap-6.0-1-arm64 91M ./build_packages 21G .
I will report same with newer haikuporter
comment:5 by , 3 years ago
It's the same size with a recent haikuporter, so probably always been like that. As noted in bug it is the sources:
haiku/arm64/objects/haiku/arm64/packaging/repositories$ du -hd2 425M ./HaikuPortsCross-build/packages 516K ./HaikuPortsCross-build/repository 80M ./HaikuPortsCross-build/app-shells 2.4G ./HaikuPortsCross-build/sys-devel 273M ./HaikuPortsCross-build/sys-apps 77M ./HaikuPortsCross-build/net-misc 427M ./HaikuPortsCross-build/dev-libs 43M ./HaikuPortsCross-build/media-libs 4.0K ./HaikuPortsCross-build/dev-java 1.2M ./HaikuPortsCross-build/haiku-apps 54M ./HaikuPortsCross-build/sys-libs 436M ./HaikuPortsCross-build/media-fonts 368M ./HaikuPortsCross-build/dev-lang 0 ./HaikuPortsCross-build/cross_tools 4.5G ./HaikuPortsCross-build 23M ./Haiku/packages 23M ./Haiku 2.7G ./HaikuPorts-sources-build/packages 52M ./HaikuPorts-sources-build/repository 100K ./HaikuPorts-sources-build/games-strategy 92K ./HaikuPorts-sources-build/haiku-games 140K ./HaikuPorts-sources-build/games-puzzle 100K ./HaikuPorts-sources-build/dev-embedded 12K ./HaikuPorts-sources-build/sci-physics 116K ./HaikuPorts-sources-build/games-engines 956K ./HaikuPorts-sources-build/haiku-apps 989M ./HaikuPorts-sources-build/dev-lang 184K ./HaikuPorts-sources-build/app-emulation 666M ./HaikuPorts-sources-build/media-libs 128K ./HaikuPorts-sources-build/games-action 20K ./HaikuPorts-sources-build/games-kids 240M ./HaikuPorts-sources-build/dev-util 112K ./HaikuPorts-sources-build/app-office 184K ./HaikuPorts-sources-build/app-misc 354M ./HaikuPorts-sources-build/sys-apps 32M ./HaikuPorts-sources-build/media-gfx 296K ./HaikuPorts-sources-build/games-emulation 8.0K ./HaikuPorts-sources-build/app-forensics 180K ./HaikuPorts-sources-build/x11-libs 264K ./HaikuPorts-sources-build/media-fonts 111M ./HaikuPorts-sources-build/app-text 8.9M ./HaikuPorts-sources-build/dev-python 52K ./HaikuPorts-sources-build/haiku-libs 256K ./HaikuPorts-sources-build/games-arcade 264K ./HaikuPorts-sources-build/sci-libs 24K ./HaikuPorts-sources-build/dev-perl 64K ./HaikuPorts-sources-build/sci-mathematics 6.4M ./HaikuPorts-sources-build/media-sound 24K ./HaikuPorts-sources-build/sys-firmware 75M ./HaikuPorts-sources-build/app-arch 60K ./HaikuPorts-sources-build/sys-fs 24K ./HaikuPorts-sources-build/games-roguelike 16K ./HaikuPorts-sources-build/dev-java 36K ./HaikuPorts-sources-build/app-i18n 36K ./HaikuPorts-sources-build/www-servers 816M ./HaikuPorts-sources-build/dev-libs 15M ./HaikuPorts-sources-build/app-crypt 48M ./HaikuPorts-sources-build/net-misc 4.8G ./HaikuPorts-sources-build/sys-devel 262M ./HaikuPorts-sources-build/sys-libs 60K ./HaikuPorts-sources-build/games-rpg 120K ./HaikuPorts-sources-build/app-dicts 16K ./HaikuPorts-sources-build/net-ftp 288K ./HaikuPorts-sources-build/kde-frameworks 20K ./HaikuPorts-sources-build/media-plugins 21M ./HaikuPorts-sources-build/net-dns 1.3M ./HaikuPorts-sources-build/media-video 12K ./HaikuPorts-sources-build/net-wireless 20K ./HaikuPorts-sources-build/games-sports 52M ./HaikuPorts-sources-build/app-shells 36K ./HaikuPorts-sources-build/app-cdr 464K ./HaikuPorts-sources-build/haiku-data 44K ./HaikuPorts-sources-build/www-client 84K ./HaikuPorts-sources-build/x11-proto 20K ./HaikuPorts-sources-build/net-irc 48K ./HaikuPorts-sources-build/net-im 256K ./HaikuPorts-sources-build/kde-apps 44K ./HaikuPorts-sources-build/app-benchmarks 64K ./HaikuPorts-sources-build/dev-games 112K ./HaikuPorts-sources-build/haiku-misc 8.0K ./HaikuPorts-sources-build/app-backup 98M ./HaikuPorts-sources-build/dev-vcs 12K ./HaikuPorts-sources-build/dev-rust 20K ./HaikuPorts-sources-build/sci-astronomy 44K ./HaikuPorts-sources-build/games-board 20K ./HaikuPorts-sources-build/dev-scheme 72K ./HaikuPorts-sources-build/games-fps 68K ./HaikuPorts-sources-build/dev-cpp 20K ./HaikuPorts-sources-build/app-admin 44K ./HaikuPorts-sources-build/games-simulation 16K ./HaikuPorts-sources-build/games-misc 12K ./HaikuPorts-sources-build/app-vim 20K ./HaikuPorts-sources-build/sys-process 12K ./HaikuPorts-sources-build/net-fs 12K ./HaikuPorts-sources-build/net-print 15M ./HaikuPorts-sources-build/dev-db 4.0K ./HaikuPorts-sources-build/app-mobilephone 158M ./HaikuPorts-sources-build/net-libs 4.0K ./HaikuPorts-sources-build/net-mail 31M ./HaikuPorts-sources-build/app-doc 4.0K ./HaikuPorts-sources-build/dev-lisp 4.0K ./HaikuPorts-sources-build/sci-biology 8.0K ./HaikuPorts-sources-build/sys-boot 40K ./HaikuPorts-sources-build/net-p2p 18M ./HaikuPorts-sources-build/app-editors 107M ./HaikuPorts-sources-build/app-accessibility 4.0K ./HaikuPorts-sources-build/dev-tcltk 48K ./HaikuPorts-sources-build/sci-electronics 8.0K ./HaikuPorts-sources-build/gnome-base 8.0K ./HaikuPorts-sources-build/sci-visualization 4.0K ./HaikuPorts-sources-build/dev-go 12K ./HaikuPorts-sources-build/sci-geosciences 56K ./HaikuPorts-sources-build/kde-base 24K ./HaikuPorts-sources-build/net-proxy 4.0K ./HaikuPorts-sources-build/net-firewall 44K ./HaikuPorts-sources-build/net-analyzer 4.0K ./HaikuPorts-sources-build/www-misc 28K ./HaikuPorts-sources-build/sys-block 4.0K ./HaikuPorts-sources-build/sys-power 16K ./HaikuPorts-sources-build/kde-misc 24K ./HaikuPorts-sources-build/sys-auth 4.0K ./HaikuPorts-sources-build/gnome-extra 8.0K ./HaikuPorts-sources-build/app-pda 2.2G ./HaikuPorts-sources-build/dev-qt 24K ./HaikuPorts-sources-build/dev-lua 8.0K ./HaikuPorts-sources-build/net-dialup 4.0K ./HaikuPorts-sources-build/sci-chemistry 24K ./HaikuPorts-sources-build/mail-client 24K ./HaikuPorts-sources-build/x11-misc 8.0K ./HaikuPorts-sources-build/games-util 4.0K ./HaikuPorts-sources-build/lxde-base 4.0K ./HaikuPorts-sources-build/meta-ports 8.0K ./HaikuPorts-sources-build/games-mud 4.0K ./HaikuPorts-sources-build/net-nds 8.0K ./HaikuPorts-sources-build/sys-cluster 4.0K ./HaikuPorts-sources-build/dev-dotnet 8.0K ./HaikuPorts-sources-build/net-news 4.0K ./HaikuPorts-sources-build/www-aps 8.0K ./HaikuPorts-sources-build/sci-calculators 4.0K ./HaikuPorts-sources-build/net-nntp 4.0K ./HaikuPorts-sources-build/net-vpn 4.0K ./HaikuPorts-sources-build/dev-haskell 4.0K ./HaikuPorts-sources-build/x11-doc 4.0K ./HaikuPorts-sources-build/x11-base 4.0K ./HaikuPorts-sources-build/net-voip 0 ./HaikuPorts-sources-build/cross_tools 14G ./HaikuPorts-sources-build 4.0K ./HaikuPorts-bootstrap 19G .
comment:6 by , 3 years ago
I think it worked by accident before the introduction of the "all" architecture. Only a few packages were enabled for most architectures (except the x86 ones), and all other packages would be skipped because of that. Now with the "all" architecture, all packages can be pulled in, exposing the bug.
Note:
See TracTickets
for help on using tickets.
hm. reading through the code, this may be a defect in haikuporter.
Our build process does the following:
This creates a list of packages to generate source files for...
Then that's passed to:
$(HOST_HAIKU_PORTER) --cross-devel-package "$haikuCrossDevelPackage" \ --all-dependencies $(HAIKU_PORTER_EXTRA_OPTIONS) \ --create-source-packages-for-bootstrap --portsfile $packageList