Recompiling unchanged files

[Adek336]

If you do jam -q on the haiku sources, and then you do it again (without changing any source files) it will update about 400 targets.

If I change build/jam/UserBuildConfig, for example change the target partition, jam wants to update almost 1900 targets, which probably needn't be updated.

It'd be nice if it wouldn't need to recompile so many unchanged sources.

[Adek336]

Possibly HAIKU_DONT_CLEAR_IMAGE = 1 helps here.

[Adek336]

or jam -q @profile update-all

[umccullough]

Since UserBuildConfig is included as part of the jam build system developed for Haiku - I'm guessing that changes to it will cause a rebuild of many targets regardless of whether any code actually changed. It may be that Jam just isn't smart enough to determine dependency changes when a file such as UserBuildConfig is altered and may affect all targets system-wide simply based on the timestamp change of the UserBuildConfig (can be tested by touching it I suppose).

AFAIK, HAIKU_DONT_CLEAR_IMAGE just determines if the image will be wiped with zeros or not before initializing it as BFS.

and I believe the "update-all" target just mounts an existing image (or partition) without re-inializing it as BFS, and copies all targets over the top of existing ones.

Thus - it seems neither of these are likely to fix the problem you're describing - which seems to be a deeper Jam dependency issue.

[bonefish]

Replying to Adek336:

Possibly HAIKU_DONT_CLEAR_IMAGE = 1 helps here.

Nope.

Replying to Adek336:

or jam -q @profile update-all

Nope.

Replying to umccullough:

Since UserBuildConfig is included as part of the jam build system developed for Haiku - I'm guessing that changes to it will cause a rebuild of many targets regardless of whether any code actually changed. It may be that Jam just isn't smart enough to determine dependency changes when a file such as UserBuildConfig is altered and may affect all targets system-wide simply based on the timestamp change of the UserBuildConfig (can be tested by touching it I suppose).

Actually that does not happen. None of the build system files (Jamfiles, Jamrules, rule files, UserBuildConfig) is declared as a dependency of anything that is built. So just touching one of them will have no effect. BTW, that's why occasionally one has to delete some generated file or directory, if a Jamfile has been changed in a certain way. It doesn't happen particularly often, though.

AFAIK, HAIKU_DONT_CLEAR_IMAGE just determines if the image will be wiped with zeros or not before initializing it as BFS.

Yep.

and I believe the "update-all" target just mounts an existing image (or partition) without re-inializing it as BFS, and copies all targets over the top of existing ones.

Exactly.

Thus - it seems neither of these are likely to fix the problem you're describing - which seems to be a deeper Jam dependency issue.

If targets are recompiled, there might indeed be a dependency problem. Jam saying that targets are updated, doesn't mean that something has been recompiled, though. There are a lot of pseudo targets involved in creating the shell scripts that populate the image. Those are always made, if you build an image, regardless of how many executables/libraries have to be updated.

[umccullough]

Perhaps it would be valuable to know what the change made to UserBuildConfig was in this particular situation.

For example, adding optional packages (especially the Bluetooth one for example) could easily add a multitude of targets to the build that weren't there before...

[Adek336]

I've finally found a way to reproduce the problem and it's got nothing to do with changing UserBuildConfig. The huge recompiling happens when I use another Haiku install to compile the files. The attachments have been created with:

when using Haiku which was just installed on disk-1_3
/haiku-dane/haiku> jam -q @disk-1_0 >budowanie.32.txt
<stderr output comes here>
(updates 1846 targets)
/haiku-dane/haiku> jam -q @disk-1_0 >budowanie.33.txt
....
(updates 386 targets)

here I rebooted to Haiku on disk-1_0:
/haiku-dane/haiku> jam -q @disk-1_3 >budowanie.34.txt
...
(updates 1846 targets, again!)

Also note another scenario

on Haiku on disk-1_0:
jam -q @disk-1_3
(minimal amount of targets updated)

reboot to Haiku on disk-1_1
jam -q @disk-1_3
(also minimal amount of targets updated!)

[Adek336]

Continued:

So like I said, I booted into disk-1_0, tried
/haiku-dane/haiku> jam -q @disk-1_3 >budowanie.34.txt
...
(here a message that it wants to update 1846 targets but I pressed Ctrl-C)
...updated 10 target(s)...

Then I rebooted into disk-1_3:
/haiku-dane/haiku> jam -q @disk-1_0 >budowanie.35.txt
...
...updated 423 target(s)...
(and not ~1840!)

[bonefish]

The build does not only depend on what is in the haiku tree, but also on headers of your host platform. The reason being that there are several build tools built by the build system that have to run on the host platform and thus have to include the host platform headers, therefore depending on them. Most other targets (libraries and executables at least) do in turn depend on the build tools. That explains why on your freshly installed disk-1_0 more targets need to be updated -- the build tools and pretty much all libraries and executables.

[Adek336]

Huh, I'd never figure that one out.. thanks for the info, indeed touching generated decreases the target count, that's what I'm gonna do from now on :-)

[umccullough]

Was there something still here to enhance? It seems like this may be invalid given the comments.

[Adek336]

Ideally, the headers gathered for installation could have the same timestamps (or a second newer so that Jam doesn't treat them as old targets the next time) as the source headers from which they have been copied (instead of the taking the time at which they were being copied). On the other hand, the gain is very low, the installed headers can easily be touched manually anyway.

[bonefish]

Closing. At least from the point of view of the build system and jam everything works as it should. :-) And I really don't think it's worth to invest time for optimizing this case.