Haiku Files - Binary Diffs

[tonestone57]

Haiku Files should offer binary diffs (using xdelta) for Haiku ISOs ( and if it works with RAW & VMDK, then these too ) based off Alpha1 ISO/RAW/VMDK with instructions to patch.

Reasons for binary diffs are: 1) Lot smaller file size ( binary patch file ), 2) Reduce bandwidth consumption

I concluded Haiku could do this and offer binary diffs from my testing. Here is what I discovered. I only tested ISOs ( binary diffs of zip & bzip2 do not work ) but RAW & VMDK may work too I believe ( untested ).

Reference image should be: Haiku Alpha1 ( diffs based off this version ). Also, all nightly post-Alpha1 images should be built with same set of optional packages included in Alpha1 to create smaller diffs (ie: same type of packages ).

I ran my initial tests with Firefox because it was smaller. I downloaded Firefox 3.5.2 & 3.5.3. Unzipped & created ISOs ( using genisoimage ).

The two diff programs I tested with were: 1) bsdiff ( really small patches, uses lots & lots of RAM, takes very long time ), 2) xdelta ( larger patch files, uses very little RAM, very fast )

NOTE: bsdiff does not work with Alpha ISOs because it requires massive amounts of RAM. xdelta worked great & what I suggest to use.

I was able to create binary diffs for Firefox with both programs & patch the original ISO to match the latest version. I checked by using "cmp" command. I will mention that bsdiff used about 150 MB RAM & took a few minutes to do Firefox patch with a resulting 531 KB patch file. xdelta was fast & used little RAM and produced larger 2.9 MB patch file. For comparison, the ISOs were 28 MB in size.

For Alpha ISOs. I luckily happened to have a Sept 9, Pre-Alpha & the Alpha ISOs. bsdiff failed because it did not have enough RAM ( 2 GB physical + 16 GB swap ). xdelta worked great and created a 2.3 MB patch for a 398 MB Alpha1 ISO. I then created a new version with the xdelta patch & used cmp to test against Alpha ISO. Both were binary identical ( no differences ) telling me it worked successfully.

NOTES2: 1) binary diffs appear to be compressed because have close to same size after using zip on them 2) I use Alpha image for reference because everyone will have this but a different Haiku image could be reference image. 3) there are other diff programs which I did not test but I believe xdelta is the best choice because the images are huge ( to create very small diffs takes lots of RAM & CPU time ). The table on page 3 here is interesting to look over for comparison & further testing ( most programs are under 1MB in uncompressed size so better to look at average compression for each or the larger sized ones ): ​http://www.daemonology.net/papers/bsdiff.pdf

[nicholas]

Courgette diffs are even smaller. Might be of use to Haiku.

​http://dev.chromium.org/developers/design-documents/software-updates-courgette

[pulkomandy]

With the package manager allowing system updates there is little use in doing this anymore.

[diver]

Actually, binary diffing for packages was one of the ideas. See PackageManagement/OldIdeas#BrainstormingResultsfromBeGeistert