trim old nightlies from download.haiku-os.org

[luroh]

As of 2015-02-09 there is 924 GB worth of old nightlies at download.haiku-os.org (and 374 GB of haiku-repositories).

In order to free up some space, it has been suggested that we drop old raw and vmdk nightly images, keeping only ISOs and anyboots.

A raw can be created from an anyboot by running:

dd if=haiku-anyboot.image of=haiku.image bs=1M skip=$(expr $(od -j 454 -N 4 -i -A n haiku-anyboot.image) / 2048)
dd if=/dev/zero of=haiku.image bs=1 seek=506 count=4 conv=notrunc

A vmdk can be created from a raw using VirtualBox:

VBoxManage convertfromraw haiku.image haiku.vmdk --format VMDK

...or qemu:

qemu-img convert -O vmdk haiku.image haiku.vmdk 

An algorithm was also mentioned, for possible future use: ​http://jekor.com/log2rotate

[kallisti5]

+1. We keep way too much cruft.

I'm actually also in favor of dropping vmdk's all together. Given we provide raw images, anyboot images, iso images, we really don't need vmdk's. It is super easy to convert raw disk images to "insert your favorite emulator here".

[waddlesplash]

Replying to kallisti5:

I'm actually also in favor of dropping vmdk's all together. Given we provide raw images, anyboot images, iso images, we really don't need vmdk's. It is super easy to convert raw disk images to "insert your favorite emulator here".

+1 on dropping them from the nightly builds, however I'm hesitant to agree to drop them from the tree as well. Then again, one less thing to worry about, so I guess it's OK.

[zooey]

Disk space was getting a bit tight, so I have dropped all the vmdk images.

Additionally, I have adjusted a test version of the script that generates the overview pages for the nightlies to always keep the last 15 nightlies and then start to thin them out. The result of the current settings would be that of the 239 available nightlies (per architecture and variant), only 25 would be kept. Does this sound about right?

[waddlesplash]

Can we just keep the Anyboots for everything, and delete everything else? Given that images are sometimes hard / impossible to replicate (at least in the case of older images) it might be a good idea to keep as many as possible...

[waddlesplash]

Oh, also -- I need to run a few tests with the ISOs and Anyboots, but it may be possible to drop the ISOs as well. I'll get back to you on that.

[pulkomandy]

We still haven't officially switched to anyboot as the main medium (IIRC alpha4 was officially an ISO, with anyboot "for testing purposes"). I think there are a few cases where ISO would work and anyboot wouldn't (or the reverse).

The raw and vmdk are definitely not needed as these can be rebuilt from the others. The command to build a raw looks almost simple enough that it could be done on the fly when a download is requested?

[haiqu]

Might want to adjust the page at download.haiku-os.org to reflect this change. It still refers to vmdk images. A quick tutorial on how to convert from another image is needed for newbies.

[zooey]

Fixed - now the last 50 nightlies are kept and then images are getting thinned out. This caused 220 from the 280 nightlies (per architecture and image variant) to get deleted, freeing a considerable amount of storage space on baron's home partition.