Opened 4 years ago

Closed 4 years ago

#11845 closed enhancement (fixed)

trim old nightlies from download.haiku-os.org

Reported by: luroh Owned by: zooey
Priority: normal Milestone: R1
Component: Sys-Admin Version: R1/Development
Keywords: Cc:
Blocked By: Blocking:
Has a Patch: no Platform: All

Description

As of 2015-02-09 there is 924 GB worth of old nightlies at download.haiku-os.org (and 374 GB of haiku-repositories).

In order to free up some space, it has been suggested that we drop old raw and vmdk nightly images, keeping only ISOs and anyboots.

A raw can be created from an anyboot by running:

dd if=haiku-anyboot.image of=haiku.image bs=1M skip=$(expr $(od -j 454 -N 4 -i -A n haiku-anyboot.image) / 2048)
dd if=/dev/zero of=haiku.image bs=1 seek=506 count=4 conv=notrunc

A vmdk can be created from a raw using VirtualBox:

VBoxManage convertfromraw haiku.image haiku.vmdk --format VMDK

...or qemu:

qemu-img convert -O vmdk haiku.image haiku.vmdk 

An algorithm was also mentioned, for possible future use: http://jekor.com/log2rotate

Change History (9)

comment:1 Changed 4 years ago by kallisti5

+1. We keep way too much cruft.

I'm actually also in favor of dropping vmdk's all together. Given we provide raw images, anyboot images, iso images, we really don't need vmdk's. It is super easy to convert raw disk images to "insert your favorite emulator here".

comment:2 in reply to:  1 Changed 4 years ago by waddlesplash

Replying to kallisti5:

I'm actually also in favor of dropping vmdk's all together. Given we provide raw images, anyboot images, iso images, we really don't need vmdk's. It is super easy to convert raw disk images to "insert your favorite emulator here".

+1 on dropping them from the nightly builds, however I'm hesitant to agree to drop them from the tree as well. Then again, one less thing to worry about, so I guess it's OK.

comment:3 Changed 4 years ago by zooey

Owner: changed from haiku-sysadmin to zooey
Status: newin-progress

comment:4 Changed 4 years ago by zooey

Disk space was getting a bit tight, so I have dropped all the vmdk images.

Additionally, I have adjusted a test version of the script that generates the overview pages for the nightlies to always keep the last 15 nightlies and then start to thin them out. The result of the current settings would be that of the 239 available nightlies (per architecture and variant), only 25 would be kept. Does this sound about right?

comment:5 Changed 4 years ago by waddlesplash

Can we just keep the Anyboots for everything, and delete everything else? Given that images are sometimes hard / impossible to replicate (at least in the case of older images) it might be a good idea to keep as many as possible...

comment:6 Changed 4 years ago by waddlesplash

Oh, also -- I need to run a few tests with the ISOs and Anyboots, but it may be possible to drop the ISOs as well. I'll get back to you on that.

comment:7 Changed 4 years ago by pulkomandy

We still haven't officially switched to anyboot as the main medium (IIRC alpha4 was officially an ISO, with anyboot "for testing purposes"). I think there are a few cases where ISO would work and anyboot wouldn't (or the reverse).

The raw and vmdk are definitely not needed as these can be rebuilt from the others. The command to build a raw looks almost simple enough that it could be done on the fly when a download is requested?

comment:8 Changed 4 years ago by haiqu

Might want to adjust the page at download.haiku-os.org to reflect this change. It still refers to vmdk images. A quick tutorial on how to convert from another image is needed for newbies.

comment:9 Changed 4 years ago by zooey

Resolution: fixed
Status: in-progressclosed

Fixed - now the last 50 nightlies are kept and then images are getting thinned out. This caused 220 from the 280 nightlies (per architecture and image variant) to get deleted, freeing a considerable amount of storage space on baron's home partition.

Note: See TracTickets for help on using tickets.