Changes between Version 16 and Version 17 of Obsolete/MovedToTree/PackageManagement/FileFormat
- Timestamp:
- May 25, 2013, 2:12:34 AM (11 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
Obsolete/MovedToTree/PackageManagement/FileFormat
v16 v17 26 26 the file, it specifies meta data of the package as a whole. 27 27 28 The TOC and Package Attributes sections aren't really separate sections, as they are stored at the end of the heap. 29 28 30 All numbers in the HPKG are stored in big endian format or [http://en.wikipedia.org/wiki/LEB128 LEB128] encoding. 29 31 … … 39 41 uint16 version; 40 42 uint64 total_size; 41 42 // package attributes section 43 uint32 attributes_compression; 44 uint32 attributes_length_compressed; 45 uint32 attributes_length_uncompressed; 43 uint16 minor_version; 44 45 uint16 heap_compression; 46 uint32 heap_chunk_size; 47 uint64 heap_size_compressed; 48 uint64 heap_size_uncompressed; 49 50 uint32 attributes_length; 46 51 uint32 attributes_strings_length; 47 52 uint32 attributes_strings_count; 48 49 // TOC section 50 uint32 toc_compression; 51 uint64 toc_length_compressed; 52 uint64 toc_length_uncompressed; 53 uint32 reserved1; 54 55 uint64 toc_length; 53 56 uint64 toc_strings_length; 54 57 uint64 toc_strings_count; … … 59 62 The string 'hpkg' (B_HPKG_MAGIC). 60 63 header_size:: 61 The size of the header. 64 The size of the header. This is also the absolute offset of the heap. 62 65 version:: 63 66 The version of the HPKG format the file conforms to. The current version is … … 65 68 total_size:: 66 69 The total file size. 67 68 attributes_compression:: 69 The compression algorithm used for the package attributes section. 70 attributes_length_compressed:: 71 The compressed size of the package attributes section. Equals 72 attributes_length_uncompressed, if the section is not compressed. 73 attributes_length_uncompressed:: 70 minor_version:: 71 The minor version of the HPKG format the file conforms to. The current minor version is 72 0 (B_HPKG_MINOR_VERSION). Additions of new attributes to the attributes or TOC sections should 73 generally only increment the minor version. When a file with a greater minor version is 74 encountered, the reader should ignore unknown attributes. 75 76 heap_compression:: 77 Compression format used for the heap. 78 heap_chunk_size:: 79 The size of the chunks the uncompressed heap data are divided into. 80 heap_size_compressed:: 81 The compressed size of the heap. This includes all administrative data (the chunk size array). 82 heap_size_uncompressed:: 83 The uncompressed size of the heap. This is only the size of the raw data (including the TOC 84 and attributes section), not including administrative data (the chunk size array). 85 86 attributes_length:: 74 87 The uncompressed size of the package attributes section. 75 88 attributes_strings_length:: … … 78 91 The number of entries in the strings subsection of the package attributes section. 79 92 80 toc_compression:: 81 The compression algorithm used for the TOC section. 82 toc_length_compressed:: 83 The compressed size of the TOC section. Equals 84 toc_length_uncompressed, if the section is not compressed. 85 toc_length_uncompressed:: 93 reserved1:: 94 Reserved for later use. 95 96 toc_length:: 86 97 The uncompressed size of the TOC section. 87 98 toc_strings_length:: … … 89 100 toc_strings_count:: 90 101 The number of entries in the strings subsection of the TOC section. 102 103 104 === Heap === 105 106 The heap provides storage for arbitrary data. Data from various sources are concatenated without padding or separator, forming the uncompressed heap. A specific section of data is usually referenced (e.g. in the TOC and attributes sections) by an offset and the number of bytes. These references always point into the uncompressed heap, even if the heap is actually stored in a compressed format. The `heap_compression` field in the header specifies which format is used. The following values are defined: 107 108 ||0||B_HPKG_COMPRESSION_NONE||no compression|| 109 ||1||B_HPKG_COMPRESSION_ZLIB||zlib (LZ77) compression|| 110 111 The uncompressed heap data are divided into equally sized chunks (64 KiB). Each individual chunk may be stored compressed or not. A uint16 array at the end of the heap contains the actual in-file (compressed) size of each chunk, save for the last one, which is omitted since it is implied. A chunk is only stored compressed, if compression actually saves space. That is if the chunk's compressed size equals its uncompressed size, the data aren't compressed. 112 113 The TOC and the package attributes sections are stored (in this order) at the end of the uncompressed heap. The offset of the package attributes section data is therefore `heap_size_uncompressed - attributes_length` and the offset of the TOC section data `heap_size_uncompressed - attributes_length - toc_length`. 91 114 92 115 … … 129 152 The strings subsections consists of a list of null-terminated UTF-8 strings. The section itself is terminated by a 0 byte. 130 153 131 Each string is implicit y assigned the (null-based) index at which theit appears in the list, i.e. the nth string has the index n - 1. The string is referenced by this index in the main TOC subsection.154 Each string is implicitly assigned the (null-based) index at which it appears in the list, i.e. the nth string has the index n - 1. The string is referenced by this index in the main TOC subsection. 132 155 133 156 ==== Main TOC ==== … … 143 166 144 167 The attribute tag encodes four pieces of information: 145 {{{(encoding << 1 0) + (hasChildren << 9) + (dataType << 6) + id + 1}}}168 {{{(encoding << 11) + (hasChildren << 10) + (dataType << 7) + id + 1}}} 146 169 147 170 encoding:: … … 173 196 174 197 ||0||B_HPKG_ATTRIBUTE_ENCODING_RAW_INLINE||unsigned LEB128: size; followed by raw bytes|| 175 ||1||B_HPKG_ATTRIBUTE_ENCODING_RAW_HEAP||unsigned LEB128: size; unsigned LEB128: offset into heap||198 ||1||B_HPKG_ATTRIBUTE_ENCODING_RAW_HEAP||unsigned LEB128: size; unsigned LEB128: offset into the uncompressed heap|| 176 199 177 200 … … 180 203 The package attributes section contains a list of attribute trees, just like 181 204 the TOC section. The structure of this section follows the TOC, i.e. there's a subsection for shared strings and a subsection that stores a list of attribute entries terminated by a 0 byte. An entry has the same format as the ones in the TOC (only using different attribute IDs). 182 183 184 === Section Compression ===185 186 The TOC and the package attributes section can be compressed. Which compression algorithm is used is specified by the {{{toc_compression}}} respectively the {{{attributes_compression}}} field in the header. The following values are defined:187 188 ||0||B_HPKG_COMPRESSION_NONE||no compression||189 ||1||B_HPKG_COMPRESSION_ZLIB||zlib (LZ77) compression||190 191 205 192 206 … … 267 281 - '''Value:''' Name of the user owning the file. 268 282 - '''Allowed Values:''' Any non-empty string. 283 - '''Default Value:''' The user owning the installation location where the package is activated. 269 284 - '''Child Attributes:''' none 270 285 … … 273 288 - '''Value:''' Name of the group owning the file. 274 289 - '''Allowed Values:''' Any non-empty string. 290 - '''Default Value:''' The group owning the installation location where the package is activated. 275 291 - '''Child Attributes:''' none 276 292 … … 331 347 - '''Type:''' data 332 348 - '''Value:''' Raw data of a file or attribute. 333 - '''Allowed Values:''' Any value, if uncompressed, otherwise see below. 334 - '''Child Attributes:''' 335 - B_HPKG_ATTRIBUTE_ID_DATA_COMPRESSION: The compression algorithm used for 336 storing the data. 337 - B_HPKG_ATTRIBUTE_ID_DATA_SIZE: The size of the uncompressed data. 338 - B_HPKG_ATTRIBUTE_ID_DATA_CHUNK_SIZE: The size of an uncompressed data chunk. 339 340 ==== B_HPKG_ATTRIBUTE_ID_DATA_COMPRESSION ("data:compression") ==== 341 - '''Type:''' uint 342 - '''Value:''' ID of the data compression algorithm. 343 - '''Allowed Values:''' 344 345 ||0||B_HPKG_COMPRESSION_NONE||no compression|| 346 ||1||B_HPKG_COMPRESSION_ZLIB||zlib (LZ77) compression|| 347 - '''Default Value:''' B_HPKG_COMPRESSION_NONE 348 - '''Child Attributes:''' none 349 350 ==== B_HPKG_ATTRIBUTE_ID_DATA_SIZE ("data:size") ==== 351 - '''Type:''' uint 352 - '''Value:''' Size of the uncompressed data. 353 - '''Allowed Values:''': Any value. 354 - '''Default Value:''' Size of the compressed data. 355 - '''Child Attributes:''' none 356 357 ==== B_HPKG_ATTRIBUTE_ID_DATA_CHUNK_SIZE ("data:chunk_size") ==== 358 - '''Type:''' uint 359 - '''Value:''' Size of an uncompressed data chunk. 360 - '''Allowed Values:''': Any value. 361 - '''Default Value:''' 362 - If not compressed: 0 363 - If B_HPKG_COMPRESSION_ZLIB compressed: 64 * 1024 364 - '''Child Attributes:''' none 349 - '''Allowed Values:''' Any value. 365 350 366 351 ==== B_HPKG_ATTRIBUTE_ID_SYMLINK_PATH ("symlink:path") ==== … … 375 360 376 361 The TOC can directly contain any number of attributes of the B_HPKG_ATTRIBUTE_ID_DIRECTORY_ENTRY type, which in turn contain descendant attributes as specified in the previous section. Any other attributes are ignored. 377 378 379 === Data Compression ===380 381 Data referred to by an B_HPKG_ATTRIBUTE_ID_DATA attribute will be the raw data, if uncompressed. If compressed, the data have a special format, that allows for fast random access.382 383 ==== B_HPKG_COMPRESSION_ZLIB ====384 385 The original data are split into equally sized chunks and compressed individually. The compressed data chunks are stored (in order) without padding, preceded by an uint64 array specifying the relative offsets of the compressed data of each chunk. The offsets are relative to the first byte following the offset array. Since the first chunk is always at offset 0, its array element is omitted. Therefore uncompressed data split into n chunks will have n - 1 offset array elements.386 387 The size of the compressed chunks is implied by the offset differences (respectively for the last chunk the difference to the total size). A compressed chunk is always shorter than the uncompressed chunk. If the chunk data couldn't be compressed, the data are stored uncompressed and the size of the stored chunk will be equal to the uncompressed chunk size.388 362 389 363 … … 494 468 ||1||B_PACKAGE_ARCHITECTURE_X86||x86, 32-bit, built with gcc4|| 495 469 ||2||B_PACKAGE_ARCHITECTURE_X86_GCC2||x86, 32-bit, built with gcc2|| 470 ||3||B_PACKAGE_ARCHITECTURE_SOURCE||source code, doesn't depend on the system architecture|| 496 471 - '''Child Attributes:''' none 497 472 … … 629 604 - '''Child Attributes:''' none 630 605 606 ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_GLOBAL_SETTINGS_FILE ("package:global-settings-file") ==== 607 - '''Type:''' string 608 - '''Value:''' Relative path of a global settings file either included in the package or created by the included software. If the settings file is included in the package, it will be installed upon activation. In this case the attribute must contain a B_HPKG_ATTRIBUTE_ID_PACKAGE_SETTINGS_FILE_UPDATE_TYPE child attribute. 609 - '''Allowed Values:''' Installation location relative path (i.e. "settings/..."). 610 - '''Child Attributes:''' 611 - B_HPKG_ATTRIBUTE_ID_PACKAGE_SETTINGS_FILE_UPDATE_TYPE: Specifies what to do with the settings file on package update. 612 613 ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_USER_SETTINGS_FILE ("package:user-settings-file") ==== 614 - '''Type:''' string 615 - '''Value:''' Relative path of a user settings file created by the included software or required by the software to be created by the user. 616 - '''Allowed Values:''' Installation location relative path (i.e. "settings/..."). 617 - '''Child Attributes:''' 618 - B_HPKG_ATTRIBUTE_ID_PACKAGE_SETTINGS_FILE_TEMPLATE: A template for the settings file. 619 620 ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_SETTINGS_FILE_UPDATE_TYPE ("package:settings-file-update-type") ==== 621 - '''Type:''' uint 622 - '''Value:''' Specifies what to do on package update when the settings file provided by the package has been changed by the user. 623 - '''Allowed Values:''' 624 625 ||0||B_SETTINGS_FILE_UPDATE_TYPE_KEEP_OLD||the old settings file shall be kept|| 626 ||1||B_SETTINGS_FILE_UPDATE_TYPE_MANUAL||the old settings file needs to be updated manually|| 627 ||2||B_SETTINGS_FILE_UPDATE_TYPE_AUTO_MERGE||an automatic three-way merge shall be attempted|| 628 - '''Child Attributes:''' none 629 630 ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_SETTINGS_FILE_TEMPLATE ("package:settings-file-template") ==== 631 - '''Type:''' string 632 - '''Value:''' Relative path of an included template file for the user settings file. 633 - '''Allowed Values:''' Installation location relative path of a file included in the package. 634 - '''Child Attributes:''' none 635 636 ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_USER ("package:user") ==== 637 - '''Type:''' string 638 - '''Value:''' Name of a user required by the package. Upon package activation the user will be created, if necessary. 639 - '''Allowed Values:''' Any valid user name, i.e. must be non-empty composed of <alphanum_underline>. 640 - '''Child Attributes:''' 641 - B_HPKG_ATTRIBUTE_ID_PACKAGE_USER_REAL_NAME: The user's real name. 642 - B_HPKG_ATTRIBUTE_ID_PACKAGE_USER_HOME: The user's home directory. 643 - B_HPKG_ATTRIBUTE_ID_PACKAGE_USER_SHELL: The user's shell. 644 - B_HPKG_ATTRIBUTE_ID_PACKAGE_USER_GROUP: The user's group(s). 645 646 ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_USER_REAL_NAME ("package:user.real-name") ==== 647 - '''Type:''' string 648 - '''Value:''' The real name of the user. 649 - '''Allowed Values:''' Any string. 650 - '''Default Value:''' The user name. 651 - '''Child Attributes:''' none 652 653 ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_USER_HOME ("package:user.home") ==== 654 - '''Type:''' string 655 - '''Value:''' The path to the home directory of the user. 656 - '''Allowed Values:''' Any valid path. 657 - '''Child Attributes:''' none 658 659 ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_USER_SHELL ("package:user.shell") ==== 660 - '''Type:''' string 661 - '''Value:''' The path to the shell to be used for the user. 662 - '''Allowed Values:''' Any valid path. 663 - '''Default Value:''' "/bin/bash". 664 - '''Child Attributes:''' none 665 666 ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_USER_GROUP ("package:user.group") ==== 667 - '''Type:''' string 668 - '''Value:''' A group the user belongs to. At least one must be specified. 669 - '''Allowed Values:''' Any valid group name, i.e. must be non-empty composed of <alphanum_underline>. 670 - '''Default Value:''' The default group for users. 671 - '''Child Attributes:''' none 672 673 ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_GROUP ("package:group") ==== 674 - '''Type:''' string 675 - '''Value:''' Name of a group required by the package. Upon package activation the group will be created, if necessary. 676 - '''Allowed Values:''' Any valid group name, i.e. must be non-empty composed of <alphanum_underline>. 677 - '''Child Attributes:''' none 678 679 ==== B_HPKG_ATTRIBUTE_ID_PACKAGE_POST_INSTALL_SCRIPT ("package:post-install-script") ==== 680 - '''Type:''' string 681 - '''Value:''' Relative path of a script that shall be executed after package activation. 682 - '''Allowed Values:''' Installation location relative path of a file included in the package. 683 - '''Child Attributes:''' none 684 685 631 686 == Haiku Package Repository Format == 632 687 … … 643 698 Header:: 644 699 Identifies the file as HPKR file and provides access to the other sections. 700 Heap:: 701 Contains the next two sections. 645 702 Repository Info:: 646 703 A section containing an archived BMessage of a BRepositoryInfo object. … … 648 705 A section just like the package attributes section of the HPKG, only that this section contains the package attributes of all the packages contained in the repository (not just one). 649 706 707 The Repository Info and Package Attributes sections aren't really separate sections, as they are stored at the end of the heap. 650 708 651 709 ==== Header ==== … … 659 717 uint16 version; 660 718 uint64 total_size; 719 uint16 minor_version; 720 721 // heap 722 uint16 heap_compression; 723 uint32 heap_chunk_size; 724 uint64 heap_size_compressed; 725 uint64 heap_size_uncompressed; 661 726 662 727 // repository info section 663 uint32 info_compression; 664 uint32 info_length_compressed; 665 uint32 info_length_uncompressed; 728 uint32 info_length; 729 uint32 reserved1; 666 730 667 731 // package attributes section 668 uint32 packages_compression; 669 uint64 packages_length_compressed; 670 uint64 packages_length_uncompressed; 732 uint64 packages_length; 671 733 uint64 packages_strings_length; 672 734 uint64 packages_strings_count; … … 677 739 The string 'hpkr' (B_HPKG_REPO_MAGIC). 678 740 header_size:: 679 The size of the header. 741 The size of the header. This is also the absolute offset of the heap. 680 742 version:: 681 743 The version of the HPKR format the file conforms to. The current version is 682 1(B_HPKG_REPO_VERSION).744 2 (B_HPKG_REPO_VERSION). 683 745 total_size:: 684 746 The total file size. 685 686 info_compression:: 687 The compression algorithm used for the repository info section. 688 info_length_compressed:: 689 The compressed size of the repository info section. Equals 690 info_length_uncompressed, if the section is not compressed. 691 info_length_uncompressed:: 747 minor_version:: 748 The minor version of the HPKR format the file conforms to. The current minor version is 749 0 (B_HPKG_REPO_MINOR_VERSION). Additions of new attributes to the attributes section should 750 generally only increment the minor version. When a file with a greater minor version is 751 encountered, the reader should ignore unknown attributes. 752 753 heap_compression:: 754 Compression format used for the heap. 755 heap_chunk_size:: 756 The size of the chunks the uncompressed heap data are divided into. 757 heap_size_compressed:: 758 The compressed size of the heap. This includes all administrative data (the chunk size array). 759 heap_size_uncompressed:: 760 The uncompressed size of the heap. This is only the size of the raw data (including the repository info 761 and attributes section), not including administrative data (the chunk size array). 762 763 info_length:: 692 764 The uncompressed size of the repository info section. 693 765 694 packages_compression:: 695 The compression algorithm used for the package attributes section. 696 packages_length_compressed:: 697 The compressed size of the package attributes section. Equals 698 attributes_length_uncompressed, if the section is not compressed. 699 packages_length_uncompressed:: 766 reserved1:: 767 Reserved for later use. 768 769 packages_length:: 700 770 The uncompressed size of the package attributes section. 701 771 packages_strings_length::
