Opened 10 years ago
Closed 10 years ago
#11306 closed bug (fixed)
"package add" no longer works
Reported by: | ttcoder | Owned by: | bonefish |
---|---|---|---|
Priority: | normal | Milestone: | R1 |
Component: | Kits/Package Kit | Version: | R1/Development |
Keywords: | Cc: | ||
Blocked By: | Blocking: | ||
Platform: | All |
Description
Got this error today when packaging even though nothing changed (well something did, see below): Failed to parse package info date from package file: Bad data
on this hrev47395 install.
Digging, I found out that package
is choking on a StyledEdit attribute of the .PackageInfo file.. only now that it is combined with a Pe attribute. Indeed a few days ago I transitioned from using StyledEdit to exclusively using Pe, so I guess I'm now getting a double serving of "caret position" attributes on my textfiiles :-).
Anyway the workaround was simply to "reset" the .PackageInfo file, create a new one, free of text editor attributes. Filing a low-priority ticket in case the underlying reason matters though..
Attachments (1)
Change History (12)
comment:2 by , 10 years ago
Priority: | low → normal |
---|---|
Summary: | "package add" odd failure wit this attribute combination → "package add" no longer works |
This has gotten worse now that I've updated to a recent hrev, package add
no longer works at all, even if the .PackageInfo has no attribute at all. Here's a self-contained reproducible case, where cat
creates a pristine info file.. If this does not get fixed soon I'm going to have to rewrite my packaging framework to use the alternate syntax, not looking forward to that :-(
mkdir -p obj3 cat << EOF > obj3/.PackageInfo name proof version 1.0-1 summary "test for packaging bug" description "see above" vendor ttcoder packager ttcoder copyrights 2014 licenses MIT architecture any provides { proof } EOF touch obj3/whatever rm proof-1.0-1-any.hpkg package create -q -C ./obj3 -b proof-1.0-1-any.hpkg package add -q -C ./obj3 proof-1.0-1-any.hpkg whatever
Run the above in a newly created folder.
You'll get this:
> sh _create.sh Failed to parse package info data from package file: Bad data > uname -a Haiku shredder 1 hrev47896 Sep 22 2014 17:04:46 BePC x86 Haiku
comment:3 by , 10 years ago
The "packager" field must be of the format "Name <something@…>" unless I'm mistaken...
follow-up: 5 comment:4 by , 10 years ago
Thanks, updated the self-contained testcase. To be clear, the "Failed to parse package info data from package file" intuitively would not seem to concern the .PackageInfo file at all, and it occurs during execution of the second line (package add, rather than package create). Appreciate if that can be fixed, I have to do my builds by hand..
comment:5 by , 10 years ago
Replying to ttcoder:
Appreciate if that can be fixed, I have to do my builds by hand..
I don't think I'll get to do this anytime soon.
Anyway, the best strategy to create a package is to prepare a directory with all package content and use package create
. This also has the advantage that you can easily use mimeset
to generate the MIME DB entries to include in the package.
There's also package update
, which is semantically similar (doesn't implicitly add directories, though) and, to my knowledge, does work.
follow-up: 7 comment:6 by , 10 years ago
update
is not listed in usage.. and non-responsive if forcing its use anyway (at least in this hrev):
~/Desktop> package update my.hpkg blah Usage: package <command> <command args> Creates, inspects, or extracts a Haiku package. Commands: add [ <options> ] <package> <entries>... (.. by contrast to "add" which responds:..) ~/Desktop> package add my.hpkg blah Failed to open package file "my.hpkg": Read-only file system
Maybe it was added in the last couple weeks ?
Anyway I guess I'll move on to the 'mainstream' way of doing things, which seems cleaner in severals aspects indeed.. And this ticket will de facto become low priority, probably noone will/is missing "package add".
Appreciate your watching over this kind of tickets BTW thx
comment:7 by , 10 years ago
Replying to ttcoder:
update
is not listed in usage.. and non-responsive if forcing its use anyway (at least in this hrev):
Uh, sorry, I seem to have recalled that incorrectly. There's indeed only add
, with the -f
switch to replace existing files. Which we use in the build system for the update
build action (e.g. jam -q @release-raw update libbe.so
).
comment:8 by , 10 years ago
I've investigated this and my conclusion is as follows:
When PackageFileHeapWriter tries to unwrite the last partial chunk in PackageFileHeapWriter::_UnwriteLastPartialChunk() it uses ReadData() which in turn calls into PackageFileHeapWriter::ReadAndDecompressChunk() with the index argument set to the last, partial, chunk (0 in case of the provided test case). Since it is the last chunk, it is assumed to be pending (as it would be once the last partial chunk was unwritten), so the pending buffer (which is yet to be filled) is copied to the uncompressed buffer instead of decompressing the actual last chunk. The pending buffer ends up containing nulls instead of the data of the last partial chunk.
Since the .PackageInfo is the only content of the package at this point, it fully resides in the last partial chunk and is therefore completely lost. Since the last chunk is lost in any case, using package add
should always be corrupting the package, even when not failing to read the .PackageInfo (i.e. when there's enough additional data so that .PackageInfo doesn't reside in the last partial chunk).
I'm attaching a workaround that uses a flag to signify whether we are unwriting the last partial chunk so that PackageFileHeapWriter::ReadAndDecompressChunk() uncompresses the last chunk instead of returning the (yet unfilled) pending buffer. I can't really tell how to fix this properly.
by , 10 years ago
Attachment: | 0001-Workaround-for-corrupting-last-partial-chunk-on-upda.patch added |
---|
follow-up: 10 comment:9 by , 10 years ago
Thanks for your analysis, Michael! Your patch looks OK, but I'd rather avoid a flag to temporarily change the behavior of a method. Since _UnwriteLastPartialChunk()
is meant to circumvent the usual mechanism of reading data, I'd rather not call ReadData()
at all. For a single known chunk ReadDataToOutput()
doesn't do much more than call ReadAndDecompressChunk()
anyway, which in turn only calls ReadAndDecompressChunkData()
. So calling the latter directly shouldn't result in much code duplication, but it would be clearer and even avoid a bit of overhead.
comment:10 by , 10 years ago
Replying to bonefish:
So calling the latter directly shouldn't result in much code duplication, but it would be clearer and even avoid a bit of overhead.
Indeed, that sounds like a nice solution. If noone beats me to it, I'll come up with a new patch and push it tonight.
comment:11 by , 10 years ago
Resolution: | → fixed |
---|---|
Status: | new → closed |
Fixed in hrev47990. As per the suggestion above, _UnwriteLastPartialChunk()
now uses ReadAndDecompressChunkData()
directly. As it already has all the needed information, this was a really simple fix.
I also tested my hypothesis that this bug would always corrupt the last partial chunk of a package on update, which was indeed the case. Since the package created in the provided test case only consisted of a single chunk, it made the error more obvious. In other use cases the corruption would probably have gone unnoticed more easily.
Thanks to ttcoder for the nice test case and Ingo for the heads up on the elegant solution.
Here's a session showing the attribute contents:
And here's the duo of packaging commands I run, the second of which being the one that fails with the mentionned error:
EDIT: I think I remember that while digging, I tried to invoke the packaging in one go (no "create" followed by "add", just a "create" line) and it worked.. it only chokes when invoked as package-add. Curiouser and curiouser.