WebPositive auto-expanding but not changing mimetype

[Pete]

There was a particular gzipped-tar file I needed to download, and used WebPositive to do so. However, Expander wouldn't handle the downloaded file. I switched to BeZilla, and got a file that unpacked correctly.

Looking at the two versions of the file, I saw that the BeZilla one was ~100kB, but the WP one was 290kB! They both had the same filename, and were "application/x-gzip". Reaching the obvious conclusion, I ran "tar -xzf..." from the terminal on the WP file and of course it worked. WP had already expanded it for me, but failed to rename it or change the mimetype!

I tried the same on a "tgz" file from my own website, and neither Bezilla nor WP had any problems. A copy renamed "...tar.gz" was just as happy.

Digging a bit further, I used "wget --save-headers..." on all the files, and found I was being served different headers in each case.

My tgz file has

Content-Type: application/x-tar

while the tar.gz one is:

Content-Type: application/x-gzip

Apache supplied neither with a "Content-Encoding:" header.

The failing file, in contrast, has:

Content-Type: application/x-tar Content-Encoding: x-gzip

FTR the file causing problems is

​http://www.fourmilab.ch/webtools/midicsv/midicsv-1.1.tar.gz

[Pete]

Sorry -- slip of the finger (and mind!)... I of course meant "tar -xf" from the terminal. No decompression required.

[ttcoder]

(saw this too)

[pulkomandy]

This will currently not happen (network backend was replaced with one that doesn't handle http-gz). However, with the content-encoding set this way, I'd say the browser is right to decompress the file. The mimetype should be set to application/x-tar. Not sure we should touch the extension, since we don't use that in the system anyway.

[pulkomandy]

Well, I'm going to close this as invalid, because the bug really is on the server side. We're served a file with type "application/x-tar", and the gzip compression is reported at the http level. What we do, unpacking the file on the fly, is expected in this case.

So, please bug the fourmilabs guys to fix their server.

[kallisti5]

side note: Midori on Linux also downloads an invalid tar.gz. Their server is indeed broken.

[Pete]

I see what you mean... Have to wonder how common this is, though, as Firefox/Bezilla seems to handle the situation. I slightly suspect they must have taken special pains to avoid it -- otherwise the "Content-Encoding: gzip" ought to tell it to gunzip the file, which it doesn't do. It gets saved still gzipped.

[pulkomandy]

I'm wondering if BeZilla supports "Content-Encoding: gzip" at all, or maybe it handles it only when showing a page and not when downloading.

[Pete]

Replying to pulkomandy:

I'm wondering if BeZilla supports "Content-Encoding: gzip" at all, or maybe it handles it only when showing a page and not when downloading.

Well, it says it supports it... A quick check with ' nc -lv -p 80' (on my Raspberry Pi, actually) shows this request string for a 'tar.gz' file:

GET /matt-1.4.tar.gz HTTP/1.1
Host: pi
User-Agent: Mozilla/5.0 (BeOS; U; BeOS BePC; en-US; rv:1.8.1.3) Gecko/20070313 BonEcho/2.0.0.3
Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip,deflate
Accept-Charset: UTF-8,*
Keep-Alive: 300
Connection: keep-alive

(That's from my BeOS machine, as it's the one I happen to be on, but both that and Haiku download the fourmilab file as an unexpanded gzip.)

[Pete]

Replying to myself Pete:

... Have to wonder how common this is, though, as Firefox/Bezilla seems to handle the situation. I slightly suspect they must have taken special pains to avoid it ...

Eh heh... For my own reasons, I wanted to grab a current rsync source, and the official repository seems to be ​http://rsync.samba.org/ftp/rsync/src/rsync-3.1.0.tar.gz. I used WP first to grab it, and once again the "tar.gz" file ended up as an actual tar file but still labelled ...gz.

Here's the response header from the site:

HTTP/1.1 200 OK
Date: Sun, 16 Feb 2014 00:02:38 GMT
Server: Apache
Last-Modified: Sun, 29 Sep 2013 03:01:29 GMT
ETag: "1126983-d7cbd-4e77cef9f5440"
Accept-Ranges: bytes
Content-Length: 883901
Content-Type: application/x-gzip
Content-Encoding: gzip
X-Pad: avoid browser bug

Once again, if I use Bezilla it gets downloaded as the stated tar.gz file. So it's not just fourmilab...