GNU bug report logs - #8870
Package installer corrupts image & zip files

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 8870 in the body.
You can then email your comments to 8870 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jambunathan K <kjambunathan <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Package installer corrupts image & zip files
Date: Wed, 15 Jun 2011 10:32:07 -0400

[ Use M-x report-emacs-bug rather than a [BUG] tag in the subject line.  ]

Would it be possible to address the following bug before my
OpenDocumentText exporter lands in elpa? I will also file a formal bug
report in few hours.

Summary:

I distribute an image file (png) and a OpenOffice extension (oxt) file
as part of the bundle. These files are getting mangled by the package
installer.

Steps for reproduction:

1. Update package-archives to point to my archive

--8<---------------cut here---------------start------------->8---
(setq package-archives
      '(("gnu" . "http://elpa.gnu.org/packages/")
	("org-odt" . "http://repo.or.cz/w/org-mode/org-jambu.git/blob_plain/HEAD:/packages/")))
--8<---------------cut here---------------end--------------->8---

2. Install the org-odt package.

3. Visit the following files:
   ~/.emacs.d/elpa/org-odt-20110613/contrib/odt/files/org-mode-unicorn.png
   ~/.emacs.d/elpa/org-odt-20110613/contrib/odt/BasicODConverter-0.8.0.oxt

   Note that the above two files are corrupt. (oxt file is a zip archive)

4. Visit these files in the downloaded tar. Note that they are sane.

Jambunathan K.

Message #8 received at 8870 <at> debbugs.gnu.org (full text, mbox):

From: Jambunathan K <kjambunathan <at> gmail.com>
To: 8870 <at> debbugs.gnu.org
Subject: Additional Info
Date: Wed, 15 Jun 2011 23:31:25 +0530

Many Thanks to whoever filed this bug report on behalf of me.

,---- Emacs Version
| GNU Emacs 24.0.50.1 (i386-mingw-nt5.1.2600) of 2011-06-13 on 3249CTO
`----

,---- tar Version
| tar (GNU tar) 1.23
| Packaged by Cygwin (1.23-1)
| Copyright (C) 2010 Free Software Foundation, Inc.
| License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>.
| This is free software: you are free to change and redistribute it.
| There is NO WARRANTY, to the extent permitted by law.
| 
| Written by John Gilmore and Jay Fenlason.
`----

Message #11 received at 8870 <at> debbugs.gnu.org (full text, mbox):

From: Chong Yidong <cyd <at> stupidchicken.com>
To: Jambunathan K <kjambunathan <at> gmail.com>
Cc: 8870 <at> debbugs.gnu.org
Subject: Re: bug#8870: Package installer corrupts image & zip files
Date: Sat, 18 Jun 2011 17:56:08 -0400

Jambunathan K <kjambunathan <at> gmail.com> writes:

> I distribute an image file (png) and a OpenOffice extension (oxt) file
> as part of the bundle. These files are getting mangled by the package
> installer.

I can't reproduce this, either with the org-odt package at your URL, nor
with a simple multi-file package containing a gif and a png file.

It's possible this is some kind of Windows problem related to tar mode.
Can you trying visiting the tar file in Emacs, and see if the image is
untarred correctly?

C-x C-f org-odt-XXXX.tar
C-s .png RET
RET

Is the image displayed correctly here?

Message #14 received at 8870 <at> debbugs.gnu.org (full text, mbox):

From: Jambunathan K <kjambunathan <at> gmail.com>
To: Chong Yidong <cyd <at> stupidchicken.com>
Cc: 8870 <at> debbugs.gnu.org
Subject: Re: bug#8870: Package installer corrupts image & zip files
Date: Sun, 19 Jun 2011 10:17:52 +0530

Chong Yidong <cyd <at> stupidchicken.com> writes:

> Jambunathan K <kjambunathan <at> gmail.com> writes:
>
>> I distribute an image file (png) and a OpenOffice extension (oxt) file
>> as part of the bundle. These files are getting mangled by the package
>> installer.
>
> I can't reproduce this, either with the org-odt package at your URL, nor
> with a simple multi-file package containing a gif and a png file.
>
> It's possible this is some kind of Windows problem related to tar mode.
> Can you trying visiting the tar file in Emacs, and see if the image is
> untarred correctly?
>
> C-x C-f org-odt-XXXX.tar
> C-s .png RET
> RET
>
> Is the image displayed correctly here?

The above sequence (which uses tar-extract) DTRT.


1 Hint for the final fix 
-------------------------

tar-untar-buffer needs to use the right coding system for
extraction. 

For example, the following "stupid change" in `tar-untar-buffer' fixes
the original bug.


  (let ((coding-system-for-write 'no-conversion)) 
    (write-region start end name))


2 Observation 
--------------
1. Visiting the png file embedded within the original tar file is OK
2. Visiting the png file in the
   ~/.emacs.d/elpa/org-odt-YYYYMMDD/contrib/odt/files/org-mode-unicorn.png
   is NOT OK.


3 Analysis 
-----------

3.1 Behaviour of tar-untar-buffer 
==================================

1. `package-install-file' uses `tar-untar-buffer' for extraction.
2. `tar-untar-buffer' invokes `write-region' and an immediate query
   for `last-coding-system-used' reports "raw-text-dos" for the png
   file.

3.2 Behaviour of tar-extract 
=============================

1. `tar-extract' uses `decode-coding-region' with coding set to
   `no-conversion' for the png file. The coding is decided by a prior
   call to `set-auto-coding'.

   I would assume that a value of `no-conversion' is picked from the
   following entry in `auto-coding-alist'.


  ("\\.\\(jpe?g\\|png\\|gif\\|tiff?\\|p[bpgn]m\\)\\'" . no-conversion)


Jambunathan K.

Message #17 received at 8870 <at> debbugs.gnu.org (full text, mbox):

From: Chong Yidong <cyd <at> stupidchicken.com>
To: Jambunathan K <kjambunathan <at> gmail.com>
Cc: 8870 <at> debbugs.gnu.org
Subject: Re: bug#8870: Package installer corrupts image & zip files
Date: Sun, 26 Jun 2011 17:01:49 -0400

Jambunathan K <kjambunathan <at> gmail.com> writes:

> tar-untar-buffer needs to use the right coding system for
> extraction.

Could you verify that a standalone `M-x tar-untar-buffer' (called
outside of package.el) also exhibits the bug?

If so, I think setting coding-system-for-write to no-conversion is
probably the correct fix, indeed.

Message #20 received at 8870 <at> debbugs.gnu.org (full text, mbox):

From: Jambunathan K <kjambunathan <at> gmail.com>
To: Chong Yidong <cyd <at> stupidchicken.com>
Cc: 8870 <at> debbugs.gnu.org
Subject: Re: bug#8870: Package installer corrupts image & zip files
Date: Tue, 28 Jun 2011 00:33:55 +0530

Chong Yidong <cyd <at> stupidchicken.com> writes:

> Jambunathan K <kjambunathan <at> gmail.com> writes:
>
>> tar-untar-buffer needs to use the right coding system for
>> extraction.
>
> Could you verify that a standalone `M-x tar-untar-buffer' (called
> outside of package.el) also exhibits the bug?

Yes, the standalone `M-x tar-untar-buffer' ALSO corrupts the zip and png
files.

> If so, I think setting coding-system-for-write to no-conversion is
> probably the correct fix, indeed.

Could you please apply this fix then?

Message #23 received at 8870 <at> debbugs.gnu.org (full text, mbox):

From: Chong Yidong <cyd <at> stupidchicken.com>
To: Jambunathan K <kjambunathan <at> gmail.com>
Cc: 8870 <at> debbugs.gnu.org
Subject: Re: bug#8870: Package installer corrupts image & zip files
Date: Mon, 27 Jun 2011 17:12:44 -0400

Jambunathan K <kjambunathan <at> gmail.com> writes:

> Chong Yidong <cyd <at> stupidchicken.com> writes:
>
>> Jambunathan K <kjambunathan <at> gmail.com> writes:
>>
>>> tar-untar-buffer needs to use the right coding system for
>>> extraction.
>>
>> Could you verify that a standalone `M-x tar-untar-buffer' (called
>> outside of package.el) also exhibits the bug?
>
> Yes, the standalone `M-x tar-untar-buffer' ALSO corrupts the zip and png
> files.

Thanks for checking.  I've committed the change.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.