GNU bug report logs - #22266
[libparted] jump instruction and boot code is corrupted with random bytes after fat is resized

Previous Next

To reply to this bug, email your comments to 22266 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tom Yan <tom.ty89 <at> gmail.com>
To: bug-parted <at> gnu.org
Subject: [libparted] jump instruction and boot code is corrupted with random
 bytes after fat is resized
Date: Tue, 29 Dec 2015 18:08:31 +0800

https://gist.github.com/tomty89/2a1ca8698bad5ae969ad/revisions?diff=split

The (last) diff compares a FAT32 resized with a working commit of
libparted and a not working one. You can see that in the latter, the
jump instruction and boot code in the boot sectors are not preserved
but completely corrupted by random bytes. (FWIW, the first byte "eb"
is crucial for Windows to recognize the filesystem)

The last working commit I can confirm is:
http://git.savannah.gnu.org/cgit/parted.git/commit/?id=d0a4cc1b57750a92afb48b229e4791154afa322b

which is the commit right before:
http://git.savannah.gnu.org/cgit/parted.git/commit/?id=80678bdd957cf49a9ccfc8b88ba3fb8b4c63fc12

where FAT resizing no longer works, until:
http://git.savannah.gnu.org/cgit/parted.git/commit/?id=5adae27101565a5d6fed4aadf28ddb39872e41f5

which is the first commit I can confirm to have this issue.

I've also tested the latest commit:
http://git.savannah.gnu.org/cgit/parted.git/commit/?id=388bab890a4e09b09d2428c0e773ed083295f91b

which does not work properly either.

See this bug report for more details:
https://bugzilla.gnome.org/show_bug.cgi?id=759916

Message #8 received at 22266 <at> debbugs.gnu.org (full text, mbox):

From: "Brian C. Lane" <bcl <at> redhat.com>
To: Tom Yan <tom.ty89 <at> gmail.com>
Cc: 22266 <at> debbugs.gnu.org
Subject: Re: bug#22266: [libparted] jump instruction and boot code is
 corrupted with random bytes after fat is resized
Date: Mon, 4 Jan 2016 09:45:24 -0800

On Tue, Dec 29, 2015 at 06:08:31PM +0800, Tom Yan wrote:
> https://gist.github.com/tomty89/2a1ca8698bad5ae969ad/revisions?diff=split
> 
> The (last) diff compares a FAT32 resized with a working commit of
> libparted and a not working one. You can see that in the latter, the
> jump instruction and boot code in the boot sectors are not preserved
> but completely corrupted by random bytes. (FWIW, the first byte "eb"
> is crucial for Windows to recognize the filesystem)
> 
> The last working commit I can confirm is:
> http://git.savannah.gnu.org/cgit/parted.git/commit/?id=d0a4cc1b57750a92afb48b229e4791154afa322b
> 
> which is the commit right before:
> http://git.savannah.gnu.org/cgit/parted.git/commit/?id=80678bdd957cf49a9ccfc8b88ba3fb8b4c63fc12
> 
> where FAT resizing no longer works, until:
> http://git.savannah.gnu.org/cgit/parted.git/commit/?id=5adae27101565a5d6fed4aadf28ddb39872e41f5
> 
> which is the first commit I can confirm to have this issue.
> 
> I've also tested the latest commit:
> http://git.savannah.gnu.org/cgit/parted.git/commit/?id=388bab890a4e09b09d2428c0e773ed083295f91b
> 
> which does not work properly either.
> 
> See this bug report for more details:
> https://bugzilla.gnome.org/show_bug.cgi?id=759916

So you are saying that commit 5adae27101565a5d6fed4aadf28ddb39872e41f5
does *not* fix the problem?

We should drop this resize code completely. I still think it was a
mistake to revive it as a library. Filesystems should be managed by the
upstream filesystem code, not parted.

-- 
Brian C. Lane | Anaconda Team | IRC: bcl #anaconda | Port Orchard, WA (PST8PDT)

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Curtis Gedak <gedakc <at> gmail.com>
To: bcl <at> redhat.com
Cc: tom.ty89 <at> gmail.com, bug-parted <at> gnu.org, 22266 <at> debbugs.gnu.org
Subject: Re: bug#22266: [libparted] jump instruction and boot code is
 corrupted with random bytes after fat is resized
Date: Mon, 4 Jan 2016 10:56:03 -0700

Hi Brian,

I agree that in an ideal world the code for resizing file systems would
exist in the file system project.  Unfortunately this does not appear to
be the case for FAT16/FAT32 file systems.

When I last looked there were no other free software or open source
options for resizing FAT16 and FAT32 file systems.  The same goes for
the code in libparted that can shrink HFS and HFS+ file systems.

Unless there is a change in this situation, I am in favour of keeping
the resizing code/library in parted.

Regards,
Curtis Gedak
(Maintainer of GParted)

On 16-01-04 10:45 AM, Brian C. Lane wrote:
> We should drop this resize code completely. I still think it was a
> mistake to revive it as a library. Filesystems should be managed by the
> upstream filesystem code, not parted.

Message #17 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tom Yan <tom.ty89 <at> gmail.com>
To: Curtis Gedak <gedakc <at> gmail.com>
Cc: bcl <at> redhat.com, bug-parted <at> gnu.org, 22266 <at> debbugs.gnu.org
Subject: Re: bug#22266: [libparted] jump instruction and boot code is
 corrupted with random bytes after fat is resized
Date: Tue, 5 Jan 2016 02:36:48 +0800

Well what I meant was commit 80678bdd957cf49a9ccfc8b88ba3fb8b4c63fc12
makes gparted simply quit with error when trying to resize FAT, and
commit 5adae27101565a5d6fed4aadf28ddb39872e41f5 fix that but
introduced the issue I am reporting now. (Well actually, I CANNOT know
exactly which of them introduced it.)

Well yeah dosfstools does not provide any resizing tool. Funny thing
is not even Windows support FAT resizing in its disk management util
(but it support that for NTFS).

On 5 January 2016 at 01:56, Curtis Gedak <gedakc <at> gmail.com> wrote:
> Hi Brian,
>
> I agree that in an ideal world the code for resizing file systems would
> exist in the file system project.  Unfortunately this does not appear to
> be the case for FAT16/FAT32 file systems.
>
> When I last looked there were no other free software or open source
> options for resizing FAT16 and FAT32 file systems.  The same goes for
> the code in libparted that can shrink HFS and HFS+ file systems.
>
> Unless there is a change in this situation, I am in favour of keeping
> the resizing code/library in parted.
>
> Regards,
> Curtis Gedak
> (Maintainer of GParted)
>
> On 16-01-04 10:45 AM, Brian C. Lane wrote:
>> We should drop this resize code completely. I still think it was a
>> mistake to revive it as a library. Filesystems should be managed by the
>> upstream filesystem code, not parted.

Message #23 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Curtis Gedak <gedakc <at> gmail.com>
To: Tom Yan <tom.ty89 <at> gmail.com>
Cc: bcl <at> redhat.com, d-eberhdt64 <at> freenet.de, bug-parted <at> gnu.org,
 22266 <at> debbugs.gnu.org
Subject: Re: bug#22266: [libparted] jump instruction and boot code is
 corrupted with random bytes after fat is resized
Date: Sun, 10 Apr 2016 14:00:33 -0600

[Message part 1 (text/plain, inline)]

Attached is a patch to fix the issue with jump instruction and boot code
is corrupted with random bytes after fat is resized.

Thanks goes to Tom Yan, Dirk Eberhardt, and others who reported the
problem and provided additional details.  This detailed information
helped with my understanding of the issue.

The portion of this email after my signature delves further into the
problem investigation and solution.

Regards,
Curtis Gedak


PROBLEM
-------

Windows does not recognize FAT32 after resizing with utilities (for
example GParted) that use parted-3.2 libraries.


CONFIRMATION
------------

Steps to confirm are:

1) Create a single FAT32 partition (e.g. 500 MiB) on a device such as
   a USB drive.

   When I created FAT32 partition using GParted the boot sector
   started with:

   $ sudo hexdump -n 7k -C /dev/sda1
   00000000  eb 58 90 6d 6b 66 73                              |.X.mkfs|
   00000007
   $

2) Use GParted linked with parted-3.2 libraries to shrink the FAT32
   partition (e.g. 450 MiB).

   After FAT32 resizing using GParted the boot sector started with:

   $ sudo hexdump -n 7k -C /dev/sda1
   00000000  d0 02 30 4d 53 57 49                              |..0MSWI|
   00000007
   $

   The important distinction is that the first three hexadecimal
   values were randomly changed.  These values represent the "boot
   jump" code and Windows requires these values to be correctly set.

3) Try to use the FAT32 partition with Windows (e.g. insert USB drive
   in Windows computer).  The FAT32 file system is not recognized.

   Regarding boot_jump (and system_id) see:
   http://git.savannah.gnu.org/cgit/parted.git/tree/doc/FAT?id=v3.2#n176

If the libraries from parted 3.1, or versions 2.4 and earlier are
used, then the resized FAT32 partition is recognized by Windows.


INVESTIGATION
-------------

The problem with Windows FAT32 recognition was introduced with:

  Fix filesystem detection on non 512 byte sectors
  80678bdd957cf49a9ccfc8b88ba3fb8b4c63fc12

The cause of the problem is that when the parted library is resizing
FAT32, there is one too many boot_sector memory allocations.

The calling structure is as follows:

fat_resize (...)
{
  ...
  ctx = create_resize_context (...);  # Allocates new boot_sector
                                      #   memory and copies content
                                      #   from old boot sector
  ...
  fat_boot_sector_generate (...)      # Allocates new boot_sector
                                      #   memory AGAIN!
                                      #   Copied content lost!
  fat_boot_sector_write (...)
  ...
}


SOLUTION
--------

Remove new boot sector memory allocation from
fat_boot_sector_generate(...).

The function fat_boot_sector_generate(...) is also called from
fat_create(...).

  Note that access to fat_create(...) via ped_file_system_create(...)
  is no longer possible since parted-3.0 because all file system
  operations were removed.  See:
  http://git.savannah.gnu.org/cgit/parted.git/tree/NEWS?id=v3.0

  In parted-3.1, access to fat_create(...) via
  ped_file_system_create(...) was not re-introduced.  See:
  http://git.savannah.gnu.org/cgit/parted.git/tree/NEWS?id=v3.1

Although fat_create(...) in inaccessible, I thought it best to move
the new boot_sector memory allocation to where it is needed in
fat_boot_sector_set_boot_code(...).  That way the code should still
work if it is ever re-instated.


REFERENCES
----------

GNU bug report logs - #22266:
[libparted] jump instruction and boot code is corrupted with random
bytes after fat is resized
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=22266

GNU bug report logs - #22710:
libparted 3.2 fat32 bootsector incompatible w. windows
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=22710

GParted Bug Report:
Bug 759916 - fat(32) resizing generates boot sector(s) with invalid
             jump instruction and pseudo-random boot code
https://bugzilla.gnome.org/show_bug.cgi?id=759916

GParted Forum:
Vista/XP don't accept a shrinked Fat32 partition (USB-stick)
http://gparted-forum.surf4.info/viewtopic.php?id=17318

[gnubug22266-boot-code-corrupted-after-fat-resize.patch (text/x-patch, attachment)]

Message #29 received at 22266 <at> debbugs.gnu.org (full text, mbox):

From: Phillip Susi <psusi <at> ubuntu.com>
To: Curtis Gedak <gedakc <at> gmail.com>, Tom Yan <tom.ty89 <at> gmail.com>
Cc: 22266 <at> debbugs.gnu.org
Subject: Re: bug#22266: [libparted] jump instruction and boot code is
 corrupted with random bytes after fat is resized
Date: Thu, 14 Apr 2016 20:30:15 -0400

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 04/10/2016 04:00 PM, Curtis Gedak wrote:
> Attached is a patch to fix the issue with jump instruction and boot
> code is corrupted with random bytes after fat is resized.
> 
> Thanks goes to Tom Yan, Dirk Eberhardt, and others who reported
> the problem and provided additional details.  This detailed
> information helped with my understanding of the issue.
> 
> The portion of this email after my signature delves further into
> the problem investigation and solution.

Thanks Curtis; patch pushed.


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQEcBAEBCgAGBQJXEDYXAAoJEBB5UWFcu6UWrdIH/1SNDXVZvz2QFgRvhqUAZhvB
AgFbgLp8TwVd7nqIIYxsBNMryF+3izBtUHl3rPE2S/2WjGvzQmgG1FZ5an0lIh0i
j6cIHof4MZ2QViAOHYzQP9npW7qurY9Jnp7MrXTNbT+kLdANJ7xm6StAFWKS0R+m
HBBSlNgKuj83C0pbYbnQkmNLlbPLLsytiaoQFE6sNBj5LwZCN7/gCiXeIO4i9KZ+
q7Kv3LWM1TbP+5zvbnr/Gs6fHKWM+LmfrgrpeyFCKlxImGTa2OH34aKRkUcIPv17
SlYKoiN96wIdePa6tztNdUhsuUmzYh30tXKBMf9eumBz1e5WxgFJUJI6hs8hvp0=
=5Noq
-----END PGP SIGNATURE-----

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.