GNU bug report logs - #25663
verify macro in system.h incorrectly expects 8 byte long for 64-bit builds on Windows/Visual Studio

Previous Next

To reply to this bug, email your comments to 25663 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Kees Dekker <Kees.Dekker <at> infor.com>
To: "bug-diffutils <at> gnu.org" <bug-diffutils <at> gnu.org>
Subject: verify macro in system.h incorrectly expects 8 byte long for 64-bit
 builds on Windows/Visual Studio
Date: Thu, 9 Feb 2017 13:15:43 +0000

[Message part 1 (text/plain, inline)]

Hi,

The verify macro in system.h:137 (diffutils3.5) fails on Windows.
This line contains the following code:

verify (sizeof (lin) <= sizeof (long int));

Because of a long on Windows remains 32-bit (for Visual Studio), even in 64-bit mode, this compile time assert fails. It is only for *NIX that a long sizes identical to the bit-ness of the build, similar to pointers.
For an integer type that properly sizes with pointers, you need to use intptr_t. See also https://msdn.microsoft.com/nl-nl/library/s3f49ktz.aspx.

I'm not sure how gcc behaves on Windows, but event then intptr_t is probably the better type to use.

If Visual Studio is used, then the _MSCVER macro is set, see also https://msdn.microsoft.com/en-us/library/b0084kay.aspx.

Can you please fix this?

Regards,
Kees

[Message part 2 (text/html, inline)]

Message #8 received at 25663 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Kees Dekker <Kees.Dekker <at> infor.com>, 25663 <at> debbugs.gnu.org
Subject: Re: [bug-diffutils] bug#25663: verify macro in system.h incorrectly
 expects 8 byte long for 64-bit builds on Windows/Visual Studio
Date: Thu, 9 Feb 2017 11:20:16 -0800

On 02/09/2017 05:15 AM, Kees Dekker wrote:
>
> Can you please fix this?
>
Just to be clear, porting GNU diffutils to Visual Studio is quite low on 
our priority list. Perhaps someday somebody will contribute 
self-contained patches for that, patches that do not significantly get 
in the way of maintaining diffutils. But it's unlikely that any 
maintainer will have time to spare to actually write the code for such a 
low-priority task. In the meantime, I suggest that you use GNU/Linux or 
some other more-GNUish platform; even Microsoft ships platforms like 
that nowadays.

Message #11 received at 25663 <at> debbugs.gnu.org (full text, mbox):

From: Kees Dekker <Kees.Dekker <at> infor.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, "25663 <at> debbugs.gnu.org"
 <25663 <at> debbugs.gnu.org>
Subject: RE: [bug-diffutils] bug#25663: verify macro in system.h incorrectly
 expects 8 byte long for 64-bit builds on Windows/Visual Studio
Date: Fri, 10 Feb 2017 06:31:58 +0000

Hi Paul,

>Just to be clear, porting GNU diffutils to Visual Studio is quite low on 
>our priority list. Perhaps someday somebody will contribute 
>self-contained patches for that, patches that do not significantly get 
>in the way of maintaining diffutils. But it's unlikely that any 
>maintainer will have time to spare to actually write the code for such a 
>low-priority task. In the meantime, I suggest that you use GNU/Linux or 
>some other more-GNUish platform; even Microsoft ships platforms like 
>that nowadays.

I've no choice to move away from Windows. If I'm able to make a fix, in which format do you like to get it. UNIX style patch?

As far as I can see, it is only util.c that did not compile. The other approach available is to port diffutils to Visual Studio 2015 (we already have it working for Visual Studio 2013).
Everything else need to move to Visual Studio 2015, so diff also need to move (otherwise end-users are forced to install different versions of the visual studio runtime).

And I even think that the build issues are related to something that was just introduced with diffutils 3.5... But I will check.

Regards,
Kees

Message #14 received at 25663 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Kees Dekker <Kees.Dekker <at> infor.com>,
 "25663 <at> debbugs.gnu.org" <25663 <at> debbugs.gnu.org>
Subject: Re: [bug-diffutils] bug#25663: verify macro in system.h incorrectly
 expects 8 byte long for 64-bit builds on Windows/Visual Studio
Date: Thu, 9 Feb 2017 22:45:28 -0800

Kees Dekker wrote:

> I've no choice to move away from Windows.

You don't need to. Microsoft ships GNU diff as part of an optional MS-Windows 
subsystem. This should be a lot less work for us maintainers than trying to 
maintain a separate MS-Windows "native" port (whatever "native" means) for GNU 
diff. See:

https://msdn.microsoft.com/en-us/commandline/wsl/about

> in which format do you like to get it. UNIX style patch?

"git format-patch" format, where the commit message uses GNU ChangeLog style. 
See the existing git repository for examples of this. The changes should be 
minimal and cleanly separable from the mainline source, and using the same style 
as what we have now.

Message #17 received at 25663 <at> debbugs.gnu.org (full text, mbox):

From: Kees Dekker <Kees.Dekker <at> infor.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, "25663 <at> debbugs.gnu.org"
 <25663 <at> debbugs.gnu.org>
Subject: RE: [bug-diffutils] bug#25663: verify macro in system.h incorrectly
 expects 8 byte long for 64-bit builds on Windows/Visual Studio
Date: Fri, 10 Feb 2017 07:50:12 +0000

>You don't need to. Microsoft ships GNU diff as part of an optional MS-Windows 
>subsystem. This should be a lot less work for us maintainers than trying to 
>maintain a separate MS-Windows "native" port (whatever "native" means) for GNU 
>diff. See:

>https://msdn.microsoft.com/en-us/commandline/wsl/about

We ship our binaries to customers. Using something of Microsoft requires that is allowed from legal point of view. And second, the software need to stay in a known location. And 3rd: the used runtime should match. The mentioned example is beta software, not reliable enough for our customers. And last: I've to stay compatible, so switching to a different approach is almost impossible (unfortunately). So really: no choice.

>> in which format do you like to get it. UNIX style patch?

>"git format-patch" format, where the commit message uses GNU ChangeLog style. 
>See the existing git repository for examples of this. The changes should be 
>minimal and cleanly separable from the mainline source, and using the same style 
>as what we have now.

If you have an additional source of information, it would be great. I'm new to git, so I will try to do so.

Message #20 received at 25663 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Kees Dekker <Kees.Dekker <at> infor.com>,
 "25663 <at> debbugs.gnu.org" <25663 <at> debbugs.gnu.org>
Subject: Re: [bug-diffutils] bug#25663: verify macro in system.h incorrectly
 expects 8 byte long for 64-bit builds on Windows/Visual Studio
Date: Fri, 10 Feb 2017 00:06:33 -0800

Kees Dekker wrote:
> If you have an additional source of information, it would be great. I'm new to git, so I will try to do so.

I'm afraid it's not written down in a central location. There are books about 
Git and it's not that hard to pick up. You can look at the Emacs CONTRIBUTE file 
for some advice about commit messages.

http://git.savannah.gnu.org/cgit/emacs.git/tree/CONTRIBUTE

Message #23 received at 25663 <at> debbugs.gnu.org (full text, mbox):

From: Kees Dekker <Kees.Dekker <at> infor.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, "25663 <at> debbugs.gnu.org"
 <25663 <at> debbugs.gnu.org>
Subject: RE: [bug-diffutils] bug#25663: verify macro in system.h incorrectly
 expects 8 byte long for 64-bit builds on Windows/Visual Studio
Date: Fri, 10 Feb 2017 12:37:29 +0000

>> If you have an additional source of information, it would be great. I'm new to git, so I will try to do so.

>I'm afraid it's not written down in a central location. There are books about 
>Git and it's not that hard to pick up. You can look at the Emacs CONTRIBUTE file 
>for some advice about commit messages.

>http://git.savannah.gnu.org/cgit/emacs.git/tree/CONTRIBUTE

The quick fix is not more or less than adding:

#ifndef _WIN64
verify (sizeof (lin) <= sizeof (long int));
#endif

Line numbers beyond 2^32 are also IMO hypothetical...

I'm fine with this quick 'fix'.

Message #26 received at 25663 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Kees Dekker <Kees.Dekker <at> infor.com>,
 "25663 <at> debbugs.gnu.org" <25663 <at> debbugs.gnu.org>
Subject: Re: [bug-diffutils] bug#25663: verify macro in system.h incorrectly
 expects 8 byte long for 64-bit builds on Windows/Visual Studio
Date: Sun, 12 Feb 2017 00:45:36 -0800

Kees Dekker wrote:
> Line numbers beyond 2^32 are also IMO hypothetical...

It's 2**31, and we don't want arbitrary limits like that in GNU code. It's easy 
to generate line numbers larger than that nowadays. Here's a contrived example:

$ rm -f big
$ truncate --size 10TiB big
$ echo x >>big
$ ls -l big
-rw-r--r--. 1 eggert eggert 10995116277762 Feb 12 00:38 big
$ grep -zn x big
10995116277761:x

Although diff is not nearly as good as grep with large files (that 'grep' 
command consumes only 4 ms real time on my old office desktop), diff shouldn't 
purposely have problems in that area.

You should be able to avoid the problem by compiling with Cygwin.

Message #29 received at 25663 <at> debbugs.gnu.org (full text, mbox):

From: Kees Dekker <Kees.Dekker <at> infor.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, "25663 <at> debbugs.gnu.org"
 <25663 <at> debbugs.gnu.org>
Subject: RE: [bug-diffutils] bug#25663: verify macro in system.h incorrectly
 expects 8 byte long for 64-bit builds on Windows/Visual Studio
Date: Mon, 13 Feb 2017 06:44:14 +0000

Hi Paul,

>> Line numbers beyond 2^32 are also IMO hypothetical...

>It's 2**31, and we don't want arbitrary limits like that in GNU code. It's easy 
>to generate line numbers larger than that nowadays. Here's a contrived example:

>$ rm -f big
>$ truncate --size 10TiB big
>$ echo x >>big
>$ ls -l big
>-rw-r--r--. 1 eggert eggert 10995116277762 Feb 12 00:38 big
>$ grep -zn x big
>10995116277761:x

I made a mistake for the limits, you are right.

>Although diff is not nearly as good as grep with large files (that 'grep' 
>command consumes only 4 ms real time on my old office desktop), diff shouldn't 
>purposely have problems in that area.

>You should be able to avoid the problem by compiling with Cygwin.

As explained before, Cygwin is not an option for us. All of our other binaries are native visual studio. We cannot force our customers to install Cygwin, just to get diff...

The code will also work (both Cygwin and Windows) if you use long long. Or even better, use the standardized types (int64).

And the provided example is IMO still hypothetical. If something is find a file by using grep, there is no editor capable in opening such large files. But this is a little bit off topic for this discussion.

Message #32 received at 25663 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Kees Dekker <Kees.Dekker <at> infor.com>,
 "25663 <at> debbugs.gnu.org" <25663 <at> debbugs.gnu.org>
Subject: Re: [bug-diffutils] bug#25663: verify macro in system.h incorrectly
 expects 8 byte long for 64-bit builds on Windows/Visual Studio
Date: Mon, 13 Feb 2017 11:11:58 -0800

On 02/12/2017 10:44 PM, Kees Dekker wrote:
> there is no editor capable in opening such large files

Sure there is. Emacs can do it (I just tried it). It's quite slow, but 
it works. And even if it didn't work, core commands like 'diff' 
shouldn't arbitrarily limit themselves to small files.

No doubt 'diff' can be fixed to work even on MS-Windows platforms. 
However, arbitrarily limiting line numbers to 32 bits is not the way to 
go about it.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.