GNU bug report logs - #60039
Update GDAL and NetCDF and include lz4 and openjpeg support

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 60039 in the body.
You can then email your comments to 60039 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Roman Scherer <roman.scherer <at> burningswell.com>
To: guix-patches <at> gnu.org
Subject: Update GDAL and NetCDF and include lz4 and openjpeg support
Date: Tue, 13 Dec 2022 19:00:32 +0100

[Message part 1 (text/plain, inline)]

Hello Guix,

this patch series updates the GDAL, Netcdf and libtiff packages. It also
adds support for lz4 and openjpeg to GDAL.

Could you please review it?

Thanks, Roman.

[0001-gnu-gdal-Update-to-3.6.0.patch (text/x-diff, attachment)]

[0002-gnu-gdal-Add-support-for-lz4-and-openjpeg.patch (text/x-diff, attachment)]

[0003-gnu-libtiff-Update-to-4.4.0.patch (text/x-diff, attachment)]

[0004-gnu-netcdf-Update-to-4.9.0.patch (text/x-diff, attachment)]

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 60039 <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: Roman Scherer <roman.scherer <at> burningswell.com>
Cc: 60039 <at> debbugs.gnu.org
Subject: Re: [bug#60039] Update GDAL and NetCDF and include lz4 and openjpeg
 support
Date: Tue, 20 Dec 2022 17:07:01 +0000

[Message part 1 (text/plain, inline)]

Roman Scherer <roman.scherer <at> burningswell.com> skribis:

> Hello Guix,
>
> this patch series updates the GDAL, Netcdf and libtiff packages. It also
> adds support for lz4 and openjpeg to GDAL.
>
> Could you please review it?
>
> Thanks, Roman.
>
> [2. text/x-diff; 0001-gnu-gdal-Update-to-3.6.0.patch]...

Hi,

Instead of this patch 1, I pushed the patch from issue 60159 that updates
gdal to 3.6.1.


> [3. text/x-diff; 0002-gnu-gdal-Add-support-for-lz4-and-openjpeg.patch]...

Patch 2 pushed as 3c6f7b53cea7ea5dc8176fec02271bc3770d7fc1.


> [4. text/x-diff; 0003-gnu-libtiff-Update-to-4.4.0.patch]...

As libtiff as over 9000 dependents, it has to be updated on the
core-updates branch instead of master (see [1]). However, libtiff is
already at version 4.4.0 on core-updates, so patch 3 is not necessary
and libtiff 4.4.0 will end up in master when core-updates gets merged.

[1] https://guix.gnu.org/manual/devel/en/guix.html#Submitting-Patches


> [5. text/x-diff; 0004-gnu-netcdf-Update-to-4.9.0.patch]...

When adding or removing a patch file to "gnu/packages/patches/...", the
'dist_patch_DATA' variable in "gnu/packages/local.mk" has to be updated
to track the necessary patch files.
Moreover, as this patch fixes only one line, it is also possible to add
a custom phase with a '(substitute* ...)' form in the 'arguments' field
instead of adding a patch file.
Could you send an updated patch 4?
Thanks.

Also, many tests are disabled in 'configure-flags'. Are they not working
at all?

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 60039 <at> debbugs.gnu.org (full text, mbox):

From: Roman Scherer <roman.scherer <at> burningswell.com>
To: Guillaume Le Vaillant <glv <at> posteo.net>
Cc: 60039 <at> debbugs.gnu.org
Subject: Re: [bug#60039] Update GDAL and NetCDF and include lz4 and openjpeg
 support
Date: Tue, 20 Dec 2022 20:10:23 +0100

[Message part 1 (text/plain, inline)]

Hi Guillaume,

here's the updated patch for Netcdf. I checked the tests again, I
believe I left some of them off by accident after trying a couple of
things. So, I enabled some of them again. They pass. However, I left the
remote and large file tests still turned off.

I believe the remote tests we do not want anyway on Guix CICD
system. Is that correct? And the large file tests seem to take ages. Can
we ignore them, because they are super annoying?

Thanks for the review.

Wdyt?

Roman.

[0005-gnu-netcdf-Update-to-4.9.0.patch (text/x-diff, attachment)]

[Message part 3 (text/plain, inline)]

Guillaume Le Vaillant <glv <at> posteo.net> writes:

> [[PGP Signed Part:Undecided]]
> Roman Scherer <roman.scherer <at> burningswell.com> skribis:
>
>> Hello Guix,
>>
>> this patch series updates the GDAL, Netcdf and libtiff packages. It also
>> adds support for lz4 and openjpeg to GDAL.
>>
>> Could you please review it?
>>
>> Thanks, Roman.
>>
>> [2. text/x-diff; 0001-gnu-gdal-Update-to-3.6.0.patch]...
>
> Hi,
>
> Instead of this patch 1, I pushed the patch from issue 60159 that updates
> gdal to 3.6.1.
>
>
>> [3. text/x-diff; 0002-gnu-gdal-Add-support-for-lz4-and-openjpeg.patch]...
>
> Patch 2 pushed as 3c6f7b53cea7ea5dc8176fec02271bc3770d7fc1.
>
>
>> [4. text/x-diff; 0003-gnu-libtiff-Update-to-4.4.0.patch]...
>
> As libtiff as over 9000 dependents, it has to be updated on the
> core-updates branch instead of master (see [1]). However, libtiff is
> already at version 4.4.0 on core-updates, so patch 3 is not necessary
> and libtiff 4.4.0 will end up in master when core-updates gets merged.
>
> [1] https://guix.gnu.org/manual/devel/en/guix.html#Submitting-Patches
>
>
>> [5. text/x-diff; 0004-gnu-netcdf-Update-to-4.9.0.patch]...
>
> When adding or removing a patch file to "gnu/packages/patches/...", the
> 'dist_patch_DATA' variable in "gnu/packages/local.mk" has to be updated
> to track the necessary patch files.
> Moreover, as this patch fixes only one line, it is also possible to add
> a custom phase with a '(substitute* ...)' form in the 'arguments' field
> instead of adding a patch file.
> Could you send an updated patch 4?
> Thanks.
>
> Also, many tests are disabled in 'configure-flags'. Are they not working
> at all?
>
> [[End of PGP Signed Part]]

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 60039 <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: Roman Scherer <roman.scherer <at> burningswell.com>
Cc: 60039 <at> debbugs.gnu.org
Subject: Re: [bug#60039] Update GDAL and NetCDF and include lz4 and openjpeg
 support
Date: Thu, 22 Dec 2022 10:41:15 +0000

[Message part 1 (text/plain, inline)]

Roman Scherer <roman.scherer <at> burningswell.com> skribis:

> Hi Guillaume,
>
> here's the updated patch for Netcdf. I checked the tests again, I
> believe I left some of them off by accident after trying a couple of
> things. So, I enabled some of them again. They pass. However, I left the
> remote and large file tests still turned off.
>
> I believe the remote tests we do not want anyway on Guix CICD
> system. Is that correct? And the large file tests seem to take ages. Can
> we ignore them, because they are super annoying?
>
> Thanks for the review.
>
> Wdyt?
>
> Roman.
>
> [2. text/x-diff; 0005-gnu-netcdf-Update-to-4.9.0.patch]...

Hi,

The build environment doesn't have network access, so indeed remote
tests have to be disabled.

The netcdf-parallel-openmpi package fails to build with the updated
netcdf. I think it's because the package definition for
netcdf-parallel-openmpi inherits from the package definition for netcdf,
so it has to be updated to take into consideration the build-system
change of netcdf.

There are also some dependents that fail to build (cdo, python-h5netcdf,
python-meshio, qgis). I saw some error messages about conflicting
versions of hdf5, probably because in your patch netcdf uses hdf5-1.12
and the dependents also have hdf5 (v1.10) in their dependency graph in
some way. Maybe this could be fixed by using hdf5 instead of hdf5-1.12
for netcdf...

Could take a look?

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 60039 <at> debbugs.gnu.org (full text, mbox):

From: Roman Scherer <roman.scherer <at> burningswell.com>
To: Guillaume Le Vaillant <glv <at> posteo.net>
Cc: 60039 <at> debbugs.gnu.org
Subject: Re: [bug#60039] Update GDAL and NetCDF and include lz4 and openjpeg
 support
Date: Thu, 22 Dec 2022 20:07:43 +0100

[Message part 1 (text/plain, inline)]

Hi Guillaume,

sorry about that. Here is another patch. The failed dependencies you
mentioned are working now. Another reason why netcdf-parallel-openmpi
failed was that I changed the build system previously to cmake. I went
back to use the original gnu build system. I changed it to cmake
initially because I saw that in Arch Linux's PKGBUILD and I remember
having had some problems initially. I guess it was related to the tests
I now patched.

While at the topic. Do we prefer any build system over the other in
general in Guix, like cmake vs gnu. Does one have more features than the
other (I heard something about cross compilation)?

And another question. How did you find the failing dependencies in the
first place? Did you build all the dependencies of the netcdf packages
with --sources=all?

Thanks for the help.

Roman

[0006-gnu-netcdf-Update-to-4.9.0.patch (text/x-diff, attachment)]

[Message part 3 (text/plain, inline)]

Guillaume Le Vaillant <glv <at> posteo.net> writes:

> [[PGP Signed Part:Undecided]]
> Roman Scherer <roman.scherer <at> burningswell.com> skribis:
>
>> Hi Guillaume,
>>
>> here's the updated patch for Netcdf. I checked the tests again, I
>> believe I left some of them off by accident after trying a couple of
>> things. So, I enabled some of them again. They pass. However, I left the
>> remote and large file tests still turned off.
>>
>> I believe the remote tests we do not want anyway on Guix CICD
>> system. Is that correct? And the large file tests seem to take ages. Can
>> we ignore them, because they are super annoying?
>>
>> Thanks for the review.
>>
>> Wdyt?
>>
>> Roman.
>>
>> [2. text/x-diff; 0005-gnu-netcdf-Update-to-4.9.0.patch]...
>
> Hi,
>
> The build environment doesn't have network access, so indeed remote
> tests have to be disabled.
>
> The netcdf-parallel-openmpi package fails to build with the updated
> netcdf. I think it's because the package definition for
> netcdf-parallel-openmpi inherits from the package definition for netcdf,
> so it has to be updated to take into consideration the build-system
> change of netcdf.
>
> There are also some dependents that fail to build (cdo, python-h5netcdf,
> python-meshio, qgis). I saw some error messages about conflicting
> versions of hdf5, probably because in your patch netcdf uses hdf5-1.12
> and the dependents also have hdf5 (v1.10) in their dependency graph in
> some way. Maybe this could be fixed by using hdf5 instead of hdf5-1.12
> for netcdf...
>
> Could take a look?
>
> [[End of PGP Signed Part]]

[signature.asc (application/pgp-signature, inline)]

Message #22 received at 60039-done <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: Roman Scherer <roman.scherer <at> burningswell.com>
Cc: 60039-done <at> debbugs.gnu.org
Subject: Re: [bug#60039] Update GDAL and NetCDF and include lz4 and openjpeg
 support
Date: Fri, 23 Dec 2022 10:58:44 +0000

[Message part 1 (text/plain, inline)]

Roman Scherer <roman.scherer <at> burningswell.com> skribis:

> Hi Guillaume,
>
> sorry about that. Here is another patch. The failed dependencies you
> mentioned are working now. Another reason why netcdf-parallel-openmpi
> failed was that I changed the build system previously to cmake. I went
> back to use the original gnu build system. I changed it to cmake
> initially because I saw that in Arch Linux's PKGBUILD and I remember
> having had some problems initially. I guess it was related to the tests
> I now patched.
>
> While at the topic. Do we prefer any build system over the other in
> general in Guix, like cmake vs gnu. Does one have more features than the
> other (I heard something about cross compilation)?

Usually the best build system is the one that upstream developers use.
If several equivalent build systems can be used, it depends what works
best or what is easier in Guix...


> And another question. How did you find the failing dependencies in the
> first place? Did you build all the dependencies of the netcdf packages
> with --sources=all?

You can find the dependents of a package with:

--8<---------------cut here---------------start------------->8---
./pre-inst-env guix refresh -l <package-name>
--8<---------------cut here---------------end--------------->8---

So you can rebuild all the dependents of a package with something like:

--8<---------------cut here---------------start------------->8---
./pre-inst-env guix build $(./pre-inst-env guix refresh -l <package-name> | cut -d ':' -f 2)
--8<---------------cut here---------------end--------------->8---


Patch pushed as 66188398c446bdf9ce044fa539536e9b54c28c60 with a complete
commit message.
Thanks.

[signature.asc (application/pgp-signature, inline)]

Message #25 received at 60039-done <at> debbugs.gnu.org (full text, mbox):

From: Roman Scherer <roman <at> burningswell.com>
To: Guillaume Le Vaillant <glv <at> posteo.net>
Cc: 60039-done <at> debbugs.gnu.org
Subject: Re: [bug#60039] Update GDAL and NetCDF and include lz4 and openjpeg
 support
Date: Fri, 23 Dec 2022 12:13:33 +0100

[Message part 1 (text/plain, inline)]

Thank you!

On Fri, Dec 23, 2022, 12:11 Guillaume Le Vaillant <glv <at> posteo.net> wrote:

> Roman Scherer <roman.scherer <at> burningswell.com> skribis:
>
> > Hi Guillaume,
> >
> > sorry about that. Here is another patch. The failed dependencies you
> > mentioned are working now. Another reason why netcdf-parallel-openmpi
> > failed was that I changed the build system previously to cmake. I went
> > back to use the original gnu build system. I changed it to cmake
> > initially because I saw that in Arch Linux's PKGBUILD and I remember
> > having had some problems initially. I guess it was related to the tests
> > I now patched.
> >
> > While at the topic. Do we prefer any build system over the other in
> > general in Guix, like cmake vs gnu. Does one have more features than the
> > other (I heard something about cross compilation)?
>
> Usually the best build system is the one that upstream developers use.
> If several equivalent build systems can be used, it depends what works
> best or what is easier in Guix...
>
>
> > And another question. How did you find the failing dependencies in the
> > first place? Did you build all the dependencies of the netcdf packages
> > with --sources=all?
>
> You can find the dependents of a package with:
>
> --8<---------------cut here---------------start------------->8---
> ./pre-inst-env guix refresh -l <package-name>
> --8<---------------cut here---------------end--------------->8---
>
> So you can rebuild all the dependents of a package with something like:
>
> --8<---------------cut here---------------start------------->8---
> ./pre-inst-env guix build $(./pre-inst-env guix refresh -l <package-name>
> | cut -d ':' -f 2)
> --8<---------------cut here---------------end--------------->8---
>
>
> Patch pushed as 66188398c446bdf9ce044fa539536e9b54c28c60 with a complete
> commit message.
> Thanks.
>

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.