From unknown Sun Jun 15 08:49:57 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#47882] [PATCH] Makefile: Reimplement download-po target Resent-From: Julien Lepiller Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sun, 18 Apr 2021 22:11:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 47882 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 47882@debbugs.gnu.org X-Debbugs-Original-To: guix-patches@gnu.org Received: via spool by submit@debbugs.gnu.org id=B.161878381720316 (code B ref -1); Sun, 18 Apr 2021 22:11:02 +0000 Received: (at submit) by debbugs.gnu.org; 18 Apr 2021 22:10:17 +0000 Received: from localhost ([127.0.0.1]:48005 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lYFcL-0005HU-4h for submit@debbugs.gnu.org; Sun, 18 Apr 2021 18:10:17 -0400 Received: from lists.gnu.org ([209.51.188.17]:42208) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lYFcJ-0005HN-W2 for submit@debbugs.gnu.org; Sun, 18 Apr 2021 18:10:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35516) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lYFcJ-0002iB-RY for guix-patches@gnu.org; Sun, 18 Apr 2021 18:10:11 -0400 Received: from lepiller.eu ([2a00:5884:8208::1]:52124) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lYFcG-0007GR-DC for guix-patches@gnu.org; Sun, 18 Apr 2021 18:10:10 -0400 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id 4ffceb64 for ; Sun, 18 Apr 2021 22:10:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date:from :to:subject:message-id:mime-version:content-type; s=dkim; bh=1HO dbailmMdwe29HPXqzU/LghhW34+gCwdzDdjVSjBM=; b=ObHtJKSx5/B8kLpx/Hm dkImKm1yK7CybjjA5hjZjMgwvM8oSQM7+/xDpHN9fK1jD876vXvQ1WC7m5nCjuqw g9O6q1DX6AWtUoTnp8oeEjZQZGqiHPkIE8PZNiRLcsEBhXVpv63666HNX4OG9wZl Q3AlZCI6LKK96zTGQd938zAgqdQ15AH4IlqJdH3ihZ/BAol83znfrSWOFXJm4zEh M7+GGGKsIXcwKQU9uSarSvWep3gJsnCcKJ3vkrDkjHiB8wp6LczvEnTJU/UFYmY6 Zi82gMNoUCa8ivGpsNAlVggJs7ETkf30Z9LrdhZCSqffxcBRPdJQSai10+l24JxW C+w== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id 562f5a69 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO) for ; Sun, 18 Apr 2021 22:10:03 +0000 (UTC) Date: Mon, 19 Apr 2021 00:09:58 +0200 From: Julien Lepiller Message-ID: <20210419000958.49968055@tachikoma.lepiller.eu> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="MP_/736Y_gLr=BoJc5_tZy53o=O" Received-SPF: pass client-ip=2a00:5884:8208::1; envelope-from=julien@lepiller.eu; helo=lepiller.eu X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) --MP_/736Y_gLr=BoJc5_tZy53o=O Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Hi Guix! The attached patch reimplements the download-po target in Makefile.am. The previous implementation was based on the implementation for the TP and would use the API of Weblate to get files. However, the rate-limit is very close to the number of files we need to download, which was a big limitation, and this target would only download existing files, ignoring new translations. With this, new translations and existing ones are added/updated. The target will first do a shallow clone of the repository behind Weblate. Then, it will check that files are not empty (contain at least one translated string) and correct (according to scheme-format at least). If they are correct, it will normalize them and copy them to the po/ subdir in the repository. Here is an example output: copied po/doc/guix-manual.ru.po. /tmp/tmp.dWvtATfWQA/translations/po/doc/guix-manual.si.po:6: AVERTISSEMENT=C2=A0: Le champ d'en-t=C3=AAte =C2=AB PO-Revision-Date =C2=BB= a encore sa valeur initiale par d=C3=A9faut WARN: po/doc/guix-manual.si.po (0 translated messages) was not added/updated. /tmp/tmp.dWvtATfWQA/translations/po/guix/nl.po:5135: Les sp=C3=A9cifications de format dans =C2=AB msgid_plural =C2=BB et =C2=AB=C2=A0msgstr[1]=C2=A0=C2= =BB ne sont pas =C3=A9quivalentes msgfmt: 1 erreur fatale trouv=C3=A9e WARN: po/guix/nl.po (1001 translated messages) was not added/updated. In this output, we can see that the Russian manual was copied, the Sinhala manual was empty, so not copied, and the Dutch translation had an issue in one string that needs to be fixed, and was not copied. I chose not to copy the file because an issue in msgfmt would break guix pull. Unfortunately, this is not enough testing, as the manual and cookbook are not checked. Checking them requires building the manual, so before commiting changes, please run "make". To document the new process: make download-po for any new file in po/guix and po/packages, add the language in a single line in po/guix/LINGUAS or po/packages/LINGUAS. for any new file in po/doc, add the file name to po/doc/local.mk, in DOC_PO_FILES or DOC_COOKBOOK_PO_FILES depending on the translation type. Add the texi file name to doc/local.mk too, in info_TEXINFOS. --MP_/736Y_gLr=BoJc5_tZy53o=O Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0001-Makefile-Reimplement-download-po-target.patch >From 5c14506d6af24b5307e03604cabf8ca10af56067 Mon Sep 17 00:00:00 2001 From: Julien Lepiller Date: Sun, 18 Apr 2021 23:56:48 +0200 Subject: [PATCH] Makefile: Reimplement `download-po` target. The weblate API rate limit is very close to the number of files we need to download. The previous implementation did not add new translations. * Makefile.am (download-po): Update target. (make-download-po, make-check-po): Remove functions. --- Makefile.am | 87 ++++++++++++++--------------------------------------- 1 file changed, 22 insertions(+), 65 deletions(-) diff --git a/Makefile.am b/Makefile.am index 17ad236655..811e47dbfc 100644 --- a/Makefile.am +++ b/Makefile.am @@ -969,71 +969,28 @@ cuirass-jobs: $(GOBJECTS) # Downloading up-to-date PO files. -# make-download-po-rule DOMAIN DIRECTORY [FILE-NAME-PREFIX] -define make-download-po-rule - -download-po.$(1): - if [ -f "$(top_srcdir)/$(2)/LINGUAS" ]; then \ - LINGUAS="`grep -v '^[[:blank:]]*#' < $(top_srcdir)/$(2)/LINGUAS`" ; \ - else \ - LINGUAS="`(cd $(top_srcdir)/$(2); \ - for i in *.po; do echo $$$$i; done) | cut -d . -f 2`" ; \ - fi ; \ - for lang in $$$$LINGUAS; do \ - if wget -nv -O "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp" \ - "https://translate.fedoraproject.org/api/translations/guix/$(1)/$$$$lang/file/" ; \ - then \ - msgfilter --no-wrap -i "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp" \ - cat > "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp2" ; \ - rm "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp" ; \ - mv "$(top_srcdir)/$(2)/$(3)$$$$lang.po"{.tmp2,} ; \ - else \ - rm "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp" ; \ - fi ; \ - done - -.PHONY: download-po.$(1) - -endef - -# Checking po files for issues. This is useful to run after downloading new -# po files. - -# make-check-po-rule DOMAIN DIRECTORY [FILE-NAME-PREFIX] -define make-check-po-rule - -check-po.$(1): - if [ -f "$(top_srcdir)/$(2)/LINGUAS" ]; then \ - LINGUAS="`grep -v '^[[:blank:]]*#' < $(top_srcdir)/$(2)/LINGUAS`" ; \ - else \ - LINGUAS="`(cd $(top_srcdir)/$(2); \ - for i in *.po; do echo $$$$i; done) | cut -d . -f 2`" ; \ - fi ; \ - for lang in $$$$LINGUAS; do \ - if [ -f "$(top_srcdir)/$(2)/$(3)$$$$lang.po" ]; \ - then \ - if ! msgfmt -c "$(top_srcdir)/$(2)/$(3)$$$$lang.po" ; \ - then \ - exit 1 ; \ - fi ; \ - fi ; \ - done - -.PHONY: check-po.$(1) - -endef - -$(eval $(call make-download-po-rule,documentation-cookbook,po/doc,guix-cookbook.)) -$(eval $(call make-download-po-rule,documentation-manual,po/doc,guix-manual.)) -$(eval $(call make-download-po-rule,guix,po/guix)) -$(eval $(call make-download-po-rule,packages,po/packages)) - -$(eval $(call make-check-po-rule,documentation-cookbook,po/doc,guix-cookbook.)) -$(eval $(call make-check-po-rule,documentation-manual,po/doc,guix-manual.)) -$(eval $(call make-check-po-rule,guix,po/guix)) -$(eval $(call make-check-po-rule,packages,po/packages)) - -download-po: $(foreach domain,guix packages documentation-manual documentation-cookbook,download-po.$(domain)) +WEBLATE_REPO="https://framagit.org/tyreunom/guix-translations" + +# shallow clone the git repository behind weblate and copy files from it if +# they contain at least one translation, and they are well-formed (scheme format +# only), warn otherwise. Copied files are converted to a canonical form. +download-po: + dir=$$(mktemp -d); \ + git clone --depth 1 "$(WEBLATE_REPO)" "$$dir/translations"; \ + for domain in po/doc po/guix po/packages; do \ + for po in "$$dir/translations/$$domain"/*.po; do \ + translated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 2>&1 | cut -f1 -d' '); \ + target=$$(basename "$$po"); \ + target="$$domain/$$target"; \ + if msgfmt -c "$$po" && [ "$$translated" != "0" ]; then \ + msgfilter --no-wrap -i "$$po" cat > "$$target"; \ + echo "copied $$target."; \ + else \ + echo "WARN: $$target ($$translated translated messages) was not added/updated."; \ + fi; \ + done; \ + done; \ + rm -rf "$$dir" .PHONY: download-po check-po: $(foreach domain,guix packages documentation-manual documentation-cookbook,check-po.$(domain)) -- 2.31.1 --MP_/736Y_gLr=BoJc5_tZy53o=O-- From unknown Sun Jun 15 08:49:57 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#47882] [PATCH] Makefile: Reimplement download-po target Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Fri, 30 Apr 2021 19:58:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 47882 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Julien Lepiller Cc: 47882@debbugs.gnu.org Received: via spool by 47882-submit@debbugs.gnu.org id=B47882.16198126362885 (code B ref 47882); Fri, 30 Apr 2021 19:58:02 +0000 Received: (at 47882) by debbugs.gnu.org; 30 Apr 2021 19:57:16 +0000 Received: from localhost ([127.0.0.1]:59879 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lcZGG-0000kS-DG for submit@debbugs.gnu.org; Fri, 30 Apr 2021 15:57:16 -0400 Received: from eggs.gnu.org ([209.51.188.92]:37110) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lcZGE-0000kE-Qo for 47882@debbugs.gnu.org; Fri, 30 Apr 2021 15:57:15 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:55779) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcZG8-00051t-N9; Fri, 30 Apr 2021 15:57:08 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=41214 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lcZG8-0005S8-6F; Fri, 30 Apr 2021 15:57:08 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20210419000958.49968055@tachikoma.lepiller.eu> Date: Fri, 30 Apr 2021 21:57:05 +0200 In-Reply-To: <20210419000958.49968055@tachikoma.lepiller.eu> (Julien Lepiller's message of "Mon, 19 Apr 2021 00:09:58 +0200") Message-ID: <87im43a6lq.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Hi! Julien Lepiller skribis: > The attached patch reimplements the download-po target in Makefile.am. > The previous implementation was based on the implementation for the TP > and would use the API of Weblate to get files. > > However, the rate-limit is very close to the number of files we need to > download, which was a big limitation, and this target would only > download existing files, ignoring new translations. OK. > To document the new process: > > make download-po > for any new file in po/guix and po/packages, add the language in a > single line in po/guix/LINGUAS or po/packages/LINGUAS. > for any new file in po/doc, add the file name to po/doc/local.mk, in > DOC_PO_FILES or DOC_COOKBOOK_PO_FILES depending on the translation > type. Add the texi file name to doc/local.mk too, in info_TEXINFOS. Alright. This looks like a nice improvement! Some nitpicking follows, but it LGTM overall. >>>From 5c14506d6af24b5307e03604cabf8ca10af56067 Mon Sep 17 00:00:00 2001 > From: Julien Lepiller > Date: Sun, 18 Apr 2021 23:56:48 +0200 > Subject: [PATCH] Makefile: Reimplement `download-po` target. > > The weblate API rate limit is very close to the number of files we need > to download. The previous implementation did not add new translations. > > * Makefile.am (download-po): Update target. > (make-download-po, make-check-po): Remove functions. Should be: (make-download-po-rule, make-check-po-rule) > - rm "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp" ; \ > - mv "$(top_srcdir)/$(2)/$(3)$$$$lang.po"{.tmp2,} ; \ That was kinda crazy, I=E2=80=99m glad you divided the number of dollars by two. :-) > +WEBLATE_REPO=3D"https://framagit.org/tyreunom/guix-translations" Since it=E2=80=99s not shell, rather: WEBLATE_REPO =3D https://framagit.org/tyreunom/guix-translations BTW, I vaguely remember we asked for a Savannah repo but I=E2=80=99m not su= re what the outcome was; did it eventually stall for some obscure reason? > +# shallow clone the git repository behind weblate and copy files from it= if > +# they contain at least one translation, and they are well-formed (schem= e format > +# only), warn otherwise. Copied files are converted to a canonical form. Please capitalize sentences and proper names. > +download-po: > + dir=3D$$(mktemp -d); \ > + git clone --depth 1 "$(WEBLATE_REPO)" "$$dir/translations"; \ > + for domain in po/doc po/guix po/packages; do \ > + for po in "$$dir/translations/$$domain"/*.po; do \ > + translated=3D$$(LANG=3Den_US.UTF-8 msgfmt --statistics "$$po" 2>&1 | = cut -f1 -d' '); \ > + target=3D$$(basename "$$po"); \ > + target=3D"$$domain/$$target"; \ > + if msgfmt -c "$$po" && [ "$$translated" !=3D "0" ]; then \ > + msgfilter --no-wrap -i "$$po" cat > "$$target"; \ Maybe write to $$target.tmp and then do =E2=80=9Cmv $$target.tmp $$target= =E2=80=9D so it=E2=80=99s atomic and we don=E2=80=99t end up with corrupt files if =E2= =80=98msgfilter=E2=80=99 fails. I would use spaces rather than tabs to indent the =E2=80=98for=E2=80=99 loo= ps. Thanks! Ludo=E2=80=99. From unknown Sun Jun 15 08:49:57 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: Julien Lepiller Subject: bug#47882: closed (Re: bug#47882: [PATCH] Makefile: Reimplement download-po target) Message-ID: References: <20210501164148.79449a60@tachikoma.lepiller.eu> <20210419000958.49968055@tachikoma.lepiller.eu> X-Gnu-PR-Message: they-closed 47882 X-Gnu-PR-Package: guix-patches X-Gnu-PR-Keywords: patch Reply-To: 47882@debbugs.gnu.org Date: Sat, 01 May 2021 14:43:02 +0000 Content-Type: multipart/mixed; boundary="----------=_1619880182-24453-1" This is a multi-part message in MIME format... ------------=_1619880182-24453-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #47882: [PATCH] Makefile: Reimplement download-po target which was filed against the guix-patches package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 47882@debbugs.gnu.org. --=20 47882: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D47882 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1619880182-24453-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 47882-done) by debbugs.gnu.org; 1 May 2021 14:42:16 +0000 Received: from localhost ([127.0.0.1]:36655 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lcqox-0006M7-Vx for submit@debbugs.gnu.org; Sat, 01 May 2021 10:42:16 -0400 Received: from lepiller.eu ([89.234.186.109]:35912) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lcqow-0006M1-AB for 47882-done@debbugs.gnu.org; Sat, 01 May 2021 10:42:15 -0400 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id 7e62c9da; Sat, 1 May 2021 14:42:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date:from :to:cc:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; s=dkim; bh=huhFN9kW8pBR oqJw3+zlQCUm1nQ8wL6yFddd8JcgwA0=; b=kKGC4FxaoXSRV7hAOa6HLwvfRhMD z49b7G8Ap2ZdvJstCrw4OS4+5x9E/BkJDElVx4zdBlIWEgilYhiTgEaGoHqPamAJ zomHveU08QMzfwemG1WthV7rB2GO3n2B2kiVDa4oJs3uQ4dH2/rzap5vv4ltqfev deYBLslBXAyoMqe6D8h0m5/0wxp2lPfMbIPhi3hP1ig+Sj0rR4iIO32rJAxYSwn2 rt8YdzQ3SgVTIvhIvWo56Q0pXqrhRgjBlMkdLiL2QjLp85p6RzNvB1n3DDJkOOLf T3KoA3e59i4GYnsOprVmqRgPDEpX6qljdi6Vosn6kgKr/dYNlROBNqDnFA== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id 4d45b873 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Sat, 1 May 2021 14:42:11 +0000 (UTC) Date: Sat, 1 May 2021 16:42:01 +0200 From: Julien Lepiller To: Ludovic =?UTF-8?B?Q291cnTDqHM=?= Subject: Re: bug#47882: [PATCH] Makefile: Reimplement download-po target Message-ID: <20210501164148.79449a60@tachikoma.lepiller.eu> In-Reply-To: <87im43a6lq.fsf@gnu.org> References: <20210419000958.49968055@tachikoma.lepiller.eu> <87im43a6lq.fsf@gnu.org> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 47882-done Cc: 47882-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Le Fri, 30 Apr 2021 21:57:05 +0200, Ludovic Court=C3=A8s a =C3=A9crit : > Hi! >=20 > Alright. This looks like a nice improvement! Some nitpicking > follows, but it LGTM overall. Thank you! I fixed your comments and pushed to master as 45549e9f31e462d0d1519fd678dd782da6ed5cb9. Thank you! > BTW, I vaguely remember we asked for a Savannah repo but I=E2=80=99m not = sure > what the outcome was; did it eventually stall for some obscure reason? Yeah, there were still some issues related to copyright. To be honest I lost my energy to push on this front... ------------=_1619880182-24453-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 18 Apr 2021 22:10:17 +0000 Received: from localhost ([127.0.0.1]:48005 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lYFcL-0005HU-4h for submit@debbugs.gnu.org; Sun, 18 Apr 2021 18:10:17 -0400 Received: from lists.gnu.org ([209.51.188.17]:42208) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lYFcJ-0005HN-W2 for submit@debbugs.gnu.org; Sun, 18 Apr 2021 18:10:12 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35516) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lYFcJ-0002iB-RY for guix-patches@gnu.org; Sun, 18 Apr 2021 18:10:11 -0400 Received: from lepiller.eu ([2a00:5884:8208::1]:52124) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1lYFcG-0007GR-DC for guix-patches@gnu.org; Sun, 18 Apr 2021 18:10:10 -0400 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id 4ffceb64 for ; Sun, 18 Apr 2021 22:10:03 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date:from :to:subject:message-id:mime-version:content-type; s=dkim; bh=1HO dbailmMdwe29HPXqzU/LghhW34+gCwdzDdjVSjBM=; b=ObHtJKSx5/B8kLpx/Hm dkImKm1yK7CybjjA5hjZjMgwvM8oSQM7+/xDpHN9fK1jD876vXvQ1WC7m5nCjuqw g9O6q1DX6AWtUoTnp8oeEjZQZGqiHPkIE8PZNiRLcsEBhXVpv63666HNX4OG9wZl Q3AlZCI6LKK96zTGQd938zAgqdQ15AH4IlqJdH3ihZ/BAol83znfrSWOFXJm4zEh M7+GGGKsIXcwKQU9uSarSvWep3gJsnCcKJ3vkrDkjHiB8wp6LczvEnTJU/UFYmY6 Zi82gMNoUCa8ivGpsNAlVggJs7ETkf30Z9LrdhZCSqffxcBRPdJQSai10+l24JxW C+w== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id 562f5a69 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO) for ; Sun, 18 Apr 2021 22:10:03 +0000 (UTC) Date: Mon, 19 Apr 2021 00:09:58 +0200 From: Julien Lepiller To: guix-patches@gnu.org Subject: [PATCH] Makefile: Reimplement download-po target Message-ID: <20210419000958.49968055@tachikoma.lepiller.eu> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="MP_/736Y_gLr=BoJc5_tZy53o=O" Received-SPF: pass client-ip=2a00:5884:8208::1; envelope-from=julien@lepiller.eu; helo=lepiller.eu X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) --MP_/736Y_gLr=BoJc5_tZy53o=O Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Hi Guix! The attached patch reimplements the download-po target in Makefile.am. The previous implementation was based on the implementation for the TP and would use the API of Weblate to get files. However, the rate-limit is very close to the number of files we need to download, which was a big limitation, and this target would only download existing files, ignoring new translations. With this, new translations and existing ones are added/updated. The target will first do a shallow clone of the repository behind Weblate. Then, it will check that files are not empty (contain at least one translated string) and correct (according to scheme-format at least). If they are correct, it will normalize them and copy them to the po/ subdir in the repository. Here is an example output: copied po/doc/guix-manual.ru.po. /tmp/tmp.dWvtATfWQA/translations/po/doc/guix-manual.si.po:6: AVERTISSEMENT=C2=A0: Le champ d'en-t=C3=AAte =C2=AB PO-Revision-Date =C2=BB= a encore sa valeur initiale par d=C3=A9faut WARN: po/doc/guix-manual.si.po (0 translated messages) was not added/updated. /tmp/tmp.dWvtATfWQA/translations/po/guix/nl.po:5135: Les sp=C3=A9cifications de format dans =C2=AB msgid_plural =C2=BB et =C2=AB=C2=A0msgstr[1]=C2=A0=C2= =BB ne sont pas =C3=A9quivalentes msgfmt: 1 erreur fatale trouv=C3=A9e WARN: po/guix/nl.po (1001 translated messages) was not added/updated. In this output, we can see that the Russian manual was copied, the Sinhala manual was empty, so not copied, and the Dutch translation had an issue in one string that needs to be fixed, and was not copied. I chose not to copy the file because an issue in msgfmt would break guix pull. Unfortunately, this is not enough testing, as the manual and cookbook are not checked. Checking them requires building the manual, so before commiting changes, please run "make". To document the new process: make download-po for any new file in po/guix and po/packages, add the language in a single line in po/guix/LINGUAS or po/packages/LINGUAS. for any new file in po/doc, add the file name to po/doc/local.mk, in DOC_PO_FILES or DOC_COOKBOOK_PO_FILES depending on the translation type. Add the texi file name to doc/local.mk too, in info_TEXINFOS. --MP_/736Y_gLr=BoJc5_tZy53o=O Content-Type: text/x-patch Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename=0001-Makefile-Reimplement-download-po-target.patch >From 5c14506d6af24b5307e03604cabf8ca10af56067 Mon Sep 17 00:00:00 2001 From: Julien Lepiller Date: Sun, 18 Apr 2021 23:56:48 +0200 Subject: [PATCH] Makefile: Reimplement `download-po` target. The weblate API rate limit is very close to the number of files we need to download. The previous implementation did not add new translations. * Makefile.am (download-po): Update target. (make-download-po, make-check-po): Remove functions. --- Makefile.am | 87 ++++++++++++++--------------------------------------- 1 file changed, 22 insertions(+), 65 deletions(-) diff --git a/Makefile.am b/Makefile.am index 17ad236655..811e47dbfc 100644 --- a/Makefile.am +++ b/Makefile.am @@ -969,71 +969,28 @@ cuirass-jobs: $(GOBJECTS) # Downloading up-to-date PO files. -# make-download-po-rule DOMAIN DIRECTORY [FILE-NAME-PREFIX] -define make-download-po-rule - -download-po.$(1): - if [ -f "$(top_srcdir)/$(2)/LINGUAS" ]; then \ - LINGUAS="`grep -v '^[[:blank:]]*#' < $(top_srcdir)/$(2)/LINGUAS`" ; \ - else \ - LINGUAS="`(cd $(top_srcdir)/$(2); \ - for i in *.po; do echo $$$$i; done) | cut -d . -f 2`" ; \ - fi ; \ - for lang in $$$$LINGUAS; do \ - if wget -nv -O "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp" \ - "https://translate.fedoraproject.org/api/translations/guix/$(1)/$$$$lang/file/" ; \ - then \ - msgfilter --no-wrap -i "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp" \ - cat > "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp2" ; \ - rm "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp" ; \ - mv "$(top_srcdir)/$(2)/$(3)$$$$lang.po"{.tmp2,} ; \ - else \ - rm "$(top_srcdir)/$(2)/$(3)$$$$lang.po.tmp" ; \ - fi ; \ - done - -.PHONY: download-po.$(1) - -endef - -# Checking po files for issues. This is useful to run after downloading new -# po files. - -# make-check-po-rule DOMAIN DIRECTORY [FILE-NAME-PREFIX] -define make-check-po-rule - -check-po.$(1): - if [ -f "$(top_srcdir)/$(2)/LINGUAS" ]; then \ - LINGUAS="`grep -v '^[[:blank:]]*#' < $(top_srcdir)/$(2)/LINGUAS`" ; \ - else \ - LINGUAS="`(cd $(top_srcdir)/$(2); \ - for i in *.po; do echo $$$$i; done) | cut -d . -f 2`" ; \ - fi ; \ - for lang in $$$$LINGUAS; do \ - if [ -f "$(top_srcdir)/$(2)/$(3)$$$$lang.po" ]; \ - then \ - if ! msgfmt -c "$(top_srcdir)/$(2)/$(3)$$$$lang.po" ; \ - then \ - exit 1 ; \ - fi ; \ - fi ; \ - done - -.PHONY: check-po.$(1) - -endef - -$(eval $(call make-download-po-rule,documentation-cookbook,po/doc,guix-cookbook.)) -$(eval $(call make-download-po-rule,documentation-manual,po/doc,guix-manual.)) -$(eval $(call make-download-po-rule,guix,po/guix)) -$(eval $(call make-download-po-rule,packages,po/packages)) - -$(eval $(call make-check-po-rule,documentation-cookbook,po/doc,guix-cookbook.)) -$(eval $(call make-check-po-rule,documentation-manual,po/doc,guix-manual.)) -$(eval $(call make-check-po-rule,guix,po/guix)) -$(eval $(call make-check-po-rule,packages,po/packages)) - -download-po: $(foreach domain,guix packages documentation-manual documentation-cookbook,download-po.$(domain)) +WEBLATE_REPO="https://framagit.org/tyreunom/guix-translations" + +# shallow clone the git repository behind weblate and copy files from it if +# they contain at least one translation, and they are well-formed (scheme format +# only), warn otherwise. Copied files are converted to a canonical form. +download-po: + dir=$$(mktemp -d); \ + git clone --depth 1 "$(WEBLATE_REPO)" "$$dir/translations"; \ + for domain in po/doc po/guix po/packages; do \ + for po in "$$dir/translations/$$domain"/*.po; do \ + translated=$$(LANG=en_US.UTF-8 msgfmt --statistics "$$po" 2>&1 | cut -f1 -d' '); \ + target=$$(basename "$$po"); \ + target="$$domain/$$target"; \ + if msgfmt -c "$$po" && [ "$$translated" != "0" ]; then \ + msgfilter --no-wrap -i "$$po" cat > "$$target"; \ + echo "copied $$target."; \ + else \ + echo "WARN: $$target ($$translated translated messages) was not added/updated."; \ + fi; \ + done; \ + done; \ + rm -rf "$$dir" .PHONY: download-po check-po: $(foreach domain,guix packages documentation-manual documentation-cookbook,check-po.$(domain)) -- 2.31.1 --MP_/736Y_gLr=BoJc5_tZy53o=O-- ------------=_1619880182-24453-1-- From unknown Sun Jun 15 08:49:57 2025 X-Loop: help-debbugs@gnu.org Subject: [bug#47882] [PATCH] Makefile: Reimplement download-po target Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= Original-Sender: "Debbugs-submit" Resent-CC: guix-patches@gnu.org Resent-Date: Sat, 01 May 2021 15:09:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 47882 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Julien Lepiller Cc: 47882-done@debbugs.gnu.org Received: via spool by 47882-done@debbugs.gnu.org id=D47882.161988171625499 (code D ref 47882); Sat, 01 May 2021 15:09:02 +0000 Received: (at 47882-done) by debbugs.gnu.org; 1 May 2021 15:08:36 +0000 Received: from localhost ([127.0.0.1]:36813 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lcrER-0006dD-Rn for submit@debbugs.gnu.org; Sat, 01 May 2021 11:08:36 -0400 Received: from eggs.gnu.org ([209.51.188.92]:44892) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1lcrEP-0006d7-BV for 47882-done@debbugs.gnu.org; Sat, 01 May 2021 11:08:35 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:44743) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1lcrEJ-0007MM-45; Sat, 01 May 2021 11:08:27 -0400 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=43904 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from ) id 1lcrEI-00016Q-JA; Sat, 01 May 2021 11:08:26 -0400 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= References: <20210419000958.49968055@tachikoma.lepiller.eu> <87im43a6lq.fsf@gnu.org> <20210501164148.79449a60@tachikoma.lepiller.eu> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 12 =?UTF-8?Q?Flor=C3=A9al?= an 229 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sat, 01 May 2021 17:08:24 +0200 In-Reply-To: <20210501164148.79449a60@tachikoma.lepiller.eu> (Julien Lepiller's message of "Sat, 1 May 2021 16:42:01 +0200") Message-ID: <87h7jm7aqf.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Julien Lepiller skribis: > Le Fri, 30 Apr 2021 21:57:05 +0200, [...] >> Alright. This looks like a nice improvement! Some nitpicking >> follows, but it LGTM overall. > > Thank you! I fixed your comments and pushed to master as > 45549e9f31e462d0d1519fd678dd782da6ed5cb9. Thank you! Cool! >> BTW, I vaguely remember we asked for a Savannah repo but I=E2=80=99m not= sure >> what the outcome was; did it eventually stall for some obscure reason? > > Yeah, there were still some issues related to copyright. To be honest I > lost my energy to push on this front... Yeah, I sympathize. It=E2=80=99s fine and more productive to use the repo = you set up. Thank you! Ludo=E2=80=99.