From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 04 19:28:30 2023 Received: (at submit) by debbugs.gnu.org; 4 Jun 2023 23:28:31 +0000 Received: from localhost ([127.0.0.1]:47562 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q5x9C-00021q-1M for submit@debbugs.gnu.org; Sun, 04 Jun 2023 19:28:30 -0400 Received: from lists.gnu.org ([209.51.188.17]:32822) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q5x99-00021g-Oz for submit@debbugs.gnu.org; Sun, 04 Jun 2023 19:28:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1q5x99-0004Ip-IN for guix-patches@gnu.org; Sun, 04 Jun 2023 19:28:27 -0400 Received: from mail-oo1-xc2b.google.com ([2607:f8b0:4864:20::c2b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1q5x96-0006PR-Sa for guix-patches@gnu.org; Sun, 04 Jun 2023 19:28:27 -0400 Received: by mail-oo1-xc2b.google.com with SMTP id 006d021491bc7-5585f2f070bso3076947eaf.0 for ; Sun, 04 Jun 2023 16:28:24 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685921303; x=1688513303; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:from:to:cc:subject:date:message-id:reply-to; bh=Ci1kZdGW4TGNATteMQhOyS2wu5DfwZ+MaxA/Kj5zC2A=; b=OnlnhbVlILBbDZUAdch9/Vi0URsYwMYzqlSyAxFol+N+ENoXP98cN1STA/6wUH23W3 8M66bJeAFc+VHZSKpeYrLc/LJbFyPLJ7IoYneR2aXAxz8uG0MeM4DafzwfAryoNjSuHi NVwc3tkt5bMPlLIzqZszgwC52f14jQepIygSZFUOXJz8xjCVEdOmNrterrPrgTmWbNkZ u7MO2cMcbDqrLpmxVAWnPtleNXeqJd6UnRUdjJXeywVYald6M1DpMJTd9LTKNeUpd2yz eqQW3xvtCCdkS7Q3zuIBC3gzf7bV7A9PdehOFvd3hoikV3hLgDTvyluRJSipK3H0mkHP J34Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685921303; x=1688513303; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=Ci1kZdGW4TGNATteMQhOyS2wu5DfwZ+MaxA/Kj5zC2A=; b=OO5qNCVLb3PVbjyx603IkhFNlPcpUOo0MBqPiK+FJ299+EiES1lEB7Rp72ohgyUpjh FH19+9/ny+5w5GaCTlFvqSAhFSeq+bii0QWdl1IZORJkzP3lcqIoNHBgd4BxsNKNhlZo bo/dNLjA7kcPDLeLrIyfzfCeam+rKWN14YUTcVqSVSuh+y5weQZuXqfjnVAq88nTT5b+ l0G2gz73osVvPcGvBj97a3J4L1LJ6oePy2yTqCZ+0fltGDwIemgZSraf2BxygNUIhsDh ZFTy7cJVHbrn8o/Fj6sEW+FLwfk092cqDSslzyZVhuWIUgsYjsPx6vy/eLS90q6V+uok GWIA== X-Gm-Message-State: AC+VfDwe+RliCARRlzlSNO+CSZc4TSTFrPLGUj0MyUaaNP0NVwNkkHHu 875Oft6Dx/JgLnCdY+GedQ87i0Af4XU= X-Google-Smtp-Source: ACHHUZ45dlSMxFShxRuFs1sUsDehHb+VKrD8LxPSfQeQwc/e/FsYaikC1SyTOHMny95lhdJt48Vm0A== X-Received: by 2002:a05:6359:67a6:b0:129:ca90:3b6c with SMTP id sq38-20020a05635967a600b00129ca903b6cmr309705rwb.17.1685921302920; Sun, 04 Jun 2023 16:28:22 -0700 (PDT) Received: from localhost.localdomain ([98.37.226.98]) by smtp.googlemail.com with ESMTPSA id i7-20020a17090adc0700b0025043a8185dsm4654974pjv.23.2023.06.04.16.28.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 04 Jun 2023 16:28:22 -0700 (PDT) From: Andy Tai To: guix-patches@gnu.org Subject: [PATCH] gnu: tensorflow_lite: Update to 2.12.0 Date: Sun, 4 Jun 2023 16:28:18 -0700 Message-Id: X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=2607:f8b0:4864:20::c2b; envelope-from=lichengtai@gmail.com; helo=mail-oo1-xc2b.google.com X-Spam_score_int: -14 X-Spam_score: -1.5 X-Spam_bar: - X-Spam_report: (-1.5 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FORGED_FROMDOMAIN=0.249, FREEMAIL_FROM=0.001, HEADER_FROM_DIFFERENT_DOMAINS=0.25, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.1 (-) X-Debbugs-Envelope-To: submit Cc: Andy Tai X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.1 (--) * gnu/packages/algebra.scm (eigen-for-tensorflow-lite): Remove variable * gnu/packages/machine-learning.scm (tensorflow-lite): Update to 2.12.0 [arguments] (configure-flags): Turn on GPU and RUY use; Use cmake to find packages already in Guix as dependencies for absl-cpp, eigen, flatbuffer, neon2ssl, cpuinfo, ruy. [arguments] (phases): Remove operations setting up absl, eigen, ruy, and neon2ssl as native inouts as now we use Guix's packages of these as dependencies [arguments] (inputs): Add cpuinfo, eigen, fp16, mesa-header, opencl, pthreadpool, ruy, vulkan and xnnpack as explicit inputs [arguments] (native-inputs): Remove local setup of neon2ssl and ruy as we now use Guix's packages of these as explicit dependencies * gnu/packages/maths.scm: Remove extra statement * gnu/packages/serialization.scm (flatbuffers-next-shared): New variable, flatbuffers-next built with -fPIC as needed by tensorflow-lite --- gnu/packages/algebra.scm | 32 -------- gnu/packages/machine-learning.scm | 118 ++++++++++++++++-------------- gnu/packages/maths.scm | 1 - gnu/packages/serialization.scm | 12 +++ 4 files changed, 75 insertions(+), 88 deletions(-) diff --git a/gnu/packages/algebra.scm b/gnu/packages/algebra.scm index 3babf90b3c..81df0c97ac 100644 --- a/gnu/packages/algebra.scm +++ b/gnu/packages/algebra.scm @@ -1155,38 +1155,6 @@ (define-public eigen-for-tensorflow (substitute* "test/CMakeLists.txt" (("ei_add_test\\(stddeque") "#"))))))))))) -(define-public eigen-for-tensorflow-lite - ;; This commit was taken from - ;; tensorflow/lite/tools/cmake/modules/eigen.cmake - (let ((commit "d10b27fe37736d2944630ecd7557cefa95cf87c9") - (revision "1")) - (package (inherit eigen) - (name "eigen-for-tensorflow-lite") - (version (git-version "3.3.7" revision commit)) - (source (origin - (method git-fetch) - (uri (git-reference - (url "https://gitlab.com/libeigen/eigen") - (commit commit))) - (sha256 - (base32 - "0v8a20cwvwmp3hw4275b37frw33v92z0mr8f4dn6y8k0rz92hrrf")) - (file-name (git-file-name name version)) - (modules '((guix build utils))) - (snippet - ;; Ther are test failures in the "unsupported" directory, but - ;; maintainers say it's unsupported anyway, so just skip - ;; them. - '(begin - (substitute* "unsupported/CMakeLists.txt" - (("add_subdirectory\\(test.*") - "# Do not build the tests for unsupported features.\n")))))) - (arguments - (substitute-keyword-arguments (package-arguments eigen) - ((#:phases phases) - `(modify-phases ,phases - (delete 'disable-some-tests)))))))) - (define-public xtensor (package (name "xtensor") diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index c9bc5e8b68..15fe4be995 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -54,6 +54,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages) #:use-module (gnu packages adns) #:use-module (gnu packages algebra) + #:use-module (gnu packages assembly) #:use-module (gnu packages audio) #:use-module (gnu packages autotools) #:use-module (gnu packages base) @@ -88,6 +89,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages ninja) #:use-module (gnu packages ocaml) #:use-module (gnu packages onc-rpc) + #:use-module (gnu packages opencl) #:use-module (gnu packages parallel) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) @@ -107,6 +109,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages swig) #:use-module (gnu packages time) #:use-module (gnu packages tls) + #:use-module (gnu packages vulkan) #:use-module (gnu packages video) #:use-module (gnu packages web) #:use-module (gnu packages xml) @@ -2642,7 +2645,7 @@ (define-public tensorflow (define-public tensorflow-lite (package (name "tensorflow-lite") - (version "2.5.0") + (version "2.12.0") (source (origin (method git-fetch) @@ -2652,35 +2655,53 @@ (define-public tensorflow-lite (file-name (git-file-name name version)) (sha256 (base32 - "1jdw2i1rq06zqd6aabh7bbm0avsg4pygnfmd7gviv0blhih9054l")))) + "051knmq6ysg4936d9baaqsx6j3rfsp4ffi2pwrxx4llan682rvpi")))) (build-system cmake-build-system) (arguments `(#:tests? #false ; no "check" target #:build-type "Release" #:configure-flags (list - "-DTFLITE_ENABLE_GPU=OFF" - "-DTFLITE_ENABLE_RUY=OFF" + ;"--trace" + ;; Use Guix's own versions of library dependencies. + "-DTFLITE_ENABLE_INSTALL=ON" ;; so cmake can be used to find this from other packages + "-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON" + "-DTFLITE_ENABLE_GPU=ON" + "-DTFLITE_ENABLE_RUY=ON" + + ;; TO DO: turn on Farmhash + ;"-DSYSTEM_FARMHASH=ON" + + + (string-append "-Dabsl_DIR=" (assoc-ref %build-inputs "abseil-cpp") + "/lib/cmake/absl") + (string-append "-DEigen3_DIR=" (assoc-ref %build-inputs "eigen") + "/share/eigen3/cmake") + (string-append "-DFlatBuffers_DIR=" (assoc-ref %build-inputs "flatbuffers") + "/lib/cmake/flatbuffers") + (string-append "-DNEON_2_SSE_DIR=" (assoc-ref %build-inputs "neon2sse") + "/lib/cmake/NEON_2_SSE") + (string-append "-Dcpuinfo_DIR=" (assoc-ref %build-inputs "cpuinfo") + "/share/cpuinfo") + (string-append "-Druy_DIR=" (assoc-ref %build-inputs "ruy") + "/lib/cmake/ruy") ;; TODO: The build system attempts to build xnnpack from source. We ;; would like to use our xnnpack package here, but this requires more ;; work. "-DTFLITE_ENABLE_XNNPACK=OFF" - ;; Pretend we've already fetched abseil. We won't actually build it - ;; but use the existing package. - "-Dabseil-cpp_POPULATED=TRUE" - - ;; Don't fetch the sources. We have already built flatbuffers. - "-Dflatbuffers_POPULATED=TRUE" + ;;; Don't fetch the sources. We have these already + "-Degl_headers_POPULATED=TRUE" + "-Dfp16_headers_POPULATED=TRUE" + "-Dopencl_headers_POPULATED=TRUE" + "-Dopengl_headers_POPULATED=TRUE" + "-Dvulkan_headers_POPULATED=TRUE" "-DFFT2D_SOURCE_DIR=/tmp/fft2d" - "-Dneon2sse_SOURCE_DIR=/tmp/neon2sse" - "-Dneon2sse_BINARY_DIR=/tmp/neon2sse-bin" "-DFARMHASH_SOURCE_DIR=/tmp/farmhash" "-Dgemmlowp_SOURCE_DIR=/tmp/gemmlowp" - (string-append "-DRUY_SOURCE_DIR=" - (assoc-ref %build-inputs "ruy-src"))) + ) #:phases (modify-phases %standard-phases (add-after 'unpack 'chdir @@ -2689,20 +2710,19 @@ (define-public tensorflow-lite (lambda* (#:key inputs #:allow-other-keys) ;; Use external cmake finders instead of these stubs that won't ;; find anything but the bundled sources. - (delete-file "tools/cmake/modules/Findabsl.cmake") - (delete-file "tools/cmake/modules/Findeigen.cmake") + ;; TO DO: properly use Guix's pthreaqdpool + ;; roght now not using pthreadpool because + ;; we are not enabling xnnpack + (substitute* "CMakeLists.txt" + (("if\\(NOT DEFINED PTHREADPOOL_SOURCE_DIR\\)") + "if(false)")) (substitute* "CMakeLists.txt" - (("find_package\\(eigen REQUIRED") - "find_package(eigen REQUIRED NAMES Eigen3")) - (substitute* "tools/cmake/modules/Findflatbuffers.cmake" - (("get_target_property.*") - (format #false "set(FLATBUFFERS_INCLUDE_DIRS ~a/include)\n" - (assoc-ref inputs "flatbuffers")))) + (("if\\(NOT TARGET pthreadpool\\)") + "if(false)")) ;; Don't fetch source code; we already have everything we need. (substitute* '("tools/cmake/modules/fft2d.cmake" - "tools/cmake/modules/ruy.cmake" "tools/cmake/modules/farmhash.cmake" "tools/cmake/modules/neon2sse.cmake" "tools/cmake/modules/gemmlowp.cmake") @@ -2718,10 +2738,9 @@ (define-public tensorflow-lite (invoke "tar" "--strip-components=1" "-xf" (assoc-ref inputs "fft2d-src"))) - (copy-recursively (assoc-ref inputs "neon2sse-src") - "/tmp/neon2sse/") (copy-recursively (assoc-ref inputs "gemmlowp-src") - "/tmp/gemmlowp/"))) + "/tmp/gemmlowp/") + )) (add-after 'copy-sources 'prepare-shared-library-build (lambda _ (chdir "c"))) (replace 'install @@ -2738,9 +2757,21 @@ (define-public tensorflow-lite (find-files "." "\\.h$"))))))))) (inputs `(("abseil-cpp" ,abseil-cpp-20200923.3) - ("eigen" ,eigen-for-tensorflow-lite) - ("flatbuffers" ,flatbuffers) - ("python" ,python))) + ("cpuinfo" ,cpuinfo) + ("eigen" ,eigen) + ("fp16" ,fp16) + ("flatbuffers" ,flatbuffers-next-shared) + ;("gemmlowp" ,gemmlowp) ;; TO DO + ("mesa-headers" ,mesa-headers) + ("neon2sse" ,neon2sse) + ("opencl-clhpp" ,opencl-clhpp) + ("opencl-headers" ,opencl-headers) + ("opencl-icd-loader" ,opencl-icd-loader) + ("pthreadpool" ,pthreadpool) + ("python" ,python) + ("ruy" ,ruy) + ;("xnnpack" ,xnnpack) ;; TO DO: properly use Guix's copy of xnnpack + ("vulkan-headers" , vulkan-headers))) (native-inputs `(("pkg-config" ,pkg-config) ("gemmlowp-src" @@ -2756,17 +2787,6 @@ (define-public tensorflow-lite (sha256 (base32 "1sbp8kmr2azwlvfbzryy1frxi99jhsh1nc93bdbxdf8zdgpv0kxl"))))) - ("neon2sse-src" - ,(let ((commit "a1652fd5253afbf3e39357b012974f93511f6108")) - (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/intel/ARM_NEON_2_x86_SSE") - (commit commit))) - (file-name (git-file-name "neon2sse" (string-take commit 8))) - (sha256 - (base32 - "1q8gkxag9wlnwdwad2pclsrkwzrdjy94hyrkayrsvxyj7szb5y8i"))))) ("farmhash-src" ,(let ((commit "816a4ae622e964763ca0862d9dbd19324a1eaf45")) (origin @@ -2782,24 +2802,12 @@ (define-public tensorflow-lite ,(origin (method url-fetch) (uri (string-append "https://storage.googleapis.com/" - "mirror.tensorflow.org/" - "www.kurims.kyoto-u.ac.jp/~ooura/fft2d.tgz")) + "mirror.tensorflow.org/github.com/petewarden/" + "OouraFFT/archive/v1.0.tar.gz")) (file-name "fft2d.tar.gz") (sha256 (base32 - "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd")))) - ("ruy-src" - ,(let ((commit "9c56af3fce210a8a103eda19bd6f47c08a9e3d90")) - (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/google/ruy") - (commit commit) - (recursive? #true))) - (file-name (git-file-name "ruy" (string-take commit 8))) - (sha256 - (base32 - "1cfd5gk6kaj8kbl3h98gx1ap8czd59y6p8qq8nr28fklpyzf5cis"))))))) + "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd")))))) (home-page "https://tensorflow.org") (synopsis "Machine learning framework") (description diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm index c6fabf1973..25a7b61156 100644 --- a/gnu/packages/maths.scm +++ b/gnu/packages/maths.scm @@ -85,7 +85,6 @@ (define-module (gnu packages maths) #:use-module (guix packages) #:use-module (guix download) #:use-module (guix git-download) - #:use-module (guix gexp) #:use-module (guix utils) #:use-module ((guix build utils) #:select (alist-replace)) #:use-module (guix build-system ant) diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm index 39954d6764..268a5d3d9b 100644 --- a/gnu/packages/serialization.scm +++ b/gnu/packages/serialization.scm @@ -748,6 +748,18 @@ (define-public flatbuffers-next (base32 "1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz")))))) +(define-public flatbuffers-next-shared + (package + (inherit flatbuffers-next) + (name "flatbuffers-shared") + (version "23.1.21") + (arguments + (substitute-keyword-arguments (package-arguments flatbuffers-next) + ((#:configure-flags flags) + ;; make C++ fiels compiled with -fPIC, needed for shared lib". + #~(cons "-DFLATBUFFERS_CXX_FLAGS=-fPIC" #$flags)))))) + + (define-public python-feather-format (package (name "python-feather-format") base-commit: eed55a6544d5bda2245ec853e5fa4b28e1865bea prerequisite-patch-id: 5e9e341f5896c5eee0f64a7a031cc57491e76eb9 prerequisite-patch-id: b17d63ac48fd66475dda11cd37f74bb63aca5cfb -- 2.40.1 From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 04 19:42:18 2023 Received: (at control) by debbugs.gnu.org; 4 Jun 2023 23:42:18 +0000 Received: from localhost ([127.0.0.1]:47568 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q5xMY-0002Om-4Z for submit@debbugs.gnu.org; Sun, 04 Jun 2023 19:42:18 -0400 Received: from mail-qt1-f179.google.com ([209.85.160.179]:52400) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q5xMX-0002OW-1S for control@debbugs.gnu.org; Sun, 04 Jun 2023 19:42:17 -0400 Received: by mail-qt1-f179.google.com with SMTP id d75a77b69052e-3f81396e435so35879301cf.0 for ; Sun, 04 Jun 2023 16:42:17 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=atai-org.20221208.gappssmtp.com; s=20221208; t=1685922131; x=1688514131; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=3hDJUAC6VY9K+dpptxF9wkNTb7BKrBaqlZ7uFDC9lJc=; b=hMVzl/GbzbMHkPT96DnYEb66M/oi/3ZrEOftUZApdNGj1VfX01nNPX1+WbMJzaqhA/ omjVThWenSNug/twmFf5PhndyaaPtRe7cwHs+rgqJ1R0tvOj79Amitt10UdfAD0hwWFG c9keRzwGte0H1F/2Nkgzh44Iht4YIBbboeJJAHCfHmxCCUuyOEbCiYqAf7sI0e09m05T GrpnSnQHtpYhDshDDiWeZ+pEbTJkWj9EunIXgLkzZdcsKQf9q1Ys6626t8ktUbdIbiAa b3zrjItXPHWGEcfNE6S+b1Guu5wPMCvVWTsG3FmixBoZUcuFm/X4TIUvcg0P6IcdNRxY rMug== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685922131; x=1688514131; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=3hDJUAC6VY9K+dpptxF9wkNTb7BKrBaqlZ7uFDC9lJc=; b=cR58pq0XGWXU7AALl+4UPnSnHZyz0XM2hXLLoR+aUln/h/gRtxOqZj5WapfL6FyPFj SXGwh9u1y/305kmo8REkOI60IhTOt86N+hX59isgBcMRwfUfozQ+itCqClYZqoFoX2xo YXRthWdgZSAvMjCDxmy8tAyKLbqWsnMUZKsuwfgZ55toXM+TNkApygbDTAg7Ayka/uZH HdqbAFaajMB7j6xEMVsxeo76S5KWqgFX5Ea4Rlhh5FNcWIHW1ThmCiQMKhDq8NsxWWIx aQFgXtj7w85vhaJqVWYVTwZ/f353Wqnr9Pfdk14zzLVKABhTrdg59YcaFYpLJxe+a26K f6XQ== X-Gm-Message-State: AC+VfDznluWxTxakz2NA6kW/2KY/pciIIK1sI+Sh0TzVkqJUCCflITTm 8OPRg8enHoiMZxzSO0J8tVsNJvuK21MVJ/Ib2kNbfAuA X-Google-Smtp-Source: ACHHUZ4kiYe9nZI/oIBD0sPj0MYlwfEadxBQaIuJYFzFvRvFZHFaENV93qwW10qxlHlk1UsQiAWJZMGw58bVeHm1o/k= X-Received: by 2002:a05:622a:591:b0:3f7:ffc8:2f72 with SMTP id c17-20020a05622a059100b003f7ffc82f72mr6018727qtb.58.1685922131291; Sun, 04 Jun 2023 16:42:11 -0700 (PDT) MIME-Version: 1.0 From: Andy Tai Date: Sun, 4 Jun 2023 16:41:35 -0700 Message-ID: Subject: To: GNU bug tracker automated control server Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 2.3 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: block 63897 by 63888, 63889 Content analysis details: (2.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (lichengtai[at]gmail.com) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [209.85.160.179 listed in list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.2 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [209.85.160.179 listed in wl.mailspike.net] 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 0.0 FREEMAIL_FORGED_FROMDOMAIN 2nd level domains in From and EnvelopeFrom freemail headers are different 2.0 BLANK_SUBJECT Subject is present but empty X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.3 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: block 63897 by 63888, 63889 Content analysis details: (1.3 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 RCVD_IN_MSPIKE_H3 RBL: Good reputation (+3) [209.85.160.179 listed in wl.mailspike.net] -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [209.85.160.179 listed in list.dnswl.org] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (lichengtai[at]gmail.com) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.2 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 0.0 FREEMAIL_FORGED_FROMDOMAIN 2nd level domains in From and EnvelopeFrom freemail headers are different 2.0 BLANK_SUBJECT Subject is present but empty -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager block 63897 by 63888, 63889 From debbugs-submit-bounces@debbugs.gnu.org Sun Jun 04 19:51:12 2023 Received: (at 63897) by debbugs.gnu.org; 4 Jun 2023 23:51:12 +0000 Received: from localhost ([127.0.0.1]:47573 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q5xV9-0005EK-4H for submit@debbugs.gnu.org; Sun, 04 Jun 2023 19:51:11 -0400 Received: from mail-pj1-f46.google.com ([209.85.216.46]:58779) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q5xV4-0005Dq-OS for 63897@debbugs.gnu.org; Sun, 04 Jun 2023 19:51:10 -0400 Received: by mail-pj1-f46.google.com with SMTP id 98e67ed59e1d1-2562cc85d3dso3435167a91.1 for <63897@debbugs.gnu.org>; Sun, 04 Jun 2023 16:51:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1685922660; x=1688514660; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:from:to:cc:subject:date:message-id:reply-to; bh=jTCtyKWhjlGiTQTDsxTE1hvVYEZr7sT3lz5Pilg0A5g=; b=mam7Expkm132LFP8a1jMpckr7AwB4vkqsgGLIjdHWPW6+CKZo2kMUs7JkghtnZaCtE MASL3+wV42eSMgwTEg8fROSWXDDnqXlr0zFGJnMZnNi4jpqgrHrOp4MrPHK/MUdNp9pG 6IWjNZNW/OZRS/tFQeuJ7NWJJ3AZkdeGu28Me+k+K241xTx9/i4asETbmnxXkB8gqRDd bixMcYoEIdTV/vT/VZ3IjLpZQdwa4v+SzUTVry02r0cNnQswOpGvqqGQ44abTDvvCK/I m8mBKvBhV2hhZBiGFwlUQRFHitOm8Oix0b5q7B8Oi2P7FbWLGTESGOIYThsPToV2jHhF kJSw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1685922660; x=1688514660; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=jTCtyKWhjlGiTQTDsxTE1hvVYEZr7sT3lz5Pilg0A5g=; b=k3xD/rs4CUja3I+nRp/Udzl5cNZSrMiD7vur6jV4PJ5OQe8e5+oUMUFQ5A+/b3+/Bo gNAo4mp37wWYk1i/LKajFdV7yV9xmos2x2891BD3JoV39j97qbHVBbc+u5XumBxwIQqB WU3QVde45yffdJ/yJDgY9bWI/pNf5CigEAIXmvEMeB7LmZTt9GnItRgnCnZxd8NTwNjs 1CB3RvOtXJjsic5qc8rALaH5cz4l0ubuCUMG5FM3aD8e8ZtTol6C+kWIOXUL3XICVGDD 9OUJTQaqy6Bq0h6fJlfL/y8Tx4iIb8txmpZupWAAlmwGHQLJO3Fj4RD6D2jEyW3MqMl0 +/og== X-Gm-Message-State: AC+VfDwNl8d86fI5SAWOjX1jcBV9vr634fTg8DAXZXCgd4KlB1Y06HxX flHExv3E4ebiqOMj7tmFR/0TZZlj6mg= X-Google-Smtp-Source: ACHHUZ75FDZJVrmlVxoTXHa1QVQnBEWITrcwUW4uoQKAxZs7b6GoRcmZRFlMJAKLHGRjCSiK6lTJwA== X-Received: by 2002:a17:902:da8c:b0:1ae:87d8:a0c3 with SMTP id j12-20020a170902da8c00b001ae87d8a0c3mr7463640plx.10.1685922660345; Sun, 04 Jun 2023 16:51:00 -0700 (PDT) Received: from localhost.localdomain ([98.37.226.98]) by smtp.googlemail.com with ESMTPSA id n13-20020a170902e54d00b001ae4d4d2676sm5150126plf.269.2023.06.04.16.50.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sun, 04 Jun 2023 16:50:59 -0700 (PDT) From: Andy Tai To: 63897@debbugs.gnu.org Subject: [PATCH v2] gnu: tensorflow_lite: Update to 2.12.0 Date: Sun, 4 Jun 2023 16:50:56 -0700 Message-Id: <580b11d16b0563a0107d742db105634e47b6e264.1685922548.git.atai@atai.org> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 63897 Cc: Andy Tai X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) * gnu/packages/algebra.scm (eigen-for-tensorflow-lite): Remove variable * gnu/packages/machine-learning.scm (tensorflow-lite): Update to 2.12.0 [arguments] (configure-flags): Turn on GPU and RUY use; Use cmake to find packages already in Guix as dependencies for absl-cpp, eigen, flatbuffer, neon2ssl, cpuinfo, ruy. [arguments] (phases): Remove operations setting up absl, eigen, ruy, and neon2ssl as native inouts as now we use Guix's packages of these as dependencies [arguments] (inputs): Add cpuinfo, eigen, fp16, mesa-header, opencl, pthreadpool, ruy, vulkan and xnnpack as explicit inputs [arguments] (native-inputs): Remove local setup of neon2ssl and ruy as we now use Guix's packages of these as explicit dependencies * gnu/packages/maths.scm: Remove extra statement * gnu/packages/serialization.scm (flatbuffers-next-shared): New variable, flatbuffers-next built with -fPIC as needed by tensorflow-lite --- gnu/packages/algebra.scm | 32 -------- gnu/packages/machine-learning.scm | 121 ++++++++++++++++-------------- gnu/packages/maths.scm | 1 - gnu/packages/serialization.scm | 12 +++ 4 files changed, 78 insertions(+), 88 deletions(-) diff --git a/gnu/packages/algebra.scm b/gnu/packages/algebra.scm index 3babf90b3c..81df0c97ac 100644 --- a/gnu/packages/algebra.scm +++ b/gnu/packages/algebra.scm @@ -1155,38 +1155,6 @@ (define-public eigen-for-tensorflow (substitute* "test/CMakeLists.txt" (("ei_add_test\\(stddeque") "#"))))))))))) -(define-public eigen-for-tensorflow-lite - ;; This commit was taken from - ;; tensorflow/lite/tools/cmake/modules/eigen.cmake - (let ((commit "d10b27fe37736d2944630ecd7557cefa95cf87c9") - (revision "1")) - (package (inherit eigen) - (name "eigen-for-tensorflow-lite") - (version (git-version "3.3.7" revision commit)) - (source (origin - (method git-fetch) - (uri (git-reference - (url "https://gitlab.com/libeigen/eigen") - (commit commit))) - (sha256 - (base32 - "0v8a20cwvwmp3hw4275b37frw33v92z0mr8f4dn6y8k0rz92hrrf")) - (file-name (git-file-name name version)) - (modules '((guix build utils))) - (snippet - ;; Ther are test failures in the "unsupported" directory, but - ;; maintainers say it's unsupported anyway, so just skip - ;; them. - '(begin - (substitute* "unsupported/CMakeLists.txt" - (("add_subdirectory\\(test.*") - "# Do not build the tests for unsupported features.\n")))))) - (arguments - (substitute-keyword-arguments (package-arguments eigen) - ((#:phases phases) - `(modify-phases ,phases - (delete 'disable-some-tests)))))))) - (define-public xtensor (package (name "xtensor") diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index c9bc5e8b68..3740206b86 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -54,6 +54,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages) #:use-module (gnu packages adns) #:use-module (gnu packages algebra) + #:use-module (gnu packages assembly) #:use-module (gnu packages audio) #:use-module (gnu packages autotools) #:use-module (gnu packages base) @@ -88,6 +89,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages ninja) #:use-module (gnu packages ocaml) #:use-module (gnu packages onc-rpc) + #:use-module (gnu packages opencl) #:use-module (gnu packages parallel) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) @@ -107,6 +109,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages swig) #:use-module (gnu packages time) #:use-module (gnu packages tls) + #:use-module (gnu packages vulkan) #:use-module (gnu packages video) #:use-module (gnu packages web) #:use-module (gnu packages xml) @@ -2642,7 +2645,7 @@ (define-public tensorflow (define-public tensorflow-lite (package (name "tensorflow-lite") - (version "2.5.0") + (version "2.12.0") (source (origin (method git-fetch) @@ -2652,35 +2655,56 @@ (define-public tensorflow-lite (file-name (git-file-name name version)) (sha256 (base32 - "1jdw2i1rq06zqd6aabh7bbm0avsg4pygnfmd7gviv0blhih9054l")))) + "051knmq6ysg4936d9baaqsx6j3rfsp4ffi2pwrxx4llan682rvpi")))) (build-system cmake-build-system) (arguments `(#:tests? #false ; no "check" target #:build-type "Release" #:configure-flags (list - "-DTFLITE_ENABLE_GPU=OFF" - "-DTFLITE_ENABLE_RUY=OFF" + ;"--trace" + "-DTFLITE_ENABLE_INSTALL=ON" + ;; so cmake can be used to find this from other packages + + "-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON" + ;; Use Guix's own packages as dependencies. + + "-DTFLITE_ENABLE_GPU=ON" + "-DTFLITE_ENABLE_RUY=ON" + + ;; TO DO: turn on Farmhash + ;"-DSYSTEM_FARMHASH=ON" + + + (string-append "-Dabsl_DIR=" (assoc-ref %build-inputs "abseil-cpp") + "/lib/cmake/absl") + (string-append "-DEigen3_DIR=" (assoc-ref %build-inputs "eigen") + "/share/eigen3/cmake") + (string-append "-DFlatBuffers_DIR=" (assoc-ref %build-inputs "flatbuffers") + "/lib/cmake/flatbuffers") + (string-append "-DNEON_2_SSE_DIR=" (assoc-ref %build-inputs "neon2sse") + "/lib/cmake/NEON_2_SSE") + (string-append "-Dcpuinfo_DIR=" (assoc-ref %build-inputs "cpuinfo") + "/share/cpuinfo") + (string-append "-Druy_DIR=" (assoc-ref %build-inputs "ruy") + "/lib/cmake/ruy") ;; TODO: The build system attempts to build xnnpack from source. We ;; would like to use our xnnpack package here, but this requires more ;; work. "-DTFLITE_ENABLE_XNNPACK=OFF" - ;; Pretend we've already fetched abseil. We won't actually build it - ;; but use the existing package. - "-Dabseil-cpp_POPULATED=TRUE" - - ;; Don't fetch the sources. We have already built flatbuffers. - "-Dflatbuffers_POPULATED=TRUE" + ;;; Don't fetch the sources. We have these already + "-Degl_headers_POPULATED=TRUE" + "-Dfp16_headers_POPULATED=TRUE" + "-Dopencl_headers_POPULATED=TRUE" + "-Dopengl_headers_POPULATED=TRUE" + "-Dvulkan_headers_POPULATED=TRUE" "-DFFT2D_SOURCE_DIR=/tmp/fft2d" - "-Dneon2sse_SOURCE_DIR=/tmp/neon2sse" - "-Dneon2sse_BINARY_DIR=/tmp/neon2sse-bin" "-DFARMHASH_SOURCE_DIR=/tmp/farmhash" "-Dgemmlowp_SOURCE_DIR=/tmp/gemmlowp" - (string-append "-DRUY_SOURCE_DIR=" - (assoc-ref %build-inputs "ruy-src"))) + ) #:phases (modify-phases %standard-phases (add-after 'unpack 'chdir @@ -2689,20 +2713,19 @@ (define-public tensorflow-lite (lambda* (#:key inputs #:allow-other-keys) ;; Use external cmake finders instead of these stubs that won't ;; find anything but the bundled sources. - (delete-file "tools/cmake/modules/Findabsl.cmake") - (delete-file "tools/cmake/modules/Findeigen.cmake") + ;; TO DO: properly use Guix's pthreaqdpool + ;; roght now not using pthreadpool because + ;; we are not enabling xnnpack + (substitute* "CMakeLists.txt" + (("if\\(NOT DEFINED PTHREADPOOL_SOURCE_DIR\\)") + "if(false)")) (substitute* "CMakeLists.txt" - (("find_package\\(eigen REQUIRED") - "find_package(eigen REQUIRED NAMES Eigen3")) - (substitute* "tools/cmake/modules/Findflatbuffers.cmake" - (("get_target_property.*") - (format #false "set(FLATBUFFERS_INCLUDE_DIRS ~a/include)\n" - (assoc-ref inputs "flatbuffers")))) + (("if\\(NOT TARGET pthreadpool\\)") + "if(false)")) ;; Don't fetch source code; we already have everything we need. (substitute* '("tools/cmake/modules/fft2d.cmake" - "tools/cmake/modules/ruy.cmake" "tools/cmake/modules/farmhash.cmake" "tools/cmake/modules/neon2sse.cmake" "tools/cmake/modules/gemmlowp.cmake") @@ -2718,10 +2741,9 @@ (define-public tensorflow-lite (invoke "tar" "--strip-components=1" "-xf" (assoc-ref inputs "fft2d-src"))) - (copy-recursively (assoc-ref inputs "neon2sse-src") - "/tmp/neon2sse/") (copy-recursively (assoc-ref inputs "gemmlowp-src") - "/tmp/gemmlowp/"))) + "/tmp/gemmlowp/") + )) (add-after 'copy-sources 'prepare-shared-library-build (lambda _ (chdir "c"))) (replace 'install @@ -2738,9 +2760,21 @@ (define-public tensorflow-lite (find-files "." "\\.h$"))))))))) (inputs `(("abseil-cpp" ,abseil-cpp-20200923.3) - ("eigen" ,eigen-for-tensorflow-lite) - ("flatbuffers" ,flatbuffers) - ("python" ,python))) + ("cpuinfo" ,cpuinfo) + ("eigen" ,eigen) + ("fp16" ,fp16) + ("flatbuffers" ,flatbuffers-next-shared) + ;("gemmlowp" ,gemmlowp) ;; TO DO + ("mesa-headers" ,mesa-headers) + ("neon2sse" ,neon2sse) + ("opencl-clhpp" ,opencl-clhpp) + ("opencl-headers" ,opencl-headers) + ("opencl-icd-loader" ,opencl-icd-loader) + ("pthreadpool" ,pthreadpool) + ("python" ,python) + ("ruy" ,ruy) + ;("xnnpack" ,xnnpack) ;; TO DO: properly use Guix's copy of xnnpack + ("vulkan-headers" , vulkan-headers))) (native-inputs `(("pkg-config" ,pkg-config) ("gemmlowp-src" @@ -2756,17 +2790,6 @@ (define-public tensorflow-lite (sha256 (base32 "1sbp8kmr2azwlvfbzryy1frxi99jhsh1nc93bdbxdf8zdgpv0kxl"))))) - ("neon2sse-src" - ,(let ((commit "a1652fd5253afbf3e39357b012974f93511f6108")) - (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/intel/ARM_NEON_2_x86_SSE") - (commit commit))) - (file-name (git-file-name "neon2sse" (string-take commit 8))) - (sha256 - (base32 - "1q8gkxag9wlnwdwad2pclsrkwzrdjy94hyrkayrsvxyj7szb5y8i"))))) ("farmhash-src" ,(let ((commit "816a4ae622e964763ca0862d9dbd19324a1eaf45")) (origin @@ -2782,24 +2805,12 @@ (define-public tensorflow-lite ,(origin (method url-fetch) (uri (string-append "https://storage.googleapis.com/" - "mirror.tensorflow.org/" - "www.kurims.kyoto-u.ac.jp/~ooura/fft2d.tgz")) + "mirror.tensorflow.org/github.com/petewarden/" + "OouraFFT/archive/v1.0.tar.gz")) (file-name "fft2d.tar.gz") (sha256 (base32 - "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd")))) - ("ruy-src" - ,(let ((commit "9c56af3fce210a8a103eda19bd6f47c08a9e3d90")) - (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/google/ruy") - (commit commit) - (recursive? #true))) - (file-name (git-file-name "ruy" (string-take commit 8))) - (sha256 - (base32 - "1cfd5gk6kaj8kbl3h98gx1ap8czd59y6p8qq8nr28fklpyzf5cis"))))))) + "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd")))))) (home-page "https://tensorflow.org") (synopsis "Machine learning framework") (description diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm index c6fabf1973..25a7b61156 100644 --- a/gnu/packages/maths.scm +++ b/gnu/packages/maths.scm @@ -85,7 +85,6 @@ (define-module (gnu packages maths) #:use-module (guix packages) #:use-module (guix download) #:use-module (guix git-download) - #:use-module (guix gexp) #:use-module (guix utils) #:use-module ((guix build utils) #:select (alist-replace)) #:use-module (guix build-system ant) diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm index 39954d6764..268a5d3d9b 100644 --- a/gnu/packages/serialization.scm +++ b/gnu/packages/serialization.scm @@ -748,6 +748,18 @@ (define-public flatbuffers-next (base32 "1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz")))))) +(define-public flatbuffers-next-shared + (package + (inherit flatbuffers-next) + (name "flatbuffers-shared") + (version "23.1.21") + (arguments + (substitute-keyword-arguments (package-arguments flatbuffers-next) + ((#:configure-flags flags) + ;; make C++ fiels compiled with -fPIC, needed for shared lib". + #~(cons "-DFLATBUFFERS_CXX_FLAGS=-fPIC" #$flags)))))) + + (define-public python-feather-format (package (name "python-feather-format") base-commit: eed55a6544d5bda2245ec853e5fa4b28e1865bea prerequisite-patch-id: 5e9e341f5896c5eee0f64a7a031cc57491e76eb9 prerequisite-patch-id: b17d63ac48fd66475dda11cd37f74bb63aca5cfb -- 2.40.1 From debbugs-submit-bounces@debbugs.gnu.org Tue Jun 06 04:41:47 2023 Received: (at 63897) by debbugs.gnu.org; 6 Jun 2023 08:41:47 +0000 Received: from localhost ([127.0.0.1]:50925 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q6SGA-0005cN-7S for submit@debbugs.gnu.org; Tue, 06 Jun 2023 04:41:47 -0400 Received: from mail-pg1-f179.google.com ([209.85.215.179]:55749) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1q6SG7-0005c7-CF for 63897@debbugs.gnu.org; Tue, 06 Jun 2023 04:41:45 -0400 Received: by mail-pg1-f179.google.com with SMTP id 41be03b00d2f7-53f7bef98b7so5194454a12.3 for <63897@debbugs.gnu.org>; Tue, 06 Jun 2023 01:41:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686040897; x=1688632897; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:from:to:cc:subject:date:message-id:reply-to; bh=gDUWrmNCDe/ID6jq82I8iRmzXzWQYJjOUVn9qQCglxc=; b=LyeFx/FXPFscgrTv5pBZ/EwbM+fiLbjsvYcA62zC2/838u8ptBxKKg1m5z8rlKP51T sSrklOBBYWwtCU6IpWVtdZlvWHMesKULoISjUpNRW9iK0rQYrjhsGyjn/rlLEpGYXmF5 Q+mgUb2RI82lzmtvPlIvQ3UEF9VEjxJC/oM6UoqaIS3hxZhTrWolQVXyHrm424Tw1lDf AM17XhOAjx8OlNSNSjQkChlaSmgac2bp8EYwSSgiznDLJw037FOKy5ipf39AqOHdZDa6 mezaixj2ZqVoNLmVNdc+rm6B3TVsCtiZU73+bK0GPc3sso0GZd2kw4+P9LU3UrptId5j IhSQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686040897; x=1688632897; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=gDUWrmNCDe/ID6jq82I8iRmzXzWQYJjOUVn9qQCglxc=; b=By80g7Xr2i6f0xEm0KAZDcNHV48mzrr1nk9szkZAqjM3QgfCCdagOWPundad0isSd9 ajSTVonArlDmHNbW4xXXquZS2QgjJsErQJDLa1ok9gH2T2xtlmyohwB0U54wNR3j1qYP UbFyzgLsTkHPyRaHtFL2q9roTlZPRSPuX//ABc3nSnjNLDMjdx4CvEiEzHbHzo5cAC4h 9o54sMz2xhnlTbYFWMIgjshNsPeAVti0XOLMRirfQIRC/fIQQgdIFs76cd3md83a5xor u3/WfHtEz1hgTJgueS3mdBq14YtZroeeaBNiWzEj34CXbz6WL56BccJYDr0nHszypEX3 cfqQ== X-Gm-Message-State: AC+VfDzSW3Vx3CTzZIKOSuwCC2f3k9MFjG73rBFsBMhVPDoEE7BFdzMf fAKZ8ItM3KRu0En0jl4v0177sLO7sxM= X-Google-Smtp-Source: ACHHUZ4RrbohXlBLCPSMWO+cuD6W3bVlCNaU5Jx2nAoujEogby0MMUZIFchcxvaEBYygAmVlrTNKBg== X-Received: by 2002:a05:6a20:918a:b0:10d:8f40:6454 with SMTP id v10-20020a056a20918a00b0010d8f406454mr2354507pzd.36.1686040896556; Tue, 06 Jun 2023 01:41:36 -0700 (PDT) Received: from localhost.localdomain ([98.37.226.98]) by smtp.googlemail.com with ESMTPSA id f25-20020aa782d9000000b0064d74808738sm6297524pfn.214.2023.06.06.01.41.35 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 06 Jun 2023 01:41:35 -0700 (PDT) From: Andy Tai To: 63897@debbugs.gnu.org Subject: [PATCH v3] gnu: tensorflow-lite: Update to 2.12.0 Date: Tue, 6 Jun 2023 01:41:32 -0700 Message-Id: X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 63897 Cc: Andy Tai X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) * gnu/packages/algebra.scm (eigen-for-tensorflow-lite): Remove variable * gnu/packages/machine-learning.scm (tensorflow-lite): Update to 2.12.0 [arguments] (configure-flags): Turn on GPU and RUY use; Use cmake to find packages already in Guix as dependencies for absl-cpp, eigen, flatbuffer, neon2ssl, cpuinfo, ruy. [arguments] (phases): Remove operations setting up absl-cpp, eigen, ruy, and neon2ssl as native inouts as now we use Guix's packages of these as dependencies. Do default build phase. Add steps to install C shared library and benchmark_model tool [arguments] (inputs): Add cpuinfo, eigen, fp16, mesa-header, opencl, pthreadpool, ruy, vulkan and xnnpack as explicit inputs [arguments] (native-inputs): Remove local setup of neon2ssl and ruy as we now use Guix's packages of these as explicit dependencies * gnu/packages/maths.scm: Remove extra statement * gnu/packages/serialization.scm (flatbuffers-next-shared): New variable, flatbuffers-next built with -fPIC as needed by tensorflow-lite --- gnu/packages/algebra.scm | 32 ------ gnu/packages/machine-learning.scm | 160 +++++++++++++++++------------- gnu/packages/maths.scm | 1 - gnu/packages/serialization.scm | 12 +++ 4 files changed, 101 insertions(+), 104 deletions(-) diff --git a/gnu/packages/algebra.scm b/gnu/packages/algebra.scm index 3babf90b3c..81df0c97ac 100644 --- a/gnu/packages/algebra.scm +++ b/gnu/packages/algebra.scm @@ -1155,38 +1155,6 @@ (define-public eigen-for-tensorflow (substitute* "test/CMakeLists.txt" (("ei_add_test\\(stddeque") "#"))))))))))) -(define-public eigen-for-tensorflow-lite - ;; This commit was taken from - ;; tensorflow/lite/tools/cmake/modules/eigen.cmake - (let ((commit "d10b27fe37736d2944630ecd7557cefa95cf87c9") - (revision "1")) - (package (inherit eigen) - (name "eigen-for-tensorflow-lite") - (version (git-version "3.3.7" revision commit)) - (source (origin - (method git-fetch) - (uri (git-reference - (url "https://gitlab.com/libeigen/eigen") - (commit commit))) - (sha256 - (base32 - "0v8a20cwvwmp3hw4275b37frw33v92z0mr8f4dn6y8k0rz92hrrf")) - (file-name (git-file-name name version)) - (modules '((guix build utils))) - (snippet - ;; Ther are test failures in the "unsupported" directory, but - ;; maintainers say it's unsupported anyway, so just skip - ;; them. - '(begin - (substitute* "unsupported/CMakeLists.txt" - (("add_subdirectory\\(test.*") - "# Do not build the tests for unsupported features.\n")))))) - (arguments - (substitute-keyword-arguments (package-arguments eigen) - ((#:phases phases) - `(modify-phases ,phases - (delete 'disable-some-tests)))))))) - (define-public xtensor (package (name "xtensor") diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index c9bc5e8b68..1f9ffc8ede 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -54,6 +54,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages) #:use-module (gnu packages adns) #:use-module (gnu packages algebra) + #:use-module (gnu packages assembly) #:use-module (gnu packages audio) #:use-module (gnu packages autotools) #:use-module (gnu packages base) @@ -88,6 +89,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages ninja) #:use-module (gnu packages ocaml) #:use-module (gnu packages onc-rpc) + #:use-module (gnu packages opencl) #:use-module (gnu packages parallel) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) @@ -107,6 +109,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages swig) #:use-module (gnu packages time) #:use-module (gnu packages tls) + #:use-module (gnu packages vulkan) #:use-module (gnu packages video) #:use-module (gnu packages web) #:use-module (gnu packages xml) @@ -2642,7 +2645,7 @@ (define-public tensorflow (define-public tensorflow-lite (package (name "tensorflow-lite") - (version "2.5.0") + (version "2.12.0") (source (origin (method git-fetch) @@ -2652,59 +2655,80 @@ (define-public tensorflow-lite (file-name (git-file-name name version)) (sha256 (base32 - "1jdw2i1rq06zqd6aabh7bbm0avsg4pygnfmd7gviv0blhih9054l")))) + "051knmq6ysg4936d9baaqsx6j3rfsp4ffi2pwrxx4llan682rvpi")))) (build-system cmake-build-system) (arguments `(#:tests? #false ; no "check" target #:build-type "Release" + + #:modules ((ice-9 match) + (guix build utils) + (guix build cmake-build-system)) + #:configure-flags (list - "-DTFLITE_ENABLE_GPU=OFF" - "-DTFLITE_ENABLE_RUY=OFF" + ;; so cmake can be used to find this from other packages + "-DTFLITE_ENABLE_INSTALL=ON" + + ;; Use Guix's own packages as dependencies. + "-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON" + + "-DTFLITE_ENABLE_GPU=ON" + "-DTFLITE_ENABLE_RUY=ON" + + ;; TO DO: turn on Farmhash + ;"-DSYSTEM_FARMHASH=ON" + + + (string-append "-Dabsl_DIR=" (assoc-ref %build-inputs "abseil-cpp") + "/lib/cmake/absl") + (string-append "-DEigen3_DIR=" (assoc-ref %build-inputs "eigen") + "/share/eigen3/cmake") + (string-append "-DFlatBuffers_DIR=" (assoc-ref %build-inputs "flatbuffers") + "/lib/cmake/flatbuffers") + (string-append "-DNEON_2_SSE_DIR=" (assoc-ref %build-inputs "neon2sse") + "/lib/cmake/NEON_2_SSE") + (string-append "-Dcpuinfo_DIR=" (assoc-ref %build-inputs "cpuinfo") + "/share/cpuinfo") + (string-append "-Druy_DIR=" (assoc-ref %build-inputs "ruy") + "/lib/cmake/ruy") ;; TODO: The build system attempts to build xnnpack from source. We ;; would like to use our xnnpack package here, but this requires more ;; work. "-DTFLITE_ENABLE_XNNPACK=OFF" - ;; Pretend we've already fetched abseil. We won't actually build it - ;; but use the existing package. - "-Dabseil-cpp_POPULATED=TRUE" - - ;; Don't fetch the sources. We have already built flatbuffers. - "-Dflatbuffers_POPULATED=TRUE" + ;;; Don't fetch the sources. We have these already + "-Degl_headers_POPULATED=TRUE" + "-Dfp16_headers_POPULATED=TRUE" + "-Dopencl_headers_POPULATED=TRUE" + "-Dopengl_headers_POPULATED=TRUE" + "-Dvulkan_headers_POPULATED=TRUE" "-DFFT2D_SOURCE_DIR=/tmp/fft2d" - "-Dneon2sse_SOURCE_DIR=/tmp/neon2sse" - "-Dneon2sse_BINARY_DIR=/tmp/neon2sse-bin" "-DFARMHASH_SOURCE_DIR=/tmp/farmhash" "-Dgemmlowp_SOURCE_DIR=/tmp/gemmlowp" - (string-append "-DRUY_SOURCE_DIR=" - (assoc-ref %build-inputs "ruy-src"))) + ) #:phases (modify-phases %standard-phases (add-after 'unpack 'chdir (lambda _ (chdir "tensorflow/lite"))) (add-after 'chdir 'copy-sources (lambda* (#:key inputs #:allow-other-keys) - ;; Use external cmake finders instead of these stubs that won't - ;; find anything but the bundled sources. - (delete-file "tools/cmake/modules/Findabsl.cmake") - (delete-file "tools/cmake/modules/Findeigen.cmake") + ;; TO DO: properly use Guix's pthreaqdpool + ;; roght now not using pthreadpool because + ;; we are not enabling xnnpack + (substitute* "CMakeLists.txt" + (("if\\(NOT DEFINED PTHREADPOOL_SOURCE_DIR\\)") + "if(false)")) (substitute* "CMakeLists.txt" - (("find_package\\(eigen REQUIRED") - "find_package(eigen REQUIRED NAMES Eigen3")) - (substitute* "tools/cmake/modules/Findflatbuffers.cmake" - (("get_target_property.*") - (format #false "set(FLATBUFFERS_INCLUDE_DIRS ~a/include)\n" - (assoc-ref inputs "flatbuffers")))) + (("if\\(NOT TARGET pthreadpool\\)") + "if(false)")) ;; Don't fetch source code; we already have everything we need. (substitute* '("tools/cmake/modules/fft2d.cmake" - "tools/cmake/modules/ruy.cmake" "tools/cmake/modules/farmhash.cmake" - "tools/cmake/modules/neon2sse.cmake" "tools/cmake/modules/gemmlowp.cmake") (("OverridableFetchContent_Populate.*") "")) @@ -2718,29 +2742,46 @@ (define-public tensorflow-lite (invoke "tar" "--strip-components=1" "-xf" (assoc-ref inputs "fft2d-src"))) - (copy-recursively (assoc-ref inputs "neon2sse-src") - "/tmp/neon2sse/") (copy-recursively (assoc-ref inputs "gemmlowp-src") "/tmp/gemmlowp/"))) - (add-after 'copy-sources 'prepare-shared-library-build - (lambda _ (chdir "c"))) - (replace 'install - (lambda* (#:key outputs #:allow-other-keys) - (let* ((out (assoc-ref outputs "out")) - (lib (string-append out "/lib")) - (headers (string-append out "/include/tensorflow/lite"))) - (install-file "../build/libtensorflowlite_c.so" lib) - (with-directory-excursion ".." - (for-each - (lambda (file) - (let ((target-dir (string-append headers "/" (dirname file)))) - (install-file file target-dir))) - (find-files "." "\\.h$"))))))))) + + (add-after 'build 'build-shared-library + (lambda* (#:key configure-flags #:allow-other-keys) + (mkdir-p "c") + (with-directory-excursion "c" + (apply invoke "cmake" (append configure-flags (list "../../lite/c"))) + (invoke "cmake" "--build" "." "-j" (number->string + (parallel-job-count)))))) + (add-after 'build-shared-library 'build-benchmark-model + (lambda _ + (invoke "cmake" "--build" "." "--target" "benchmark_model" + "-j" (number->string (parallel-job-count))))) + + (add-after 'install 'install-extra + (lambda* (#:key outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (lib (string-append out "/lib")) + (bin (string-append out "/bin"))) + (install-file "../build/c/libtensorflowlite_c.so" lib) + (install-file "../build/tools/benchmark/benchmark_model" bin)))) + ))) (inputs `(("abseil-cpp" ,abseil-cpp-20200923.3) - ("eigen" ,eigen-for-tensorflow-lite) - ("flatbuffers" ,flatbuffers) - ("python" ,python))) + ("cpuinfo" ,cpuinfo) + ("eigen" ,eigen) + ("fp16" ,fp16) + ("flatbuffers" ,flatbuffers-next-shared) + ;("gemmlowp" ,gemmlowp) ;; TO DO + ("mesa-headers" ,mesa-headers) + ("neon2sse" ,neon2sse) + ("opencl-clhpp" ,opencl-clhpp) + ("opencl-headers" ,opencl-headers) + ("opencl-icd-loader" ,opencl-icd-loader) + ("pthreadpool" ,pthreadpool) + ("python" ,python) + ("ruy" ,ruy) + ;("xnnpack" ,xnnpack) ;; TO DO: properly use Guix's copy of xnnpack + ("vulkan-headers" , vulkan-headers))) (native-inputs `(("pkg-config" ,pkg-config) ("gemmlowp-src" @@ -2756,17 +2797,6 @@ (define-public tensorflow-lite (sha256 (base32 "1sbp8kmr2azwlvfbzryy1frxi99jhsh1nc93bdbxdf8zdgpv0kxl"))))) - ("neon2sse-src" - ,(let ((commit "a1652fd5253afbf3e39357b012974f93511f6108")) - (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/intel/ARM_NEON_2_x86_SSE") - (commit commit))) - (file-name (git-file-name "neon2sse" (string-take commit 8))) - (sha256 - (base32 - "1q8gkxag9wlnwdwad2pclsrkwzrdjy94hyrkayrsvxyj7szb5y8i"))))) ("farmhash-src" ,(let ((commit "816a4ae622e964763ca0862d9dbd19324a1eaf45")) (origin @@ -2782,24 +2812,12 @@ (define-public tensorflow-lite ,(origin (method url-fetch) (uri (string-append "https://storage.googleapis.com/" - "mirror.tensorflow.org/" - "www.kurims.kyoto-u.ac.jp/~ooura/fft2d.tgz")) + "mirror.tensorflow.org/github.com/petewarden/" + "OouraFFT/archive/v1.0.tar.gz")) (file-name "fft2d.tar.gz") (sha256 (base32 - "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd")))) - ("ruy-src" - ,(let ((commit "9c56af3fce210a8a103eda19bd6f47c08a9e3d90")) - (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/google/ruy") - (commit commit) - (recursive? #true))) - (file-name (git-file-name "ruy" (string-take commit 8))) - (sha256 - (base32 - "1cfd5gk6kaj8kbl3h98gx1ap8czd59y6p8qq8nr28fklpyzf5cis"))))))) + "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd")))))) (home-page "https://tensorflow.org") (synopsis "Machine learning framework") (description diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm index 7904d9e8e3..37480a8397 100644 --- a/gnu/packages/maths.scm +++ b/gnu/packages/maths.scm @@ -85,7 +85,6 @@ (define-module (gnu packages maths) #:use-module (guix packages) #:use-module (guix download) #:use-module (guix git-download) - #:use-module (guix gexp) #:use-module (guix utils) #:use-module ((guix build utils) #:select (alist-replace)) #:use-module (guix build-system ant) diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm index 39954d6764..268a5d3d9b 100644 --- a/gnu/packages/serialization.scm +++ b/gnu/packages/serialization.scm @@ -748,6 +748,18 @@ (define-public flatbuffers-next (base32 "1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz")))))) +(define-public flatbuffers-next-shared + (package + (inherit flatbuffers-next) + (name "flatbuffers-shared") + (version "23.1.21") + (arguments + (substitute-keyword-arguments (package-arguments flatbuffers-next) + ((#:configure-flags flags) + ;; make C++ fiels compiled with -fPIC, needed for shared lib". + #~(cons "-DFLATBUFFERS_CXX_FLAGS=-fPIC" #$flags)))))) + + (define-public python-feather-format (package (name "python-feather-format") base-commit: 8937898af9433c0b7a8d95a78c95e21576ae22e8 prerequisite-patch-id: 6a623f7c3bdc733b110fd6f59f31032adb7d987d prerequisite-patch-id: 5e9e341f5896c5eee0f64a7a031cc57491e76eb9 -- 2.40.1 From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 16 01:36:15 2023 Received: (at control) by debbugs.gnu.org; 16 Jun 2023 05:36:15 +0000 Received: from localhost ([127.0.0.1]:48441 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qA286-0003fB-Og for submit@debbugs.gnu.org; Fri, 16 Jun 2023 01:36:15 -0400 Received: from mail-vk1-f171.google.com ([209.85.221.171]:62720) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qA284-0003ew-Mm for control@debbugs.gnu.org; Fri, 16 Jun 2023 01:36:13 -0400 Received: by mail-vk1-f171.google.com with SMTP id 71dfb90a1353d-466021212d0so1161483e0c.1 for ; Thu, 15 Jun 2023 22:36:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=atai-org.20221208.gappssmtp.com; s=20221208; t=1686893767; x=1689485767; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=5djRKLfPp3ZC26JAlBgTEJbsdac1GsvlspPE91mv+fk=; b=mL/1V6ax9BJnbG4jZQPN7KfOUCiZ2/qPKTId8Do+8mf2mSjAmp1EBkCb0rc1h0oq4j tk6recgWBWwsBcJCGmCG34VZgfyEFtHhdo2iUsYvHfUYJ/zWFBQntpA/PogpBUUPch51 iza4C0GYIBsIjvSXoH0J9SrsVu9gqcvKPl1qQdZEb84w/hgZL+rHrKLvj15oTpR1IeZc mctbESZiLVF5XlN4alZgVMU0ftkdkNy23DUb515yLQs/ZZn/5xeGUxjhLN88kUA3cB0R UEGQohWbqU5fFuIxsIfR+XEX1u6hhJHJWw5Fn/48ZF7RKOyqS7+n1SrxRk9kj86pk+2q qTMw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686893767; x=1689485767; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=5djRKLfPp3ZC26JAlBgTEJbsdac1GsvlspPE91mv+fk=; b=Qnj9uVJzN7jbkdwLa9LwarOBUXtFtkWg42aUuvAk1qrVInOrzKsrYA7kivoSs9oDc8 2xTozxUUbkOkQkgIIoAWmyu6fdRavtiu630sVXAemvoUQ/LrLn2A/LZgUpFuV+Nmi3bC 6YH7Y3LxtrWTrWt696nfSEyI1hj+Pnyy3AGxow6PQgHswevOFQgGHCdY5WiQ1JpI9V95 zXGqceufvGltUP4ZCty49Lt6Wzginec+Wthq1qKirGiNsj1gjqtNohoOjUJTwVpdO0gm /ypRzAHZrZ77Hg6uMBQj0GpBqw0rWVIunEV2/PSLMGcmEtm35/hc6V14bbJ9NhH6oMUK TRvA== X-Gm-Message-State: AC+VfDxU8hCp0aG5sb/exuR96Q6nzJ1Lr/XSgjmFoCQdeyyyAM0mvifp EEhg4Fk5w6m+DoIb8tOylEiTBao7OQ/je6RoVSieJ9xk X-Google-Smtp-Source: ACHHUZ7y007FR6uv3RGlaGHijYBfJ/U3/3hZomiv4wkZwDZTMvU+QGbYepC/rwjZ2QGxtplA+162EcO6EHEskv39SCc= X-Received: by 2002:a05:6122:99a:b0:46e:51bc:310b with SMTP id g26-20020a056122099a00b0046e51bc310bmr3457354vkd.4.1686893766702; Thu, 15 Jun 2023 22:36:06 -0700 (PDT) MIME-Version: 1.0 From: Andy Tai Date: Thu, 15 Jun 2023 22:35:30 -0700 Message-ID: Subject: To: GNU bug tracker automated control server Content-Type: text/plain; charset="UTF-8" X-Spam-Score: 2.2 (++) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: block 63897 by 56202 Content analysis details: (2.2 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (lichengtai[at]gmail.com) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.2 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [209.85.221.171 listed in wl.mailspike.net] -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [209.85.221.171 listed in list.dnswl.org] 0.0 FREEMAIL_FORGED_FROMDOMAIN 2nd level domains in From and EnvelopeFrom freemail headers are different 2.0 BLANK_SUBJECT Subject is present but empty X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 1.2 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: block 63897 by 56202 Content analysis details: (1.2 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_MSPIKE_H2 RBL: Average reputation (+2) [209.85.221.171 listed in wl.mailspike.net] -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/, no trust [209.85.221.171 listed in list.dnswl.org] 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (lichengtai[at]gmail.com) 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record -0.0 SPF_PASS SPF: sender matches SPF record 0.2 HEADER_FROM_DIFFERENT_DOMAINS From and EnvelopeFrom 2nd level mail domains are different 0.0 FREEMAIL_FORGED_FROMDOMAIN 2nd level domains in From and EnvelopeFrom freemail headers are different 2.0 BLANK_SUBJECT Subject is present but empty -1.0 MAILING_LIST_MULTI Multiple indicators imply a widely-seen list manager block 63897 by 56202 From debbugs-submit-bounces@debbugs.gnu.org Fri Jun 16 02:15:24 2023 Received: (at 63897) by debbugs.gnu.org; 16 Jun 2023 06:15:24 +0000 Received: from localhost ([127.0.0.1]:48502 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qA2jz-0007JI-D0 for submit@debbugs.gnu.org; Fri, 16 Jun 2023 02:15:24 -0400 Received: from mail-pl1-f179.google.com ([209.85.214.179]:42166) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qA2jw-0007Ix-Ap for 63897@debbugs.gnu.org; Fri, 16 Jun 2023 02:15:22 -0400 Received: by mail-pl1-f179.google.com with SMTP id d9443c01a7336-1b3be39e666so3428255ad.0 for <63897@debbugs.gnu.org>; Thu, 15 Jun 2023 23:15:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1686896114; x=1689488114; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:from:to:cc:subject:date:message-id:reply-to; bh=NHa+cQJ6TD/uo4osvVdVL3W6DB2iI1+qo5i/bHNfsEk=; b=Gq0fgegbx6KF6SwUYQ9Wm/uxBWGiikrvRcN7h73Nk6WvdkrA5qRvArRtCHGGtympvr 30LRilqFPhQxKln7ZymVurwOFp8QBEcBbQYkmEY3QejG1b2twpHB+1LQuDDqqvPdHt08 Zn67J4jc5zEaGRyl2+3gRWnsCwDCX/+pfAyGdNXuBJmh9HMnT13Y5/2MTcJJjApeaBYY RYsA5hOMQWaP7pKTa8IV1HSZusV0lV1ULpttLGRseI4v6mCJdqxjsn+Fs3EG21tEXazD v8U0EgPIuWbIMY2eQ7K9H86E+51CuRKqGoLnLMei/1XxRyb8npx2cJbvKSAP2R801fKK sn3w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1686896114; x=1689488114; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=NHa+cQJ6TD/uo4osvVdVL3W6DB2iI1+qo5i/bHNfsEk=; b=CxIyFjO8xEgKLc5F6Md0lBfrY8n9qy1z34XQGEpRX0PaqoDrxt2NY/3e2cQ4T5LbkT iHP7hrqPE0St4cTA2plU6HPYkiYtpmNJHrnj82qK5sOpBHhjj/UYO4C+xerTj3DVwG+M CK1IMWPPRcNDS74EG9C3S7qfCtwA76dT82jMuKl403BoQmizAin7hZyQS7nXIcEDkmy2 9/YSQUVWQcNMZJKaL4XdI5OG72/6GPJAUVopcXiqtXeR5yxAWbq2ysbaL5OWNFdte3Zm uQfMQU9ORx85ZtVlNbUCfVdNdgLzCv112Nv9coHZZfBNHak9mkMf2enM2TGryC1Si0Bq lCjw== X-Gm-Message-State: AC+VfDwR7KQB1Sc+M15zuGCONGeojHcX4UKhmsZZbXk85MmPtannk3Vk GnRxqOd4o+8FRzr59ZmxpwbRXlJuVB0= X-Google-Smtp-Source: ACHHUZ5K2HmrJfoBfaoCi4DX+PliTQEx7M2Pt1k9QCN+mOvZjQTkCDVDV940gzE4Xr+s3lq6UzCeJQ== X-Received: by 2002:a17:902:f68a:b0:1b4:5699:aac8 with SMTP id l10-20020a170902f68a00b001b45699aac8mr8593483plg.21.1686896113741; Thu, 15 Jun 2023 23:15:13 -0700 (PDT) Received: from localhost.localdomain ([98.37.226.98]) by smtp.googlemail.com with ESMTPSA id jo6-20020a170903054600b001aae625e422sm14998683plb.37.2023.06.15.23.15.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 15 Jun 2023 23:15:13 -0700 (PDT) From: Andy Tai To: 63897@debbugs.gnu.org Subject: [PATCH v4] gnu: tensorflow-lite: Update to 2.12.0 Date: Thu, 15 Jun 2023 23:15:09 -0700 Message-Id: <91734fdfc45da111d2a58ae4a1e4f09abf4d298d.1686895970.git.atai@atai.org> X-Mailer: git-send-email 2.40.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.3 (/) X-Debbugs-Envelope-To: 63897 Cc: Andy Tai X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.7 (/) * gnu/packages/algebra.scm (eigen-for-tensorflow-lite): Remove variable * gnu/packages/machine-learning.scm (tensorflow-lite): Update to 2.12.0 [arguments] (configure-flags): Turn on GPU and RUY use; Use cmake to find packages already in Guix as dependencies for absl-cpp, eigen, flatbuffer, neon2ssl, cpuinfo, ruy. [arguments] (phases): Remove operations setting up absl-cpp, eigen, ruy, and neon2ssl as native inouts as now we use Guix's packages of these as dependencies. Do default build phase. Add steps to install C shared library and benchmark_model tool [arguments] (inputs): Add cpuinfo, eigen, fp16, mesa-header, opencl, pthreadpool, ruy, vulkan and xnnpack as explicit inputs [arguments] (native-inputs): Remove local setup of neon2ssl and ruy as we now use Guix's packages of these as explicit dependencies * gnu/packages/maths.scm: Remove extra statement * gnu/packages/serialization.scm (flatbuffers-next-shared): New variable, flatbuffers-next built with -fPIC as needed by tensorflow-lite --- gnu/packages/algebra.scm | 32 ------ gnu/packages/machine-learning.scm | 177 ++++++++++++++++++------------ gnu/packages/maths.scm | 1 - gnu/packages/serialization.scm | 12 ++ 4 files changed, 117 insertions(+), 105 deletions(-) diff --git a/gnu/packages/algebra.scm b/gnu/packages/algebra.scm index 3babf90b3c..81df0c97ac 100644 --- a/gnu/packages/algebra.scm +++ b/gnu/packages/algebra.scm @@ -1155,38 +1155,6 @@ (define-public eigen-for-tensorflow (substitute* "test/CMakeLists.txt" (("ei_add_test\\(stddeque") "#"))))))))))) -(define-public eigen-for-tensorflow-lite - ;; This commit was taken from - ;; tensorflow/lite/tools/cmake/modules/eigen.cmake - (let ((commit "d10b27fe37736d2944630ecd7557cefa95cf87c9") - (revision "1")) - (package (inherit eigen) - (name "eigen-for-tensorflow-lite") - (version (git-version "3.3.7" revision commit)) - (source (origin - (method git-fetch) - (uri (git-reference - (url "https://gitlab.com/libeigen/eigen") - (commit commit))) - (sha256 - (base32 - "0v8a20cwvwmp3hw4275b37frw33v92z0mr8f4dn6y8k0rz92hrrf")) - (file-name (git-file-name name version)) - (modules '((guix build utils))) - (snippet - ;; Ther are test failures in the "unsupported" directory, but - ;; maintainers say it's unsupported anyway, so just skip - ;; them. - '(begin - (substitute* "unsupported/CMakeLists.txt" - (("add_subdirectory\\(test.*") - "# Do not build the tests for unsupported features.\n")))))) - (arguments - (substitute-keyword-arguments (package-arguments eigen) - ((#:phases phases) - `(modify-phases ,phases - (delete 'disable-some-tests)))))))) - (define-public xtensor (package (name "xtensor") diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 927b531cf2..4b915e509e 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -54,12 +54,14 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages) #:use-module (gnu packages adns) #:use-module (gnu packages algebra) + #:use-module (gnu packages assembly) #:use-module (gnu packages audio) #:use-module (gnu packages autotools) #:use-module (gnu packages base) #:use-module (gnu packages bash) #:use-module (gnu packages boost) #:use-module (gnu packages bdw-gc) + #:use-module (gnu packages c) #:use-module (gnu packages check) #:use-module (gnu packages compression) #:use-module (gnu packages cmake) @@ -88,6 +90,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages ninja) #:use-module (gnu packages ocaml) #:use-module (gnu packages onc-rpc) + #:use-module (gnu packages opencl) #:use-module (gnu packages parallel) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) @@ -99,6 +102,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages python-science) #:use-module (gnu packages python-web) #:use-module (gnu packages python-xyz) + #:use-module (gnu packages regex) #:use-module (gnu packages rpc) #:use-module (gnu packages serialization) #:use-module (gnu packages sphinx) @@ -107,6 +111,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages swig) #:use-module (gnu packages time) #:use-module (gnu packages tls) + #:use-module (gnu packages vulkan) #:use-module (gnu packages video) #:use-module (gnu packages web) #:use-module (gnu packages xml) @@ -2833,7 +2838,7 @@ (define-public tensorflow (define-public tensorflow-lite (package (name "tensorflow-lite") - (version "2.5.0") + (version "2.12.0") (source (origin (method git-fetch) @@ -2843,59 +2848,85 @@ (define-public tensorflow-lite (file-name (git-file-name name version)) (sha256 (base32 - "1jdw2i1rq06zqd6aabh7bbm0avsg4pygnfmd7gviv0blhih9054l")))) + "051knmq6ysg4936d9baaqsx6j3rfsp4ffi2pwrxx4llan682rvpi")))) (build-system cmake-build-system) (arguments - `(#:tests? #false ; no "check" target + `(#:tests? #false ;tests not building now #:build-type "Release" + + #:modules ((ice-9 match) + (guix build utils) + (guix build cmake-build-system)) + #:configure-flags (list - "-DTFLITE_ENABLE_GPU=OFF" - "-DTFLITE_ENABLE_RUY=OFF" + ; "-DTFLITE_KERNEL_TEST=ON" ; TO DO: build tests + ;; so cmake can be used to find this from other packages + "-DTFLITE_ENABLE_INSTALL=ON" + + ;; Use Guix's own packages as dependencies. + "-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON" + + "-DTFLITE_ENABLE_GPU=ON" + "-DTFLITE_ENABLE_RUY=ON" + + ;; TO DO: turn on Farmhash + ;"-DSYSTEM_FARMHASH=ON" + + + (string-append "-Dabsl_DIR=" (assoc-ref %build-inputs "abseil-cpp") + "/lib/cmake/absl") + (string-append "-DEigen3_DIR=" (assoc-ref %build-inputs "eigen") + "/share/eigen3/cmake") + (string-append "-DFlatBuffers_DIR=" (assoc-ref %build-inputs "flatbuffers") + "/lib/cmake/flatbuffers") + (string-append "-DNEON_2_SSE_DIR=" (assoc-ref %build-inputs "neon2sse") + "/lib/cmake/NEON_2_SSE") + (string-append "-Dcpuinfo_DIR=" (assoc-ref %build-inputs "cpuinfo") + "/share/cpuinfo") + (string-append "-Druy_DIR=" (assoc-ref %build-inputs "ruy") + "/lib/cmake/ruy") ;; TODO: The build system attempts to build xnnpack from source. We ;; would like to use our xnnpack package here, but this requires more ;; work. "-DTFLITE_ENABLE_XNNPACK=OFF" - ;; Pretend we've already fetched abseil. We won't actually build it - ;; but use the existing package. - "-Dabseil-cpp_POPULATED=TRUE" - - ;; Don't fetch the sources. We have already built flatbuffers. - "-Dflatbuffers_POPULATED=TRUE" + ;;; Don't fetch the sources. We have these already + "-Degl_headers_POPULATED=TRUE" + "-Dfp16_headers_POPULATED=TRUE" + "-Dopencl_headers_POPULATED=TRUE" + "-Dopengl_headers_POPULATED=TRUE" + "-Dvulkan_headers_POPULATED=TRUE" + "-Dgoogletest_POPULATED=TRUE" + "-Dgoogle_benchmark_POPULATED=TRUE" + "-Dnsync_POPULATED=TRUE" + "-Dre2_POPULATED=TRUE" "-DFFT2D_SOURCE_DIR=/tmp/fft2d" - "-Dneon2sse_SOURCE_DIR=/tmp/neon2sse" - "-Dneon2sse_BINARY_DIR=/tmp/neon2sse-bin" "-DFARMHASH_SOURCE_DIR=/tmp/farmhash" "-Dgemmlowp_SOURCE_DIR=/tmp/gemmlowp" - (string-append "-DRUY_SOURCE_DIR=" - (assoc-ref %build-inputs "ruy-src"))) + ) #:phases (modify-phases %standard-phases (add-after 'unpack 'chdir (lambda _ (chdir "tensorflow/lite"))) (add-after 'chdir 'copy-sources (lambda* (#:key inputs #:allow-other-keys) - ;; Use external cmake finders instead of these stubs that won't - ;; find anything but the bundled sources. - (delete-file "tools/cmake/modules/Findabsl.cmake") - (delete-file "tools/cmake/modules/Findeigen.cmake") + ;; TO DO: properly use Guix's pthreaqdpool + ;; roght now not using pthreadpool because + ;; we are not enabling xnnpack (substitute* "CMakeLists.txt" - (("find_package\\(eigen REQUIRED") - "find_package(eigen REQUIRED NAMES Eigen3")) - (substitute* "tools/cmake/modules/Findflatbuffers.cmake" - (("get_target_property.*") - (format #false "set(FLATBUFFERS_INCLUDE_DIRS ~a/include)\n" - (assoc-ref inputs "flatbuffers")))) + (("if\\(NOT DEFINED PTHREADPOOL_SOURCE_DIR\\)") + "if(false)")) + (substitute* "CMakeLists.txt" + (("if\\(NOT TARGET pthreadpool\\)") + "if(false)")) ;; Don't fetch source code; we already have everything we need. (substitute* '("tools/cmake/modules/fft2d.cmake" - "tools/cmake/modules/ruy.cmake" "tools/cmake/modules/farmhash.cmake" - "tools/cmake/modules/neon2sse.cmake" "tools/cmake/modules/gemmlowp.cmake") (("OverridableFetchContent_Populate.*") "")) @@ -2909,31 +2940,56 @@ (define-public tensorflow-lite (invoke "tar" "--strip-components=1" "-xf" (assoc-ref inputs "fft2d-src"))) - (copy-recursively (assoc-ref inputs "neon2sse-src") - "/tmp/neon2sse/") (copy-recursively (assoc-ref inputs "gemmlowp-src") "/tmp/gemmlowp/"))) - (add-after 'copy-sources 'prepare-shared-library-build - (lambda _ (chdir "c"))) - (replace 'install - (lambda* (#:key outputs #:allow-other-keys) - (let* ((out (assoc-ref outputs "out")) - (lib (string-append out "/lib")) - (headers (string-append out "/include/tensorflow/lite"))) - (install-file "../build/libtensorflowlite_c.so" lib) - (with-directory-excursion ".." - (for-each - (lambda (file) - (let ((target-dir (string-append headers "/" (dirname file)))) - (install-file file target-dir))) - (find-files "." "\\.h$"))))))))) + + (add-after 'build 'build-shared-library + (lambda* (#:key configure-flags #:allow-other-keys) + (mkdir-p "c") + (with-directory-excursion "c" + (apply invoke "cmake" (append configure-flags (list "../../lite/c"))) + (invoke "cmake" "--build" "." "-j" (number->string + (parallel-job-count)))))) + (add-after 'build-shared-library 'build-benchmark-model + (lambda _ + (invoke "cmake" "--build" "." "--target" "benchmark_model" + "-j" (number->string (parallel-job-count))))) + + (add-after 'install 'install-extra + (lambda* (#:key outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (lib (string-append out "/lib")) + (bin (string-append out "/bin"))) + (install-file "../build/c/libtensorflowlite_c.so" lib) + (install-file "../build/tools/benchmark/benchmark_model" bin)))) + ; TO DO: test not building now + ; (replace 'check + ; (lambda* (#:key tests? #:allow-other-keys) + ; (when tests? + ; (invoke "ctest" "-L" "plain")))) + ))) (inputs `(("abseil-cpp" ,abseil-cpp-20200923.3) - ("eigen" ,eigen-for-tensorflow-lite) - ("flatbuffers" ,flatbuffers) - ("python" ,python))) + ("cpuinfo" ,cpuinfo) + ("eigen" ,eigen) + ("fp16" ,fp16) + ("flatbuffers" ,flatbuffers-next-shared) + ;("gemmlowp" ,gemmlowp) ;; TO DO + ("mesa-headers" ,mesa-headers) + ("neon2sse" ,neon2sse) + ("nsync" ,nsync) + ("opencl-clhpp" ,opencl-clhpp) + ("opencl-headers" ,opencl-headers) + ("opencl-icd-loader" ,opencl-icd-loader) + ("pthreadpool" ,pthreadpool) + ("python" ,python) + ("ruy" ,ruy) + ("re2" ,re2) + ;("xnnpack" ,xnnpack) ;; TO DO: properly use Guix's copy of xnnpack + ("vulkan-headers" , vulkan-headers))) (native-inputs `(("pkg-config" ,pkg-config) + ("googletest" ,googletest) ("gemmlowp-src" ;; The commit hash is taken from ;; "tensorflow/lite/tools/cmake/modules/gemmlowp.cmake". @@ -2947,17 +3003,6 @@ (define-public tensorflow-lite (sha256 (base32 "1sbp8kmr2azwlvfbzryy1frxi99jhsh1nc93bdbxdf8zdgpv0kxl"))))) - ("neon2sse-src" - ,(let ((commit "a1652fd5253afbf3e39357b012974f93511f6108")) - (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/intel/ARM_NEON_2_x86_SSE") - (commit commit))) - (file-name (git-file-name "neon2sse" (string-take commit 8))) - (sha256 - (base32 - "1q8gkxag9wlnwdwad2pclsrkwzrdjy94hyrkayrsvxyj7szb5y8i"))))) ("farmhash-src" ,(let ((commit "816a4ae622e964763ca0862d9dbd19324a1eaf45")) (origin @@ -2973,24 +3018,12 @@ (define-public tensorflow-lite ,(origin (method url-fetch) (uri (string-append "https://storage.googleapis.com/" - "mirror.tensorflow.org/" - "www.kurims.kyoto-u.ac.jp/~ooura/fft2d.tgz")) + "mirror.tensorflow.org/github.com/petewarden/" + "OouraFFT/archive/v1.0.tar.gz")) (file-name "fft2d.tar.gz") (sha256 (base32 - "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd")))) - ("ruy-src" - ,(let ((commit "9c56af3fce210a8a103eda19bd6f47c08a9e3d90")) - (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/google/ruy") - (commit commit) - (recursive? #true))) - (file-name (git-file-name "ruy" (string-take commit 8))) - (sha256 - (base32 - "1cfd5gk6kaj8kbl3h98gx1ap8czd59y6p8qq8nr28fklpyzf5cis"))))))) + "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd")))))) (home-page "https://tensorflow.org") (synopsis "Machine learning framework") (description diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm index fd0e8eae3a..c7996e12a9 100644 --- a/gnu/packages/maths.scm +++ b/gnu/packages/maths.scm @@ -85,7 +85,6 @@ (define-module (gnu packages maths) #:use-module (guix packages) #:use-module (guix download) #:use-module (guix git-download) - #:use-module (guix gexp) #:use-module (guix utils) #:use-module ((guix build utils) #:select (alist-replace)) #:use-module (guix build-system ant) diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm index 39954d6764..268a5d3d9b 100644 --- a/gnu/packages/serialization.scm +++ b/gnu/packages/serialization.scm @@ -748,6 +748,18 @@ (define-public flatbuffers-next (base32 "1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz")))))) +(define-public flatbuffers-next-shared + (package + (inherit flatbuffers-next) + (name "flatbuffers-shared") + (version "23.1.21") + (arguments + (substitute-keyword-arguments (package-arguments flatbuffers-next) + ((#:configure-flags flags) + ;; make C++ fiels compiled with -fPIC, needed for shared lib". + #~(cons "-DFLATBUFFERS_CXX_FLAGS=-fPIC" #$flags)))))) + + (define-public python-feather-format (package (name "python-feather-format") base-commit: 31336e9f5d68512a9c1c6826bce9f17c892a2125 prerequisite-patch-id: 5e9e341f5896c5eee0f64a7a031cc57491e76eb9 prerequisite-patch-id: 6a623f7c3bdc733b110fd6f59f31032adb7d987d prerequisite-patch-id: f672f4219dc90516d7e83bc2801e4a8dd51a9839 -- 2.40.1 From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 28 04:44:30 2023 Received: (at 63897) by debbugs.gnu.org; 28 Jul 2023 08:44:30 +0000 Received: from localhost ([127.0.0.1]:43597 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qPJ5J-0004Wc-1P for submit@debbugs.gnu.org; Fri, 28 Jul 2023 04:44:29 -0400 Received: from mail-qk1-x72a.google.com ([2607:f8b0:4864:20::72a]:55741) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qPJ5G-0004WK-B5 for 63897@debbugs.gnu.org; Fri, 28 Jul 2023 04:44:27 -0400 Received: by mail-qk1-x72a.google.com with SMTP id af79cd13be357-76c4890a220so142366685a.3 for <63897@debbugs.gnu.org>; Fri, 28 Jul 2023 01:44:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1690533860; x=1691138660; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:from:to:cc:subject:date:message-id:reply-to; bh=qf8jt7cxX75JL2Mj1YNHEvW7auI3ljSMHx8NHlUtAyE=; b=kIrysZTveWvMruGWZFxZcQ6mRNRJQaS0m1wouHHBhV/8k3WmADld/BnCXDv010OwX5 W1uj+fmzNer7jW0yNCeiohs90GKhE3J4Udx+c7qzzX+ckEkeqEJob1kSHRqzpd1FLqpG oc3ckvBAF2ZKM9jpNSwP2jPHwwkFZy69IWKR2hLmkCKU3bkQV3gndWqujehZb8Spa1Dp OXQUcindN9BxElwkIHkmJMh2NWX6tcB8uqj6UeyhYEmqs8UtlB1LJRr0ALaTf6ez83Gw uyW30gghNU4x0ux/IZysYI/qi2XIc2dlZAX1bA31n+AdEoc7iOEa7kGXxNXTKc+u8S7U QPpA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1690533860; x=1691138660; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=qf8jt7cxX75JL2Mj1YNHEvW7auI3ljSMHx8NHlUtAyE=; b=F1rTp0LvzmbL4u7sWSUv1TtLv3iU0fp/XtbbshL61VuOhOn32VVZLdAYiwqjNC9RLw n2qqq6K5foEvxquE5xtkD0Ux9cAWJzTT/Bj0PujUjx6HXmEK9i7sU9pOzUofCNHXoh0Q cTKUEPRVS67I0NAhopvJXEjkXVO71nWH1qWJoiu3SQ4waYnBaUqV91l6hsM4jGdMrr3Y mhR0GmjltUz0B7xaGzkBcY8WW2ERSpfDcWPpO3za0WGqJibKy5WkzeQCGbnUA1bUEkxz Or/WR1JfiFMu7NKVQjYjFAhTUWyOJKiBbQoEX6tx0NKq1VwNmSvV1tFHJgiWQ6A2TOH5 hfKQ== X-Gm-Message-State: ABy/qLadEk9xxyxHivdCEODLytGDa5mrvKlcdYZZNtzfeXJAgLsTYN3X heKS76j2tQLJvt5ifU2pqm8l3vLSPYFC5g== X-Google-Smtp-Source: APBJJlH0kn1mol501PwW+pzorq5APYTj9WgjfMrCGpWismlp+r+euH7lFUPZFp8vb1df3TalGVK0eA== X-Received: by 2002:a05:620a:130b:b0:767:614f:322c with SMTP id o11-20020a05620a130b00b00767614f322cmr1834258qkj.18.1690533860041; Fri, 28 Jul 2023 01:44:20 -0700 (PDT) Received: from localhost.localdomain ([98.37.226.98]) by smtp.googlemail.com with ESMTPSA id y8-20020a17090a474800b0025ea87b97b9sm21516634pjg.0.2023.07.28.01.44.19 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 28 Jul 2023 01:44:19 -0700 (PDT) From: Andy Tai To: 63897@debbugs.gnu.org Subject: [PATCH v5] gnu: tensorflow-lite: Update to 2.12.1 Date: Fri, 28 Jul 2023 01:44:14 -0700 Message-ID: <60f8a10d1291b1283b17af6b034a6aeff1e503a3.1690533806.git.atai@atai.org> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-Debbugs-Cc: Andreas Enge , Efraim Flashner , Eric Bavier Content-Transfer-Encoding: 8bit X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 63897 Cc: Andy Tai X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) * gnu/packages/algebra.scm (eigen-for-tensorflow-lite): Remove variable * gnu/packages/machine-learning.scm (tensorflow-lite): Update to 2.12.1 [arguments] (configure-flags): Turn on GPU and RUY use; Use cmake to find packages already in Guix as dependencies for absl-cpp, eigen, flatbuffer, neon2ssl, cpuinfo, ruy. [arguments] (phases): Remove operations setting up absl-cpp, eigen, ruy, and neon2ssl as native inouts as now we use Guix's packages of these as dependencies. Do default build phase. Add steps to install C shared library and benchmark_model tool [arguments] (inputs): Add cpuinfo, eigen, fp16, mesa-header, opencl, pthreadpool, ruy, vulkan and xnnpack as explicit inputs [arguments] (native-inputs): Remove local setup of neon2ssl and ruy as we now use Guix's packages of these as explicit dependencies * gnu/packages/maths.scm: Remove extra statement * gnu/packages/serialization.scm (flatbuffers-next-shared): New variable, flatbuffers-next built with -fPIC as needed by tensorflow-lite --- gnu/packages/algebra.scm | 32 ------ gnu/packages/machine-learning.scm | 177 ++++++++++++++++++------------ gnu/packages/maths.scm | 1 - gnu/packages/serialization.scm | 12 ++ 4 files changed, 117 insertions(+), 105 deletions(-) diff --git a/gnu/packages/algebra.scm b/gnu/packages/algebra.scm index a717750c7b..b1ab755144 100644 --- a/gnu/packages/algebra.scm +++ b/gnu/packages/algebra.scm @@ -1154,38 +1154,6 @@ (define-public eigen-for-tensorflow (substitute* "test/CMakeLists.txt" (("ei_add_test\\(stddeque") "#"))))))))))) -(define-public eigen-for-tensorflow-lite - ;; This commit was taken from - ;; tensorflow/lite/tools/cmake/modules/eigen.cmake - (let ((commit "d10b27fe37736d2944630ecd7557cefa95cf87c9") - (revision "1")) - (package (inherit eigen) - (name "eigen-for-tensorflow-lite") - (version (git-version "3.3.7" revision commit)) - (source (origin - (method git-fetch) - (uri (git-reference - (url "https://gitlab.com/libeigen/eigen") - (commit commit))) - (sha256 - (base32 - "0v8a20cwvwmp3hw4275b37frw33v92z0mr8f4dn6y8k0rz92hrrf")) - (file-name (git-file-name name version)) - (modules '((guix build utils))) - (snippet - ;; Ther are test failures in the "unsupported" directory, but - ;; maintainers say it's unsupported anyway, so just skip - ;; them. - '(begin - (substitute* "unsupported/CMakeLists.txt" - (("add_subdirectory\\(test.*") - "# Do not build the tests for unsupported features.\n")))))) - (arguments - (substitute-keyword-arguments (package-arguments eigen) - ((#:phases phases) - `(modify-phases ,phases - (delete 'disable-some-tests)))))))) - (define-public xtensor (package (name "xtensor") diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index e8225c76f3..6ae9c52625 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -55,12 +55,14 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages) #:use-module (gnu packages adns) #:use-module (gnu packages algebra) + #:use-module (gnu packages assembly) #:use-module (gnu packages audio) #:use-module (gnu packages autotools) #:use-module (gnu packages base) #:use-module (gnu packages bash) #:use-module (gnu packages boost) #:use-module (gnu packages bdw-gc) + #:use-module (gnu packages c) #:use-module (gnu packages check) #:use-module (gnu packages compression) #:use-module (gnu packages cmake) @@ -89,6 +91,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages ninja) #:use-module (gnu packages ocaml) #:use-module (gnu packages onc-rpc) + #:use-module (gnu packages opencl) #:use-module (gnu packages parallel) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) @@ -100,6 +103,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages python-science) #:use-module (gnu packages python-web) #:use-module (gnu packages python-xyz) + #:use-module (gnu packages regex) #:use-module (gnu packages rpc) #:use-module (gnu packages serialization) #:use-module (gnu packages sphinx) @@ -108,6 +112,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages swig) #:use-module (gnu packages time) #:use-module (gnu packages tls) + #:use-module (gnu packages vulkan) #:use-module (gnu packages video) #:use-module (gnu packages web) #:use-module (gnu packages xml) @@ -2908,7 +2913,7 @@ (define-public tensorflow (define-public tensorflow-lite (package (name "tensorflow-lite") - (version "2.5.0") + (version "2.12.1") (source (origin (method git-fetch) @@ -2918,59 +2923,85 @@ (define-public tensorflow-lite (file-name (git-file-name name version)) (sha256 (base32 - "1jdw2i1rq06zqd6aabh7bbm0avsg4pygnfmd7gviv0blhih9054l")))) + "0jkgljdagdqllnxygl35r5bh3f9qmbczymfj357gm9krh59g2kmd")))) (build-system cmake-build-system) (arguments - `(#:tests? #false ; no "check" target + `(#:tests? #false ;tests not building now #:build-type "Release" + + #:modules ((ice-9 match) + (guix build utils) + (guix build cmake-build-system)) + #:configure-flags (list - "-DTFLITE_ENABLE_GPU=OFF" - "-DTFLITE_ENABLE_RUY=OFF" + ; "-DTFLITE_KERNEL_TEST=ON" ; TO DO: build tests + ;; so cmake can be used to find this from other packages + "-DTFLITE_ENABLE_INSTALL=ON" + + ;; Use Guix's own packages as dependencies. + "-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON" + + "-DTFLITE_ENABLE_GPU=ON" + "-DTFLITE_ENABLE_RUY=ON" + + ;; TO DO: turn on Farmhash + ;"-DSYSTEM_FARMHASH=ON" + + + (string-append "-Dabsl_DIR=" (assoc-ref %build-inputs "abseil-cpp") + "/lib/cmake/absl") + (string-append "-DEigen3_DIR=" (assoc-ref %build-inputs "eigen") + "/share/eigen3/cmake") + (string-append "-DFlatBuffers_DIR=" (assoc-ref %build-inputs "flatbuffers") + "/lib/cmake/flatbuffers") + (string-append "-DNEON_2_SSE_DIR=" (assoc-ref %build-inputs "neon2sse") + "/lib/cmake/NEON_2_SSE") + (string-append "-Dcpuinfo_DIR=" (assoc-ref %build-inputs "cpuinfo") + "/share/cpuinfo") + (string-append "-Druy_DIR=" (assoc-ref %build-inputs "ruy") + "/lib/cmake/ruy") ;; TODO: The build system attempts to build xnnpack from source. We ;; would like to use our xnnpack package here, but this requires more ;; work. "-DTFLITE_ENABLE_XNNPACK=OFF" - ;; Pretend we've already fetched abseil. We won't actually build it - ;; but use the existing package. - "-Dabseil-cpp_POPULATED=TRUE" - - ;; Don't fetch the sources. We have already built flatbuffers. - "-Dflatbuffers_POPULATED=TRUE" + ;;; Don't fetch the sources. We have these already + "-Degl_headers_POPULATED=TRUE" + "-Dfp16_headers_POPULATED=TRUE" + "-Dopencl_headers_POPULATED=TRUE" + "-Dopengl_headers_POPULATED=TRUE" + "-Dvulkan_headers_POPULATED=TRUE" + "-Dgoogletest_POPULATED=TRUE" + "-Dgoogle_benchmark_POPULATED=TRUE" + "-Dnsync_POPULATED=TRUE" + "-Dre2_POPULATED=TRUE" "-DFFT2D_SOURCE_DIR=/tmp/fft2d" - "-Dneon2sse_SOURCE_DIR=/tmp/neon2sse" - "-Dneon2sse_BINARY_DIR=/tmp/neon2sse-bin" "-DFARMHASH_SOURCE_DIR=/tmp/farmhash" "-Dgemmlowp_SOURCE_DIR=/tmp/gemmlowp" - (string-append "-DRUY_SOURCE_DIR=" - (assoc-ref %build-inputs "ruy-src"))) + ) #:phases (modify-phases %standard-phases (add-after 'unpack 'chdir (lambda _ (chdir "tensorflow/lite"))) (add-after 'chdir 'copy-sources (lambda* (#:key inputs #:allow-other-keys) - ;; Use external cmake finders instead of these stubs that won't - ;; find anything but the bundled sources. - (delete-file "tools/cmake/modules/Findabsl.cmake") - (delete-file "tools/cmake/modules/Findeigen.cmake") + ;; TO DO: properly use Guix's pthreaqdpool + ;; roght now not using pthreadpool because + ;; we are not enabling xnnpack (substitute* "CMakeLists.txt" - (("find_package\\(eigen REQUIRED") - "find_package(eigen REQUIRED NAMES Eigen3")) - (substitute* "tools/cmake/modules/Findflatbuffers.cmake" - (("get_target_property.*") - (format #false "set(FLATBUFFERS_INCLUDE_DIRS ~a/include)\n" - (assoc-ref inputs "flatbuffers")))) + (("if\\(NOT DEFINED PTHREADPOOL_SOURCE_DIR\\)") + "if(false)")) + (substitute* "CMakeLists.txt" + (("if\\(NOT TARGET pthreadpool\\)") + "if(false)")) ;; Don't fetch source code; we already have everything we need. (substitute* '("tools/cmake/modules/fft2d.cmake" - "tools/cmake/modules/ruy.cmake" "tools/cmake/modules/farmhash.cmake" - "tools/cmake/modules/neon2sse.cmake" "tools/cmake/modules/gemmlowp.cmake") (("OverridableFetchContent_Populate.*") "")) @@ -2984,31 +3015,56 @@ (define-public tensorflow-lite (invoke "tar" "--strip-components=1" "-xf" (assoc-ref inputs "fft2d-src"))) - (copy-recursively (assoc-ref inputs "neon2sse-src") - "/tmp/neon2sse/") (copy-recursively (assoc-ref inputs "gemmlowp-src") "/tmp/gemmlowp/"))) - (add-after 'copy-sources 'prepare-shared-library-build - (lambda _ (chdir "c"))) - (replace 'install - (lambda* (#:key outputs #:allow-other-keys) - (let* ((out (assoc-ref outputs "out")) - (lib (string-append out "/lib")) - (headers (string-append out "/include/tensorflow/lite"))) - (install-file "../build/libtensorflowlite_c.so" lib) - (with-directory-excursion ".." - (for-each - (lambda (file) - (let ((target-dir (string-append headers "/" (dirname file)))) - (install-file file target-dir))) - (find-files "." "\\.h$"))))))))) + + (add-after 'build 'build-shared-library + (lambda* (#:key configure-flags #:allow-other-keys) + (mkdir-p "c") + (with-directory-excursion "c" + (apply invoke "cmake" (append configure-flags (list "../../lite/c"))) + (invoke "cmake" "--build" "." "-j" (number->string + (parallel-job-count)))))) + (add-after 'build-shared-library 'build-benchmark-model + (lambda _ + (invoke "cmake" "--build" "." "--target" "benchmark_model" + "-j" (number->string (parallel-job-count))))) + + (add-after 'install 'install-extra + (lambda* (#:key outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (lib (string-append out "/lib")) + (bin (string-append out "/bin"))) + (install-file "../build/c/libtensorflowlite_c.so" lib) + (install-file "../build/tools/benchmark/benchmark_model" bin)))) + ; TO DO: test not building now + ; (replace 'check + ; (lambda* (#:key tests? #:allow-other-keys) + ; (when tests? + ; (invoke "ctest" "-L" "plain")))) + ))) (inputs `(("abseil-cpp" ,abseil-cpp-20200923.3) - ("eigen" ,eigen-for-tensorflow-lite) - ("flatbuffers" ,flatbuffers) - ("python" ,python))) + ("cpuinfo" ,cpuinfo) + ("eigen" ,eigen) + ("fp16" ,fp16) + ("flatbuffers" ,flatbuffers-next-shared) + ;("gemmlowp" ,gemmlowp) ;; TO DO + ("mesa-headers" ,mesa-headers) + ("neon2sse" ,neon2sse) + ("nsync" ,nsync) + ("opencl-clhpp" ,opencl-clhpp) + ("opencl-headers" ,opencl-headers) + ("opencl-icd-loader" ,opencl-icd-loader) + ("pthreadpool" ,pthreadpool) + ("python" ,python) + ("ruy" ,ruy) + ("re2" ,re2) + ;("xnnpack" ,xnnpack) ;; TO DO: properly use Guix's copy of xnnpack + ("vulkan-headers" , vulkan-headers))) (native-inputs `(("pkg-config" ,pkg-config) + ("googletest" ,googletest) ("gemmlowp-src" ;; The commit hash is taken from ;; "tensorflow/lite/tools/cmake/modules/gemmlowp.cmake". @@ -3022,17 +3078,6 @@ (define-public tensorflow-lite (sha256 (base32 "1sbp8kmr2azwlvfbzryy1frxi99jhsh1nc93bdbxdf8zdgpv0kxl"))))) - ("neon2sse-src" - ,(let ((commit "a1652fd5253afbf3e39357b012974f93511f6108")) - (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/intel/ARM_NEON_2_x86_SSE") - (commit commit))) - (file-name (git-file-name "neon2sse" (string-take commit 8))) - (sha256 - (base32 - "1q8gkxag9wlnwdwad2pclsrkwzrdjy94hyrkayrsvxyj7szb5y8i"))))) ("farmhash-src" ,(let ((commit "816a4ae622e964763ca0862d9dbd19324a1eaf45")) (origin @@ -3048,24 +3093,12 @@ (define-public tensorflow-lite ,(origin (method url-fetch) (uri (string-append "https://storage.googleapis.com/" - "mirror.tensorflow.org/" - "www.kurims.kyoto-u.ac.jp/~ooura/fft2d.tgz")) + "mirror.tensorflow.org/github.com/petewarden/" + "OouraFFT/archive/v1.0.tar.gz")) (file-name "fft2d.tar.gz") (sha256 (base32 - "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd")))) - ("ruy-src" - ,(let ((commit "9c56af3fce210a8a103eda19bd6f47c08a9e3d90")) - (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/google/ruy") - (commit commit) - (recursive? #true))) - (file-name (git-file-name "ruy" (string-take commit 8))) - (sha256 - (base32 - "1cfd5gk6kaj8kbl3h98gx1ap8czd59y6p8qq8nr28fklpyzf5cis"))))))) + "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd")))))) (home-page "https://tensorflow.org") (synopsis "Machine learning framework") (description diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm index b2cb0a7308..4c92b22bac 100644 --- a/gnu/packages/maths.scm +++ b/gnu/packages/maths.scm @@ -86,7 +86,6 @@ (define-module (gnu packages maths) #:use-module (guix packages) #:use-module (guix download) #:use-module (guix git-download) - #:use-module (guix gexp) #:use-module (guix utils) #:use-module ((guix build utils) #:select (alist-replace)) #:use-module (guix build-system ant) diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm index 1f8c128eba..0a45920894 100644 --- a/gnu/packages/serialization.scm +++ b/gnu/packages/serialization.scm @@ -807,6 +807,18 @@ (define-public flatbuffers-next (base32 "1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz")))))) +(define-public flatbuffers-next-shared + (package + (inherit flatbuffers-next) + (name "flatbuffers-shared") + (version "23.1.21") + (arguments + (substitute-keyword-arguments (package-arguments flatbuffers-next) + ((#:configure-flags flags) + ;; make C++ fiels compiled with -fPIC, needed for shared lib". + #~(cons "-DFLATBUFFERS_CXX_FLAGS=-fPIC" #$flags)))))) + + (define-public python-feather-format (package (name "python-feather-format") base-commit: 35c8d631d2bd63c64e863b5509f29fdcb7f856e6 prerequisite-patch-id: 36485dd2de3da6fe04218b9a9fb621d749609628 prerequisite-patch-id: 6a623f7c3bdc733b110fd6f59f31032adb7d987d prerequisite-patch-id: 4b8245be9f9ab7b2cf9713a404b7d1632d945278 -- 2.41.0 From debbugs-submit-bounces@debbugs.gnu.org Mon Aug 07 11:28:08 2023 Received: (at 63897) by debbugs.gnu.org; 7 Aug 2023 15:28:08 +0000 Received: from localhost ([127.0.0.1]:34104 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qT29Q-0000qO-HD for submit@debbugs.gnu.org; Mon, 07 Aug 2023 11:28:08 -0400 Received: from hera.aquilenet.fr ([185.233.100.1]:45964) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qT29O-0000pu-44 for 63897@debbugs.gnu.org; Mon, 07 Aug 2023 11:28:07 -0400 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id BF8E3E09; Mon, 7 Aug 2023 17:27:59 +0200 (CEST) X-Virus-Scanned: Debian amavisd-new at hera.aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QMjxbH9V8alB; Mon, 7 Aug 2023 17:27:59 +0200 (CEST) Received: from jurong (unknown [IPv6:2001:861:c4:f2f0::c64]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 1BEAD123; Mon, 7 Aug 2023 17:27:59 +0200 (CEST) Date: Mon, 7 Aug 2023 17:27:57 +0200 From: Andreas Enge To: Andy Tai Subject: Re: [bug#63897] [PATCH v5] gnu: tensorflow-lite: Update to 2.12.1 Message-ID: References: <60f8a10d1291b1283b17af6b034a6aeff1e503a3.1690533806.git.atai@atai.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <60f8a10d1291b1283b17af6b034a6aeff1e503a3.1690533806.git.atai@atai.org> X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 63897 Cc: Ricardo Wurmus , 63897@debbugs.gnu.org, Efraim Flashner , Eric Bavier X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hello, it looks like I was in cc because of your touching maths.scm and/or algebra.scm. I am definitely fine with removing eigen-for-tensorflow if it is not needed any more! Removal of the duplicate line in maths.scm is unrelated and should be done in a separate commit, which I have just pushed; thanks for noticing it! However I do not feel competent about tensorflow itself, so I am not going to review your patch. I am adding Ricardo in cc, who has been working on the package in the past. Andreas From debbugs-submit-bounces@debbugs.gnu.org Sun Sep 03 00:55:43 2023 Received: (at 63897) by debbugs.gnu.org; 3 Sep 2023 04:55:43 +0000 Received: from localhost ([127.0.0.1]:39052 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qcf9C-0005J6-Bn for submit@debbugs.gnu.org; Sun, 03 Sep 2023 00:55:43 -0400 Received: from mail-pl1-x62f.google.com ([2607:f8b0:4864:20::62f]:49533) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qcf99-0005Is-Lq for 63897@debbugs.gnu.org; Sun, 03 Sep 2023 00:55:41 -0400 Received: by mail-pl1-x62f.google.com with SMTP id d9443c01a7336-1c06f6f98c0so3110855ad.3 for <63897@debbugs.gnu.org>; Sat, 02 Sep 2023 21:55:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1693716923; x=1694321723; darn=debbugs.gnu.org; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:from:to:cc:subject:date:message-id:reply-to; bh=zySLwNhukDldcNlbGJCxtCSTcs7XqwaSsNxzTkX/ZEU=; b=qTsb3LRUjquOdfg2BgeDI68Rsf8ys7pjJyP6UKWSQoHZ7pLz8rpeA8/vsjMT8usBgp T7sASWfonPRcpxzCV/BmEZudrO90N1hSl8QVRIR2GUOdsCMzDmB97wc3GK+kFHd4aZEb 2LYuhtnAuIxPhCWSvOtSxinmY8vkkjo0LlYsXq9tOHwuIyEkbmylVX05bSO/Qe0Hjg2o DAAkZkNkrth2YPK8FVr9vSfM2ncC+wkjGp5QtoY30D1IzXZ3YjJolgaMyIhxG/HwYidA mJhDPiRN5t0hTExyq9ZZwtkZezM7D5Mt+2BSJgMJrzGTL1M/DOLZKkcfAbjTEFsuWjkP qoWw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693716923; x=1694321723; h=content-transfer-encoding:mime-version:message-id:date:subject:cc :to:from:sender:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=zySLwNhukDldcNlbGJCxtCSTcs7XqwaSsNxzTkX/ZEU=; b=gF+RqXixnTWKIWbtF24K1Fv4Gg2040RKrolGHjxNddRJV3DlU3DTAUL4uIeuV0Xlsn OmFx3InzyV6Y8o09FhLrEIGxm+OYqGBPFOQh8KFnAiXEkAFBJc6zRBf3n3nVC8rzAQ9y MkNCAEsDUL+c9Ra+pF2MEI4duiDqjoizDS4MzxrUDlZ2zC1gMivdwmawgqwnuH+4kMWi 4dDk0S5migo4XAk/NcUklgvzuYFwPXixksbxSQrZyWs+D0VOrxoqmhSivuM3gxsu1Hki +IdfOjZzkXp6phLKeIrx0rdS9Ka0k9zwMspR1ny5K2G16yH390H6Mzj1wY/B805Q7stf 2ykA== X-Gm-Message-State: AOJu0Yx09myyiVFg2WbcBnhvQyuVGybB6Rg4JMHSWB+ot68oe9yk1ayh 6+NmFicaoN/VhJ/cMRIXKAbMag+rrh4FfA== X-Google-Smtp-Source: AGHT+IFEX+IBxvnNpVCHYyoqhWwwUfOGp1QhiuoKtGU4wSNO5AAhi0l38d7sP01r6tDurWEUf8cGOg== X-Received: by 2002:a17:902:dac4:b0:1bc:6c8:cde3 with SMTP id q4-20020a170902dac400b001bc06c8cde3mr8290498plx.57.1693716922793; Sat, 02 Sep 2023 21:55:22 -0700 (PDT) Received: from localhost.localdomain ([98.37.226.98]) by smtp.googlemail.com with ESMTPSA id t11-20020a170902a5cb00b001bfd92ec592sm5315393plq.292.2023.09.02.21.55.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 02 Sep 2023 21:55:22 -0700 (PDT) From: Andy Tai To: 63897@debbugs.gnu.org Subject: [PATCH v6] gnu: tensorflow-lite: Update to 2.12.1 Date: Sat, 2 Sep 2023 21:55:18 -0700 Message-ID: <9c48f700175bf6dfc257faf34de9e8abfae4ca23.1693716893.git.atai@atai.org> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 X-Debbugs-Cc: Andreas Enge , Efraim Flashner , Eric Bavier Content-Transfer-Encoding: 8bit X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 63897 Cc: Andy Tai X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) * gnu/packages/algebra.scm (eigen-for-tensorflow-lite): Remove variable. * gnu/packages/machine-learning.scm (tensorflow-lite): Update to 2.12.1 [arguments] (configure-flags): Turn on GPU and RUY use; Use cmake to find packages already in Guix as dependencies for absl-cpp, eigen, flatbuffer, neon2ssl, cpuinfo, ruy. [arguments] (phases): Remove operations setting up absl-cpp, eigen, ruy, and neon2ssl as native inouts as now we use Guix's packages of these as dependencies. Do default build phase. Add steps to install C shared library and benchmark_model tool. [arguments] (inputs): Add cpuinfo, eigen, fp16, mesa-header, opencl, pthreadpool, ruy, vulkan and xnnpack as explicit inputs. [arguments] (native-inputs): Remove local setup of neon2ssl and ruy as we now use Guix's packages of these as explicit dependencies. * gnu/packages/serialization.scm (flatbuffers-next-shared): New variable, flatbuffers-next built with -fPIC as needed by tensorflow-lite. --- gnu/packages/algebra.scm | 32 ------ gnu/packages/machine-learning.scm | 177 ++++++++++++++++++------------ gnu/packages/serialization.scm | 12 ++ 3 files changed, 117 insertions(+), 104 deletions(-) diff --git a/gnu/packages/algebra.scm b/gnu/packages/algebra.scm index a717750c7b..b1ab755144 100644 --- a/gnu/packages/algebra.scm +++ b/gnu/packages/algebra.scm @@ -1154,38 +1154,6 @@ (define-public eigen-for-tensorflow (substitute* "test/CMakeLists.txt" (("ei_add_test\\(stddeque") "#"))))))))))) -(define-public eigen-for-tensorflow-lite - ;; This commit was taken from - ;; tensorflow/lite/tools/cmake/modules/eigen.cmake - (let ((commit "d10b27fe37736d2944630ecd7557cefa95cf87c9") - (revision "1")) - (package (inherit eigen) - (name "eigen-for-tensorflow-lite") - (version (git-version "3.3.7" revision commit)) - (source (origin - (method git-fetch) - (uri (git-reference - (url "https://gitlab.com/libeigen/eigen") - (commit commit))) - (sha256 - (base32 - "0v8a20cwvwmp3hw4275b37frw33v92z0mr8f4dn6y8k0rz92hrrf")) - (file-name (git-file-name name version)) - (modules '((guix build utils))) - (snippet - ;; Ther are test failures in the "unsupported" directory, but - ;; maintainers say it's unsupported anyway, so just skip - ;; them. - '(begin - (substitute* "unsupported/CMakeLists.txt" - (("add_subdirectory\\(test.*") - "# Do not build the tests for unsupported features.\n")))))) - (arguments - (substitute-keyword-arguments (package-arguments eigen) - ((#:phases phases) - `(modify-phases ,phases - (delete 'disable-some-tests)))))))) - (define-public xtensor (package (name "xtensor") diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 712abe9aff..f4d4b45084 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -55,12 +55,14 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages) #:use-module (gnu packages adns) #:use-module (gnu packages algebra) + #:use-module (gnu packages assembly) #:use-module (gnu packages audio) #:use-module (gnu packages autotools) #:use-module (gnu packages base) #:use-module (gnu packages bash) #:use-module (gnu packages boost) #:use-module (gnu packages bdw-gc) + #:use-module (gnu packages c) #:use-module (gnu packages check) #:use-module (gnu packages compression) #:use-module (gnu packages cmake) @@ -89,6 +91,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages ninja) #:use-module (gnu packages ocaml) #:use-module (gnu packages onc-rpc) + #:use-module (gnu packages opencl) #:use-module (gnu packages parallel) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) @@ -100,6 +103,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages python-science) #:use-module (gnu packages python-web) #:use-module (gnu packages python-xyz) + #:use-module (gnu packages regex) #:use-module (gnu packages rpc) #:use-module (gnu packages serialization) #:use-module (gnu packages sphinx) @@ -108,6 +112,7 @@ (define-module (gnu packages machine-learning) #:use-module (gnu packages swig) #:use-module (gnu packages time) #:use-module (gnu packages tls) + #:use-module (gnu packages vulkan) #:use-module (gnu packages video) #:use-module (gnu packages web) #:use-module (gnu packages xml) @@ -2936,7 +2941,7 @@ (define-public tensorflow (define-public tensorflow-lite (package (name "tensorflow-lite") - (version "2.5.0") + (version "2.12.1") (source (origin (method git-fetch) @@ -2946,59 +2951,85 @@ (define-public tensorflow-lite (file-name (git-file-name name version)) (sha256 (base32 - "1jdw2i1rq06zqd6aabh7bbm0avsg4pygnfmd7gviv0blhih9054l")))) + "0jkgljdagdqllnxygl35r5bh3f9qmbczymfj357gm9krh59g2kmd")))) (build-system cmake-build-system) (arguments - `(#:tests? #false ; no "check" target + `(#:tests? #false ;tests not building now #:build-type "Release" + + #:modules ((ice-9 match) + (guix build utils) + (guix build cmake-build-system)) + #:configure-flags (list - "-DTFLITE_ENABLE_GPU=OFF" - "-DTFLITE_ENABLE_RUY=OFF" + ; "-DTFLITE_KERNEL_TEST=ON" ; TO DO: build tests + ;; so cmake can be used to find this from other packages + "-DTFLITE_ENABLE_INSTALL=ON" + + ;; Use Guix's own packages as dependencies. + "-DCMAKE_FIND_PACKAGE_PREFER_CONFIG=ON" + + "-DTFLITE_ENABLE_GPU=ON" + "-DTFLITE_ENABLE_RUY=ON" + + ;; TO DO: turn on Farmhash + ;"-DSYSTEM_FARMHASH=ON" + + + (string-append "-Dabsl_DIR=" (assoc-ref %build-inputs "abseil-cpp") + "/lib/cmake/absl") + (string-append "-DEigen3_DIR=" (assoc-ref %build-inputs "eigen") + "/share/eigen3/cmake") + (string-append "-DFlatBuffers_DIR=" (assoc-ref %build-inputs "flatbuffers") + "/lib/cmake/flatbuffers") + (string-append "-DNEON_2_SSE_DIR=" (assoc-ref %build-inputs "neon2sse") + "/lib/cmake/NEON_2_SSE") + (string-append "-Dcpuinfo_DIR=" (assoc-ref %build-inputs "cpuinfo") + "/share/cpuinfo") + (string-append "-Druy_DIR=" (assoc-ref %build-inputs "ruy") + "/lib/cmake/ruy") ;; TODO: The build system attempts to build xnnpack from source. We ;; would like to use our xnnpack package here, but this requires more ;; work. "-DTFLITE_ENABLE_XNNPACK=OFF" - ;; Pretend we've already fetched abseil. We won't actually build it - ;; but use the existing package. - "-Dabseil-cpp_POPULATED=TRUE" - - ;; Don't fetch the sources. We have already built flatbuffers. - "-Dflatbuffers_POPULATED=TRUE" + ;;; Don't fetch the sources. We have these already + "-Degl_headers_POPULATED=TRUE" + "-Dfp16_headers_POPULATED=TRUE" + "-Dopencl_headers_POPULATED=TRUE" + "-Dopengl_headers_POPULATED=TRUE" + "-Dvulkan_headers_POPULATED=TRUE" + "-Dgoogletest_POPULATED=TRUE" + "-Dgoogle_benchmark_POPULATED=TRUE" + "-Dnsync_POPULATED=TRUE" + "-Dre2_POPULATED=TRUE" "-DFFT2D_SOURCE_DIR=/tmp/fft2d" - "-Dneon2sse_SOURCE_DIR=/tmp/neon2sse" - "-Dneon2sse_BINARY_DIR=/tmp/neon2sse-bin" "-DFARMHASH_SOURCE_DIR=/tmp/farmhash" "-Dgemmlowp_SOURCE_DIR=/tmp/gemmlowp" - (string-append "-DRUY_SOURCE_DIR=" - (assoc-ref %build-inputs "ruy-src"))) + ) #:phases (modify-phases %standard-phases (add-after 'unpack 'chdir (lambda _ (chdir "tensorflow/lite"))) (add-after 'chdir 'copy-sources (lambda* (#:key inputs #:allow-other-keys) - ;; Use external cmake finders instead of these stubs that won't - ;; find anything but the bundled sources. - (delete-file "tools/cmake/modules/Findabsl.cmake") - (delete-file "tools/cmake/modules/Findeigen.cmake") + ;; TO DO: properly use Guix's pthreaqdpool + ;; roght now not using pthreadpool because + ;; we are not enabling xnnpack (substitute* "CMakeLists.txt" - (("find_package\\(eigen REQUIRED") - "find_package(eigen REQUIRED NAMES Eigen3")) - (substitute* "tools/cmake/modules/Findflatbuffers.cmake" - (("get_target_property.*") - (format #false "set(FLATBUFFERS_INCLUDE_DIRS ~a/include)\n" - (assoc-ref inputs "flatbuffers")))) + (("if\\(NOT DEFINED PTHREADPOOL_SOURCE_DIR\\)") + "if(false)")) + (substitute* "CMakeLists.txt" + (("if\\(NOT TARGET pthreadpool\\)") + "if(false)")) ;; Don't fetch source code; we already have everything we need. (substitute* '("tools/cmake/modules/fft2d.cmake" - "tools/cmake/modules/ruy.cmake" "tools/cmake/modules/farmhash.cmake" - "tools/cmake/modules/neon2sse.cmake" "tools/cmake/modules/gemmlowp.cmake") (("OverridableFetchContent_Populate.*") "")) @@ -3012,31 +3043,56 @@ (define-public tensorflow-lite (invoke "tar" "--strip-components=1" "-xf" (assoc-ref inputs "fft2d-src"))) - (copy-recursively (assoc-ref inputs "neon2sse-src") - "/tmp/neon2sse/") (copy-recursively (assoc-ref inputs "gemmlowp-src") "/tmp/gemmlowp/"))) - (add-after 'copy-sources 'prepare-shared-library-build - (lambda _ (chdir "c"))) - (replace 'install - (lambda* (#:key outputs #:allow-other-keys) - (let* ((out (assoc-ref outputs "out")) - (lib (string-append out "/lib")) - (headers (string-append out "/include/tensorflow/lite"))) - (install-file "../build/libtensorflowlite_c.so" lib) - (with-directory-excursion ".." - (for-each - (lambda (file) - (let ((target-dir (string-append headers "/" (dirname file)))) - (install-file file target-dir))) - (find-files "." "\\.h$"))))))))) + + (add-after 'build 'build-shared-library + (lambda* (#:key configure-flags #:allow-other-keys) + (mkdir-p "c") + (with-directory-excursion "c" + (apply invoke "cmake" (append configure-flags (list "../../lite/c"))) + (invoke "cmake" "--build" "." "-j" (number->string + (parallel-job-count)))))) + (add-after 'build-shared-library 'build-benchmark-model + (lambda _ + (invoke "cmake" "--build" "." "--target" "benchmark_model" + "-j" (number->string (parallel-job-count))))) + + (add-after 'install 'install-extra + (lambda* (#:key outputs #:allow-other-keys) + (let* ((out (assoc-ref outputs "out")) + (lib (string-append out "/lib")) + (bin (string-append out "/bin"))) + (install-file "../build/c/libtensorflowlite_c.so" lib) + (install-file "../build/tools/benchmark/benchmark_model" bin)))) + ; TO DO: test not building now + ; (replace 'check + ; (lambda* (#:key tests? #:allow-other-keys) + ; (when tests? + ; (invoke "ctest" "-L" "plain")))) + ))) (inputs `(("abseil-cpp" ,abseil-cpp-20200923.3) - ("eigen" ,eigen-for-tensorflow-lite) - ("flatbuffers" ,flatbuffers) - ("python" ,python))) + ("cpuinfo" ,cpuinfo) + ("eigen" ,eigen) + ("fp16" ,fp16) + ("flatbuffers" ,flatbuffers-next-shared) + ;("gemmlowp" ,gemmlowp) ;; TO DO + ("mesa-headers" ,mesa-headers) + ("neon2sse" ,neon2sse) + ("nsync" ,nsync) + ("opencl-clhpp" ,opencl-clhpp) + ("opencl-headers" ,opencl-headers) + ("opencl-icd-loader" ,opencl-icd-loader) + ("pthreadpool" ,pthreadpool) + ("python" ,python) + ("ruy" ,ruy) + ("re2" ,re2) + ;("xnnpack" ,xnnpack) ;; TO DO: properly use Guix's copy of xnnpack + ("vulkan-headers" , vulkan-headers))) (native-inputs `(("pkg-config" ,pkg-config) + ("googletest" ,googletest) ("gemmlowp-src" ;; The commit hash is taken from ;; "tensorflow/lite/tools/cmake/modules/gemmlowp.cmake". @@ -3050,17 +3106,6 @@ (define-public tensorflow-lite (sha256 (base32 "1sbp8kmr2azwlvfbzryy1frxi99jhsh1nc93bdbxdf8zdgpv0kxl"))))) - ("neon2sse-src" - ,(let ((commit "a1652fd5253afbf3e39357b012974f93511f6108")) - (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/intel/ARM_NEON_2_x86_SSE") - (commit commit))) - (file-name (git-file-name "neon2sse" (string-take commit 8))) - (sha256 - (base32 - "1q8gkxag9wlnwdwad2pclsrkwzrdjy94hyrkayrsvxyj7szb5y8i"))))) ("farmhash-src" ,(let ((commit "816a4ae622e964763ca0862d9dbd19324a1eaf45")) (origin @@ -3076,24 +3121,12 @@ (define-public tensorflow-lite ,(origin (method url-fetch) (uri (string-append "https://storage.googleapis.com/" - "mirror.tensorflow.org/" - "www.kurims.kyoto-u.ac.jp/~ooura/fft2d.tgz")) + "mirror.tensorflow.org/github.com/petewarden/" + "OouraFFT/archive/v1.0.tar.gz")) (file-name "fft2d.tar.gz") (sha256 (base32 - "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd")))) - ("ruy-src" - ,(let ((commit "9c56af3fce210a8a103eda19bd6f47c08a9e3d90")) - (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/google/ruy") - (commit commit) - (recursive? #true))) - (file-name (git-file-name "ruy" (string-take commit 8))) - (sha256 - (base32 - "1cfd5gk6kaj8kbl3h98gx1ap8czd59y6p8qq8nr28fklpyzf5cis"))))))) + "1jfflzi74fag9z4qmgwvp90aif4dpbr1657izmxlgvf4hy8fk9xd")))))) (home-page "https://tensorflow.org") (synopsis "Machine learning framework") (description diff --git a/gnu/packages/serialization.scm b/gnu/packages/serialization.scm index 9c114aaf39..0090d650ac 100644 --- a/gnu/packages/serialization.scm +++ b/gnu/packages/serialization.scm @@ -811,6 +811,18 @@ (define-public flatbuffers-next (base32 "1z3a6l8g2y53i5xzraswfs2i0i3kk52zv7nzc2q3fgisbyiri3pz")))))) +(define-public flatbuffers-next-shared + (package + (inherit flatbuffers-next) + (name "flatbuffers-shared") + (version "23.1.21") + (arguments + (substitute-keyword-arguments (package-arguments flatbuffers-next) + ((#:configure-flags flags) + ;; make C++ fiels compiled with -fPIC, needed for shared lib". + #~(cons "-DFLATBUFFERS_CXX_FLAGS=-fPIC" #$flags)))))) + + (define-public python-feather-format (package (name "python-feather-format") base-commit: 4818ad5ba204e094fd8995c90c16ee1940e76210 prerequisite-patch-id: 4b8245be9f9ab7b2cf9713a404b7d1632d945278 prerequisite-patch-id: 6a623f7c3bdc733b110fd6f59f31032adb7d987d -- 2.41.0 From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 05 03:58:36 2023 Received: (at 63897) by debbugs.gnu.org; 5 Sep 2023 07:58:36 +0000 Received: from localhost ([127.0.0.1]:53324 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdQxI-00038g-AP for submit@debbugs.gnu.org; Tue, 05 Sep 2023 03:58:36 -0400 Received: from sender3-of-o57.zoho.com ([136.143.184.57]:21719) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdQxG-00038X-SD for 63897@debbugs.gnu.org; Tue, 05 Sep 2023 03:58:35 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1693900711; cv=none; d=zohomail.com; s=zohoarc; b=b8GcHYlx5bQ+wrQRURiCpWe1pPG7MYXOZl0+6Won9JE7sc/3UMrmJzXRk9zj3SEQIMmuWMyiWEwiSHp47a0+vtLOJeeApvT3CgYFc5togCvEOr6mxQwoTDTGvKxQXxvao/1/1EtLzov78nVuPx7CLGTBgWHyVFoYDGUG7/PSz8c= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1693900711; h=Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:From:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:Subject:To:To:Message-Id:Reply-To; bh=g/tRhjqUhPO7hRaA/YKTlSagi47pMyzcBS/Ckt7Px4w=; b=MyvlXTI+EEfiP+sJW5BrMN1KEk5FiTfcrNw3bHOAEqydPFkVYk1fIbHfgrfPm/Z4KBTcIvcO3zYQ5pxtcr1YnjK6WUC4qMu3Lc5kdS6fzJLTx2BUAWEtWmkcLEq6jK6qatiy2ku577NXFrvhO+Nvphlncs8eL4KBc2341nr4P4g= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1693900711; s=zoho; d=elephly.net; i=rekado@elephly.net; h=References:From:From:To:To:Cc:Cc:Subject:Subject:Date:Date:In-reply-to:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To; bh=g/tRhjqUhPO7hRaA/YKTlSagi47pMyzcBS/Ckt7Px4w=; b=ZCSvrxchQuBu+iFqM45UBsCL6PQTp0PjsqwMz+UJrRWvy9nrPq6pkSn9lKGrpciY F22nDYdQyZiZKGsrRX/sw9pnIpanhUNu1gg4u6hcs6xitL2IpiWgg5bkSpYIrsF2cZ6 ABpm+VZG1UT7O4zAvvYl0opxZ6HVT1TP7GSTu+Vc= Received: from localhost (113-111-142-46.pool.kielnet.net [46.142.111.113]) by mx.zohomail.com with SMTPS id 1693900708844369.2850260002208; Tue, 5 Sep 2023 00:58:28 -0700 (PDT) References: <9c48f700175bf6dfc257faf34de9e8abfae4ca23.1693716893.git.atai@atai.org> User-agent: mu4e 1.10.5; emacs 28.2 From: Ricardo Wurmus To: Andy Tai Subject: Re: [bug#63897] [PATCH v6] gnu: tensorflow-lite: Update to 2.12.1 Date: Tue, 05 Sep 2023 09:50:31 +0200 In-reply-to: Message-ID: <877cp59hlq.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 63897 Cc: 63897@debbugs.gnu.org, Andreas Enge X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Hi, The update looks good to me. My only concerns are cosmetic: - the indentation throughout is rather wonky. Would be good to reformat this with Emacs C-M-q or =E2=80=9Cguix style=E2=80=9D. - =E2=80=9CTO DO=E2=80=9D =E2=80=94> =E2=80=9CTODO=E2=80=9D (so we can find= it easily) - please use =E2=80=9C;;=E2=80=9D for line comments; a single semicolon is = only used for margin comments - it would be good not to use =E2=80=9Cassoc-ref %build-inputs=E2=80=9D and= use G-exps instead. I could make all these changes before pushing it. Actually, there is one question I have that is not of cosmetic nature: What=E2=80=99s up with the commented gemmlowp in the inputs? We have gemmlowp-src in the native-inputs. If you could answer this question I can take care of the rest and apply it. Thank you for your patience! --=20 Ricardo From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 05 08:46:37 2023 Received: (at 63897) by debbugs.gnu.org; 5 Sep 2023 12:46:37 +0000 Received: from localhost ([127.0.0.1]:54880 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdVS1-0002Ca-7L for submit@debbugs.gnu.org; Tue, 05 Sep 2023 08:46:37 -0400 Received: from mail-oo1-xc35.google.com ([2607:f8b0:4864:20::c35]:57597) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdVRz-0002CN-5O for 63897@debbugs.gnu.org; Tue, 05 Sep 2023 08:46:36 -0400 Received: by mail-oo1-xc35.google.com with SMTP id 006d021491bc7-572a7141434so1491088eaf.2 for <63897@debbugs.gnu.org>; Tue, 05 Sep 2023 05:46:34 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=atai-org.20230601.gappssmtp.com; s=20230601; t=1693917989; x=1694522789; darn=debbugs.gnu.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=l+5XbypcM6tzU+cVzwrbhW72bW4Roxn8mUviWKiIVbk=; b=TEWroQ4VC2HUlIJoogatIzT0v6dLaRj7kRKok2Pj9GQp5KyBZ+1Xbiifb7KUrWGkQn yxBb2QrpPGngjqKLJCa0PAZqkCEeYN0eNKbw4Z0rAKy6DKOOMDrnMcnPVouyVEA+XEW8 TeCzn25eXn2ai+9Q/zRjimHIBglhw5a1CfnaCDSVBE38mavpae4DcmBxtxLs2VZbJpyP 7ULn8nxzM1ilsy06TCSKj4ZLgINp/V/dQOAZ5syB3Rt/WQ2dQTqZ7pXHO47yQwB1lI3+ fnmqR/8cVeGNK19AFz8iTc9qX91PTiTn4Ke3KRvKAGTfV2RazMArz3fI+u/dIMwdrnnq rWhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693917989; x=1694522789; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=l+5XbypcM6tzU+cVzwrbhW72bW4Roxn8mUviWKiIVbk=; b=lqjO1S9so9XC/fRCA3odIr6G2fx5FTCaL97mVDcARNQH+MCqyqvR4EFWxsZJnsSYDv bqNI4voy1O0/SaBBwTzEFQDpVmYhx2tTipShUyYodkNbx/q1TME3AODqReHjJEwqQNj+ hQkYsyKjtnkPG+tv3pmY6y2CZVfB2ZuhdUfaUI6Q/KQXEO4o8HU3AfqrUrAmK1ClxBa7 rITF/kHtgA5AsNJWnuGBmLJIzChWzaGFpFOmSnKxSVA7Ey+tnF8DBnAXeyfcMbi24eTX SVM6/4YHBsehIKpog0ogLj/gB5cT+zfWClXzzSSZcM9yBHPmtDD4ChNq5D33Ae9CBKRk qm6Q== X-Gm-Message-State: AOJu0Yy5n9kiFQM32nRtlDrAhMm53zsZmKL7eXd6uWGMJ1hdgfuWqhwI il7/Yd1VOk5oluE1NUs9CYyZ6a0HHW9jfmPvia4= X-Google-Smtp-Source: AGHT+IFG1w4Y+ofFnUyJiM5g0NnzarrQ+fiRBT9tLkz2wuQXCoVF7wSt9xhcfnN/3zQIUUzAe4MSECcr7BlHaiaIvKM= X-Received: by 2002:a4a:6202:0:b0:571:aceb:26d2 with SMTP id x2-20020a4a6202000000b00571aceb26d2mr9500933ooc.3.1693917988586; Tue, 05 Sep 2023 05:46:28 -0700 (PDT) MIME-Version: 1.0 References: <9c48f700175bf6dfc257faf34de9e8abfae4ca23.1693716893.git.atai@atai.org> <877cp59hlq.fsf@elephly.net> In-Reply-To: <877cp59hlq.fsf@elephly.net> From: Andy Tai Date: Tue, 5 Sep 2023 05:45:52 -0700 Message-ID: Subject: Re: [bug#63897] [PATCH v6] gnu: tensorflow-lite: Update to 2.12.1 To: Ricardo Wurmus Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 63897 Cc: 63897@debbugs.gnu.org, Andreas Enge X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) Hi, I did created a package definition in Guix for gemmlowp and that was merged. So I was trying to refer to that. This is not in this patch but feel free to get that (and everything else) right and to push. On Tue, Sep 5, 2023 at 12:58=E2=80=AFAM Ricardo Wurmus = wrote: > > Actually, there is one question I have that is not of cosmetic nature: > What=E2=80=99s up with the commented gemmlowp in the inputs? We have > gemmlowp-src in the native-inputs. > > If you could answer this question I can take care of the rest and apply > it. Thank you for your patience! > > -- > Ricardo From debbugs-submit-bounces@debbugs.gnu.org Tue Sep 05 10:55:52 2023 Received: (at 63897) by debbugs.gnu.org; 5 Sep 2023 14:55:52 +0000 Received: from localhost ([127.0.0.1]:57662 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdXT5-0003DR-Mq for submit@debbugs.gnu.org; Tue, 05 Sep 2023 10:55:52 -0400 Received: from mail-oo1-xc34.google.com ([2607:f8b0:4864:20::c34]:57748) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qdXT2-0003DD-KS for 63897@debbugs.gnu.org; Tue, 05 Sep 2023 10:55:50 -0400 Received: by mail-oo1-xc34.google.com with SMTP id 006d021491bc7-5711f6dff8cso1574185eaf.3 for <63897@debbugs.gnu.org>; Tue, 05 Sep 2023 07:55:47 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=atai-org.20230601.gappssmtp.com; s=20230601; t=1693925742; x=1694530542; darn=debbugs.gnu.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=XA7u2L3v67dQsNiLNpK1J4XS2DtcYz+4wOpy5VuVhAM=; b=V3WngdLGyiNKmSBJJIsJ5U5zE9beFOBceD2f8tvLQOcmKPx/oFj865033X6qAnDYdE SLVlK2Z3T3Zz0h7YCdYFgPyEQYgtdToJbv6O6V9Ol+nPM8hkGGRRZUR0EIT7AMsxqK4l yJUynXdiujEoyui4fLOsXXb86WNsbRp4Q28EKeQ0aKrT+5XujYkRS9Eq3qMDvtlNqQhm 1aeCLD/IryLzdLBv1GiLF3yqkbc3AQB29lly9WFVUidEM8BN15k15+QV8ebuYrhU3N1r A7xvCN7Ak/Lrb46UE71Q8M5Ntj1InXtBHz0RzUFeZLjIub0Sez11IDNVIKe1jVY3WVa6 INCA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1693925742; x=1694530542; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=XA7u2L3v67dQsNiLNpK1J4XS2DtcYz+4wOpy5VuVhAM=; b=Z8H+8qyakm+TFUkuGCNokyACQCwC2oGqRdlwbfhiLCDNEY1/RcYAUtzU3h7ifGzhQF Bkwxsyx+eCa7t1kW+GTX3ef1fadsKCI8jCLThJw4bZx8Z89ocC5E6fgWzxejq6T4HufQ Aai3CYuMoho18QBJEfvOMdaHBu5PTUolMCVJUpFYZ09DkTKtJsEyX9CN9JsnT15RzG/J U+Hqk/k+C77+a7MxJUQlRxvjxnz19YLzAUCDqsyw7J0TJBR7GPcyzlk4VMMOT7kuC+z9 qcVfk/BnDfigJaWyImUAPT0V0cw0UImRpztC2Pvmi6yHn54FbAey+UZ9UcB+EBWIUQS4 gH+Q== X-Gm-Message-State: AOJu0Yzrneo5wY1HxxkWtyphtvAqpwCbqCzCv/26JmxpO6MNZUFuATo9 IBy4gkL4gEAC6RYWCqls2QEZc5sQRPTv/QQBf59f/pCqG18gnA== X-Google-Smtp-Source: AGHT+IEj5NUgEIgPuthIWQnn82ETuLyRjMS8UEKOJbnTPByOLa03SSa1IqCrzL1C6f1H2q5/hxgudMQI22E5UYoYtck= X-Received: by 2002:a4a:3008:0:b0:56c:e856:8b2c with SMTP id q8-20020a4a3008000000b0056ce8568b2cmr10723432oof.9.1693925742190; Tue, 05 Sep 2023 07:55:42 -0700 (PDT) MIME-Version: 1.0 References: <9c48f700175bf6dfc257faf34de9e8abfae4ca23.1693716893.git.atai@atai.org> <877cp59hlq.fsf@elephly.net> In-Reply-To: From: Andy Tai Date: Tue, 5 Sep 2023 07:55:06 -0700 Message-ID: Subject: Re: [bug#63897] [PATCH v6] gnu: tensorflow-lite: Update to 2.12.1 To: Ricardo Wurmus Content-Type: multipart/alternative; boundary="00000000000023bd2b06049dd214" X-Spam-Score: 0.5 (/) X-Debbugs-Envelope-To: 63897 Cc: 63897@debbugs.gnu.org, Andreas Enge X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.5 (/) --00000000000023bd2b06049dd214 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Also note this patch requires two other patches as prerequisites. These are marked with block relationships in debugs but here are them for reference https://issues.guix.gnu.org/63889#4 https://issues.guix.gnu.org/56202#5 On Tue, Sep 5, 2023 at 5:45=E2=80=AFAM Andy Tai wrote: > Hi, I did created a package definition in Guix for gemmlowp and that > was merged. So I was trying to refer to that. This is not in this > patch but feel free to get that (and everything else) right and to > push. > > On Tue, Sep 5, 2023 at 12:58=E2=80=AFAM Ricardo Wurmus wrote: > > > > Actually, there is one question I have that is not of cosmetic nature: > > What=E2=80=99s up with the commented gemmlowp in the inputs? We have > > gemmlowp-src in the native-inputs. > > > > If you could answer this question I can take care of the rest and apply > > it. Thank you for your patience! > > > > -- > > Ricardo > --=20 Andy Tai, atai@atai.org, Skype: licheng.tai, Line: andy_tai, WeChat: andytai1010 Year 2023 =E6=B0=91=E5=9C=8B112=E5=B9=B4 =E8=87=AA=E5=8B=95=E7=9A=84=E7=B2=BE=E7=A5=9E=E5=8A=9B=E6=98=AF=E4=BF=A1=E4= =BB=B0=E8=88=87=E8=A6=BA=E6=82=9F =E8=87=AA=E5=8B=95=E7=9A=84=E8=A1=8C=E7=82=BA=E5=8A=9B=E6=98=AF=E5=8B=9E=E5= =8B=95=E8=88=87=E6=8A=80=E8=83=BD --00000000000023bd2b06049dd214 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Also note this patch requires two other patches as prer= equisites. These are marked with block relationships in debugs but here are= them for reference

https://issues.guix.gnu.org/63889#4
https://issues.guix.gnu.org/56202#5<= /div>

On Tue, Sep 5, 2023 at 5:45=E2=80=AFAM Andy Tai <atai@atai.org> wrote:
Hi, I did created a package definition in Guix = for gemmlowp and that
was merged.=C2=A0 So I was trying to refer to that.=C2=A0 This is not in th= is
patch but feel free to get that (and everything else) right and to
push.

On Tue, Sep 5, 2023 at 12:58=E2=80=AFAM Ricardo Wurmus <rekado@elephly.net> wrote: >
> Actually, there is one question I have that is not of cosmetic nature:=
> What=E2=80=99s up with the commented gemmlowp in the inputs?=C2=A0 We = have
> gemmlowp-src in the native-inputs.
>
> If you could answer this question I can take care of the rest and appl= y
> it.=C2=A0 Thank you for your patience!
>
> --
> Ricardo


--
=
Andy Tai, atai@atai.org, Skype: li= cheng.tai, Line: andy_tai, WeChat: andytai1010
Year 2023 =E6=B0=91=E5=9C= =8B112=E5=B9=B4
=E8=87=AA=E5=8B=95=E7=9A=84=E7=B2=BE=E7=A5=9E=E5=8A=9B= =E6=98=AF=E4=BF=A1=E4=BB=B0=E8=88=87=E8=A6=BA=E6=82=9F
=E8=87=AA=E5=8B= =95=E7=9A=84=E8=A1=8C=E7=82=BA=E5=8A=9B=E6=98=AF=E5=8B=9E=E5=8B=95=E8=88=87= =E6=8A=80=E8=83=BD
--00000000000023bd2b06049dd214-- From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 08 04:55:55 2023 Received: (at 63897-done) by debbugs.gnu.org; 8 Sep 2023 08:55:56 +0000 Received: from localhost ([127.0.0.1]:42191 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qeXHP-0000uW-O6 for submit@debbugs.gnu.org; Fri, 08 Sep 2023 04:55:55 -0400 Received: from sender4-of-o50.zoho.com ([136.143.188.50]:21078) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1qeXHJ-0000uK-VU for 63897-done@debbugs.gnu.org; Fri, 08 Sep 2023 04:55:54 -0400 ARC-Seal: i=1; a=rsa-sha256; t=1694163345; cv=none; d=zohomail.com; s=zohoarc; b=DW8s7c34rxmq6cMCkK+4boH/rg3KiavVOz/hP/qwCM1FIQRTmIAvUzd2F15N7ZUqcw0CkZpPFnNgYOK8aIQTnjdn9Fo8WZEq/kKrZvc+pBqtXSe7iL2uS35BHi4XVBEx8p20o0tQ3Y03pc7nGCzU59akZahT0sTWY7z23u1/4Dc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1694163345; h=Content-Type:Content-Transfer-Encoding:Date:Date:From:From:MIME-Version:Message-ID:Subject:Subject:To:To:Message-Id:Reply-To:Cc; bh=S+7KTbq0C0JJhrjPIw1awR29gziislSRtZom+UCGPPk=; b=FpHzSSwBUXkrgDEPsE/LoN7/FLAgVLPfq6ezqYlxixJuz/tPIaAG9CcHgrxLJqoYNyRGAOz9kjUrmdVvFD7fC0m/u5sW6nXTitccMSHNWrOHbEM62BTTt3Fnzt3GlguQ78g/1jaunPTSprmTjRgatLXsZeDloQf44w7tt37JvXg= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1694163345; s=zoho; d=elephly.net; i=rekado@elephly.net; h=From:From:To:To:Subject:Subject:Date:Date:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding:Message-Id:Reply-To:Cc; bh=S+7KTbq0C0JJhrjPIw1awR29gziislSRtZom+UCGPPk=; b=D4i2Ymu1yFTtXOfaiySGcJXuMAy9yeGVgAnBoHeZA3RYE7FB3JY/dJhof8QH+zxw Lb1MjPFuZPJrgIAF3xXS1ycRZxiFX0vwwKi/+gHu38zlFG0HE0vz5mQqYv7q162f6YD hhfO7S6UJZ8S2JehvFlbq8IwrQIL2Q7/v2c4R4Ww= Received: from localhost (133-122-142-46.pool.kielnet.net [46.142.122.133]) by mx.zohomail.com with SMTPS id 1694163343941231.5284420373738; Fri, 8 Sep 2023 01:55:43 -0700 (PDT) User-agent: mu4e 1.10.5; emacs 28.2 From: Ricardo Wurmus To: 63897-done@debbugs.gnu.org Subject: [PATCH] gnu: tensorflow_lite: Update to 2.12.0 Date: Fri, 08 Sep 2023 10:54:40 +0200 Message-ID: <87a5tx6o36.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 63897-done X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Thanks for the patches and your patience. I=E2=80=99ve made the changes I mentioned earlier and pushed the update tog= ether with the other two patches this depends on. --=20 Ricardo From unknown Wed Jun 18 00:25:51 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Fri, 06 Oct 2023 11:24:20 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator