From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 01 23:39:39 2021 Received: (at submit) by debbugs.gnu.org; 2 Oct 2021 03:39:39 +0000 Received: from localhost ([127.0.0.1]:57670 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWVsB-0003FR-MG for submit@debbugs.gnu.org; Fri, 01 Oct 2021 23:39:39 -0400 Received: from lists.gnu.org ([209.51.188.17]:47938) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWVs7-0003FH-J4 for submit@debbugs.gnu.org; Fri, 01 Oct 2021 23:39:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:50500) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mWVs7-0002g4-3e for guix-patches@gnu.org; Fri, 01 Oct 2021 23:39:35 -0400 Received: from mout01.posteo.de ([185.67.36.65]:40485) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mWVs4-0007Mj-7G for guix-patches@gnu.org; Fri, 01 Oct 2021 23:39:34 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 80D34240026 for ; Sat, 2 Oct 2021 05:39:27 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1633145967; bh=uLK9Q2FvVSvDB8fCEyyYzQaMNflEY5fERxAOCtcS3UU=; h=From:To:Cc:Subject:Date:From; b=j1kCLU5Vc04XE4CyOvJXv9PWlzDE+9tzTj2VYNf+i+jndAxskBXjrwevyzdB/f3bt 3V8A9tvISWGeAccJnjpKGZ/jul3sotrR85CwdJ7Vvdl7cjW1+3jcp0T7/bvKWJBo8Y BI2MzS9ZgwVDChnQVRPytYX1/QZ9tR41hhXTQrDxOJ6fIhsd0zo9BpkvmudKspQUKk /2qWejh9zonW2rGcgKDGYXDMm7gQt6PGiB2wPfzUSJ6eLvlDNu097gz2u2YRKOKe+R FTCDgdDV/GiCj1HwipEXrExymp6Flbq323O0qviWYotkMTLDT8VxJaRH1b58ct5AP+ Wks3G0Gf6RELg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4HLt5B2hrBz6tm5; Sat, 2 Oct 2021 05:39:26 +0200 (CEST) From: Vinicius Monego To: guix-patches@gnu.org Subject: [PATCH 0/3] Add Optuna. Date: Sat, 2 Oct 2021 03:39:16 +0000 Message-Id: <20211002033916.64690-1-monego@posteo.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=185.67.36.65; envelope-from=monego@posteo.net; helo=mout01.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit Cc: Vinicius Monego X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -2.3 (--) This patchset adds the Optuna hyperparameter optimization framework. Optuna requires a number of ML frameworks for the whole test suite. I deleted some test files and added comments on missing packages. I packaged Skorch in the series but it's an optional dependency that is only used in the integration tests, which are completely disabled. The latest version of scikit-optimize (0.8.1), another optional dependency, is incompatible with the scikit-learn version in Guix (0.24.x). I am waiting for the next release to package it. The other dependencies are trickier. The executable in bin/ throws an error coming from pygobject, which I couldn't update. While there is a CLI, Optuna is mainly used as a library, that is why I prefix it with python-. Vinicius Monego (3): gnu: Add python-cma. gnu: Add python-skorch. gnu: Add python-optuna. gnu/packages/machine-learning.scm | 166 ++++++++++++++++++++++++++++++ 1 file changed, 167 insertions(+) -- 2.30.2 From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 01 23:42:42 2021 Received: (at 50956) by debbugs.gnu.org; 2 Oct 2021 03:42:42 +0000 Received: from localhost ([127.0.0.1]:57675 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWVv8-0003Km-4T for submit@debbugs.gnu.org; Fri, 01 Oct 2021 23:42:42 -0400 Received: from mout01.posteo.de ([185.67.36.65]:45773) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWVv5-0003KT-Po for 50956@debbugs.gnu.org; Fri, 01 Oct 2021 23:42:40 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 3336A240026 for <50956@debbugs.gnu.org>; Sat, 2 Oct 2021 05:42:34 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1633146154; bh=y29X890m8/++sqAC3IwRGKk1CiYMnc2QNpHBmgU7Kes=; h=From:To:Cc:Subject:Date:From; b=cIpNG03B53XvHXzPSpaXKJm81aQx3KwEUYiuJJn1ICFvGV7ECeQ2b172SHYd3O8CK OX/i7pHlGU1YbpavcJq3zQCOXNAlYr2A9YTRQaf+3Z/bbksY7Cujdoxym2MvtAiNIo MoSh7JWwF0hHQ8ljefjLippJCUmyPRHaQdA/hYh79EoTNLn45ludyvS5rL2qAe23eG s/xU2sgqrGdMA4RKkzH4Ti2kvoD6OpdM4v0RxZ0XqQGPoOeVfy1PjyhOfuT49JxO+P /lm8UqOtUAwhQlniNQMM5K4m99Z1mriqVN21t/aox6FCt2/j+medfvECZIyVaR1QGv EUZCiSTtxluZQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4HLt8n0rYgz9rxK; Sat, 2 Oct 2021 05:42:32 +0200 (CEST) From: Vinicius Monego To: 50956@debbugs.gnu.org Subject: [PATCH 1/3] gnu: Add python-cma. Date: Sat, 2 Oct 2021 03:41:15 +0000 Message-Id: <20211002034117.64876-1-monego@posteo.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 50956 Cc: Vinicius Monego X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * gnu/packages/machine-learning.scm (python-cma): New variable. --- gnu/packages/machine-learning.scm | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index e34de5df43..281c88d6f8 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -1150,6 +1150,33 @@ good at identifying feature interactions that are normally overlooked by standard feature selection algorithms.") (license license:expat))) +(define-public python-cma + (package + (name "python-cma") + (version "3.1.0") + (source + (origin + (method url-fetch) + (uri (pypi-uri "cma" version)) + (sha256 + (base32 "1ip32lnilbhmv1fyvmmdn5rcf084c0ps4q9dr3cf2ax5wdzhg0rv")))) + (build-system python-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (replace 'check + (lambda* (#:key inputs outputs tests? #:allow-other-keys) + (when tests? + (add-installed-pythonpath inputs outputs) + (invoke "python" "-m" "cma.test"))))))) + (propagated-inputs + `(("python-numpy" ,python-numpy))) + (home-page "https://github.com/CMA-ES/pycma") + (synopsis "Python implementation of CMA-ES") + (description "@code{pycma} is a Python implementation of CMA-ES and a few +related numerical optimization tools.") + (license license:bsd-3))) + (define-public python-cmaes (package (name "python-cmaes") -- 2.30.2 From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 01 23:42:45 2021 Received: (at 50956) by debbugs.gnu.org; 2 Oct 2021 03:42:45 +0000 Received: from localhost ([127.0.0.1]:57678 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWVvB-0003L4-C2 for submit@debbugs.gnu.org; Fri, 01 Oct 2021 23:42:45 -0400 Received: from mout01.posteo.de ([185.67.36.65]:44103) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWVv9-0003Ka-RF for 50956@debbugs.gnu.org; Fri, 01 Oct 2021 23:42:44 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 3D36E240026 for <50956@debbugs.gnu.org>; Sat, 2 Oct 2021 05:42:38 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1633146158; bh=T4z3KWZ1061fhGPEg9zQR9I+rRwOFuC9So8K6/irQWM=; h=From:To:Cc:Subject:Date:From; b=bO2qZD2eKy+gbAhiYgQh8ATK1VKh9E+ThVvKjHxqf63LAML1bIW8gnSPltzVHwWoM oToxA7tBx4LteID/NOUKszE8t5vo2cXw9VB9B3a3klMHz1P3gdzeI/NoET2hgIdY1d TccFsmYxu3YaPTNVsBOnBUyYNj5xkABHlf+lE9Z1BtmN45qMDhKxgtapbsi0rq5mNG KDFhCnjtn8FUua/BSZQGJelLU1bweoQN5snKLDX7UjAqPY8jn8SMQPJ75843bGb0bO xOuX5j88L5ikrs+ziYcWOn0eVGTA7WnsklrjAI9x+weQlzk85OP8YKFVhigsloKXVJ UqwMhXGppxkLg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4HLt8s22lNz9rxK; Sat, 2 Oct 2021 05:42:37 +0200 (CEST) From: Vinicius Monego To: 50956@debbugs.gnu.org Subject: [PATCH 2/3] gnu: Add python-skorch. Date: Sat, 2 Oct 2021 03:41:16 +0000 Message-Id: <20211002034117.64876-2-monego@posteo.net> In-Reply-To: <20211002034117.64876-1-monego@posteo.net> References: <20211002034117.64876-1-monego@posteo.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 50956 Cc: Vinicius Monego X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * gnu/packages/machine-learning.scm (python-skorch): New variable. --- gnu/packages/machine-learning.scm | 42 +++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 281c88d6f8..fd3e6b2090 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -1050,6 +1050,48 @@ number of threads used in the threadpool-backed of common native libraries used for scientific computing and data science (e.g. BLAS and OpenMP).") (license license:bsd-3))) +(define-public python-skorch + (package + (name "python-skorch") + (version "0.10.0") + (source + (origin + (method url-fetch) + (uri (pypi-uri "skorch" version)) + (sha256 + (base32 "196hr0q5nw1nzckwanfv27myasayfdxxhx80iv9whm7675rzj44r")))) + (build-system python-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (replace 'check + (lambda* (#:key inputs outputs tests? #:allow-other-keys) + (when tests? + (add-installed-pythonpath inputs outputs) + (invoke "python" "-m" "pytest" "--pyargs" "skorch" "-k" + (string-append + ;; Errors because of missing weight and pickle files. + "not test_load_cuda_params_to_cpu" + " and not test_pickle_load")))))))) + (propagated-inputs + `(("python-numpy" ,python-numpy) + ("python-scikit-learn" ,python-scikit-learn) + ("python-scipy" ,python-scipy) + ("python-tabulate" ,python-tabulate) + ("python-tqdm" ,python-tqdm))) + (native-inputs + `(("python-flaky" ,python-flaky) + ("python-pandas" ,python-pandas) + ("python-pytest" ,python-pytest) + ("python-pytest-cov" ,python-pytest-cov) + ("python-pytorch" ,python-pytorch))) + (home-page "https://github.com/skorch-dev/skorch") + (synopsis "Scikit-learn compatible neural network library for PyTorch") + (description "Skorch is a scikit-learn compatible neural network library +that wraps PyTorch. It allows to build and train PyTorch models using a +scikit-learn-like API.") + (license license:bsd-3))) + (define-public python-pynndescent (package (name "python-pynndescent") -- 2.30.2 From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 01 23:43:00 2021 Received: (at 50956) by debbugs.gnu.org; 2 Oct 2021 03:43:00 +0000 Received: from localhost ([127.0.0.1]:57681 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWVvH-0003LN-Ku for submit@debbugs.gnu.org; Fri, 01 Oct 2021 23:43:00 -0400 Received: from mout01.posteo.de ([185.67.36.65]:53825) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1mWVvF-0003Kv-AT for 50956@debbugs.gnu.org; Fri, 01 Oct 2021 23:42:51 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id B4631240026 for <50956@debbugs.gnu.org>; Sat, 2 Oct 2021 05:42:43 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1633146163; bh=6pSVPkSUrkJQ3Qr33QOug4S78cMFd4671bn1/o6Lq/M=; h=From:To:Cc:Subject:Date:From; b=OkVwMb2Z6K7YoxZ4RGHeXS12SLVo61pBVRELQcDBLEcZya/oL2WAbx+IQQ3MQG+LB yXTQOvTY7oj7WkgTEwuYveda0zeWwFO7JBBycXIYuVLxwe8aCjJSz+ouC8YfLIgcgF lNcWrUobzjDg9nR5utTwMPzqtvjfuAlLnnddBcFHJS5Vy8rQ7mWVi+5WueNMazEj2M pjOOsgQhHE46V+73aRbmHQTNPU7wjpy5xcjt0Ous8wLoJElrhe9dLrQ5W9WeQqSETY x92KSfVUdKxw9E/onwnRhKKmvHIudpwyqiyFoiJqKcO81cbttSgtjDT3ScbAAVYWYZ w+bGNAK13JPsg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4HLt8y5HCMz6tm5; Sat, 2 Oct 2021 05:42:42 +0200 (CEST) From: Vinicius Monego To: 50956@debbugs.gnu.org Subject: [PATCH 3/3] gnu: Add python-optuna. Date: Sat, 2 Oct 2021 03:41:17 +0000 Message-Id: <20211002034117.64876-3-monego@posteo.net> In-Reply-To: <20211002034117.64876-1-monego@posteo.net> References: <20211002034117.64876-1-monego@posteo.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 50956 Cc: Vinicius Monego X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) * gnu/packages/machine-learning.scm (python-optuna): New variable. --- gnu/packages/machine-learning.scm | 96 +++++++++++++++++++++++++++++++ 1 file changed, 96 insertions(+) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index fd3e6b2090..3b6f709c4e 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -74,6 +74,7 @@ #:use-module (gnu packages ocaml) #:use-module (gnu packages onc-rpc) #:use-module (gnu packages parallel) + #:use-module (gnu packages openstack) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) #:use-module (gnu packages protobuf) @@ -938,6 +939,101 @@ computing environments.") (home-page "http://dlib.net") (license license:boost1.0))) +(define-public python-optuna + (package + (name "python-optuna") + (version "2.9.1") + (source + (origin + ;; No tests in the PyPI tarball. + (method git-fetch) + (uri (git-reference + (url "https://github.com/optuna/optuna") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 "1fx80qjrkmnvn2mg9fx26qn3sjlwnwqlmkaf6sqhdw79pn6khlpi")))) + (build-system python-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (add-after 'unpack 'dont-check-deps + ;; Don't check for dependencies we don't have or don't need. + ;; TODO: Package and enable some of these. + (lambda _ + (substitute* "setup.py" + ((".*allennlp.*") "") + ((".*bokeh.*") "") + ((".*catalyst.*") "") + ((".*chainer.*") "") + ((".*fastai.*") "") + ((".*keras.*") "") + ((".*lightgbm.*") "") + ((".*mlflow.*") "") + ((".*mxnet.*") "") + ((".*plotly.*") "") + ((".*scikit-optimize.*") "") + ((".*skorch.*") "") + ((".*tensorflow.*") "") + ((".*torch.*") "") + ((".*xgboost.*") "")))) + (add-after 'unpack 'disable-some-tests + (lambda _ + ;; Integration tests require most of the dependencies above. + (delete-file-recursively "tests/integration_tests") + ;; TODO: Requires scikit-optimize. + (delete-file "tests/samplers_tests/test_samplers.py") + ;; TODO: Requires bokeh. + (delete-file "tests/test_dashboard.py") + ;; FIXME: "Optuna" executable is not found. + (delete-file "tests/test_cli.py") + ;; FIXME: Files below require plotly but setup fails to identify + ;; plotly version and suggests an upgrade to >= 4.0.0. + (delete-file "tests/visualization_tests/test_contour.py") + (delete-file (string-append "tests/multi_objective_tests/" + "visualization_tests/" + "test_pareto_front.py")) + (delete-file-recursively "tests/visualization_tests"))) + (replace 'check + (lambda* (#:key inputs outputs tests? #:allow-other-keys) + (when tests? + (add-installed-pythonpath inputs outputs) + (invoke "python" "-m" "pytest" "-k" + ;; redis.exceptions.ResponseError: unknown command 'TIME'. + (string-append + "not test_retry_failed_trial_callback" + " and not test_failed_trial_callback" + " and not test_fail_stale_trials_with_optimize")))))))) + (propagated-inputs + `(("python-alembic" ,python-alembic) + ("python-cliff" ,python-cliff) + ("python-cmaes" ,python-cmaes) + ("python-colorlog" ,python-colorlog) + ("python-numpy" ,python-numpy) + ("python-packaging" ,python-packaging) + ("python-pyyaml" ,python-pyyaml) + ("python-scipy" ,python-scipy) + ("python-sqlalchemy" ,python-sqlalchemy) + ("python-tqdm" ,python-tqdm))) + (native-inputs + `(("python-cma" ,python-cma) + ("python-fakeredis" ,python-fakeredis) + ("python-matplotlib" ,python-matplotlib) + ("python-mpi4py" ,python-mpi4py) + ("python-pandas" ,python-pandas) + ("python-pytest" ,python-pytest) + ("python-redis" ,python-redis) + ("python-scikit-learn" ,python-scikit-learn) + ("which" ,which))) + (home-page "https://optuna.org/") + (synopsis "Hyperparameter optimization framework") + (description "Optuna is an automatic hyperparameter optimization software +framework, particularly designed for machine learning. It features an +imperative, @emph{define-by-run} style user API. Thanks to it, the code +written with Optuna enjoys high modularity, and the user of Optuna can +dynamically construct the search spaces for the hyperparameters.") + (license license:expat))) + (define-public python-scikit-learn (package (name "python-scikit-learn") -- 2.30.2 From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 12 22:00:59 2021 Received: (at 50956) by debbugs.gnu.org; 13 Oct 2021 02:00:59 +0000 Received: from localhost ([127.0.0.1]:56698 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1maTZj-0007So-7y for submit@debbugs.gnu.org; Tue, 12 Oct 2021 22:00:59 -0400 Received: from mout01.posteo.de ([185.67.36.65]:51741) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1maTZh-0007Sa-P7 for 50956@debbugs.gnu.org; Tue, 12 Oct 2021 22:00:58 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 6E744240027 for <50956@debbugs.gnu.org>; Wed, 13 Oct 2021 04:00:51 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1634090451; bh=JH8sakxZTK0wQ8Oy0C8anuvzIF6Ga+2mU7ye15XFGKo=; h=From:To:Cc:Subject:Date:From; b=Ri2i8PTuNukZo5zvoi0mQNIJ8NgFHcgGnfN8+5Hwe6a5U4jcoadIvkYwcbeC+iWaF rHiFrtWv353L5eGO65/okWSb3tHss1GyZ9N4zsyhvUPk2UcY9PgLHin8j/2RWoGEwI rmFDZWB/DSSlGbm6dzV+lsDQv2aoIX/8/xC947oAcWHUPsRlbgY/MoO3RJbSesqyQe aLpo2YrAiuglhPAVKSgn7OgHzhJPqnUPp4m64PRAkQNkJdOu7e6pI0p5ee3DGfnP6r JwzbBkBhkbm044EqDMJYdoT0WDJF3wQQee3y6tl21ZuT+rViVd5mOZjgIYPf9zFZ5r VJA2C2/PBKSsw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4HTbNL361dz9rxM; Wed, 13 Oct 2021 04:00:49 +0200 (CEST) From: Vinicius Monego To: 50956@debbugs.gnu.org Subject: [PATCH v2 1/4] gnu: Add python-scikit-optimize. Date: Wed, 13 Oct 2021 02:00:32 +0000 Message-Id: <20211013020035.53000-1-monego@posteo.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 50956 Cc: Vinicius Monego X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * gnu/packages/machine-learning.scm (python-scikit-optimize): New variable. --- scikit-optimize 0.9.0 was released today, added it as a patch in the series. gnu/packages/machine-learning.scm | 38 +++++++++++++++++++++++++++++++ 1 file changed, 38 insertions(+) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 029422677a..478e9548e9 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -1218,6 +1218,44 @@ main intended application of Autograd is gradient-based optimization.") (define-public python2-autograd (package-with-python2 python-autograd)) +(define-public python-scikit-optimize + (package + (name "python-scikit-optimize") + (version "0.9.0") + (source + (origin + (method git-fetch) ; no tests in PyPI tarball + (uri (git-reference + (url "https://github.com/scikit-optimize/scikit-optimize") + (commit (string-append "v" version)))) + (sha256 + (base32 "0hsq6pmryimxc275yrcy4bv217bx7ma6rz0q6m4138bv4zgq18d1")) + (file-name (git-file-name name version)))) + (build-system python-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (replace 'check + (lambda* (#:key inputs outputs tests? #:allow-other-keys) + (when tests? + (add-installed-pythonpath inputs outputs) + (invoke "python" "-m" "pytest"))))))) + (native-inputs + `(("python-pytest" ,python-pytest))) + (propagated-inputs + `(("python-joblib" ,python-joblib) + ("python-numpy" ,python-numpy) + ("python-pyaml" ,python-pyaml) + ("python-scikit-learn" ,python-scikit-learn) + ("python-scipy" ,python-scipy))) + (home-page "https://scikit-optimize.github.io/") + (synopsis "Sequential model-based optimization") + (description + "Scikit-Optimize, or @code{skopt}, is a library to minimize (very) +expensive and noisy black-box functions. It implements several methods +for sequential model-based optimization.") + (license license:bsd-3))) + (define-public lightgbm (package (name "lightgbm") -- 2.30.2 From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 12 22:01:05 2021 Received: (at 50956) by debbugs.gnu.org; 13 Oct 2021 02:01:05 +0000 Received: from localhost ([127.0.0.1]:56703 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1maTZp-0007Tf-HZ for submit@debbugs.gnu.org; Tue, 12 Oct 2021 22:01:05 -0400 Received: from mout02.posteo.de ([185.67.36.66]:34497) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1maTZn-0007Sh-6Y for 50956@debbugs.gnu.org; Tue, 12 Oct 2021 22:01:04 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id 99947240101 for <50956@debbugs.gnu.org>; Wed, 13 Oct 2021 04:00:56 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1634090456; bh=24fNlFoVpculwpel8TS/ttJnVY/dZLcgtIOFf5Nd7WU=; h=From:To:Cc:Subject:Date:From; b=btvqmdUTiWnEHDSJn1FZZxMGEkxTp9se2AAyl6MudqJcNPwIoyDNkwx3A8s1RCkGX CTsGqRT7SrCroxGAOKAZ5uToIEJbkCAonLP8cuZQdM21WvmtX4ue7QAWoXkWylN8rG xgGFZrXp300EepL7TujLPrpCBcTcrrEn//YwYeae6PrkcI40Hh5Uuf5IG2JOgtocRy gOzqkmW1HylVO5xS1KU8zqIftQr9ekZdOOJANgLq2PyalJIau1O5f9yK5BKu2HR0D1 MlvGt9/ZfPvP4gik6McUW7znw9tznYFH7mRVBtKDYBugu08AFnetPYA2XvlHOkIRMe 3Z/1g7dJ7aDtw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4HTbNR3SxVz6tm5; Wed, 13 Oct 2021 04:00:54 +0200 (CEST) From: Vinicius Monego To: 50956@debbugs.gnu.org Subject: [PATCH v2 2/4] gnu: Add python-cma. Date: Wed, 13 Oct 2021 02:00:33 +0000 Message-Id: <20211013020035.53000-2-monego@posteo.net> In-Reply-To: <20211013020035.53000-1-monego@posteo.net> References: <20211013020035.53000-1-monego@posteo.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 50956 Cc: Vinicius Monego X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * gnu/packages/machine-learning.scm (python-cma): New variable. --- gnu/packages/machine-learning.scm | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 478e9548e9..6b7736702a 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -1150,6 +1150,33 @@ good at identifying feature interactions that are normally overlooked by standard feature selection algorithms.") (license license:expat))) +(define-public python-cma + (package + (name "python-cma") + (version "3.1.0") + (source + (origin + (method url-fetch) + (uri (pypi-uri "cma" version)) + (sha256 + (base32 "1ip32lnilbhmv1fyvmmdn5rcf084c0ps4q9dr3cf2ax5wdzhg0rv")))) + (build-system python-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (replace 'check + (lambda* (#:key inputs outputs tests? #:allow-other-keys) + (when tests? + (add-installed-pythonpath inputs outputs) + (invoke "python" "-m" "cma.test"))))))) + (propagated-inputs + `(("python-numpy" ,python-numpy))) + (home-page "https://github.com/CMA-ES/pycma") + (synopsis "Python implementation of CMA-ES") + (description "@code{pycma} is a Python implementation of CMA-ES and a few +related numerical optimization tools.") + (license license:bsd-3))) + (define-public python-cmaes (package (name "python-cmaes") -- 2.30.2 From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 12 22:01:09 2021 Received: (at 50956) by debbugs.gnu.org; 13 Oct 2021 02:01:09 +0000 Received: from localhost ([127.0.0.1]:56709 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1maTZs-0007Tz-PE for submit@debbugs.gnu.org; Tue, 12 Oct 2021 22:01:08 -0400 Received: from mout01.posteo.de ([185.67.36.65]:55959) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1maTZq-0007Sy-5H for 50956@debbugs.gnu.org; Tue, 12 Oct 2021 22:01:06 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 7D6E9240026 for <50956@debbugs.gnu.org>; Wed, 13 Oct 2021 04:01:00 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1634090460; bh=CaHgLvowGVd87UXUGUHCyX0GZce6K0J05wOB8ZOZ6B4=; h=From:To:Cc:Subject:Date:From; b=UHonUI1rOg1APPnJ7O3uuPvhnNP4ikbCsDjCGoutx/HNzBgPYEpZ36qqQUYsbdRDr exRCSE8h0RGlrzHZYHADAPOAWm+MGCOS6/MGR34MWiMTpXRxjgYCMu1uVqbLTfTQCX wTzdoeoZofGPwVnvWNiC9b+JFB5fyXFR9IVYCRQmsa5rbu+sPhuA2PV597PXEaadNy UzMUtLrTet99a4H8SKU33p7dbThkzh9DfVSyYbSJJ92r4pq6FEJxZPTieIwplPSS7l 3yO75Axd/q6L8W6jz4qfxTA7i0pE+sv8Dg1S7FToSipmPl2cbz+OQ5OFVqPdH+ip1V hTS8cq/EhknVg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4HTbNW31C4z6tm4; Wed, 13 Oct 2021 04:00:59 +0200 (CEST) From: Vinicius Monego To: 50956@debbugs.gnu.org Subject: [PATCH v2 3/4] gnu: Add python-skorch. Date: Wed, 13 Oct 2021 02:00:34 +0000 Message-Id: <20211013020035.53000-3-monego@posteo.net> In-Reply-To: <20211013020035.53000-1-monego@posteo.net> References: <20211013020035.53000-1-monego@posteo.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 50956 Cc: Vinicius Monego X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * gnu/packages/machine-learning.scm (python-skorch): New variable. --- Swapped order of native and propagated inputs. gnu/packages/machine-learning.scm | 42 +++++++++++++++++++++++++++++++ 1 file changed, 42 insertions(+) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 6b7736702a..09767584bb 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -1050,6 +1050,48 @@ number of threads used in the threadpool-backed of common native libraries used for scientific computing and data science (e.g. BLAS and OpenMP).") (license license:bsd-3))) +(define-public python-skorch + (package + (name "python-skorch") + (version "0.10.0") + (source + (origin + (method url-fetch) + (uri (pypi-uri "skorch" version)) + (sha256 + (base32 "196hr0q5nw1nzckwanfv27myasayfdxxhx80iv9whm7675rzj44r")))) + (build-system python-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (replace 'check + (lambda* (#:key inputs outputs tests? #:allow-other-keys) + (when tests? + (add-installed-pythonpath inputs outputs) + (invoke "python" "-m" "pytest" "--pyargs" "skorch" "-k" + (string-append + ;; Errors because of missing weight and pickle files. + "not test_load_cuda_params_to_cpu" + " and not test_pickle_load")))))))) + (native-inputs + `(("python-flaky" ,python-flaky) + ("python-pandas" ,python-pandas) + ("python-pytest" ,python-pytest) + ("python-pytest-cov" ,python-pytest-cov) + ("python-pytorch" ,python-pytorch))) + (propagated-inputs + `(("python-numpy" ,python-numpy) + ("python-scikit-learn" ,python-scikit-learn) + ("python-scipy" ,python-scipy) + ("python-tabulate" ,python-tabulate) + ("python-tqdm" ,python-tqdm))) + (home-page "https://github.com/skorch-dev/skorch") + (synopsis "Scikit-learn compatible neural network library for PyTorch") + (description "Skorch is a scikit-learn compatible neural network library +that wraps PyTorch. It allows to build and train PyTorch models using a +scikit-learn-like API.") + (license license:bsd-3))) + (define-public python-pynndescent (package (name "python-pynndescent") -- 2.30.2 From debbugs-submit-bounces@debbugs.gnu.org Tue Oct 12 22:01:20 2021 Received: (at 50956) by debbugs.gnu.org; 13 Oct 2021 02:01:20 +0000 Received: from localhost ([127.0.0.1]:56713 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1maTZx-0007UG-0v for submit@debbugs.gnu.org; Tue, 12 Oct 2021 22:01:20 -0400 Received: from mout01.posteo.de ([185.67.36.65]:43841) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1maTZu-0007Td-5u for 50956@debbugs.gnu.org; Tue, 12 Oct 2021 22:01:11 -0400 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 7A9CA240026 for <50956@debbugs.gnu.org>; Wed, 13 Oct 2021 04:01:04 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1634090464; bh=8xJUsyzM9fq8i0xJa2kKKcWdMC1ZSKf1v21V8kjgB28=; h=From:To:Cc:Subject:Date:From; b=Z4W5pKjXQKHT0UHtqzAt26fKRTyIXxi+J6YY86+FxhGG/qrBdxDjxW4cQ/6CpDv/O Er3iXXIKNaiGA1acl3znYPcPvyuuztgjYCyi5dVWXh2lzM6rD8oZmNis8Dj5IGZTxm BUmNeHT4mEkDAuzOZJhK+vTARKsWlgGP3FzyjhcXGf5+x94UrLDasCPbjDLQ4KLCBE XDoOGHF6joqJDmNsaSPcHWblWAMXk7VjY1IoGy5SkPC0iSRaKEaT8lSo04RR60B0a7 i4lIhpdNESMTefmW2b2Cdalg8jC1At8KDG8blt2LiZmmUheEBimPnREM+CCR4aWbYs OZVimnqVdmfaw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4HTbNb414vz9rxM; Wed, 13 Oct 2021 04:01:03 +0200 (CEST) From: Vinicius Monego To: 50956@debbugs.gnu.org Subject: [PATCH v2 4/4] gnu: Add python-optuna. Date: Wed, 13 Oct 2021 02:00:35 +0000 Message-Id: <20211013020035.53000-4-monego@posteo.net> In-Reply-To: <20211013020035.53000-1-monego@posteo.net> References: <20211013020035.53000-1-monego@posteo.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 50956 Cc: Vinicius Monego X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) * gnu/packages/machine-learning.scm (python-optuna): New variable. --- Updated to 2.10.0, added scikit-optimize to native-inputs don't delete the test file that requires it. Swapped order of native and propagated inputs. gnu/packages/machine-learning.scm | 93 +++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 09767584bb..44033cfb77 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -74,6 +74,7 @@ #:use-module (gnu packages ocaml) #:use-module (gnu packages onc-rpc) #:use-module (gnu packages parallel) + #:use-module (gnu packages openstack) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) #:use-module (gnu packages protobuf) @@ -938,6 +939,98 @@ computing environments.") (home-page "http://dlib.net") (license license:boost1.0))) +(define-public python-optuna + (package + (name "python-optuna") + (version "2.10.0") + (source + (origin + (method git-fetch) ; no tests in PyPI tarball + (uri (git-reference + (url "https://github.com/optuna/optuna") + (commit (string-append "v" version)))) + (file-name (git-file-name name version)) + (sha256 + (base32 "0fha0pwxq6n3mbpvpz3vk8hh61zqncj5cnq063kzfl5d8rd48vcd")))) + (build-system python-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (add-after 'unpack 'dont-check-deps + ;; Don't check for test dependencies we don't have. + ;; TODO: Package and enable some of these. + (lambda _ + (substitute* "setup.py" + ((".*allennlp.*") "") + ((".*bokeh.*") "") + ((".*catalyst.*") "") + ((".*chainer.*") "") + ((".*fastai.*") "") + ((".*keras.*") "") + ((".*lightgbm.*") "") + ((".*mlflow.*") "") + ((".*mxnet.*") "") + ((".*plotly.*") "") + ((".*skorch.*") "") + ((".*tensorflow.*") "") + ((".*torch.*") "") + ((".*xgboost.*") "")))) + (add-after 'unpack 'disable-some-tests + (lambda _ + ;; Integration tests require most of the dependencies above. + (delete-file-recursively "tests/integration_tests") + ;; TODO: Requires bokeh. + (delete-file "tests/test_dashboard.py") + ;; FIXME: "Optuna" executable is not found. + (delete-file "tests/test_cli.py") + ;; FIXME: Files below require plotly but setup fails to identify + ;; plotly version and suggests an upgrade to >= 4.0.0. + (delete-file "tests/visualization_tests/test_contour.py") + (delete-file (string-append "tests/multi_objective_tests/" + "visualization_tests/" + "test_pareto_front.py")) + (delete-file-recursively "tests/visualization_tests"))) + (replace 'check + (lambda* (#:key inputs outputs tests? #:allow-other-keys) + (when tests? + (add-installed-pythonpath inputs outputs) + (invoke "python" "-m" "pytest" "-k" + ;; redis.exceptions.ResponseError: unknown command 'TIME'. + (string-append + "not test_retry_failed_trial_callback" + " and not test_failed_trial_callback" + " and not test_fail_stale_trials_with_optimize")))))))) + (native-inputs + `(("python-cma" ,python-cma) + ("python-fakeredis" ,python-fakeredis) + ("python-matplotlib" ,python-matplotlib) + ("python-mpi4py" ,python-mpi4py) + ("python-pandas" ,python-pandas) + ("python-pytest" ,python-pytest) + ("python-redis" ,python-redis) + ("python-scikit-learn" ,python-scikit-learn) + ("python-scikit-optimize" ,python-scikit-optimize) + ("which" ,which))) + (propagated-inputs + `(("python-alembic" ,python-alembic) + ("python-cliff" ,python-cliff) + ("python-cmaes" ,python-cmaes) + ("python-colorlog" ,python-colorlog) + ("python-numpy" ,python-numpy) + ("python-packaging" ,python-packaging) + ("python-pyyaml" ,python-pyyaml) + ("python-scipy" ,python-scipy) + ("python-sqlalchemy" ,python-sqlalchemy) + ("python-tqdm" ,python-tqdm))) + (home-page "https://optuna.org/") + (synopsis "Hyperparameter optimization framework") + (description "Optuna is an automatic hyperparameter optimization software +framework, particularly designed for machine learning. It features an +imperative, @emph{define-by-run} style user API. Thanks to it, the code +written with Optuna enjoys high modularity, and the user of Optuna can +dynamically construct the search spaces for the hyperparameters.") + (license license:expat))) + (define-public python-scikit-learn (package (name "python-scikit-learn") -- 2.30.2