From unknown Fri Sep 05 16:58:34 2025 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.509 (Entity 5.509) Content-Type: text/plain; charset=utf-8 From: bug#70175 <70175@debbugs.gnu.org> To: bug#70175 <70175@debbugs.gnu.org> Subject: Status: [PATCH] gnu: llama-cpp: support OpenBLAS for faster prompt processing Reply-To: bug#70175 <70175@debbugs.gnu.org> Date: Fri, 05 Sep 2025 23:58:34 +0000 retitle 70175 [PATCH] gnu: llama-cpp: support OpenBLAS for faster prompt pr= ocessing reassign 70175 guix-patches submitter 70175 John Fremlin severity 70175 normal tag 70175 patch thanks From debbugs-submit-bounces@debbugs.gnu.org Wed Apr 03 23:48:40 2024 Received: (at submit) by debbugs.gnu.org; 4 Apr 2024 03:48:41 +0000 Received: from localhost ([127.0.0.1]:60100 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rsE5f-0003kB-F0 for submit@debbugs.gnu.org; Wed, 03 Apr 2024 23:48:40 -0400 Received: from lists.gnu.org ([2001:470:142::17]:46118) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rsE5c-0003ib-2w for submit@debbugs.gnu.org; Wed, 03 Apr 2024 23:48:38 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rsE5R-0002DY-Ht for guix-patches@gnu.org; Wed, 03 Apr 2024 23:48:25 -0400 Received: from spribog.eb.pe ([5.9.0.72]) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rsE5E-0003Db-MW for guix-patches@gnu.org; Wed, 03 Apr 2024 23:48:25 -0400 Received: by spribog.eb.pe (Postfix, from userid 1000) id 03154162B75E; Thu, 4 Apr 2024 05:48:05 +0200 (CEST) From: John Fremlin To: guix-patches@gnu.org Subject: [PATCH] gnu: llama-cpp: support OpenBLAS for faster prompt processing Date: Wed, 3 Apr 2024 23:46:25 -0400 Message-ID: <7f9fe9d92aff101a12057e8571772428028dc6ae.1712202385.git.john@fremlin.org> X-Mailer: git-send-email 2.41.0 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: none client-ip=5.9.0.72; envelope-from=john@spribog.eb.pe; helo=spribog.eb.pe X-Spam_score_int: -15 X-Spam_score: -1.6 X-Spam_bar: - X-Spam_report: (-1.6 / 5.0 requ) BAYES_00=-1.9, HEADER_FROM_DIFFERENT_DOMAINS=0.25, SPF_HELO_NONE=0.001, SPF_NONE=0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.2 (/) X-Debbugs-Envelope-To: submit Cc: John Fremlin X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.8 (/) OpenBLAS is recommended by https://github.com/ggerganov/llama.cpp Change-Id: Iaf6f22252da13e2d6f503992878b35b0da7de0aa --- gnu/packages/machine-learning.scm | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/gnu/packages/machine-learning.scm b/gnu/packages/machine-learning.scm index 225bff0ca2..ea3674ce3e 100644 --- a/gnu/packages/machine-learning.scm +++ b/gnu/packages/machine-learning.scm @@ -542,6 +542,8 @@ (define-public llama-cpp (build-system cmake-build-system) (arguments (list + #:configure-flags + '(list "-DLLAMA_BLAS=ON" "-DLLAMA_BLAS_VENDOR=OpenBLAS") #:modules '((ice-9 textual-ports) (guix build utils) ((guix build python-build-system) #:prefix python:) @@ -576,8 +578,9 @@ (define-public llama-cpp (lambda _ (copy-file "bin/main" (string-append #$output "/bin/llama"))))))) (inputs (list python)) + (native-inputs (list pkg-config)) (propagated-inputs - (list python-numpy python-pytorch python-sentencepiece)) + (list python-numpy python-pytorch python-sentencepiece openblas)) (home-page "https://github.com/ggerganov/llama.cpp") (synopsis "Port of Facebook's LLaMA model in C/C++") (description "This package provides a port to Facebook's LLaMA collection base-commit: 1441a205b1ebb610ecfae945b5770734cbe8478c -- 2.41.0 From debbugs-submit-bounces@debbugs.gnu.org Fri Apr 05 07:35:37 2024 Received: (at submit) by debbugs.gnu.org; 5 Apr 2024 11:35:38 +0000 Received: from localhost ([127.0.0.1]:35328 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rshr6-00017c-49 for submit@debbugs.gnu.org; Fri, 05 Apr 2024 07:35:37 -0400 Received: from lists.gnu.org ([2001:470:142::17]:48480) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1rshr4-00016l-47 for submit@debbugs.gnu.org; Fri, 05 Apr 2024 07:35:34 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1rshqt-0001m4-6o for guix-patches@gnu.org; Fri, 05 Apr 2024 07:35:23 -0400 Received: from mira.cbaines.net ([212.71.252.8]) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1rshqd-0007AW-PN for guix-patches@gnu.org; Fri, 05 Apr 2024 07:35:22 -0400 Received: from localhost (unknown [212.132.255.10]) by mira.cbaines.net (Postfix) with ESMTPSA id C01F727BBE2; Fri, 5 Apr 2024 12:35:02 +0100 (BST) Received: from felis (localhost.lan [127.0.0.1]) by localhost (OpenSMTPD) with ESMTP id 4b249bca; Fri, 5 Apr 2024 11:35:02 +0000 (UTC) From: Christopher Baines To: John Fremlin via Guix-patches via Subject: Re: [bug#70175] [PATCH] gnu: llama-cpp: support OpenBLAS for faster prompt processing In-Reply-To: <7f9fe9d92aff101a12057e8571772428028dc6ae.1712202385.git.john@fremlin.org> (John Fremlin via Guix-patches via's message of "Wed, 3 Apr 2024 23:46:25 -0400") References: <7f9fe9d92aff101a12057e8571772428028dc6ae.1712202385.git.john@fremlin.org> User-Agent: mu4e 1.12.2; emacs 29.3 Date: Fri, 05 Apr 2024 12:35:02 +0100 Message-ID: <871q7kghzt.fsf@cbaines.net> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=212.71.252.8; envelope-from=mail@cbaines.net; helo=mira.cbaines.net X-Spam_score_int: -18 X-Spam_score: -1.9 X-Spam_bar: - X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-Debbugs-Envelope-To: submit Cc: John Fremlin , 70175-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) --=-=-= Content-Type: text/plain John Fremlin via Guix-patches via writes: > OpenBLAS is recommended by https://github.com/ggerganov/llama.cpp > > Change-Id: Iaf6f22252da13e2d6f503992878b35b0da7de0aa > --- > gnu/packages/machine-learning.scm | 5 ++++- > 1 file changed, 4 insertions(+), 1 deletion(-) Looks good to me, I tweaked the commit message a bit and pushed this to master as d8a63bbcee616f224c10462dbfb117ec009c50d8. Chris --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKlBAEBCgCPFiEEPonu50WOcg2XVOCyXiijOwuE9XcFAmYP4eZfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcRHG1haWxAY2Jh aW5lcy5uZXQACgkQXiijOwuE9XfO6BAAtxZdsEz/eVJTh/1r+U0chxX+DVw8a9y9 R3qG2Eiwi9Q9Pt4xhys/CB8KGFK7USrk22931rLLBib4PRyUFjxv2URtJ66eBMVB DSUN2kbhKjcljUnfo1/pgi9bVkVSaoWcEH7b52QfoV0AhPxbTWOFGxlNn9tCCv43 x/DLaAiRulKn51FgZBxpgtJYvaJ0GvFg7+ufyZWXpu05544sErYeMl+hpnsAgPMw H0YrnRbSV/Sym42hgft0+rkQACMZHbHrX8Pa0PjJnCqtiy394YwH2Dx/g3P/MYeH K2rYQ9YUKX/Fq6P510kcr1FPfN8h9S0VA3q+9Ao0LJf68ejSHSPhTxOLqJqOseZK XbdU7q5jO+ak0hU/7cgyOx0g0SlSM2tHCt+c09Goa3osfRE9G1stnl4Ib1OYAwyl 9YPIq3r84pQ7MBiFKwWH2qs4n8u680ah1LeJPhWXQTrAuAFh+UrZ/p76j185+aFv VmYJL6jeKSvaYILU/8t7FpNrM9UO9dr0NC/2j3oeXb1A3tp/ygtmsTCm+r9wnodK BWR9hMrU4O4SAqcr8y3xVK5cnN59yTSkVnVLTTSuIKeFdx8bL1jHcEBqJHaL4/kX +k/C1WU6Xdz8ZPqaRiG4BsbGorWfp1u2HLBKiS5aRZbv/CJAYEOcX+zn37Pyxh2F /W5FvZwW+Ak= =jNzz -----END PGP SIGNATURE----- --=-=-=-- From unknown Fri Sep 05 16:58:34 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sat, 04 May 2024 11:24:14 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator