From unknown Fri Aug 15 04:07:44 2025 X-Loop: help-debbugs@gnu.org Subject: bug#78846: Emacs hangs non-deterministically when eglot and clangd are used Resent-From: "admin@sonictk.com" Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Jun 2025 05:10:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 78846 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 78846@debbugs.gnu.org X-Debbugs-Original-To: "bug-gnu-emacs@gnu.org" Received: via spool by submit@debbugs.gnu.org id=B.17503961662330 (code B ref -1); Fri, 20 Jun 2025 05:10:02 +0000 Received: (at submit) by debbugs.gnu.org; 20 Jun 2025 05:09:26 +0000 Received: from localhost ([127.0.0.1]:43986 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1uSU0E-0000bT-3t for submit@debbugs.gnu.org; Fri, 20 Jun 2025 01:09:26 -0400 Received: from lists.gnu.org ([2001:470:142::17]:56346) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1uSU0B-0000aT-F5 for submit@debbugs.gnu.org; Fri, 20 Jun 2025 01:09:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSU03-0000Hn-Bc for bug-gnu-emacs@gnu.org; Fri, 20 Jun 2025 01:09:15 -0400 Received: from caracal.ash.relay.mailchannels.net ([23.83.222.30]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSU00-0007yr-MF for bug-gnu-emacs@gnu.org; Fri, 20 Jun 2025 01:09:14 -0400 X-Sender-Id: dreamhost|x-authsender|admin@sonictk.com Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 638CE2C4C42 for ; Fri, 20 Jun 2025 05:09:07 +0000 (UTC) Received: from pdx1-sub0-mail-a203.dreamhost.com (trex-green-2.trex.outbound.svc.cluster.local [100.96.75.209]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 0DACE2C5DD6 for ; Fri, 20 Jun 2025 05:09:07 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1750396147; a=rsa-sha256; cv=none; b=C3naDW2KwY7YAsnrWRxkXXX5QGgpYzx/YRaWjmRCW2cMaJC5hM7g0Wbi8iFeO/lkCpX+Vx 75O/ce+/e0vSlfspaisxPK9xRF/6cKwN48Bt+IoiDp4/QWFmvlErP6cbQRwo6U6u9JVFIB Sx46edRIRw41txuinxRVRRJhTc2n4xxqK2qklPy5laK2YVgx9tacge4Avq76fd5qyF527d baOTsQIEvTjPQgYdMCSYA2M3ag5rUxfbNm2i2WbRx2CjIk8uqpGYiDEv4NcH7O3Wg8l8Hl f37Ulybs8wikuHFkQoWkyr2asIeeQ9oMVA7LZoriXt34lt3IaNE4lhO/1rHXVg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1750396147; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding:dkim-signature; bh=X3ZlldxicZfMOYlxmjtdkFzYRfvpBISfqIFUpCC2KAk=; b=BX0I3isbXUzgZcT4vuqx4sYy6JYWw8NNuj1OxwkKFgWjvQ3itSGXhpjo9/r407q2F5AKIx qM1dc2ct8bUHClU0ox2BSiSWyG8nusijPV0DXO+K28KM5gh/wrJY46T9F+uWaTEuOsO56l XM9Kq9hAP4bcMOP6Aenj95udrLeNMUadCrKpB3NrW0596sM7G/jend9WMNuLkkCmDuBvfx HQ+ojMsF2jbC/HjY3fqKdqppKouXTPsQOvTyJ4wZXqtJhz1Z4oS/lC0VLX+7k1YPPdjxHu gSmFFastMJhOwfygAWtxkuqEHGYQFwL7Fi8EqJQLpwncvyWFBI5ASlH1SbsWxg== ARC-Authentication-Results: i=1; rspamd-6597f9cdc7-pqms9; auth=pass smtp.auth=dreamhost smtp.mailfrom=admin@sonictk.com X-Sender-Id: dreamhost|x-authsender|admin@sonictk.com X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|admin@sonictk.com X-MailChannels-Auth-Id: dreamhost X-Chemical-Oafish: 3486943f7e0a93e3_1750396147256_145218882 X-MC-Loop-Signature: 1750396147256:1120351296 X-MC-Ingress-Time: 1750396147256 Received: from pdx1-sub0-mail-a203.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.96.75.209 (trex/7.0.3); Fri, 20 Jun 2025 05:09:07 +0000 Received: from MN2PR20MB3133.namprd20.prod.outlook.com (unknown [52.96.64.37]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: admin@sonictk.com) by pdx1-sub0-mail-a203.dreamhost.com (Postfix) with ESMTPSA id 4bNlrL5hX3z27 for ; Thu, 19 Jun 2025 22:09:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sonictk.com; s=dreamhost; t=1750396146; bh=X3ZlldxicZfMOYlxmjtdkFzYRfvpBISfqIFUpCC2KAk=; h=From:To:Subject:Date:Content-Type:Content-Transfer-Encoding; b=FaEQvRMAMqtJSTobSld/ItMr2b/UoQ8RVfRldjHdKYYn1kHAWPbGE5lbP5H/Po99R LalVIv3Ky+z6sIhr1IAAvbBCFS6JjH5MAONhQ3Oekp379jtRTUI3odVvfZn8g1oFTi gI455rtHltk01Z1KmjVd0tZ4VfU4J/EYpowAQt6h16fRcy+J0gsn7ytPHtbvNpgZ4H SZmJonXnquZRxCqB18O4T9xFHVVk5pof8s/uVENDc7cSfIVEKgzg26J8mXaQmJtcZV 2a50lHZsOZ2Pq6nYgZs3IgjtFThGVdCgXbzmGlWyVBmUp3zBSwJW8fsHCA6j58Rzzb sxIerEGLLIITQ== From: "admin@sonictk.com" Thread-Topic: Emacs hangs non-deterministically when eglot and clangd are used Thread-Index: AQHb4aBjT5QqzDCOz0eKbpbsg4uYUg== X-MS-Exchange-MessageSentRepresentingType: 1 Date: Fri, 20 Jun 2025 05:09:06 +0000 Message-ID: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-Exchange-Organization-SCL: -1 X-MS-TNEF-Correlator: X-MS-Exchange-Organization-RecordReviewCfmType: 0 msip_labels: Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 Received-SPF: pass client-ip=23.83.222.30; envelope-from=admin@sonictk.com; helo=caracal.ash.relay.mailchannels.net X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, RCVD_IN_VALIDITY_SAFE_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: 0.9 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.1 (/) Hi:=0A= =0A= This has been a problem ever since as long as I can remember, but using `eg= lot`=A0with `clangd` results in Emacs sometimes just=A0hanging. I work in t= he Unreal Engine codebase, which results in some pretty massive `clangd` me= mory usage (sometimes 200+ GB) and thus response times, along with having t= o spin up multiple `clangd` servers simultaneously.=0A= =0A= As far as I can tell, Emacs can hang in cases with as little as two `clangd= ` servers spun up, but generally this problem repros more often when I have= multiple servers spun up. It usually happens on some LSP request, and it's= not always deterministic which request it is.=0A= =0A= I work on Windows, and I grabbed a callstack of Emacs's main thread when th= e hang occurs in WinDbg:=0A= =0A= ```=0A= [0x0] ntdll!NtWaitForMultipleObjects+0x14 0x3df75fdea8 0x7ffdf18abaf0= =0A= [0x1] KERNELBASE!WaitForMultipleObjectsEx+0xf0 0x3df75fdeb0 0x7ffdf18= ab9ee =0A= [0x2] KERNELBASE!WaitForMultipleObjects+0xe 0x3df75fe1a0 0x7ff635a921= f7 =0A= [0x3] emacs!sys_select+0x12b7 0x3df75fe1e0 0x7ff635a1147e =0A= [0x4] emacs!really_call_select+0x5e 0x3df75fe2a0 0x7ff635a1273d =0A= [0x5] emacs!thread_select+0x9d 0x3df75fe300 0x7ff6359d9aec =0A= [0x6] emacs!wait_reading_process_output+0x111c 0x3df75fe450 0x7ff6359= db4c9 =0A= [0x7] emacs!send_process+0x269 0x3df75fe9c0 0x7ff6359dbd11 =0A= [0x8] emacs!Fprocess_send_string+0xb1 0x3df75fea70 0x7ff6359b5c5e = =0A= [0x9] emacs!exec_byte_code+0x7ee 0x3df75feab0 0x7ff63592eefe =0A= [0xa] emacs!Ffuncall+0xfe 0x3df75feb90 0x7ff63592f3a1 =0A= [0xb] emacs!Fapply+0x141 0x3df75fec20 0x7ff6359b5c5e =0A= [0xc] emacs!exec_byte_code+0x7ee 0x3df75fed00 0x7ff63592eefe =0A= [0xd] emacs!Ffuncall+0xfe 0x3df75fede0 0x7ffd967e587e =0A= [0xe] jsonrpc_e62a9c36_c0dd1fe3!F6a736f6e7270632d6e6f74696679_jsonrpc_not= ify_0+0x4e 0x3df75fee70 0x7ff63592eefe =0A= [0xf] emacs!Ffuncall+0xfe 0x3df75feed0 0x7ffd96ebc1b7 =0A= [0x10] eglot_3726620f_1a2b9d18!F65676c6f742d2d63616e63656c2d696e666c69676= 8742d6173796e632d7265717565737473_eglot__cancel_inflight_async_requests_0+0= x1a7 0x3df75fef60 0x7ff63592eefe =0A= [0x11] emacs!Ffuncall+0xfe 0x3df75ff030 0x7ffd96ec657e =0A= [0x12] eglot_3726620f_1a2b9d18!F65676c6f742d2d7072652d636f6d6d616e642d686= f6f6b_eglot__pre_command_hook_0+0x3e 0x3df75ff0c0 0x7ff63592eefe =0A= [0x13] emacs!Ffuncall+0xfe 0x3df75ff100 0x7ff635927a05 =0A= [0x14] emacs!internal_condition_case_n+0x45 0x3df75ff190 0x7ff63584f3= 84 =0A= [0x15] emacs!safe_run_hook_funcall+0xa4 0x3df75ff1d0 0x7ff635927ffb = =0A= [0x16] emacs!run_hook_with_args+0x9b 0x3df75ff260 0x7ff635868e7c = =0A= [0x17] emacs!command_loop_1+0x3cc 0x3df75ff2d0 0x7ff635927839 =0A= [0x18] emacs!internal_condition_case+0x39 0x3df75ff4b0 0x7ff63584ca56= =0A= [0x19] emacs!command_loop_2+0x26 0x3df75ff4f0 0x7ff6359277a7 =0A= [0x1a] emacs!internal_catch+0x37 0x3df75ff520 0x7ff63584c9ec =0A= [0x1b] emacs!command_loop+0x15c 0x3df75ff560 0x7ff6358571e7 =0A= [0x1c] emacs!recursive_edit_1+0x87 0x3df75ff670 0x7ff6358575e4 =0A= [0x1d] emacs!Frecursive_edit+0xe4 0x3df75ff6c0 0x7ff635aeac77 =0A= [0x1e] emacs!main+0x2787 0x3df75ff6f0 0x7ff635701318 =0A= [0x1f] emacs!__tmainCRTStartup+0x168 0x3df75ffc20 0x7ff635701426 = =0A= [0x20] emacs!mainCRTStartup+0x16 0x3df75ffc70 0x7ffdf3d37374 =0A= [0x21] KERNEL32!BaseThreadInitThunk+0x14 0x3df75ffca0 0x7ffdf3edcc91 = =0A= [0x22] ntdll!RtlUserThreadStart+0x21 0x3df75ffcd0 0x0 =0A= ```=0A= =0A= The only workaround, without killing Emacs, is to kill all `clangd.exe` pro= cesses - this usually gets Emacs unstuck after a second or so. Sometimes, t= hough, this trick doesn't work, and then the only recourse I have is to kil= l Emacs.exe entirely.=0A= =0A= I've been trying to find time on and off to debug this properly, but today = I finally gave up and decided to report this in the hope that maybe someone= will see this callstack and know immediately where the bug is.=0A= =0A= My emacs build was built from source. Version information is as follows:=0A= =0A= ```=0A= In GNU Emacs 31.0.50 (build 5, x86_64-w64-mingw32) of 2025-06-03 built=0A= on CDW-AQRHE1HHT39=0A= Repository revision: eb788fd8fd2026fa4d29b918ff95b12d8e3e0bab=0A= Repository branch: master=0A= Windowing system distributor 'Microsoft Corp.', version 10.0.19045=0A= System Description: Microsoft Windows 10 Enterprise (v10.0.2009.19045.5965)= =0A= =0A= Configured using:=0A= 'configure --without-pop --with-imagemagick=0A= --without-compress-install -without-dbus --with-gnutls=0A= --with-tree-sitter --without-gconf --with-rsvg --without-gsettings=0A= --with-mailutils --with-native-compilation --with-modules --with-xml2=0A= --with-wide-int 'CFLAGS=3D-O3 -ggdb -fno-math-errno=0A= -funsafe-math-optimizations -fno-finite-math-only -fno-trapping-math=0A= -freciprocal-math -fno-rounding-math -fno-signaling-nans=0A= -fassociative-math -fno-signed-zeros -frename-registers=0A= -funroll-loops -mtune=3Dnative -march=3Dnative -fomit-frame-pointer=0A= -fallow-store-data-races -fno-semantic-interposition=0A= -floop-parallelize-all -ftree-parallelize-loops=3D4'=0A= PKG_CONFIG_PATH=3D/mingw64/lib/pkgconfig:/mingw64/share/pkgconfig'=0A= ```=0A= =0A= I haven't been able to find an isolated test case for this to happen determ= inistically.=0A= =0A= Anyone who has ideas for me to try/look in debugging this issue, happy to g= ive them a go. I encounter this fairly frequently, so I should be able to c= atch it in action.=0A= =0A= Thanks.=0A= =0A= - Yi Liang= From unknown Fri Aug 15 04:07:44 2025 X-Loop: help-debbugs@gnu.org Subject: bug#78846: Emacs hangs non-deterministically when eglot and clangd are used Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Fri, 20 Jun 2025 07:48:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 78846 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: "admin@sonictk.com" Cc: 78846@debbugs.gnu.org Received: via spool by 78846-submit@debbugs.gnu.org id=B78846.17504056427320 (code B ref 78846); Fri, 20 Jun 2025 07:48:02 +0000 Received: (at 78846) by debbugs.gnu.org; 20 Jun 2025 07:47:22 +0000 Received: from localhost ([127.0.0.1]:45976 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1uSWT3-0001tz-Ou for submit@debbugs.gnu.org; Fri, 20 Jun 2025 03:47:22 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41844) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1uSWSz-0001tf-Ou for 78846@debbugs.gnu.org; Fri, 20 Jun 2025 03:47:19 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uSWSs-00033z-Il; Fri, 20 Jun 2025 03:47:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=YfU7AGSB7d28fDa0/s/IjsaruVtPEV+x2VNGEJBis8k=; b=hz+5o1rKRyUFZXQfWk69 FVRqI/8AWNsAvkf06IVWzfS2hXLluxU7VYF05Nso9AAezyvy7f1K9rA2Mwr4H95JoOvwsIUzSVgBE h4xpPheRfIyb1bzXf0DW4MfeGd92W44yKRgF7scB2T6gYewsBCFY7YEoptUEsmtLGXlrGPY5AKkhN 5FuT1QfaXrwFww9uO2435YUq3pO33rMrwhXCWi+u3CBk9bFkmUJNAw1AIA7C4z+GjtSnxfo4VEjiE VbDhqmHyvw9HbUU3FPrH1jHTwR2e5MKIvOCUB2q6Xq+gQe2G55+ShdB/qzKwoUEC2bNAGz27JZPrx b2IS/OhZG9tkeQ==; Date: Fri, 20 Jun 2025 10:46:51 +0300 Message-Id: <86y0tmkg7o.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: (admin@sonictk.com) References: MIME-version: 1.0 Content-type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: "admin@sonictk.com" > Date: Fri, 20 Jun 2025 05:09:06 +0000 > > This has been a problem ever since as long as I can remember, but > using `eglot` with `clangd` results in Emacs sometimes just hanging. Does this happen only in Emacs 31, or did you see that in older versions as well? > I work in the Unreal Engine codebase, which results in some pretty massive `clangd` memory usage (sometimes 200+ GB) and thus response times, along with having to spin up multiple `clangd` servers simultaneously. How much VM do you have on that system, if memory consumption can be 200+ GB? And what is the memory footprint of Emacs in those cases? > As far as I can tell, Emacs can hang in cases with as little as two `clangd` servers spun up, but generally this problem repros more often when I have multiple servers spun up. It usually happens on some LSP request, and it's not always deterministic which request it is. How many LSP servers could you have simultaneously? > I work on Windows, and I grabbed a callstack of Emacs's main thread when the hang occurs in WinDbg: This doesn't tell the whole story, because there must be other threads of interest in this case (since one or more clangd processes are being read from and written to). > ``` > [0x0] ntdll!NtWaitForMultipleObjects+0x14 0x3df75fdea8 0x7ffdf18abaf0 > [0x1] KERNELBASE!WaitForMultipleObjectsEx+0xf0 0x3df75fdeb0 0x7ffdf18ab9ee > [0x2] KERNELBASE!WaitForMultipleObjects+0xe 0x3df75fe1a0 0x7ff635a921f7 > [0x3] emacs!sys_select+0x12b7 0x3df75fe1e0 0x7ff635a1147e > [0x4] emacs!really_call_select+0x5e 0x3df75fe2a0 0x7ff635a1273d > [0x5] emacs!thread_select+0x9d 0x3df75fe300 0x7ff6359d9aec > [0x6] emacs!wait_reading_process_output+0x111c 0x3df75fe450 0x7ff6359db4c9 > [0x7] emacs!send_process+0x269 0x3df75fe9c0 0x7ff6359dbd11 > [0x8] emacs!Fprocess_send_string+0xb1 0x3df75fea70 0x7ff6359b5c5e This says that Emacs called process-send-string, and it loops waiting for the queue of sent material to be emptied. This information is not enough to analyze the reason for the hang. > The only workaround, without killing Emacs, is to kill all `clangd.exe` processes - this usually gets Emacs unstuck after a second or so. Sometimes, though, this trick doesn't work, and then the only recourse I have is to kill Emacs.exe entirely. > > I've been trying to find time on and off to debug this properly, but today I finally gave up and decided to report this in the hope that maybe someone will see this callstack and know immediately where the bug is. > > My emacs build was built from source. Version information is as follows: > > ``` > In GNU Emacs 31.0.50 (build 5, x86_64-w64-mingw32) of 2025-06-03 built > on CDW-AQRHE1HHT39 > Repository revision: eb788fd8fd2026fa4d29b918ff95b12d8e3e0bab > Repository branch: master > Windowing system distributor 'Microsoft Corp.', version 10.0.19045 > System Description: Microsoft Windows 10 Enterprise (v10.0.2009.19045.5965) This build is from 2 weeks ago. Please update from master and rebuild, so that the source-level information you report will be easy to follow by looking at the current sources. > Configured using: > 'configure --without-pop --with-imagemagick > --without-compress-install -without-dbus --with-gnutls > --with-tree-sitter --without-gconf --with-rsvg --without-gsettings > --with-mailutils --with-native-compilation --with-modules --with-xml2 > --with-wide-int 'CFLAGS=-O3 -ggdb -fno-math-errno First, please remove the --with-wide-int, it is not needed for 64-bit builds (and is supposed to be a no-op, but who knows?). Also, please don't use -O3, but -O2 instead. The -O3 switch could cause unsafe optimizations. > -funsafe-math-optimizations -fno-finite-math-only -fno-trapping-math > -freciprocal-math -fno-rounding-math -fno-signaling-nans > -fassociative-math -fno-signed-zeros -frename-registers > -funroll-loops -mtune=native -march=native -fomit-frame-pointer > -fallow-store-data-races -fno-semantic-interposition > -floop-parallelize-all -ftree-parallelize-loops=4' Please also drop all these -fSOMETHING and -f-noSOMETHING switches; they are not needed with -O2. And fomit-frame-pointer is actually dangerous in Emacs. > PKG_CONFIG_PATH=/mingw64/lib/pkgconfig:/mingw64/share/pkgconfig' Don't know what that is, or why do you need it. > Anyone who has ideas for me to try/look in debugging this issue, happy to give them a go. I encounter this fairly frequently, so I should be able to catch it in action. Next time Emacs hangs like that, attach GDB to it, then type at the GDB prompt: (gdb) thread apply all bt and post here everything that GDB produces as result. This will show us the C-level backtraces of all the threads that run in the process, not only of the main thread. When Emacs communicates with external processes, it uses additional threads for that purpose, and it is important to know their state and status. (Let me know if you need instructions about attaching GDB to a running Emacs process.) Thanks. From unknown Fri Aug 15 04:07:44 2025 X-Loop: help-debbugs@gnu.org Subject: bug#78846: Emacs hangs non-deterministically when eglot and clangd are used Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 05 Jul 2025 08:16:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 78846 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: admin@sonictk.com Cc: 78846@debbugs.gnu.org Received: via spool by 78846-submit@debbugs.gnu.org id=B78846.175170332432536 (code B ref 78846); Sat, 05 Jul 2025 08:16:02 +0000 Received: (at 78846) by debbugs.gnu.org; 5 Jul 2025 08:15:24 +0000 Received: from localhost ([127.0.0.1]:39545 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1uXy3Q-0008Sg-7o for submit@debbugs.gnu.org; Sat, 05 Jul 2025 04:15:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:36968) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1uXy3M-0008RN-JT for 78846@debbugs.gnu.org; Sat, 05 Jul 2025 04:15:22 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uXy3F-0000ou-BY; Sat, 05 Jul 2025 04:15:14 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From: Date; bh=yBLwWSGKGmYtHKyX0AW53bXqqSyULsZvPn/vZhgI2wU=; b=RYAqa0msO2LDnybMZtCh BuTQ68rlqPrkHXQsZS+rs3UVrZ53ZFzQhDaZyvyXqJK/fQxplgvpU3lro4uVVs/r/AU4m0+DFdI+9 pVffBo3bQuVgzYbF/y5jffFQXGEVSdT2zU4ThjW4K7bY94iYnEyl6zuKclkLLJAvXZau+MAevIwWG XsLQKQNuYVnqHX8ex0PG/eMSo7tMQ8FZdNJ1S1gKip8FI0NpNMziMs2fctLWCqJx5fIk5tPIkIzqq QHc11DgODMT35/4IZQFtY12YUZkDoAcB4BFY/xaAmsjoMeqNC1m/gQFvlPtWynyd1a4K/d6JhF7Aw Mt2Tu9JFRgdv9w==; Date: Sat, 05 Jul 2025 11:14:46 +0300 Message-Id: <86ikk7vytl.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: <86y0tmkg7o.fsf@gnu.org> (message from Eli Zaretskii on Fri, 20 Jun 2025 10:46:51 +0300) References: <86y0tmkg7o.fsf@gnu.org> MIME-version: 1.0 Content-type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ping! Could you please answer my questions, and perhaps provide the additional information I asked for? I'd like to make some progress in investigating this problem. > Date: Fri, 20 Jun 2025 10:46:51 +0300 > From: Eli Zaretskii > Cc: 78846@debbugs.gnu.org > > > From: "admin@sonictk.com" > > Date: Fri, 20 Jun 2025 05:09:06 +0000 > > > > This has been a problem ever since as long as I can remember, but > > using `eglot` with `clangd` results in Emacs sometimes just hanging. > > Does this happen only in Emacs 31, or did you see that in older > versions as well? > > > I work in the Unreal Engine codebase, which results in some pretty massive `clangd` memory usage (sometimes 200+ GB) and thus response times, along with having to spin up multiple `clangd` servers simultaneously. > > How much VM do you have on that system, if memory consumption can be > 200+ GB? And what is the memory footprint of Emacs in those cases? > > > As far as I can tell, Emacs can hang in cases with as little as two `clangd` servers spun up, but generally this problem repros more often when I have multiple servers spun up. It usually happens on some LSP request, and it's not always deterministic which request it is. > > How many LSP servers could you have simultaneously? > > > I work on Windows, and I grabbed a callstack of Emacs's main thread when the hang occurs in WinDbg: > > This doesn't tell the whole story, because there must be other > threads of interest in this case (since one or more clangd processes > are being read from and written to). > > > ``` > > [0x0] ntdll!NtWaitForMultipleObjects+0x14 0x3df75fdea8 0x7ffdf18abaf0 > > [0x1] KERNELBASE!WaitForMultipleObjectsEx+0xf0 0x3df75fdeb0 0x7ffdf18ab9ee > > [0x2] KERNELBASE!WaitForMultipleObjects+0xe 0x3df75fe1a0 0x7ff635a921f7 > > [0x3] emacs!sys_select+0x12b7 0x3df75fe1e0 0x7ff635a1147e > > [0x4] emacs!really_call_select+0x5e 0x3df75fe2a0 0x7ff635a1273d > > [0x5] emacs!thread_select+0x9d 0x3df75fe300 0x7ff6359d9aec > > [0x6] emacs!wait_reading_process_output+0x111c 0x3df75fe450 0x7ff6359db4c9 > > [0x7] emacs!send_process+0x269 0x3df75fe9c0 0x7ff6359dbd11 > > [0x8] emacs!Fprocess_send_string+0xb1 0x3df75fea70 0x7ff6359b5c5e > > This says that Emacs called process-send-string, and it loops waiting > for the queue of sent material to be emptied. This information is not > enough to analyze the reason for the hang. > > > The only workaround, without killing Emacs, is to kill all `clangd.exe` processes - this usually gets Emacs unstuck after a second or so. Sometimes, though, this trick doesn't work, and then the only recourse I have is to kill Emacs.exe entirely. > > > > I've been trying to find time on and off to debug this properly, but today I finally gave up and decided to report this in the hope that maybe someone will see this callstack and know immediately where the bug is. > > > > My emacs build was built from source. Version information is as follows: > > > > ``` > > In GNU Emacs 31.0.50 (build 5, x86_64-w64-mingw32) of 2025-06-03 built > > on CDW-AQRHE1HHT39 > > Repository revision: eb788fd8fd2026fa4d29b918ff95b12d8e3e0bab > > Repository branch: master > > Windowing system distributor 'Microsoft Corp.', version 10.0.19045 > > System Description: Microsoft Windows 10 Enterprise (v10.0.2009.19045.5965) > > This build is from 2 weeks ago. Please update from master and > rebuild, so that the source-level information you report will be easy > to follow by looking at the current sources. > > > Configured using: > > 'configure --without-pop --with-imagemagick > > --without-compress-install -without-dbus --with-gnutls > > --with-tree-sitter --without-gconf --with-rsvg --without-gsettings > > --with-mailutils --with-native-compilation --with-modules --with-xml2 > > --with-wide-int 'CFLAGS=-O3 -ggdb -fno-math-errno > > First, please remove the --with-wide-int, it is not needed for 64-bit > builds (and is supposed to be a no-op, but who knows?). Also, please > don't use -O3, but -O2 instead. The -O3 switch could cause unsafe > optimizations. > > > -funsafe-math-optimizations -fno-finite-math-only -fno-trapping-math > > -freciprocal-math -fno-rounding-math -fno-signaling-nans > > -fassociative-math -fno-signed-zeros -frename-registers > > -funroll-loops -mtune=native -march=native -fomit-frame-pointer > > -fallow-store-data-races -fno-semantic-interposition > > -floop-parallelize-all -ftree-parallelize-loops=4' > > Please also drop all these -fSOMETHING and -f-noSOMETHING switches; > they are not needed with -O2. And fomit-frame-pointer is actually > dangerous in Emacs. > > > PKG_CONFIG_PATH=/mingw64/lib/pkgconfig:/mingw64/share/pkgconfig' > > Don't know what that is, or why do you need it. > > > Anyone who has ideas for me to try/look in debugging this issue, happy to give them a go. I encounter this fairly frequently, so I should be able to catch it in action. > > Next time Emacs hangs like that, attach GDB to it, then type at the > GDB prompt: > > (gdb) thread apply all bt > > and post here everything that GDB produces as result. This will show > us the C-level backtraces of all the threads that run in the process, > not only of the main thread. When Emacs communicates with external > processes, it uses additional threads for that purpose, and it is > important to know their state and status. > > (Let me know if you need instructions about attaching GDB to a running > Emacs process.) > > Thanks. > > > > From unknown Fri Aug 15 04:07:44 2025 X-Loop: help-debbugs@gnu.org Subject: bug#78846: Emacs hangs non-deterministically when eglot and clangd are used Resent-From: "admin@sonictk.com" Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 07 Jul 2025 06:46:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 78846 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Eli Zaretskii Cc: "78846@debbugs.gnu.org" <78846@debbugs.gnu.org> Received: via spool by 78846-submit@debbugs.gnu.org id=B78846.175187074813529 (code B ref 78846); Mon, 07 Jul 2025 06:46:02 +0000 Received: (at 78846) by debbugs.gnu.org; 7 Jul 2025 06:45:48 +0000 Received: from localhost ([127.0.0.1]:60844 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1uYfbl-0003Vq-SW for submit@debbugs.gnu.org; Mon, 07 Jul 2025 02:45:48 -0400 Received: from crocodile.elm.relay.mailchannels.net ([23.83.212.45]:59869) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1uYfbi-0003VT-Nb for 78846@debbugs.gnu.org; Mon, 07 Jul 2025 02:45:44 -0400 X-Sender-Id: dreamhost|x-authsender|admin@sonictk.com Received: from relay.mailchannels.net (localhost [127.0.0.1]) by relay.mailchannels.net (Postfix) with ESMTP id 8D67322F52; Mon, 7 Jul 2025 06:45:40 +0000 (UTC) Received: from pdx1-sub0-mail-a312.dreamhost.com (100-100-179-10.trex-nlb.outbound.svc.cluster.local [100.100.179.10]) (Authenticated sender: dreamhost) by relay.mailchannels.net (Postfix) with ESMTPA id 2A30922B49; Mon, 7 Jul 2025 06:45:40 +0000 (UTC) ARC-Seal: i=1; s=arc-2022; d=mailchannels.net; t=1751870740; a=rsa-sha256; cv=none; b=43KZtT16lcxRRvUzENkAco4VV5v98tMcaex8nGT6ptMR/5oFZhCX4crEjNFVwBanLhdG34 xSXHCjO3Q6HIj5SfzsTYpSyQXXtZKIseQZeql0bmGFF0nj8/cO53rQQVYtxhT/uOoUTc1/ j5ZXfW+Yeiz6jZMYKCmfpTjJ2oXa836FK0EAH0/bIZ84Env22ylAN0+KVvjVcRcvx/quGO KGcAgj4xQblGq5TzcNubv4/6qTAf1noD9QRKJmmh2U3vxPhIPOmXRMmHAs7JWuOCeobHDd qhzoNXTUcij1tkfZMIqt/DmubpE/uxgcdIQHo1kuhBbFyCQFB/YIq3t14nmlmQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=mailchannels.net; s=arc-2022; t=1751870740; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references:dkim-signature; bh=AMJmRmBCpJWOof9ldB/IS1Qf5n14V5F0RwCNFXtwgkI=; b=m8pHa69eY575aroVHTSaazxaXHklrHHgh2XjP5UCypG5+Ml0qTgMle50LLc6wuK5F2xZHe i72Lzi/khE4KnKLBPGIA2c7TFDOuf1Mr87aFcrkK0N9ihBwv6iZOoJJME7gI8qcEY46BCm /r3+aHoUo6r/HkwOkKtwta74NZTSb1OI/+KuMbq+Bp0RM0WGXIIiFwWkW5K+YFl+8n99IW EZvVDQN/vjWuqotPLpG7Wnpn4s9ikz03s7P0+zbQsaDnVcyfmfapiUucGihCFsJOHA1EJ8 wAWnG4yzLDVqK4bAT65oMXh5ZaJx9f82QZI0KG7XsBTWzYlNrro99GZgTYfMTw== ARC-Authentication-Results: i=1; rspamd-5c976dc8b-8gsv4; auth=pass smtp.auth=dreamhost smtp.mailfrom=admin@sonictk.com X-Sender-Id: dreamhost|x-authsender|admin@sonictk.com X-MC-Relay: Neutral X-MailChannels-SenderId: dreamhost|x-authsender|admin@sonictk.com X-MailChannels-Auth-Id: dreamhost X-Average-Abortive: 4a93020a7d324303_1751870740422_4097049331 X-MC-Loop-Signature: 1751870740422:1046255386 X-MC-Ingress-Time: 1751870740422 Received: from pdx1-sub0-mail-a312.dreamhost.com (pop.dreamhost.com [64.90.62.162]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384) by 100.100.179.10 (trex/7.1.3); Mon, 07 Jul 2025 06:45:40 +0000 Received: from MN2PR20MB3133.namprd20.prod.outlook.com (unknown [52.96.64.37]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) (Authenticated sender: admin@sonictk.com) by pdx1-sub0-mail-a312.dreamhost.com (Postfix) with ESMTPSA id 4bbF9v56BLzMP; Sun, 6 Jul 2025 23:45:39 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sonictk.com; s=dreamhost; t=1751870740; bh=AMJmRmBCpJWOof9ldB/IS1Qf5n14V5F0RwCNFXtwgkI=; h=From:To:CC:Subject:Date:Content-Type; b=BxLuH0Ziq9tB7CYVKnifMcQAOK+Q/8C9llCWsoUPyrPGiMibqvMF3vEu9W4MtRxYh yn8oAoYTaT237P6At9vFHenurOBLF++VvijdSC7bX4/6r5G5wU3l+bk6PY1hVlINUB GdAxTTC0ZdolP6MKMdXuu81bk0SK/uv1aW2nR6rJmFStIa6zJEOhJ3qJzjJ6ax6wXC 3uyrRitW+kAKSOE6SI4ZnHleuY9ZU4bhiFF4RqaWxWvbBwkA5g+5gXJd8aNPP5iD5R DCsMBoWOHkPLXTDx1J7IpFmVmDvphXPxIMD4pa8uEzw29Hn6MFYF2murmCUXvtpVSF ZOFoDP3iau/EQ== From: "admin@sonictk.com" Thread-Topic: bug#78846: Emacs hangs non-deterministically when eglot and clangd are used Thread-Index: AQHb4aBjT5QqzDCOz0eKbpbsg4uYUmZmb2dtZmZvZ21mZmx5N6qMoMj7 X-MS-Exchange-MessageSentRepresentingType: 1 Date: Mon, 7 Jul 2025 05:26:31 +0000 Message-ID: References: <86y0tmkg7o.fsf@gnu.org> <86ikk7vytl.fsf@gnu.org> In-Reply-To: <86ikk7vytl.fsf@gnu.org> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-Exchange-Organization-SCL: -1 X-MS-TNEF-Correlator: X-MS-Exchange-Organization-RecordReviewCfmType: 0 msip_labels: Content-Type: multipart/alternative; boundary="_000_MN2PR20MB3133DC2450DD62245F4F5834AB4FAMN2PR20MB3133namp_" MIME-Version: 1.0 X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --_000_MN2PR20MB3133DC2450DD62245F4F5834AB4FAMN2PR20MB3133namp_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Hi Eli: Sorry for the delayed response. I actually find that after removing a bunch= of the optimization flags (those were just from some recipe that I copied = for compiling Emacs on Windows), the specific problem of Emacs itself deadl= ocking goes away, though I still find Emacs will occasionally hang on the m= ain thread, and trying to attach gdb to it via MSYS2 (which is what I use t= o compile Emacs with) doesn't work - however, it will recover after a non-s= pecific amount of time has elapsed (which doesn't seem to have any relation= to any of the timeout settings for eglot . I haven't had a chance to dig i= nto this further yet, but it's definitely an improvement over having to ter= minate all clangd processes or restarting Emacs. * Yi Liang ________________________________ From: Eli Zaretskii Sent: Saturday, July 5, 2025 1:14 AM To: admin@sonictk.com Cc: 78846@debbugs.gnu.org <78846@debbugs.gnu.org> Subject: Re: bug#78846: Emacs hangs non-deterministically when eglot and cl= angd are used Ping! Could you please answer my questions, and perhaps provide the additional information I asked for? I'd like to make some progress in investigating this problem. > Date: Fri, 20 Jun 2025 10:46:51 +0300 > From: Eli Zaretskii > Cc: 78846@debbugs.gnu.org > > > From: "admin@sonictk.com" > > Date: Fri, 20 Jun 2025 05:09:06 +0000 > > > > This has been a problem ever since as long as I can remember, but > > using `eglot` with `clangd` results in Emacs sometimes just hanging. > > Does this happen only in Emacs 31, or did you see that in older > versions as well? > > > I work in the Unreal Engine codebase, which results in some pretty mass= ive `clangd` memory usage (sometimes 200+ GB) and thus response times, alon= g with having to spin up multiple `clangd` servers simultaneously. > > How much VM do you have on that system, if memory consumption can be > 200+ GB? And what is the memory footprint of Emacs in those cases? > > > As far as I can tell, Emacs can hang in cases with as little as two `cl= angd` servers spun up, but generally this problem repros more often when I = have multiple servers spun up. It usually happens on some LSP request, and = it's not always deterministic which request it is. > > How many LSP servers could you have simultaneously? > > > I work on Windows, and I grabbed a callstack of Emacs's main thread whe= n the hang occurs in WinDbg: > > This doesn't tell the whole story, because there must be other > threads of interest in this case (since one or more clangd processes > are being read from and written to). > > > ``` > > [0x0] ntdll!NtWaitForMultipleObjects+0x14 0x3df75fdea8 0x7ffdf18a= baf0 > > [0x1] KERNELBASE!WaitForMultipleObjectsEx+0xf0 0x3df75fdeb0 0x7ff= df18ab9ee > > [0x2] KERNELBASE!WaitForMultipleObjects+0xe 0x3df75fe1a0 0x7ff635= a921f7 > > [0x3] emacs!sys_select+0x12b7 0x3df75fe1e0 0x7ff635a1147e > > [0x4] emacs!really_call_select+0x5e 0x3df75fe2a0 0x7ff635a1273d > > [0x5] emacs!thread_select+0x9d 0x3df75fe300 0x7ff6359d9aec > > [0x6] emacs!wait_reading_process_output+0x111c 0x3df75fe450 0x7ff= 6359db4c9 > > [0x7] emacs!send_process+0x269 0x3df75fe9c0 0x7ff6359dbd11 > > [0x8] emacs!Fprocess_send_string+0xb1 0x3df75fea70 0x7ff6359b5c5e > > This says that Emacs called process-send-string, and it loops waiting > for the queue of sent material to be emptied. This information is not > enough to analyze the reason for the hang. > > > The only workaround, without killing Emacs, is to kill all `clangd.exe`= processes - this usually gets Emacs unstuck after a second or so. Sometime= s, though, this trick doesn't work, and then the only recourse I have is to= kill Emacs.exe entirely. > > > > I've been trying to find time on and off to debug this properly, but to= day I finally gave up and decided to report this in the hope that maybe som= eone will see this callstack and know immediately where the bug is. > > > > My emacs build was built from source. Version information is as follows= : > > > > ``` > > In GNU Emacs 31.0.50 (build 5, x86_64-w64-mingw32) of 2025-06-03 built > > on CDW-AQRHE1HHT39 > > Repository revision: eb788fd8fd2026fa4d29b918ff95b12d8e3e0bab > > Repository branch: master > > Windowing system distributor 'Microsoft Corp.', version 10.0.19045 > > System Description: Microsoft Windows 10 Enterprise (v10.0.2009.19045.5= 965) > > This build is from 2 weeks ago. Please update from master and > rebuild, so that the source-level information you report will be easy > to follow by looking at the current sources. > > > Configured using: > > 'configure --without-pop --with-imagemagick > > --without-compress-install -without-dbus --with-gnutls > > --with-tree-sitter --without-gconf --with-rsvg --without-gsettings > > --with-mailutils --with-native-compilation --with-modules --with-xml2 > > --with-wide-int 'CFLAGS=3D-O3 -ggdb -fno-math-errno > > First, please remove the --with-wide-int, it is not needed for 64-bit > builds (and is supposed to be a no-op, but who knows?). Also, please > don't use -O3, but -O2 instead. The -O3 switch could cause unsafe > optimizations. > > > -funsafe-math-optimizations -fno-finite-math-only -fno-trapping-math > > -freciprocal-math -fno-rounding-math -fno-signaling-nans > > -fassociative-math -fno-signed-zeros -frename-registers > > -funroll-loops -mtune=3Dnative -march=3Dnative -fomit-frame-pointer > > -fallow-store-data-races -fno-semantic-interposition > > -floop-parallelize-all -ftree-parallelize-loops=3D4' > > Please also drop all these -fSOMETHING and -f-noSOMETHING switches; > they are not needed with -O2. And fomit-frame-pointer is actually > dangerous in Emacs. > > > PKG_CONFIG_PATH=3D/mingw64/lib/pkgconfig:/mingw64/share/pkgconfig' > > Don't know what that is, or why do you need it. > > > Anyone who has ideas for me to try/look in debugging this issue, happy = to give them a go. I encounter this fairly frequently, so I should be able = to catch it in action. > > Next time Emacs hangs like that, attach GDB to it, then type at the > GDB prompt: > > (gdb) thread apply all bt > > and post here everything that GDB produces as result. This will show > us the C-level backtraces of all the threads that run in the process, > not only of the main thread. When Emacs communicates with external > processes, it uses additional threads for that purpose, and it is > important to know their state and status. > > (Let me know if you need instructions about attaching GDB to a running > Emacs process.) > > Thanks. > > > > --_000_MN2PR20MB3133DC2450DD62245F4F5834AB4FAMN2PR20MB3133namp_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable
Hi Eli:

Sorry for the delayed response. I actually find that after removing a bunch= of the optimization flags (those were just from some recipe that I copied = for compiling Emacs on Windows), the specific problem of Emacs itself deadl= ocking goes away, though I still find Emacs will occasionally hang on the main thread, and trying to attach= gdb to it via MSYS2 (which is what I use to compile Emacs with)= doesn't work - however, it will recover after a non-specific amount of tim= e has elapsed (which doesn't seem to have any relation to any of the timeou= t settings for eglot . I haven't had a chance to dig into this further yet, b= ut it's definitely an improvement over having to terminate all clangd proce= sses or restarting Emacs.

  • Yi Liang
From: Eli Zaretskii <eli= z@gnu.org>
Sent: Saturday, July 5, 2025 1:14 AM
To: admin@sonictk.com <admin@sonictk.com>
Cc: 78846@debbugs.gnu.org <78846@debbugs.gnu.org>
Subject: Re: bug#78846: Emacs hangs non-deterministically when eglot= and clangd are used
 
Ping! Could you please answer my questions, and pe= rhaps provide the
additional information I asked for?  I'd like to make some progress in=
investigating this problem.

> Date: Fri, 20 Jun 2025 10:46:51 +0300
> From: Eli Zaretskii <eliz@gnu.org>
> Cc: 78846@debbugs.gnu.org
>
> > From: "admin@sonictk.com" <admin@sonictk.com>
> > Date: Fri, 20 Jun 2025 05:09:06 +0000
> >
> > This has been a problem ever since as long as I can remember, but=
> > using `eglot` with `clangd` results in Emacs sometimes just&= nbsp;hanging.
>
> Does this happen only in Emacs 31, or did you see that in older
> versions as well?
>
> > I work in the Unreal Engine codebase, which results in some prett= y massive `clangd` memory usage (sometimes 200+ GB) and thus response times= , along with having to spin up multiple `clangd` servers simultaneously. >
> How much VM do you have on that system, if memory consumption can be > 200+ GB?  And what is the memory footprint of Emacs in those case= s?
>
> > As far as I can tell, Emacs can hang in cases with as little as t= wo `clangd` servers spun up, but generally this problem repros more often w= hen I have multiple servers spun up. It usually happens on some LSP request= , and it's not always deterministic which request it is.
>
> How many LSP servers could you have simultaneously?
>
> > I work on Windows, and I grabbed a callstack of Emacs's main thre= ad when the hang occurs in WinDbg:
>
> This doesn't tell the whole story, because there must be other
> threads of interest in this case (since one or more clangd processes > are being read from and written to).
>
> > ```
> > [0x0]   ntdll!NtWaitForMultipleObjects+0x14  = 0x3df75fdea8   0x7ffdf18abaf0  
> > [0x1]   KERNELBASE!WaitForMultipleObjectsEx+0xf0 &= nbsp; 0x3df75fdeb0   0x7ffdf18ab9ee  
> > [0x2]   KERNELBASE!WaitForMultipleObjects+0xe &nbs= p; 0x3df75fe1a0   0x7ff635a921f7  
> > [0x3]   emacs!sys_select+0x12b7   0x3df75fe1e= 0   0x7ff635a1147e  
> > [0x4]   emacs!really_call_select+0x5e   0x3df= 75fe2a0   0x7ff635a1273d  
> > [0x5]   emacs!thread_select+0x9d   0x3df75fe3= 00   0x7ff6359d9aec  
> > [0x6]   emacs!wait_reading_process_output+0x111c &= nbsp; 0x3df75fe450   0x7ff6359db4c9  
> > [0x7]   emacs!send_process+0x269   0x3df75fe9= c0   0x7ff6359dbd11  
> > [0x8]   emacs!Fprocess_send_string+0xb1   0x3= df75fea70   0x7ff6359b5c5e  
>
> This says that Emacs called process-send-string, and it loops waiting<= br> > for the queue of sent material to be emptied.  This information i= s not
> enough to analyze the reason for the hang.
>
> > The only workaround, without killing Emacs, is to kill all `clang= d.exe` processes - this usually gets Emacs unstuck after a second or so. So= metimes, though, this trick doesn't work, and then the only recourse I have= is to kill Emacs.exe entirely.
> >
> > I've been trying to find time on and off to debug this properly, = but today I finally gave up and decided to report this in the hope that may= be someone will see this callstack and know immediately where the bug is. > >
> > My emacs build was built from source. Version information is as f= ollows:
> >
> > ```
> > In GNU Emacs 31.0.50 (build 5, x86_64-w64-mingw32) of 2025-06-03 = built
> >  on CDW-AQRHE1HHT39
> > Repository revision: eb788fd8fd2026fa4d29b918ff95b12d8e3e0bab
> > Repository branch: master
> > Windowing system distributor 'Microsoft Corp.', version 10.0.1904= 5
> > System Description: Microsoft Windows 10 Enterprise (v10.0.2009.1= 9045.5965)
>
> This build is from 2 weeks ago.  Please update from master and > rebuild, so that the source-level information you report will be easy<= br> > to follow by looking at the current sources.
>
> > Configured using:
> >  'configure --without-pop --with-imagemagick
> >  --without-compress-install -without-dbus --with-gnutls
> >  --with-tree-sitter --without-gconf --with-rsvg --without-gs= ettings
> >  --with-mailutils --with-native-compilation --with-modules -= -with-xml2
> >  --with-wide-int 'CFLAGS=3D-O3 -ggdb -fno-math-errno
>
> First, please remove the --with-wide-int, it is not needed for 64-bit<= br> > builds (and is supposed to be a no-op, but who knows?).  Also, pl= ease
> don't use -O3, but -O2 instead.  The -O3 switch could cause unsaf= e
> optimizations.
>
> >  -funsafe-math-optimizations -fno-finite-math-only -fno-trap= ping-math
> >  -freciprocal-math -fno-rounding-math -fno-signaling-nans > >  -fassociative-math -fno-signed-zeros -frename-registers
> >  -funroll-loops -mtune=3Dnative -march=3Dnative -fomit-frame= -pointer
> >  -fallow-store-data-races -fno-semantic-interposition
> >  -floop-parallelize-all -ftree-parallelize-loops=3D4'
>
> Please also drop all these -fSOMETHING and -f-noSOMETHING switches; > they are not needed with -O2.  And fomit-frame-pointer is actuall= y
> dangerous in Emacs.
>
> >  PKG_CONFIG_PATH=3D/mingw64/lib/pkgconfig:/mingw64/share/pkg= config'
>
> Don't know what that is, or why do you need it.
>
> > Anyone who has ideas for me to try/look in debugging this issue, = happy to give them a go. I encounter this fairly frequently, so I should be= able to catch it in action.
>
> Next time Emacs hangs like that, attach GDB to it, then type at the > GDB prompt:
>
>   (gdb) thread apply all bt
>
> and post here everything that GDB produces as result.  This will = show
> us the C-level backtraces of all the threads that run in the process,<= br> > not only of the main thread.  When Emacs communicates with extern= al
> processes, it uses additional threads for that purpose, and it is
> important to know their state and status.
>
> (Let me know if you need instructions about attaching GDB to a running=
> Emacs process.)
>
> Thanks.
>
>
>
>

--_000_MN2PR20MB3133DC2450DD62245F4F5834AB4FAMN2PR20MB3133namp_-- From unknown Fri Aug 15 04:07:44 2025 X-Loop: help-debbugs@gnu.org Subject: bug#78846: Emacs hangs non-deterministically when eglot and clangd are used Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Mon, 07 Jul 2025 14:12:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 78846 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: "admin@sonictk.com" Cc: 78846@debbugs.gnu.org Received: via spool by 78846-submit@debbugs.gnu.org id=B78846.175189748820555 (code B ref 78846); Mon, 07 Jul 2025 14:12:01 +0000 Received: (at 78846) by debbugs.gnu.org; 7 Jul 2025 14:11:28 +0000 Received: from localhost ([127.0.0.1]:36082 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1uYmZ6-0005LR-GO for submit@debbugs.gnu.org; Mon, 07 Jul 2025 10:11:28 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:53900) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1uYmZ3-0005Kr-Ga for 78846@debbugs.gnu.org; Mon, 07 Jul 2025 10:11:26 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uYmYx-0001kz-OY; Mon, 07 Jul 2025 10:11:19 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=h3MEa/6FrWc8luHmxICRhBGKZkTtOcTg0JSFcrQvYwg=; b=WJUCfIX2RgCY dSrfpeIYZBvYumTr1DB/ifLQSTidLdB1ddZY5HmVq9y3jMWlc9KZr11Ox+G6q21j/RbHPk957xYSu f8zLeKXumyA/PFHFLdPXw777gb80pCEllNhr/CQeXYzejQAEGWPxubRNgp3DxSYuKL5wGNPoLSfWQ jeiEs15JHZC5X0Kgo4Oxh55mVZwpxuufPTLOqyhUy5PXOs+u4K3+UixQ9xuMNbyYff8NyMDIP2tcP OOC9wi4+db1VO31Q/Sg08i+T/SaHdcBcRfagdrgYTPek4BYyt16g0YK77lwHmWCYlM1t3L+U8h6T1 vTux5Ic0LQ9cBIJOFG21vw==; Date: Mon, 07 Jul 2025 17:11:16 +0300 Message-Id: <864ivot7jv.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: (admin@sonictk.com) References: <86y0tmkg7o.fsf@gnu.org> <86ikk7vytl.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) > From: "admin@sonictk.com" > CC: "78846@debbugs.gnu.org" <78846@debbugs.gnu.org> > Date: Mon, 7 Jul 2025 05:26:31 +0000 > > Sorry for the delayed response. I actually find that after removing a bunch of the optimization flags (those > were just from some recipe that I copied for compiling Emacs on Windows), the specific problem of Emacs > itself deadlocking goes away, though I still find Emacs will occasionally hang on the main thread, and trying > to attach gdb to it via MSYS2 (which is what I use to compile Emacs with) doesn't work - however, it will > recover after a non-specific amount of time has elapsed (which doesn't seem to have any relation to any of > the timeout settings for eglot . I haven't had a chance to dig into this further yet, but it's definitely an > improvement over having to terminate all clangd processes or restarting Emacs. Thanks, but could you please try answering my other questions: . Does this happen only in Emacs 31, or did you see that in older versions as well? . How much VM do you have on that system, if memory consumption can be 200+ GB? And what is the memory footprint of Emacs in those cases? . How many LSP servers could you have simultaneously in your sessions? . Can you rebuild Emacs without --with-wide-int, with -O2 instead of -O3, and without all the -fSOMETHING and -fno-SOMETHING switches, and see if the problem goes away (or maybe you already did that -- if so, please show the updated build information) From unknown Fri Aug 15 04:07:44 2025 X-Loop: help-debbugs@gnu.org Subject: bug#78846: Emacs hangs non-deterministically when eglot and clangd are used Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 19 Jul 2025 07:15:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 78846 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: admin@sonictk.com Cc: 78846@debbugs.gnu.org Received: via spool by 78846-submit@debbugs.gnu.org id=B78846.17529092455435 (code B ref 78846); Sat, 19 Jul 2025 07:15:02 +0000 Received: (at 78846) by debbugs.gnu.org; 19 Jul 2025 07:14:05 +0000 Received: from localhost ([127.0.0.1]:39196 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1ud1lk-0001Pa-M9 for submit@debbugs.gnu.org; Sat, 19 Jul 2025 03:14:05 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:35640) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1ud1lh-0001Oa-HW for 78846@debbugs.gnu.org; Sat, 19 Jul 2025 03:14:02 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1ud1lc-0000kg-40; Sat, 19 Jul 2025 03:13:56 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=fo2CSXBzMf1wt6dl5O8/5StqAfVGnKFJAw8cX/2DCx0=; b=kk6PnezREiDg LjwyfAVX2fFYzWzu7D0kEDjeVwY6oB7pMs70cUwDZ0k5e8Y3gB1EKtMgCKHh5oZ1AfBhS33AY3Wv6 j2LpM8nBXWIGfTQyn6ETTI38uzaV7Ysd9wT0WiHxKcaM6LkYKe/IZFCsGrCHJSo2HUZYt0v5EpbMs q423pA6WYQb8hSmmWSvGL7d1JaCgW/tSB3y4c9o9MeoMjq1Y2jgzPkRA6ABJ120IhbqDadNeG/xJt cJ4Z6pjlXiO5ruBY2A+ThTfgwX9Gt14xKEvw8zEl6TxcBhUPEzveF9RCS4B9JYz1fMiv5Zi282rsE TBb3hiwBButwmdnxGefIEg==; Date: Sat, 19 Jul 2025 10:13:47 +0300 Message-Id: <86tt38ejpw.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: <864ivot7jv.fsf@gnu.org> (message from Eli Zaretskii on Mon, 07 Jul 2025 17:11:16 +0300) References: <86y0tmkg7o.fsf@gnu.org> <86ikk7vytl.fsf@gnu.org> <864ivot7jv.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) Ping! Could you please answer my questions below? > Cc: 78846@debbugs.gnu.org > Date: Mon, 07 Jul 2025 17:11:16 +0300 > From: Eli Zaretskii > > > From: "admin@sonictk.com" > > CC: "78846@debbugs.gnu.org" <78846@debbugs.gnu.org> > > Date: Mon, 7 Jul 2025 05:26:31 +0000 > > > > Sorry for the delayed response. I actually find that after removing a bunch of the optimization flags (those > > were just from some recipe that I copied for compiling Emacs on Windows), the specific problem of Emacs > > itself deadlocking goes away, though I still find Emacs will occasionally hang on the main thread, and trying > > to attach gdb to it via MSYS2 (which is what I use to compile Emacs with) doesn't work - however, it will > > recover after a non-specific amount of time has elapsed (which doesn't seem to have any relation to any of > > the timeout settings for eglot . I haven't had a chance to dig into this further yet, but it's definitely an > > improvement over having to terminate all clangd processes or restarting Emacs. > > Thanks, but could you please try answering my other questions: > > . Does this happen only in Emacs 31, or did you see that in older > versions as well? > . How much VM do you have on that system, if memory consumption can > be 200+ GB? And what is the memory footprint of Emacs in those > cases? > . How many LSP servers could you have simultaneously in your > sessions? > . Can you rebuild Emacs without --with-wide-int, with -O2 instead of > -O3, and without all the -fSOMETHING and -fno-SOMETHING switches, > and see if the problem goes away (or maybe you already did that -- > if so, please show the updated build information) > > > > From unknown Fri Aug 15 04:07:44 2025 X-Loop: help-debbugs@gnu.org Subject: bug#78846: Emacs hangs non-deterministically when eglot and clangd are used Resent-From: Eli Zaretskii Original-Sender: "Debbugs-submit" Resent-CC: bug-gnu-emacs@gnu.org Resent-Date: Sat, 02 Aug 2025 13:56:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 78846 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: admin@sonictk.com Cc: 78846@debbugs.gnu.org Received: via spool by 78846-submit@debbugs.gnu.org id=B78846.175414292528477 (code B ref 78846); Sat, 02 Aug 2025 13:56:01 +0000 Received: (at 78846) by debbugs.gnu.org; 2 Aug 2025 13:55:25 +0000 Received: from localhost ([127.0.0.1]:35651 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1uiCho-0007PF-TM for submit@debbugs.gnu.org; Sat, 02 Aug 2025 09:55:25 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:40192) by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.84_2) (envelope-from ) id 1uiChm-0007Ml-Oe; Sat, 02 Aug 2025 09:55:23 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1uiChf-0005WJ-JJ; Sat, 02 Aug 2025 09:55:17 -0400 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date: mime-version; bh=mZam9OQUAh7+J5Mi/b9q3xmJis15ZVs9cgDoPuddg0A=; b=m9iGJt4qgPck QnBhKwxiHz6w1A4ZJtu1IX5bnTgviP33k+1cMeZ44Lmi8VW6k6mFhyCRX1n6SKOirzfmc0WeEYGZd hokW6HGghlBNW4qR1zca78jVa4g/41JfBgTVm7+YtsCC8gLs02rxF1O68kNwf1Uy3xD6oORFTabDU ZYQfw7IocDs1y4DfDmAllNZspnx8mnj5p3FFEJGhjGk3GfVvtAJtZwS9cHlSbyqATbEzws/IqppOH M5nS82S6uER0bR8YmghoE5fuFoEA/M9rtX/6pC2V52SzAfBkYRtnbyackb6bcOq2CMDavsWjPsfcS kboilD6O+JGKxhp3xc37Kw==; Date: Sat, 02 Aug 2025 16:55:13 +0300 Message-Id: <86y0s1rfpa.fsf@gnu.org> From: Eli Zaretskii In-Reply-To: <86tt38ejpw.fsf@gnu.org> (message from Eli Zaretskii on Sat, 19 Jul 2025 10:13:47 +0300) References: <86y0tmkg7o.fsf@gnu.org> <86ikk7vytl.fsf@gnu.org> <864ivot7jv.fsf@gnu.org> <86tt38ejpw.fsf@gnu.org> X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -3.3 (---) tags 78846 unreproducible close 78846 thanks > Cc: 78846@debbugs.gnu.org > Date: Sat, 19 Jul 2025 10:13:47 +0300 > From: Eli Zaretskii > > Ping! Could you please answer my questions below? No further comments within several weeks, so I'm now closing this bug as unreproducible. If you see it again, we can reopen with the new data. > > Cc: 78846@debbugs.gnu.org > > Date: Mon, 07 Jul 2025 17:11:16 +0300 > > From: Eli Zaretskii > > > > > From: "admin@sonictk.com" > > > CC: "78846@debbugs.gnu.org" <78846@debbugs.gnu.org> > > > Date: Mon, 7 Jul 2025 05:26:31 +0000 > > > > > > Sorry for the delayed response. I actually find that after removing a bunch of the optimization flags (those > > > were just from some recipe that I copied for compiling Emacs on Windows), the specific problem of Emacs > > > itself deadlocking goes away, though I still find Emacs will occasionally hang on the main thread, and trying > > > to attach gdb to it via MSYS2 (which is what I use to compile Emacs with) doesn't work - however, it will > > > recover after a non-specific amount of time has elapsed (which doesn't seem to have any relation to any of > > > the timeout settings for eglot . I haven't had a chance to dig into this further yet, but it's definitely an > > > improvement over having to terminate all clangd processes or restarting Emacs. > > > > Thanks, but could you please try answering my other questions: > > > > . Does this happen only in Emacs 31, or did you see that in older > > versions as well? > > . How much VM do you have on that system, if memory consumption can > > be 200+ GB? And what is the memory footprint of Emacs in those > > cases? > > . How many LSP servers could you have simultaneously in your > > sessions? > > . Can you rebuild Emacs without --with-wide-int, with -O2 instead of > > -O3, and without all the -fSOMETHING and -fno-SOMETHING switches, > > and see if the problem goes away (or maybe you already did that -- > > if so, please show the updated build information) > > > > > > > > > > > >