From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 20 14:07:31 2018 Received: (at submit) by debbugs.gnu.org; 20 Sep 2018 18:07:31 +0000 Received: from localhost ([127.0.0.1]:47305 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g33MQ-0004ky-NJ for submit@debbugs.gnu.org; Thu, 20 Sep 2018 14:07:31 -0400 Received: from eggs.gnu.org ([208.118.235.92]:41880) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g32WG-0003J7-4Q for submit@debbugs.gnu.org; Thu, 20 Sep 2018 13:13:36 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g32W7-0001IW-SU for submit@debbugs.gnu.org; Thu, 20 Sep 2018 13:13:31 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: * X-Spam-Status: No, score=1.8 required=5.0 tests=BAYES_50,FROM_EXCESS_BASE64, T_DKIM_INVALID autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:39961) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g32W1-0001FL-1B for submit@debbugs.gnu.org; Thu, 20 Sep 2018 13:13:24 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:34584) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1g32Vz-0006O9-OW for bug-guile@gnu.org; Thu, 20 Sep 2018 13:13:20 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1g32Vw-00013t-9x for bug-guile@gnu.org; Thu, 20 Sep 2018 13:13:19 -0400 Received: from k.imm.uran.ru ([195.19.132.74]:49582) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1g32Vv-0000x3-Fc for bug-guile@gnu.org; Thu, 20 Sep 2018 13:13:16 -0400 Received: from k (k.imm.uran.ru [195.19.132.74]) by k.imm.uran.ru (Postfix) with ESMTPSA id 5497D173C5 for ; Thu, 20 Sep 2018 22:13:08 +0500 (+05) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=k.imm.uran.ru; s=kite; t=1537463588; bh=PirV7KFUmj0JDSVNYRus7sri4JwYXBsFW5qa7XNtAH4=; h=Date:From:To:Subject; b=Zf7PNpQhv7B4vkFD7qgjgaXygARdCZtdehWS4Za3Yb0ikxRSaFr8MVnKFToJLHJsV HwLDh6LybCvuYoPcmsmWUvUfxWFwMIau1bSDPMlzwrfx7oN7nWKmzU+LzR2a+2R+Xp DKaPa9ewX0G9EaIkPIar0olAi06Yy07ZMVTkk3/46AbkKFRzwqsWPln7LYYfxHSZDo eQL68ui0fnZ1D7UOmuVAF/3/Sg2l4sS+/uOMqrzu234VFrW9See1CPqp+IrA4dszX/ NQS+G+ut+MEnspSUgxVOlw+dWvVxjm/oDL9o03gNYqd/LoXiCi1BNcQFrURGp6nm0e AvF6gyqRm964Hf5AOhldbeYcRADaGna0K63erC56eHiBPwf50XJCNIVNMuSRh7uacr lSI+836oH2nZ8gra456sh1/hcXULsMU3KADGxfTo1Bas5PNWUAsGwuH1XVN9rn3fPt WRtn/3zpHR5Z9OPeviY3g3jHdOey0YffVoAdIB30f/2B3PhBLX7vIjsE4sROxMIoui zLc1qYGvJmEQ3UAeZTBtfHEWE+EnnjL82meIg/dLI4Fa1dlkSj8d8nxj2oSmwXKl1q jt5r9q/3rrseIogm+t5CH8GnVx5xXFx6cfai3UOXUj7L7gc/rLCGI8tyUj1Crhx4vG sBuK/uxd13cFnlcfYf+dkQO8= Date: Thu, 20 Sep 2018 22:13:06 +0500 From: =?utf-8?B?0JzQuNGF0LDQuNC7INCR0LDRhdGC0LXRgNC10LI=?= To: bug-guile@gnu.org Subject: (system* ...) call somehow interferes with atomic-box on armv7l Message-ID: <20180920171306.GA9185@k> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="Kj7319i9nmIyA2yE" Content-Disposition: inline User-Agent: Mutt/1.10.1 (2018-07-13) Content-Transfer-Encoding: 7bit X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.9 (----) X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Thu, 20 Sep 2018 14:07:29 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.9 (-----) --Kj7319i9nmIyA2yE Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Greetings. I attach the code sample, which expected behaviour is: 1. to periodically restart thread, which executes =ABsleep 1s=BB with system* call; 2. and then to check if it should repeat the execution. =20 The flag, which controls that re-execution is managed by another thread through atomic-box primitive. The code does not run as expected: the sleep executing sleep starts once, and change seems to remain invisible to main thread. The code works as expected, when: 1. (sleep 1) is used instead (system* "sleep" "1s"); 2. running the code on x86_64 cpus. The use of affinity mask taskset 0x01 guile test.scm does not help. - MB. Respectfully P.S. Additional information: $ guile --version =20 guile (GNU Guile) 2.2.4 Copyright (C) 2018 Free Software Foundation, Inc. License LGPLv3+: GNU LGPL 3 or later . This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. $ sh config.guess armv7l-unknown-linux-gnueabihf $ pacman -Qi guile=20 Name : guile Version : 2.2.4-1 Description : Portable, embeddable Scheme implementation written in C Architecture : armv7h URL : https://www.gnu.org/software/guile/ Licenses : GPL Groups : None Provides : None Depends On : gmp libltdl ncurses texinfo libunistring gc libff= i Optional Deps : None Required By : make Optional For : gnutls Conflicts With : None Replaces : None Installed Size : 40.83 MiB Packager : Arch Linux ARM Build System Build Date : Tue 03 Jul 2018 04:47:53 PM +05 Install Date : Mon 09 Jul 2018 01:02:48 PM +05 Install Reason : Installed as a dependency for another package Install Script : No Validated By : Signature --Kj7319i9nmIyA2yE Content-Type: application/vnd.lotus-screencam Content-Disposition: attachment; filename="test.scm" Content-Transfer-Encoding: quoted-printable (use-modules (ice-9 atomic)=0A (ice-9 threads))=0A=0A(define du= mp=0A (let ((p (current-error-port)))=0A (lambda (fmt . args) (apply fo= rmat p fmt args))))=0A=0A(define state (make-atomic-box #:nothing-to-do))= =0A=0A(define (expect e v)=0A (when (not (eq? e v))=0A (error "Protocol= violation; (expecting . got):" (cons e v)))) =0A=0A(define (sleep-loop)=0A= (expect #:need-to-sleep (atomic-box-ref state))=0A=0A (dump "R: Going to= (sleep 1)~%")=0A (atomic-box-set! state #:accepted)=0A =0A ; SOMETHING = WRONG IS HERE=0A (system* "sleep" "1s")=0A ; (sleep 1)=0A=0A (let ((v (a= tomic-box-compare-and-swap! state #:accepted #:nothing-to-do)))=0A (when= (not (eq? #:accepted v))=0A (dump "S: Repeating sleep~%")=0A (sl= eep-loop)))=0A=0A (dump "R: sleep-loop finished~%"))=0A=0A(define (main-lo= op)=0A (define (run-thread)=0A (dump "M: new sleep thread~%")=0A (at= omic-box-set! state #:need-to-sleep)=0A (call-with-new-thread sleep-loop= )) =0A=0A (dump "M: (sleep 3)~%")=0A (sleep 3)=0A=0A (dump "M: protocol = exchange~%")=0A (let ((st (atomic-box-ref state)))=0A (case st =0A = ((#:nothing-to-do) (run-thread))=0A ((#:accepted) (let ((v (atomic-bo= x-compare-and-swap! state #:accepted #:need-to-sleep)))=0A = (when (not (eq? #:accepted v))=0A (expect #:acc= epted v)=0A (run-thread))))=0A (else (expect #:= need-to-sleep st))))=0A=0A (main-loop))=0A=0A(main-loop)=0A --Kj7319i9nmIyA2yE Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename=cpuinfo processor : 0 model name : ARMv7 Processor rev 10 (v7l) BogoMIPS : 7.54 Features : half thumb fastmult vfp edsp thumbee neon vfpv3 tls vfpd32 CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x2 CPU part : 0xc09 CPU revision : 10 processor : 1 model name : ARMv7 Processor rev 10 (v7l) BogoMIPS : 7.54 Features : half thumb fastmult vfp edsp thumbee neon vfpv3 tls vfpd32 CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x2 CPU part : 0xc09 CPU revision : 10 processor : 2 model name : ARMv7 Processor rev 10 (v7l) BogoMIPS : 7.54 Features : half thumb fastmult vfp edsp thumbee neon vfpv3 tls vfpd32 CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x2 CPU part : 0xc09 CPU revision : 10 processor : 3 model name : ARMv7 Processor rev 10 (v7l) BogoMIPS : 7.54 Features : half thumb fastmult vfp edsp thumbee neon vfpv3 tls vfpd32 CPU implementer : 0x41 CPU architecture: 7 CPU variant : 0x2 CPU part : 0xc09 CPU revision : 10 Hardware : Freescale i.MX6 Quad/DualLite (Device Tree) Revision : 0000 Serial : 0000000000000000 --Kj7319i9nmIyA2yE-- From debbugs-submit-bounces@debbugs.gnu.org Fri Sep 21 17:26:48 2018 Received: (at 32786) by debbugs.gnu.org; 21 Sep 2018 21:26:48 +0000 Received: from localhost ([127.0.0.1]:48492 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g3Swq-0004Cx-Ei for submit@debbugs.gnu.org; Fri, 21 Sep 2018 17:26:48 -0400 Received: from world.peace.net ([64.112.178.59]:40778) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g3Swo-0004Ck-IQ for 32786@debbugs.gnu.org; Fri, 21 Sep 2018 17:26:47 -0400 Received: from mhw by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1g3Swh-0004kl-0e; Fri, 21 Sep 2018 17:26:39 -0400 From: Mark H Weaver To: =?utf-8?B?0JzQuNGF0LDQuNC7INCR0LDRhdGC0LXRgNC10LI=?= Subject: Re: bug#32786: (system* ...) call somehow interferes with atomic-box on armv7l References: <20180920171306.GA9185@k> Date: Fri, 21 Sep 2018 17:26:27 -0400 In-Reply-To: <20180920171306.GA9185@k> (=?utf-8?B?ItCc0LjRhdCw0LjQuyDQkdCw?= =?utf-8?B?0YXRgtC10YDQtdCyIidz?= message of "Thu, 20 Sep 2018 22:13:06 +0500") Message-ID: <87fty2v8xo.fsf@netris.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi, Михаил Бахтерев writes: > I attach the code sample, which expected behaviour is: > > 1. to periodically restart thread, which executes «sleep 1s» with > system* call; > > 2. and then to check if it should repeat the execution. > > The flag, which controls that re-execution is managed by another thread > through atomic-box primitive. > > The code does not run as expected: the sleep executing sleep starts > once, and change seems to remain invisible to main thread. [...] Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 1.8 FUZZY_XPILL BODY: Attempt to obfuscate words in spam X-Debbugs-Envelope-To: 32786 Cc: 32786@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.8 (/) Hi, =D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB =D0=91=D0=B0=D1=85=D1=82=D0=B5=D1=80= =D0=B5=D0=B2 writes: > I attach the code sample, which expected behaviour is: > > 1. to periodically restart thread, which executes =C2=ABsleep 1s=C2=BB = with > system* call; > > 2. and then to check if it should repeat the execution. >=20=20=20 > The flag, which controls that re-execution is managed by another thread > through atomic-box primitive. > > The code does not run as expected: the sleep executing sleep starts > once, and change seems to remain invisible to main thread. Can you describe in more detail the behavior you are seeing? What messages do you see on stderr from your 'dump' calls when it gets stuck? What state is the atomic box left in? I see a couple of problems with the code below: * Two threads write concurrently to the same port, namely (current-error-port). You should use thread synchronization to ensure that operations to a given port are serialized. * There's a race condition when the atomic box is in the #:accepted state. If the main thread finds the box in that state, it changes the state to #:need-to-sleep, which will apparently cause the other thread to sleep again. I'm not sure if these problems could explain what you're seeing. Regards, Mark > The code works as expected, when: > > 1. (sleep 1) is used instead (system* "sleep" "1s"); > 2. running the code on x86_64 cpus. > > The use of affinity mask > > taskset 0x01 guile test.scm > > does not help. > > - MB. Respectfully > > P.S. Additional information: > > $ guile --version=20=20 > guile (GNU Guile) 2.2.4 > Copyright (C) 2018 Free Software Foundation, Inc. > > License LGPLv3+: GNU LGPL 3 or later . > This is free software: you are free to change and redistribute it. > There is NO WARRANTY, to the extent permitted by law. > > $ sh config.guess > armv7l-unknown-linux-gnueabihf > > $ pacman -Qi guile=20 > Name : guile > Version : 2.2.4-1 > Description : Portable, embeddable Scheme implementation written in C > Architecture : armv7h > URL : https://www.gnu.org/software/guile/ > Licenses : GPL > Groups : None > Provides : None > Depends On : gmp libltdl ncurses texinfo libunistring gc libffi > Optional Deps : None > Required By : make > Optional For : gnutls > Conflicts With : None > Replaces : None > Installed Size : 40.83 MiB > Packager : Arch Linux ARM Build System > Build Date : Tue 03 Jul 2018 04:47:53 PM +05 > Install Date : Mon 09 Jul 2018 01:02:48 PM +05 > Install Reason : Installed as a dependency for another package > Install Script : No > Validated By : Signature > > (use-modules (ice-9 atomic) > (ice-9 threads)) > > (define dump > (let ((p (current-error-port))) > (lambda (fmt . args) (apply format p fmt args)))) > > (define state (make-atomic-box #:nothing-to-do)) > > (define (expect e v) > (when (not (eq? e v)) > (error "Protocol violation; (expecting . got):" (cons e v))))=20 > > (define (sleep-loop) > (expect #:need-to-sleep (atomic-box-ref state)) > > (dump "R: Going to (sleep 1)~%") > (atomic-box-set! state #:accepted) >=20=20=20 > ; SOMETHING WRONG IS HERE > (system* "sleep" "1s") > ; (sleep 1) > > (let ((v (atomic-box-compare-and-swap! state #:accepted #:nothing-to-do= ))) > (when (not (eq? #:accepted v)) > (dump "S: Repeating sleep~%") > (sleep-loop))) > > (dump "R: sleep-loop finished~%")) > > (define (main-loop) > (define (run-thread) > (dump "M: new sleep thread~%") > (atomic-box-set! state #:need-to-sleep) > (call-with-new-thread sleep-loop))=20 > > (dump "M: (sleep 3)~%") > (sleep 3) > > (dump "M: protocol exchange~%") > (let ((st (atomic-box-ref state))) > (case st=20 > ((#:nothing-to-do) (run-thread)) > ((#:accepted) (let ((v (atomic-box-compare-and-swap! state #:accept= ed #:need-to-sleep))) > (when (not (eq? #:accepted v)) > (expect #:accepted v) > (run-thread)))) > (else (expect #:need-to-sleep st)))) > > (main-loop)) > > (main-loop) From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 22 01:05:39 2018 Received: (at 32786) by debbugs.gnu.org; 22 Sep 2018 05:05:40 +0000 Received: from localhost ([127.0.0.1]:48640 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g3a6t-0001K9-Gw for submit@debbugs.gnu.org; Sat, 22 Sep 2018 01:05:39 -0400 Received: from k.imm.uran.ru ([195.19.132.74]:38334) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g3Zhm-0000dp-Qg for 32786@debbugs.gnu.org; Sat, 22 Sep 2018 00:39:43 -0400 Received: from blacky (88.226-224-87.telenet.ru [87.224.226.88]) by k.imm.uran.ru (Postfix) with ESMTPSA id B50BB19352; Sat, 22 Sep 2018 09:39:39 +0500 (+05) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=k.imm.uran.ru; s=kite; t=1537591179; bh=0n2Md/lMgViMYoTlX9BP9m+CTeKMOclhw3iM/k8VAO0=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=IfAgMSynYBICy7kQYeAKNSzyaJqYbxU/s5oSAyiiwKOccrGDfR88GiJ4MJbOW7NXA Z/9V0iwwRs3wJVQkuoF7OlnQgFvbKQ2wa5piYHaOZDkpPvAQasGacsMfqn5g6vseu4 1e0bcb5YOZT6th/znWnTbe5y50686UndVt5lCl2nn6F/L/Jyv6k9tywmpGbi+ndEUX MBNVKxVmTutY4gVK7cPoAjXXaS+Nf/clLyxoFVVF/Nqc3xygSt3L5mI83apTvKOXsB VcX0TPYYS7Fap5HDMsle53NuWgXM5dJW2D/3TbUBZwqCXPk3s918FJ5LvOJgmtQrpY nYvJ8gBzckxrty7wWDLg0Lafdik+lPGJlIViNhmEIzWfFHfDaKvrfQCt7f/KcMsqjD c7IeNDOUCGRSNWeh5yrlwf0x/r2h3w+XuhmjFW1zpNGqpyc570O3mvz2PIFJwBwHfm Fjm1gGGPhyAcijTa7ghtxRhoaK2YVwPvuzslDTR2N38huyWOWcSiS5dzXyc8WZMKsD MVrTEkQEZgdad/PzOu+4IcdKI/8rV4dphN55vN5uqHW/PSrbaAyl/Jd4m9xn4BNjqV 7odqT5xAIrdMi9nusvtMyEomJnMSJjYB+Q1yVN8U0ZVImmydI5e/WMdvwraEYWSbSS eqmoi+u/woVuCW3Ub3vsPyzM= Date: Sat, 22 Sep 2018 09:39:38 +0500 From: =?utf-8?B?0JzQuNGF0LDQuNC7INCR0LDRhdGC0LXRgNC10LI=?= To: Mark H Weaver Subject: Re: bug#32786: (system* ...) call somehow interferes with atomic-box on armv7l Message-ID: <20180922043938.GA4539@blacky> References: <20180920171306.GA9185@k> <87fty2v8xo.fsf@netris.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87fty2v8xo.fsf@netris.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Score: 0.1 (/) X-Debbugs-Envelope-To: 32786 X-Mailman-Approved-At: Sat, 22 Sep 2018 01:05:38 -0400 Cc: 32786@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.9 (/) On Fri, Sep 21, 2018 at 05:26:27PM -0400, Mark H Weaver wrote: > Hi, > > Can you describe in more detail the behavior you are seeing? What > messages do you see on stderr from your 'dump' calls when it gets stuck? > What state is the atomic box left in? > > I see a couple of problems with the code below: > > * Two threads write concurrently to the same port, namely > (current-error-port). You should use thread synchronization to ensure > that operations to a given port are serialized. > > * There's a race condition when the atomic box is in the #:accepted > state. If the main thread finds the box in that state, it changes the > state to #:need-to-sleep, which will apparently cause the other thread > to sleep again. > > I'm not sure if these problems could explain what you're seeing. > > Regards, > Mark > The code is supposed to model the following behaviour. The main thread tracks some events (in the example: the end of (sleep 3)). If one or multiple events occurred some long-lasting external command should be executed. Long-lasting means that multiple events may occur during execution. To perform that the main thread should: 1. either to run new work thread (sleep-loop procedure), which will call (system* ...); 2. or to communicate the fact of events occurrences to the work thread, which is supposed to detect this and just repeat the command. If multiple events have occurred during command execution the command should be re-executed only once. The event occurrence is communicated through simple protocol with three states: #:nothing-to-do < #:accepted < #:need-to-sleep. The main thread pulls state up, and work thread pulls it down. When the main thread detects event, and the state is #:accepted, it means, that work thread accepted the previous job, and executing it now, so the main thread pulls the state up to need-to-sleep, and the work thread re-executes command (sleeps again in the test case), when the previous external command run has finished. This is intended behaviour. I expect the program to print some garbage, showing that work thread is restarted periodically. Something like that (it is result of using (sleep 1) instead of (system* "sleep" "1s")): $ guile test.scm M: (sleep 3) M: protocol exchange M: new sleep thread M: (sleep 3) R: Going to (sleep 1) R: sleep-loop finished M: protocol exchange M: new sleep thread M: (sleep 3) : (sleep 3)R: Going to (sleep 1) R: sleep-loop finished M: protocol exchange M: new sleep thread M: (sleep 3) R: Going to (sleep 1) R: sleep-loop finished M: protocol exchange M: new sleep thread M: (sleep 3) : (sleep 3)R: Going to (sleep 1) R: sleep-loop finished But what i get with (system* "sleep" "1s") is this: $ guile test.scm M: (sleep 3) M: protocol exchange M: new sleep thread M: (sleep 3) R: Going to (sleep 1) R: sleep-loop finished M: protocol exchange M: (sleep 3) M: protocol exchange M: (sleep 3) M: protocol exchange M: (sleep 3) So, the state of the atomic-box is stuck in #:need-to-sleep somehow. - MB. Respectfully From debbugs-submit-bounces@debbugs.gnu.org Sat Sep 22 01:05:40 2018 Received: (at 32786) by debbugs.gnu.org; 22 Sep 2018 05:05:40 +0000 Received: from localhost ([127.0.0.1]:48642 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g3a6t-0001KC-Ue for submit@debbugs.gnu.org; Sat, 22 Sep 2018 01:05:40 -0400 Received: from k.imm.uran.ru ([195.19.132.74]:38356) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g3Zrw-0000wG-UC for 32786@debbugs.gnu.org; Sat, 22 Sep 2018 00:50:13 -0400 Received: from blacky (88.226-224-87.telenet.ru [87.224.226.88]) by k.imm.uran.ru (Postfix) with ESMTPSA id 8CCDB1937D; Sat, 22 Sep 2018 09:50:11 +0500 (+05) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=k.imm.uran.ru; s=kite; t=1537591811; bh=JbwFNmPiPblQ68cJ+BY3lOhZHrzbDYeh6WmC6qy1NwM=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=2PlyqKb6vaQq+K2TgJE7T7FUAdSNWRIjew2+eBVM2jbwCeQ4MnbVbzOFlKl405tw6 vd8/Zu8D4AZghEO9OXYafVQwewflpxnUGd8k0nvZLz1ZgQt+MKP6RA+6a1xC2fS5a7 mjGPo4Rmdd1Jj97CPQfrkC1Q5G2nJf+/4PfQR5ivwnxX08FF0DVwb+U8Sf8S+6BtyN FpLK5kpehN8/8Ay4P258HCzF0GStij8zkHj7TzdTeGID3d9L6fx+iJgJZccN226cbh EJIVMnntFMJJIRl7hixNAgfXoaB77z+S2NvBTvFXZhLCZ7PvkdSStoiQGwiBZKNtnT iiIQHkJYKMHvEe797sVvpyP+9eRqdnFB7OnoAi6BvMpFZ62mfmeESybrHAiz6yU1Q2 kU5Qc/O9gARskkwCIOp49lHzrJwcmtMKMZRJeNokSq90liwOpXd8VbX9L23SstvEjK s6kWx1zhp8a5xCCg2mtljbspYZEX9RUaVy8H7ExQAqzHPSTTbppoW9y6SCOl3gwD3d pHImsEjubxHyqAJi699QMIJTVzXWp5KzRKUfuOmmrrsTCnJ/YwiGkpWS39/jXqXcDr PZm+FqhEFXeQAxcMwuvqYrSda85ox0rOmckTkenQFQDRAnN+uXE7W7YLnx75P7MxhF 1hoRQZ1i4JRh4RaVGTm65TsI= Date: Sat, 22 Sep 2018 09:50:09 +0500 From: =?utf-8?B?0JzQuNGF0LDQuNC7INCR0LDRhdGC0LXRgNC10LI=?= To: Mark H Weaver Subject: Re: bug#32786: (system* ...) call somehow interferes with atomic-box on armv7l Message-ID: <20180922045009.GB4539@blacky> References: <20180920171306.GA9185@k> <87fty2v8xo.fsf@netris.org> <20180922043938.GA4539@blacky> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <20180922043938.GA4539@blacky> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Score: 1.9 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: By the way, if i change (system* "sleep" "1s") to the (system* "true"), nothing changes. The log of execution is the same: $ guile test.scm M: (sleep 3) M: protocol exchange M: new sleep thread M: (sleep 3) R: Going to (sleep 1) R: sleep-loop finished M: protocol exchange M: (sleep 3) M: protocol exchange M: (sleep 3) [...] Content analysis details: (1.9 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 1.8 FUZZY_XPILL BODY: Attempt to obfuscate words in spam 0.1 FROM_EXCESS_BASE64 From: base64 encoded unnecessarily 0.0 T_DKIM_INVALID DKIM-Signature header exists but is not valid X-Debbugs-Envelope-To: 32786 X-Mailman-Approved-At: Sat, 22 Sep 2018 01:05:38 -0400 Cc: 32786@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.9 (/) By the way, if i change (system* "sleep" "1s") to the (system* "true"), nothing changes. The log of execution is the same: $ guile test.scm M: (sleep 3) M: protocol exchange M: new sleep thread M: (sleep 3) R: Going to (sleep 1) R: sleep-loop finished M: protocol exchange M: (sleep 3) M: protocol exchange M: (sleep 3) But 3 seconds are more than enough to finish the work thread and pull the state down to the #:nothing-to-do, which should cause the main thread re-run work thread. - MB. Respectfully On Sat, Sep 22, 2018 at 09:39:38AM +0500, Михаил Бахтерев wrote: > On Fri, Sep 21, 2018 at 05:26:27PM -0400, Mark H Weaver wrote: > > Hi, > > > > Can you describe in more detail the behavior you are seeing? What > > messages do you see on stderr from your 'dump' calls when it gets stuck? > > What state is the atomic box left in? > > > > I see a couple of problems with the code below: > > > > * Two threads write concurrently to the same port, namely > > (current-error-port). You should use thread synchronization to ensure > > that operations to a given port are serialized. > > > > * There's a race condition when the atomic box is in the #:accepted > > state. If the main thread finds the box in that state, it changes the > > state to #:need-to-sleep, which will apparently cause the other thread > > to sleep again. > > > > I'm not sure if these problems could explain what you're seeing. > > > > Regards, > > Mark > > > > The code is supposed to model the following behaviour. > > The main thread tracks some events (in the example: the end of (sleep > 3)). If one or multiple events occurred some long-lasting external > command should be executed. Long-lasting means that multiple events may > occur during execution. > > To perform that the main thread should: > > 1. either to run new work thread (sleep-loop procedure), which will > call (system* ...); > > 2. or to communicate the fact of events occurrences to the work > thread, which is supposed to detect this and just repeat the command. > > If multiple events have occurred during command execution the command > should be re-executed only once. > > The event occurrence is communicated through simple protocol with three > states: #:nothing-to-do < #:accepted < #:need-to-sleep. The main thread > pulls state up, and work thread pulls it down. When the main thread > detects event, and the state is #:accepted, it means, that work thread > accepted the previous job, and executing it now, so the main thread > pulls the state up to need-to-sleep, and the work thread re-executes > command (sleeps again in the test case), when the previous external > command run has finished. This is intended behaviour. > > I expect the program to print some garbage, showing that work thread is > restarted periodically. Something like that (it is result of using > (sleep 1) instead of (system* "sleep" "1s")): > > $ guile test.scm > M: (sleep 3) > M: protocol exchange > M: new sleep thread > M: (sleep 3) > > R: Going to (sleep 1) > R: sleep-loop finished > M: protocol exchange > M: new sleep thread > M: (sleep 3) > : (sleep 3)R: Going to (sleep 1) > > R: sleep-loop finished > M: protocol exchange > M: new sleep thread > M: (sleep 3) > > R: Going to (sleep 1) > R: sleep-loop finished > M: protocol exchange > M: new sleep thread > M: (sleep 3) > > : (sleep 3)R: Going to (sleep 1) > R: sleep-loop finished > > But what i get with (system* "sleep" "1s") is this: > > $ guile test.scm > M: (sleep 3) > M: protocol exchange > M: new sleep thread > M: (sleep 3) > R: Going to (sleep 1) > R: sleep-loop finished > M: protocol exchange > M: (sleep 3) > M: protocol exchange > M: (sleep 3) > M: protocol exchange > M: (sleep 3) > > So, the state of the atomic-box is stuck in #:need-to-sleep somehow. > > - MB. Respectfully From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 27 01:49:44 2018 Received: (at 32786) by debbugs.gnu.org; 27 Sep 2018 05:49:44 +0000 Received: from localhost ([127.0.0.1]:55837 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g5PBI-0005t2-4r for submit@debbugs.gnu.org; Thu, 27 Sep 2018 01:49:44 -0400 Received: from world.peace.net ([64.112.178.59]:40328) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g5PBG-0005sn-T5 for 32786@debbugs.gnu.org; Thu, 27 Sep 2018 01:49:43 -0400 Received: from mhw by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1g5PBA-0003vR-KO; Thu, 27 Sep 2018 01:49:36 -0400 From: Mark H Weaver To: =?utf-8?B?0JzQuNGF0LDQuNC7INCR0LDRhdGC0LXRgNC10LI=?= Subject: Re: bug#32786: (system* ...) call somehow interferes with atomic-box on armv7l References: <20180920171306.GA9185@k> <87fty2v8xo.fsf@netris.org> <20180922043938.GA4539@blacky> Date: Thu, 27 Sep 2018 01:49:23 -0400 In-Reply-To: <20180922043938.GA4539@blacky> (=?utf-8?B?ItCc0LjRhdCw0Lg=?= =?utf-8?B?0Lsg0JHQsNGF0YLQtdGA0LXQsiIncw==?= message of "Sat, 22 Sep 2018 09:39:38 +0500") Message-ID: <87sh1vcwws.fsf@netris.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="=-=-=" X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 32786 Cc: 32786@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) --=-=-= Content-Type: text/plain Hi, Thanks for the additional details. I was able to reproduce the bug, and I believe I now see the problem. 'atomic-box-compare-and-swap!' is implemented using 'atomic_compare_exchange_weak' (if available), but neglects to handle the case where 'atomic_compare_exchange_weak' may spuriously fail. In that case, the box is left unchanged, although the observed value was equal to the expected value. What's happening here is that the 'atomic-box-compare-and-swap!' in 'sleep-loop' fails spuriously, leaving the box in state #:accepted although it returns #:accepted as its result. When the main loop discovers this, it changes the state to #:need-to-sleep, although the thread has already ended. To confirm this hypothesis, I added a print statement to the main loop showing the state of the box that it observed during the protocol exchange, and indeed it sees #:accepted the first time it checks, and #:need-to-sleep in all later iterations. I've attached a proposed patch that I hope will fix this problem. If you'd be willing to test it, I'd be grateful, but otherwise I'll try to test it in the next week or so. My access to armv7l boxes is somewhat limited. Thanks for this report. Mark --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=0001-UNTESTED-Fix-atomic-box-compare-and-swap.patch Content-Description: [PATCH] UNTESTED: Fix 'atomic-box-compare-and-swap!' >From 958d37686a9ac65f48cb2ca32a341cf182c24b5a Mon Sep 17 00:00:00 2001 From: Mark H Weaver Date: Thu, 27 Sep 2018 01:00:11 -0400 Subject: [PATCH] UNTESTED: Fix 'atomic-box-compare-and-swap!'. Fixes . 'scm_atomic_compare_and_swap_scm' uses 'atomic_compare_exchange_weak' if it's available on the host platform. 'atomic_compare_exchange_weak' may fail spuriously, leaving the atomic object unchanged even when it contained the expected value. 'atomic-box-compare-and-swap!' uses 'scm_atomic_compare_and_swap_scm' without checking its return value, and therefore may return the expected value while leaving the box unchanged. This contradicts the documentation, which explicitly states that "you can know if the swap worked by checking if the return value is 'eq?' to EXPECTED.". * libguile/atomic.c (scm_atomic_box_compare_and_swap_x): If 'scm_atomic_compare_and_swap_scm' returns false and the observed value is equal to 'expected', then try again. * libguile/vm-engine.c (atomic-box-compare-and-swap!): Ditto. --- libguile/atomic.c | 13 +++++++++---- libguile/vm-engine.c | 13 ++++++++----- 2 files changed, 17 insertions(+), 9 deletions(-) diff --git a/libguile/atomic.c b/libguile/atomic.c index 950874030..504643912 100644 --- a/libguile/atomic.c +++ b/libguile/atomic.c @@ -1,4 +1,4 @@ -/* Copyright (C) 2016 Free Software Foundation, Inc. +/* Copyright (C) 2016, 2018 Free Software Foundation, Inc. * * This library is free software; you can redistribute it and/or * modify it under the terms of the GNU Lesser General Public License @@ -95,10 +95,15 @@ SCM_DEFINE (scm_atomic_box_compare_and_swap_x, "if the return value is @code{eq?} to @var{expected}.") #define FUNC_NAME s_scm_atomic_box_compare_and_swap_x { + SCM result = expected; + SCM_VALIDATE_ATOMIC_BOX (1, box); - scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (box), - &expected, desired); - return expected; + while (!scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (box), + &result, desired) + && scm_is_eq (result, expected)) + { + } + return result; } #undef FUNC_NAME diff --git a/libguile/vm-engine.c b/libguile/vm-engine.c index 19ff3e498..9650e33eb 100644 --- a/libguile/vm-engine.c +++ b/libguile/vm-engine.c @@ -3868,16 +3868,19 @@ VM_NAME (scm_i_thread *thread, struct scm_vm *vp, { scm_t_uint16 dst, box; scm_t_uint32 expected, desired; - SCM scm_box, scm_expected; + SCM scm_box, scm_expected, scm_result; UNPACK_12_12 (op, dst, box); UNPACK_24 (ip[1], expected); UNPACK_24 (ip[2], desired); scm_box = SP_REF (box); VM_VALIDATE_ATOMIC_BOX (scm_box, "atomic-box-compare-and-swap!"); - scm_expected = SP_REF (expected); - scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (scm_box), - &scm_expected, SP_REF (desired)); - SP_SET (dst, scm_expected); + scm_result = scm_expected = SP_REF (expected); + while (!scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (scm_box), + &scm_result, SP_REF (desired)) + && scm_is_eq (scm_result, scm_expected)) + { + } + SP_SET (dst, scm_result); NEXT (3); } -- 2.19.0 --=-=-=-- From debbugs-submit-bounces@debbugs.gnu.org Thu Sep 27 02:18:21 2018 Received: (at 32786) by debbugs.gnu.org; 27 Sep 2018 06:18:22 +0000 Received: from localhost ([127.0.0.1]:55851 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g5Pcz-0000BU-MW for submit@debbugs.gnu.org; Thu, 27 Sep 2018 02:18:21 -0400 Received: from k.imm.uran.ru ([195.19.132.74]:48586) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g5Pcy-0000BM-E3 for 32786@debbugs.gnu.org; Thu, 27 Sep 2018 02:18:21 -0400 Received: from blacky (88.226-224-87.telenet.ru [87.224.226.88]) by k.imm.uran.ru (Postfix) with ESMTPSA id 1C57D19369; Thu, 27 Sep 2018 11:18:18 +0500 (+05) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=k.imm.uran.ru; s=kite; t=1538029098; bh=dNXqzu7iNnyIfV3Vp6Kyv19uFmJeMLNyQsJtXw1w2GM=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=nlQdruqVz70z2P7HtckF0OMHQh43wBOpjY2lIge7Cbjot62l/SuWkk5jo0wzwaGhT LX7omGpzvScrQH8K3m6HCAZzy7hKacChOJNfOt/SIvSuHcehZIf3GrHnbeorkY2FIz az3NIgMquH7VdN/0sUE3Zq69ZMUzGFgLKPAPSCV/oxgu+ba0d0baLBKErslHj+c395 UbnFTRzddrd5ts9qiJiWOZqvqeHcgwW6YkkKM4dYuqyYWkuEPsj7x4xyiooOyZWdm9 3x/CddOMB2p5S5rU+TLCwqwKTDVt9xDpyDb8glyQCX0Oef2zcCqPkeOXz0Q3Do2ROT NtmmGB9rjTV4XtVzKUD0dxdQKK4C3XAtkhAfACTP1OwlEB4R4jZNn7XPpN0f+wJwdm TJtkuRBky98qvYcgFewVxc/3ydaRET2rzvX1dfr/+aWQH7cjBXTsBwMHXxkeaxmD2s CFfZnOwhmF3XsAsnsAnj7QHLf/gc2PL6cqH3Yk5v7G5xZOdpwIiq90b172WNuKzIiL vLbWh6NmTJc+tN6uolVRO7GIxbB/YCTH27SCuWhl+VFKFYUX1NkywpVzFJdFcJ7kbc b1SJCJejyddAcm21IVUU4aaHqwEc1jGWn1WAtwMYFgZE6xsuIIMa+As8gWex82n7ec HIwTW5TqZiGYBdM9V5Rc1rG4= Date: Thu, 27 Sep 2018 11:18:16 +0500 From: =?utf-8?B?0JzQuNGF0LDQuNC7INCR0LDRhdGC0LXRgNC10LI=?= To: Mark H Weaver Subject: Re: bug#32786: (system* ...) call somehow interferes with atomic-box on armv7l Message-ID: <20180927061816.GB9085@blacky> References: <20180920171306.GA9185@k> <87fty2v8xo.fsf@netris.org> <20180922043938.GA4539@blacky> <87sh1vcwws.fsf@netris.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <87sh1vcwws.fsf@netris.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Score: 0.1 (/) X-Debbugs-Envelope-To: 32786 Cc: 32786@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.9 (/) Thanks for clarification! I'll be able to test the patch in a couple of days. - MB. Respectfully On Thu, Sep 27, 2018 at 01:49:23AM -0400, Mark H Weaver wrote: > Hi, > > Thanks for the additional details. I was able to reproduce the bug, and > I believe I now see the problem. > > 'atomic-box-compare-and-swap!' is implemented using > 'atomic_compare_exchange_weak' (if available), but neglects to handle > the case where 'atomic_compare_exchange_weak' may spuriously fail. In > that case, the box is left unchanged, although the observed value was > equal to the expected value. > > What's happening here is that the 'atomic-box-compare-and-swap!' in > 'sleep-loop' fails spuriously, leaving the box in state #:accepted > although it returns #:accepted as its result. When the main loop > discovers this, it changes the state to #:need-to-sleep, although the > thread has already ended. > > To confirm this hypothesis, I added a print statement to the main loop > showing the state of the box that it observed during the protocol > exchange, and indeed it sees #:accepted the first time it checks, and > #:need-to-sleep in all later iterations. > > I've attached a proposed patch that I hope will fix this problem. If > you'd be willing to test it, I'd be grateful, but otherwise I'll try to > test it in the next week or so. My access to armv7l boxes is somewhat > limited. > > Thanks for this report. > > Mark > > From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 05 15:44:04 2018 Received: (at 32786) by debbugs.gnu.org; 5 Oct 2018 19:44:05 +0000 Received: from localhost ([127.0.0.1]:38211 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g8W16-0007GS-K3 for submit@debbugs.gnu.org; Fri, 05 Oct 2018 15:44:04 -0400 Received: from k.imm.uran.ru ([195.19.132.74]:47412) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g8W14-0007Fq-Qn for 32786@debbugs.gnu.org; Fri, 05 Oct 2018 15:44:03 -0400 Received: from k (k.imm.uran.ru [195.19.132.74]) by k.imm.uran.ru (Postfix) with ESMTPSA id A20681B2B9; Sat, 6 Oct 2018 00:43:58 +0500 (+05) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=k.imm.uran.ru; s=kite; t=1538768638; bh=BPn8CDr3rBmL0pGZKmn5FVdVJU8ZzCLliBaMVmVdCx0=; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=09tX1fDZJIqmdjEpdWKYpXsAV2muQbqsBpxedfLAYM2Krw4EzxuHDVLh4JGdUshHp rOxa5glLgOjM2bQSnZru79B0wp4La8cu2jGNcNEyIeBWPRZJrB3ZluQg7aaq3vzYp8 kSRu00ExVUNJQ2xY9JZPKO3LpmQzSYaRLwi2Xa1uGXO0TKHQ+Q2Il8nRYlxqr+1PZT qJc5ILSaooNT9myGMdpM289cydEx5O1SLSLHx3Pt7L4acjyD4FwOnWargXVPJA3KOf KHbjiIz17ioxzd7ThWMzL1nOyvEKCKx6GVqxTZuO2aJJe4ELpiPUjaHVp3k/8Buawy 6CuAuXmbJf5dhwVVA/u+5pDfRNa5hZwZmre7gtVP/9oc7j6iLalaZecbMwZtylFNHu bHZkKGJ0dA435tQMq0WVkrZcUeK1ycEETULVfELroToWYQR64DoGKlCt2AB9wDzoyn uHzn/+bRl+eG8DoYi/PofHbJjs45utkknsvH1VEQsXEYwno0BRe0Hx5VqfF+IH7lkh YFueaUDZbX2QuFcxxrozr7wpZfbhTqpBqGNTXQmOy6sE5jMLmfwnH89wqjt6TPmHWC b82GetpkXGl/WzaoWZabUUBvjdbkSovL27q6+ymGHBZZjDlU9cUT+OlZO6F5NsbnFx GlQOzNRJvc5GwR1TAM9uHgNc= Date: Sat, 6 Oct 2018 00:43:57 +0500 From: =?utf-8?B?0JzQuNGF0LDQuNC7INCR0LDRhdGC0LXRgNC10LI=?= To: Mark H Weaver Subject: Re: bug#32786: (system* ...) call somehow interferes with atomic-box on armv7l Message-ID: <20181005194357.GA542@k> References: <20180920171306.GA9185@k> <87fty2v8xo.fsf@netris.org> <20180922043938.GA4539@blacky> <87sh1vcwws.fsf@netris.org> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <87sh1vcwws.fsf@netris.org> User-Agent: Mutt/1.10.1 (2018-07-13) X-Spam-Score: 0.1 (/) X-Debbugs-Envelope-To: 32786 Cc: 32786@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.9 (/) Hi. I am sorry for delay. I've tested the patch (i've applied it to the v2.2.4 source code). The test sample works now as expected with both test «payloads»: (sleep N-seconds) and (system* "any" "command" "here") calls. Thanks for fixing it. - MB. Respectfully On Thu, Sep 27, 2018 at 01:49:23AM -0400, Mark H Weaver wrote: > Hi, > > Thanks for the additional details. I was able to reproduce the bug, and > I believe I now see the problem. > > 'atomic-box-compare-and-swap!' is implemented using > 'atomic_compare_exchange_weak' (if available), but neglects to handle > the case where 'atomic_compare_exchange_weak' may spuriously fail. In > that case, the box is left unchanged, although the observed value was > equal to the expected value. > > What's happening here is that the 'atomic-box-compare-and-swap!' in > 'sleep-loop' fails spuriously, leaving the box in state #:accepted > although it returns #:accepted as its result. When the main loop > discovers this, it changes the state to #:need-to-sleep, although the > thread has already ended. > > To confirm this hypothesis, I added a print statement to the main loop > showing the state of the box that it observed during the protocol > exchange, and indeed it sees #:accepted the first time it checks, and > #:need-to-sleep in all later iterations. > > I've attached a proposed patch that I hope will fix this problem. If > you'd be willing to test it, I'd be grateful, but otherwise I'll try to > test it in the next week or so. My access to armv7l boxes is somewhat > limited. > > Thanks for this report. > > Mark > > > From 958d37686a9ac65f48cb2ca32a341cf182c24b5a Mon Sep 17 00:00:00 2001 > From: Mark H Weaver > Date: Thu, 27 Sep 2018 01:00:11 -0400 > Subject: [PATCH] UNTESTED: Fix 'atomic-box-compare-and-swap!'. > > Fixes . > > 'scm_atomic_compare_and_swap_scm' uses 'atomic_compare_exchange_weak' if > it's available on the host platform. 'atomic_compare_exchange_weak' may > fail spuriously, leaving the atomic object unchanged even when it > contained the expected value. 'atomic-box-compare-and-swap!' uses > 'scm_atomic_compare_and_swap_scm' without checking its return value, and > therefore may return the expected value while leaving the box unchanged. > This contradicts the documentation, which explicitly states that "you > can know if the swap worked by checking if the return value is 'eq?' to > EXPECTED.". > > * libguile/atomic.c (scm_atomic_box_compare_and_swap_x): If > 'scm_atomic_compare_and_swap_scm' returns false and the observed value > is equal to 'expected', then try again. > * libguile/vm-engine.c (atomic-box-compare-and-swap!): Ditto. > --- > libguile/atomic.c | 13 +++++++++---- > libguile/vm-engine.c | 13 ++++++++----- > 2 files changed, 17 insertions(+), 9 deletions(-) > > diff --git a/libguile/atomic.c b/libguile/atomic.c > index 950874030..504643912 100644 > --- a/libguile/atomic.c > +++ b/libguile/atomic.c > @@ -1,4 +1,4 @@ > -/* Copyright (C) 2016 Free Software Foundation, Inc. > +/* Copyright (C) 2016, 2018 Free Software Foundation, Inc. > * > * This library is free software; you can redistribute it and/or > * modify it under the terms of the GNU Lesser General Public License > @@ -95,10 +95,15 @@ SCM_DEFINE (scm_atomic_box_compare_and_swap_x, > "if the return value is @code{eq?} to @var{expected}.") > #define FUNC_NAME s_scm_atomic_box_compare_and_swap_x > { > + SCM result = expected; > + > SCM_VALIDATE_ATOMIC_BOX (1, box); > - scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (box), > - &expected, desired); > - return expected; > + while (!scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (box), > + &result, desired) > + && scm_is_eq (result, expected)) > + { > + } > + return result; > } > #undef FUNC_NAME > > diff --git a/libguile/vm-engine.c b/libguile/vm-engine.c > index 19ff3e498..9650e33eb 100644 > --- a/libguile/vm-engine.c > +++ b/libguile/vm-engine.c > @@ -3868,16 +3868,19 @@ VM_NAME (scm_i_thread *thread, struct scm_vm *vp, > { > scm_t_uint16 dst, box; > scm_t_uint32 expected, desired; > - SCM scm_box, scm_expected; > + SCM scm_box, scm_expected, scm_result; > UNPACK_12_12 (op, dst, box); > UNPACK_24 (ip[1], expected); > UNPACK_24 (ip[2], desired); > scm_box = SP_REF (box); > VM_VALIDATE_ATOMIC_BOX (scm_box, "atomic-box-compare-and-swap!"); > - scm_expected = SP_REF (expected); > - scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (scm_box), > - &scm_expected, SP_REF (desired)); > - SP_SET (dst, scm_expected); > + scm_result = scm_expected = SP_REF (expected); > + while (!scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (scm_box), > + &scm_result, SP_REF (desired)) > + && scm_is_eq (scm_result, scm_expected)) > + { > + } > + SP_SET (dst, scm_result); > NEXT (3); > } > > -- > 2.19.0 > From debbugs-submit-bounces@debbugs.gnu.org Fri Oct 05 19:22:35 2018 Received: (at 32786-done) by debbugs.gnu.org; 5 Oct 2018 23:22:35 +0000 Received: from localhost ([127.0.0.1]:38253 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g8ZQZ-0005tr-90 for submit@debbugs.gnu.org; Fri, 05 Oct 2018 19:22:35 -0400 Received: from world.peace.net ([64.112.178.59]:38276) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1g8ZQW-0005tc-Ck for 32786-done@debbugs.gnu.org; Fri, 05 Oct 2018 19:22:34 -0400 Received: from mhw by world.peace.net with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89) (envelope-from ) id 1g8ZQP-000383-Vm; Fri, 05 Oct 2018 19:22:26 -0400 From: Mark H Weaver To: =?utf-8?B?0JzQuNGF0LDQuNC7INCR0LDRhdGC0LXRgNC10LI=?= Subject: Re: bug#32786: (system* ...) call somehow interferes with atomic-box on armv7l References: <20180920171306.GA9185@k> <87fty2v8xo.fsf@netris.org> <20180922043938.GA4539@blacky> <87sh1vcwws.fsf@netris.org> <20181005194357.GA542@k> Date: Fri, 05 Oct 2018 19:22:13 -0400 In-Reply-To: <20181005194357.GA542@k> (=?utf-8?B?ItCc0LjRhdCw0LjQuyDQkdCw?= =?utf-8?B?0YXRgtC10YDQtdCyIidz?= message of "Sat, 6 Oct 2018 00:43:57 +0500") Message-ID: <87lg7cf07u.fsf@netris.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.8 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Hi, Михаил Бахтерев writes: > I've tested the patch (i've applied it to the v2.2.4 source code). The > test sample works now as expected with both test «payloads»: (sleep > N-seconds) and (system* "any" "command" "here") calls. [...] Content analysis details: (1.8 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 1.8 FUZZY_XPILL BODY: Attempt to obfuscate words in spam X-Debbugs-Envelope-To: 32786-done Cc: 32786-done@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: 0.8 (/) Hi, =D0=9C=D0=B8=D1=85=D0=B0=D0=B8=D0=BB =D0=91=D0=B0=D1=85=D1=82=D0=B5=D1=80= =D0=B5=D0=B2 writes: > I've tested the patch (i've applied it to the v2.2.4 source code). The > test sample works now as expected with both test =C2=ABpayloads=C2=BB: (s= leep > N-seconds) and (system* "any" "command" "here") calls. Thanks for letting us know the results of your testing, this is very helpful. I just pushed a similar fix (same code, improved comments) to the stable-2.2 branch, commit 2d09e0513fc11e2305077ba3653f6e4c2f266ddb, which will be in the upcoming guile-2.2.5 release. Thanks, Mark > > Thanks for fixing it. > > - MB. Respectfully > > On Thu, Sep 27, 2018 at 01:49:23AM -0400, Mark H Weaver wrote: >> Hi, >>=20 >> Thanks for the additional details. I was able to reproduce the bug, and >> I believe I now see the problem. >>=20 >> 'atomic-box-compare-and-swap!' is implemented using >> 'atomic_compare_exchange_weak' (if available), but neglects to handle >> the case where 'atomic_compare_exchange_weak' may spuriously fail. In >> that case, the box is left unchanged, although the observed value was >> equal to the expected value. >>=20 >> What's happening here is that the 'atomic-box-compare-and-swap!' in >> 'sleep-loop' fails spuriously, leaving the box in state #:accepted >> although it returns #:accepted as its result. When the main loop >> discovers this, it changes the state to #:need-to-sleep, although the >> thread has already ended. >>=20 >> To confirm this hypothesis, I added a print statement to the main loop >> showing the state of the box that it observed during the protocol >> exchange, and indeed it sees #:accepted the first time it checks, and >> #:need-to-sleep in all later iterations. >>=20 >> I've attached a proposed patch that I hope will fix this problem. If >> you'd be willing to test it, I'd be grateful, but otherwise I'll try to >> test it in the next week or so. My access to armv7l boxes is somewhat >> limited. >>=20 >> Thanks for this report. >>=20 >> Mark >>=20 >>=20 > >> From 958d37686a9ac65f48cb2ca32a341cf182c24b5a Mon Sep 17 00:00:00 2001 >> From: Mark H Weaver >> Date: Thu, 27 Sep 2018 01:00:11 -0400 >> Subject: [PATCH] UNTESTED: Fix 'atomic-box-compare-and-swap!'. >>=20 >> Fixes . >>=20 >> 'scm_atomic_compare_and_swap_scm' uses 'atomic_compare_exchange_weak' if >> it's available on the host platform. 'atomic_compare_exchange_weak' may >> fail spuriously, leaving the atomic object unchanged even when it >> contained the expected value. 'atomic-box-compare-and-swap!' uses >> 'scm_atomic_compare_and_swap_scm' without checking its return value, and >> therefore may return the expected value while leaving the box unchanged. >> This contradicts the documentation, which explicitly states that "you >> can know if the swap worked by checking if the return value is 'eq?' to >> EXPECTED.". >>=20 >> * libguile/atomic.c (scm_atomic_box_compare_and_swap_x): If >> 'scm_atomic_compare_and_swap_scm' returns false and the observed value >> is equal to 'expected', then try again. >> * libguile/vm-engine.c (atomic-box-compare-and-swap!): Ditto. >> --- >> libguile/atomic.c | 13 +++++++++---- >> libguile/vm-engine.c | 13 ++++++++----- >> 2 files changed, 17 insertions(+), 9 deletions(-) >>=20 >> diff --git a/libguile/atomic.c b/libguile/atomic.c >> index 950874030..504643912 100644 >> --- a/libguile/atomic.c >> +++ b/libguile/atomic.c >> @@ -1,4 +1,4 @@ >> -/* Copyright (C) 2016 Free Software Foundation, Inc. >> +/* Copyright (C) 2016, 2018 Free Software Foundation, Inc. >> * >> * This library is free software; you can redistribute it and/or >> * modify it under the terms of the GNU Lesser General Public License >> @@ -95,10 +95,15 @@ SCM_DEFINE (scm_atomic_box_compare_and_swap_x, >> "if the return value is @code{eq?} to @var{expected}.") >> #define FUNC_NAME s_scm_atomic_box_compare_and_swap_x >> { >> + SCM result =3D expected; >> + >> SCM_VALIDATE_ATOMIC_BOX (1, box); >> - scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (box), >> - &expected, desired); >> - return expected; >> + while (!scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (box), >> + &result, desired) >> + && scm_is_eq (result, expected)) >> + { >> + } >> + return result; >> } >> #undef FUNC_NAME >>=20=20 >> diff --git a/libguile/vm-engine.c b/libguile/vm-engine.c >> index 19ff3e498..9650e33eb 100644 >> --- a/libguile/vm-engine.c >> +++ b/libguile/vm-engine.c >> @@ -3868,16 +3868,19 @@ VM_NAME (scm_i_thread *thread, struct scm_vm *vp, >> { >> scm_t_uint16 dst, box; >> scm_t_uint32 expected, desired; >> - SCM scm_box, scm_expected; >> + SCM scm_box, scm_expected, scm_result; >> UNPACK_12_12 (op, dst, box); >> UNPACK_24 (ip[1], expected); >> UNPACK_24 (ip[2], desired); >> scm_box =3D SP_REF (box); >> VM_VALIDATE_ATOMIC_BOX (scm_box, "atomic-box-compare-and-swap!"); >> - scm_expected =3D SP_REF (expected); >> - scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (scm_box), >> - &scm_expected, SP_REF (desired)); >> - SP_SET (dst, scm_expected); >> + scm_result =3D scm_expected =3D SP_REF (expected); >> + while (!scm_atomic_compare_and_swap_scm (scm_atomic_box_loc (scm_= box), >> + &scm_result, SP_REF (des= ired)) >> + && scm_is_eq (scm_result, scm_expected)) >> + { >> + } >> + SP_SET (dst, scm_result); >> NEXT (3); >> } >>=20=20 >> --=20 >> 2.19.0 >>=20 From unknown Wed Jun 18 00:27:06 2025 Received: (at fakecontrol) by fakecontrolmessage; To: internal_control@debbugs.gnu.org From: Debbugs Internal Request Subject: Internal Control Message-Id: bug archived. Date: Sat, 03 Nov 2018 11:24:05 +0000 User-Agent: Fakemail v42.6.9 # This is a fake control message. # # The action: # bug archived. thanks # This fakemail brought to you by your local debbugs # administrator