From unknown Mon Jun 23 23:52:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort. Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Sat, 10 Jul 2010 01:09:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 6600 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: Chen Guo Cc: eggert@cs.ucla.edu, 6600@debbugs.gnu.org, glen.lenker@gmail.com, mykphyre@gmail.com, quaker4lyf@gmail.com, cdickens@ucla.edu X-Debbugs-Original-Cc: Paul Eggert , Bug Coreutils , Glen Lenker , Mike Nichols , Gene Auyeung , Chris Dickens Received: via spool by submit@debbugs.gnu.org id=B.12787241262758 (code B ref -1); Sat, 10 Jul 2010 01:09:02 +0000 Received: (at submit) by debbugs.gnu.org; 10 Jul 2010 01:08:46 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OXOYf-0000iR-AP for submit@debbugs.gnu.org; Fri, 09 Jul 2010 21:08:45 -0400 Received: from mx10.gnu.org ([199.232.76.166]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OXOYc-0000iK-TE for submit@debbugs.gnu.org; Fri, 09 Jul 2010 21:08:43 -0400 Received: from lists.gnu.org ([199.232.76.165]:49413) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1OXOYZ-0003lg-0E for submit@debbugs.gnu.org; Fri, 09 Jul 2010 21:08:39 -0400 Received: from [140.186.70.92] (port=57143 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OXOYX-0000Wc-Hi for bug-coreutils@gnu.org; Fri, 09 Jul 2010 21:08:38 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=unavailable version=3.3.1 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OXOYW-0003qp-5r for bug-coreutils@gnu.org; Fri, 09 Jul 2010 21:08:37 -0400 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:1061) by eggs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1OXOYV-0003qL-SU for bug-coreutils@gnu.org; Fri, 09 Jul 2010 21:08:36 -0400 Received: (qmail 45007 invoked from network); 10 Jul 2010 01:08:33 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 10 Jul 2010 01:08:33 -0000 Message-ID: <4C37C7D9.2030909@draigBrady.com> Date: Sat, 10 Jul 2010 02:07:37 +0100 From: =?UTF-8?Q?P=C3=A1draig?= Brady User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> In-Reply-To: <535984.88146.qm@web180006.mail.gq1.yahoo.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9 X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4) X-Spam-Score: -3.5 (---) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -4.8 (----) On 08/03/10 10:39, Chen Guo wrote: > Hi Padraig, > >> You previously mentioned a thread bug with memcoll. Is that worked around? > > That happened when more than one instance of memcoll is called on the same > line at once, since memcoll replaces the eolchar with '\0'. Under our approach, > the same line shouldn't ever be compared at the same time, so we're fine. > On top of that, Professor Eggert suggested NUL delimiting all lines as they're > read in, so memcoll doesn't have to; hence the patch to gnulib, which introduces > xmemcoll_nul and memcoll_nul, for when input is known to be NUL delimited, thus > no replacement of the eolchar is needed, making memcoll threadsafe. Note the current xmemcoll0() in gnulib requires the length _including_ the terminating NUL to be passed, whereas one usually does not include the terminating char in the length passed to xmemcoll(). I accordingly updated the lengths passed to xmemcoll0() by your latest patch. However there are still writes done to the source text in the keycompare() function. So I'm thinking of dropping the whole xmemcoll0() thing altogether assuming your statement above is correct, that a particular line will not be used at the same time by multiple threads. I did try to copy the text to the stack before comparing, but that introduced a significant overhead noted below. Your patch is still performing well on a single core machine: ----------- before --------------------- $ time ./src/sort < nums.list >/dev/null real 0m8.644s user 0m8.307s sys 0m0.292s $ time ./src/sort -g < nums.list >/dev/null real 0m11.046s user 0m10.652s sys 0m0.295s $ time ./src/sort -n < nums.list >/dev/null real 0m4.909s user 0m4.567s sys 0m0.298s $ time LANG=C ./src/sort < nums.list >/dev/null real 0m1.959s user 0m1.657s sys 0m0.285s ------------ after --------------------- $ time ./src/sort < nums.list >/dev/null real 0m8.686s user 0m8.300s sys 0m0.232s $ time ./src/sort -g < nums.list >/dev/null real 0m10.196s user 0m9.850s sys 0m0.221s $ time ./src/sort -n < nums.list >/dev/null real 0m2.958s user 0m2.664s sys 0m0.221s $ time LANG=C ./src/sort < nums.list >/dev/null real 0m1.985s user 0m1.750s sys 0m0.217s After copying the text to the stack as mentioned above there is a significant performance drop: $ time ./src/sort -n < nums.list >/dev/null real 0m4.086s user 0m3.848s sys 0m0.218s cheers, Pádraig. From unknown Mon Jun 23 23:52:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort. Resent-From: Paul Eggert Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Sat, 10 Jul 2010 02:55:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6600 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: =?UTF-8?Q?P=C3=A1draig?= Brady Cc: Bug Coreutils , Chen Guo , Glen Lenker , Mike Nichols , Gene Auyeung , Chris Dickens Received: via spool by submit@debbugs.gnu.org id=B.12787304845356 (code B ref -1); Sat, 10 Jul 2010 02:55:01 +0000 Received: (at submit) by debbugs.gnu.org; 10 Jul 2010 02:54:44 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OXQDD-0001OL-NQ for submit@debbugs.gnu.org; Fri, 09 Jul 2010 22:54:44 -0400 Received: from mail.gnu.org ([199.232.76.166] helo=mx10.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OXQDB-0001OG-3d for submit@debbugs.gnu.org; Fri, 09 Jul 2010 22:54:41 -0400 Received: from lists.gnu.org ([199.232.76.165]:39317) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1OXQD7-0004nO-Lo for submit@debbugs.gnu.org; Fri, 09 Jul 2010 22:54:37 -0400 Received: from [140.186.70.92] (port=55389 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OXQD6-000704-8o for bug-coreutils@gnu.org; Fri, 09 Jul 2010 22:54:37 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_RP_MATCHES_RCVD autolearn=unavailable version=3.3.1 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OXQD5-0005WR-4Z for bug-coreutils@gnu.org; Fri, 09 Jul 2010 22:54:36 -0400 Received: from kiwi.cs.ucla.edu ([131.179.128.19]:52653) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OXQD4-0005W5-PV for bug-coreutils@gnu.org; Fri, 09 Jul 2010 22:54:35 -0400 Received: from [131.179.64.200] (Penguin.CS.UCLA.EDU [131.179.64.200]) by kiwi.cs.ucla.edu (8.13.8+Sun/8.13.8/UCLACS-6.0) with ESMTP id o6A2sQoF002843; Fri, 9 Jul 2010 19:54:27 -0700 (PDT) Message-ID: <4C37E0E2.3060205@cs.ucla.edu> Date: Fri, 09 Jul 2010 19:54:26 -0700 From: Paul Eggert Organization: UCLA Computer Science Department User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.10) Gecko/20100527 Thunderbird/3.0.5 MIME-Version: 1.0 References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> <4C37C7D9.2030909@draigBrady.com> In-Reply-To: <4C37C7D9.2030909@draigBrady.com> Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by kiwi.cs.ucla.edu id o6A2sQoF002843 X-detected-operating-system: by eggs.gnu.org: Solaris 10 (beta) X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4) X-Spam-Score: -4.8 (----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -4.8 (----) On 07/09/10 18:07, P=E1draig Brady wrote: > Chen Guo wrote: >> That happened when more than one instance of memcoll is called on the = same >> line at once, since memcoll replaces the eolchar with '\0'. Under our = approach, >> the same line shouldn't ever be compared at the same time, so we're fi= ne. Ah, sorry, I wasn't aware of that. > I'm thinking of dropping > the whole xmemcoll0() thing altogether assuming your > statement above is correct, that a particular line will > not be used at the same time by multiple threads. Yes, that makes sense. We can revert that change from gnulib, since it makes gnulib bigger unnecessarily. From unknown Mon Jun 23 23:52:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort. Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Sat, 10 Jul 2010 09:31:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6600 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: Chen Guo Cc: Paul Eggert , Bug Coreutils , Glen Lenker , Mike Nichols , Gene Auyeung , Chris Dickens Received: via spool by submit@debbugs.gnu.org id=B.127875421615086 (code B ref -1); Sat, 10 Jul 2010 09:31:02 +0000 Received: (at submit) by debbugs.gnu.org; 10 Jul 2010 09:30:16 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OXWO0-0003vH-Il for submit@debbugs.gnu.org; Sat, 10 Jul 2010 05:30:16 -0400 Received: from mail.gnu.org ([199.232.76.166] helo=mx10.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OXWNx-0003vC-7J for submit@debbugs.gnu.org; Sat, 10 Jul 2010 05:30:14 -0400 Received: from lists.gnu.org ([199.232.76.165]:47815) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1OXWNt-00010x-AB for submit@debbugs.gnu.org; Sat, 10 Jul 2010 05:30:09 -0400 Received: from [140.186.70.92] (port=53408 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OXWNp-00032U-LJ for bug-coreutils@gnu.org; Sat, 10 Jul 2010 05:30:07 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=unavailable version=3.3.1 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OXWNn-00053f-UJ for bug-coreutils@gnu.org; Sat, 10 Jul 2010 05:30:05 -0400 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:13450) by eggs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1OXWNn-00053F-LS for bug-coreutils@gnu.org; Sat, 10 Jul 2010 05:30:03 -0400 Received: (qmail 96387 invoked from network); 10 Jul 2010 09:30:01 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 10 Jul 2010 09:30:01 -0000 Message-ID: <4C383D5F.4090703@draigBrady.com> Date: Sat, 10 Jul 2010 10:29:03 +0100 From: =?UTF-8?Q?P=C3=A1draig?= Brady User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> <4C37C7D9.2030909@draigBrady.com> <4C37E0E2.3060205@cs.ucla.edu> In-Reply-To: X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9 X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4) X-Spam-Score: -4.8 (----) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -4.8 (----) On 10/07/10 05:23, Chen Guo wrote: > 2010/7/9 Paul Eggert : >> On 07/09/10 18:07, Pádraig Brady wrote: >>> Chen Guo wrote: >>>> That happened when more than one instance of memcoll is called on the same >>>> line at once, since memcoll replaces the eolchar with '\0'. Under our approach, >>>> the same line shouldn't ever be compared at the same time, so we're fine. >> >> Ah, sorry, I wasn't aware of that. >> >>> I'm thinking of dropping >>> the whole xmemcoll0() thing altogether assuming your >>> statement above is correct, that a particular line will >>> not be used at the same time by multiple threads. >> >> Yes, that makes sense. We can revert that change from gnulib, since it >> makes gnulib bigger unnecessarily. >> > > Actually, the '\0' saves about 5% off runtime last I checked. This is because > EACH TIME sort compares two lines memcoll would replace the last byte. If we > set them all to NUL anyway at the start, memcoll_nul wouldn't need to do that > replacement for each compare. When we output, we'd simply put the \n back. > > I could be wrong though, this is going off memory from 4-5 months ago. But 5% > is about what I remember, when sorting 1M lines on 8 cores. Well for the whole line comparison where it works it's 2.9% faster (or 2.3% adding in the NUL checks to xmemcoll0). Also xmemcoll0() is probably generally useful since it's a readonly function. So I'll leave it and fix up the keycompare() calls to it (while documenting that keycompare() needs to work on a particular line in line one thread. thanks, Pádraig. From unknown Mon Jun 23 23:52:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort. Resent-From: Chen Guo Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Sat, 10 Jul 2010 16:30:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6600 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: Paul Eggert Cc: Bug Coreutils , Glen Lenker , Mike Nichols , Gene Auyeung , Chris Dickens , =?UTF-8?Q?P=C3=A1draig?= Brady Received: via spool by submit@debbugs.gnu.org id=B.127877934428854 (code B ref -1); Sat, 10 Jul 2010 16:30:02 +0000 Received: (at submit) by debbugs.gnu.org; 10 Jul 2010 16:29:04 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OXcvH-0007VL-0o for submit@debbugs.gnu.org; Sat, 10 Jul 2010 12:29:03 -0400 Received: from mx10.gnu.org ([199.232.76.166]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OXRbW-0001vV-OS for submit@debbugs.gnu.org; Sat, 10 Jul 2010 00:23:55 -0400 Received: from lists.gnu.org ([199.232.76.165]:39306) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1OXRbT-0005gB-8u for submit@debbugs.gnu.org; Sat, 10 Jul 2010 00:23:51 -0400 Received: from [140.186.70.92] (port=52869 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OXRbR-0001Zb-DY for bug-coreutils@gnu.org; Sat, 10 Jul 2010 00:23:50 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.3 required=5.0 tests=BAYES_00, FREEMAIL_ENVFROM_END_DIGIT, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE, T_DKIM_INVALID autolearn=no version=3.3.1 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OXRbQ-0005sK-3s for bug-coreutils@gnu.org; Sat, 10 Jul 2010 00:23:49 -0400 Received: from mail-pv0-f169.google.com ([74.125.83.169]:33082) by eggs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OXRbP-0005sG-T2 for bug-coreutils@gnu.org; Sat, 10 Jul 2010 00:23:48 -0400 Received: by pvc30 with SMTP id 30so2510326pvc.0 for ; Fri, 09 Jul 2010 21:23:46 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=AocMZLpSpYOpiyP9DgmzJn42whcvPIoDCOSGecskCMI=; b=grt7ovdCLMC3bnsyh6O6/GfoWXmrM4qoIwj+yrRITjp6kPmUEcm2Vl+kxL/hu5zOC9 b0rERZ9QOhTrXBq40ktf/jX672mvaeFq8X060xmtNrqGhAQyEmKHu1vlfXVUlLB123L7 NjS4t4PO3CrsSAskN2H+1oEJgZsDVihmxUtlI= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=HRMEZg+CctXiLfjn8GciyoqJbgZmzLCveTdTc9khskP0WfuLK/MBs9B9LexpoaHsLY G11OIz+qNnx2KD7njpkO62w5TI3UQod5qNQb7WUoFkXwDsDDRqH4B5PhC6XdAJ2D4gzP 12LBOBo2WVS0Mv9m1c8QvoFMtr7f/qVULpV4k= MIME-Version: 1.0 Received: by 10.142.170.2 with SMTP id s2mr12850973wfe.243.1278735826309; Fri, 09 Jul 2010 21:23:46 -0700 (PDT) Received: by 10.142.192.12 with HTTP; Fri, 9 Jul 2010 21:23:46 -0700 (PDT) In-Reply-To: <4C37E0E2.3060205@cs.ucla.edu> References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> <4C37C7D9.2030909@draigBrady.com> <4C37E0E2.3060205@cs.ucla.edu> Date: Fri, 9 Jul 2010 21:23:46 -0700 Message-ID: From: Chen Guo Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6 (newer, 2) X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4) X-Spam-Score: -5.9 (-----) X-Mailman-Approved-At: Sat, 10 Jul 2010 12:29:01 -0400 X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -5.9 (-----) 2010/7/9 Paul Eggert : > On 07/09/10 18:07, P=E1draig Brady wrote: >> Chen Guo wrote: >>> That happened when more than one instance of memcoll is called on the s= ame >>> line at once, since memcoll replaces the eolchar with '\0'. Under our a= pproach, >>> the same line shouldn't ever be compared at the same time, so we're fin= e. > > Ah, sorry, I wasn't aware of that. > >> I'm thinking of dropping >> the whole xmemcoll0() thing altogether assuming your >> statement above is correct, that a particular line will >> not be used at the same time by multiple threads. > > Yes, that makes sense. =A0We can revert that change from gnulib, since it > makes gnulib bigger unnecessarily. > Actually, the '\0' saves about 5% off runtime last I checked. This is becau= se EACH TIME sort compares two lines memcoll would replace the last byte. If w= e set them all to NUL anyway at the start, memcoll_nul wouldn't need to do th= at replacement for each compare. When we output, we'd simply put the \n back. I could be wrong though, this is going off memory from 4-5 months ago. But = 5% is about what I remember, when sorting 1M lines on 8 cores. From unknown Mon Jun 23 23:52:43 2025 MIME-Version: 1.0 X-Mailer: MIME-tools 5.427 (Entity 5.427) X-Loop: help-debbugs@gnu.org From: help-debbugs@gnu.org (GNU bug Tracking System) To: =?UTF-8?Q?P=C3=A1draig?= Brady Subject: bug#6600: closed (Re: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort.) Message-ID: References: <4C3BBA60.9020201@draigBrady.com> <4C37C7D9.2030909@draigBrady.com> X-Gnu-PR-Message: they-closed 6600 X-Gnu-PR-Package: coreutils X-Gnu-PR-Keywords: patch Reply-To: 6600@debbugs.gnu.org Date: Tue, 13 Jul 2010 01:01:01 +0000 Content-Type: multipart/mixed; boundary="----------=_1278982861-19756-1" This is a multi-part message in MIME format... ------------=_1278982861-19756-1 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset="utf-8" Your bug report #6600: [PATCH] sort: add --threads option to parallelize internal sort. which was filed against the coreutils package, has been closed. The explanation is attached below, along with your original report. If you require more details, please reply to 6600@debbugs.gnu.org. --=20 6600: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D6600 GNU Bug Tracking System Contact help-debbugs@gnu.org with problems ------------=_1278982861-19756-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at 6600-done) by debbugs.gnu.org; 13 Jul 2010 01:00:22 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OYTrC-00058O-7I for submit@debbugs.gnu.org; Mon, 12 Jul 2010 21:00:22 -0400 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1OYTr9-00058J-MX for 6600-done@debbugs.gnu.org; Mon, 12 Jul 2010 21:00:20 -0400 Received: (qmail 15223 invoked from network); 13 Jul 2010 01:00:22 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 13 Jul 2010 01:00:22 -0000 Message-ID: <4C3BBA60.9020201@draigBrady.com> Date: Tue, 13 Jul 2010 01:59:12 +0100 From: =?ISO-8859-1?Q?P=E1draig_Brady?= User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 To: 6600-done@debbugs.gnu.org Subject: Re: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort. References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> <4C37C7D9.2030909@draigBrady.com> <4C37E0E2.3060205@cs.ucla.edu> <4C383D5F.4090703@draigBrady.com> In-Reply-To: <4C383D5F.4090703@draigBrady.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.8 (--) X-Debbugs-Envelope-To: 6600-done Cc: Paul Eggert , Chen Guo , Glen Lenker , Mike Nichols , Gene Auyeung , Chris Dickens X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.8 (--) I've finally applied the patch. http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commit;h=9face836 I made a few comment tweaks and added some dependencies for the heap module. I also removed the xmemcoll0() calls which are separate to this concurrent functionality. I will add those back in Chen's name after updating to the latest gnulib. Thanks everyone for their work on this! Pádraig. ------------=_1278982861-19756-1 Content-Type: message/rfc822 Content-Disposition: inline Content-Transfer-Encoding: 7bit Received: (at submit) by debbugs.gnu.org; 10 Jul 2010 01:08:46 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OXOYf-0000iR-AP for submit@debbugs.gnu.org; Fri, 09 Jul 2010 21:08:45 -0400 Received: from mx10.gnu.org ([199.232.76.166]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OXOYc-0000iK-TE for submit@debbugs.gnu.org; Fri, 09 Jul 2010 21:08:43 -0400 Received: from lists.gnu.org ([199.232.76.165]:49413) by monty-python.gnu.org with esmtps (TLS-1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.60) (envelope-from ) id 1OXOYZ-0003lg-0E for submit@debbugs.gnu.org; Fri, 09 Jul 2010 21:08:39 -0400 Received: from [140.186.70.92] (port=57143 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1OXOYX-0000Wc-Hi for bug-coreutils@gnu.org; Fri, 09 Jul 2010 21:08:38 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00 autolearn=unavailable version=3.3.1 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.69) (envelope-from ) id 1OXOYW-0003qp-5r for bug-coreutils@gnu.org; Fri, 09 Jul 2010 21:08:37 -0400 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]:1061) by eggs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1OXOYV-0003qL-SU for bug-coreutils@gnu.org; Fri, 09 Jul 2010 21:08:36 -0400 Received: (qmail 45007 invoked from network); 10 Jul 2010 01:08:33 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 10 Jul 2010 01:08:33 -0000 Message-ID: <4C37C7D9.2030909@draigBrady.com> Date: Sat, 10 Jul 2010 02:07:37 +0100 From: =?ISO-8859-1?Q?P=E1draig_Brady?= User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 To: Chen Guo Subject: Re: [PATCH] sort: add --threads option to parallelize internal sort. References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> In-Reply-To: <535984.88146.qm@web180006.mail.gq1.yahoo.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-detected-operating-system: by eggs.gnu.org: FreeBSD 4.6-4.9 X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4) X-Spam-Score: -3.5 (---) X-Debbugs-Envelope-To: submit Cc: Paul Eggert , Bug Coreutils , Glen Lenker , Mike Nichols , Gene Auyeung , Chris Dickens X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -4.8 (----) On 08/03/10 10:39, Chen Guo wrote: > Hi Padraig, > >> You previously mentioned a thread bug with memcoll. Is that worked around? > > That happened when more than one instance of memcoll is called on the same > line at once, since memcoll replaces the eolchar with '\0'. Under our approach, > the same line shouldn't ever be compared at the same time, so we're fine. > On top of that, Professor Eggert suggested NUL delimiting all lines as they're > read in, so memcoll doesn't have to; hence the patch to gnulib, which introduces > xmemcoll_nul and memcoll_nul, for when input is known to be NUL delimited, thus > no replacement of the eolchar is needed, making memcoll threadsafe. Note the current xmemcoll0() in gnulib requires the length _including_ the terminating NUL to be passed, whereas one usually does not include the terminating char in the length passed to xmemcoll(). I accordingly updated the lengths passed to xmemcoll0() by your latest patch. However there are still writes done to the source text in the keycompare() function. So I'm thinking of dropping the whole xmemcoll0() thing altogether assuming your statement above is correct, that a particular line will not be used at the same time by multiple threads. I did try to copy the text to the stack before comparing, but that introduced a significant overhead noted below. Your patch is still performing well on a single core machine: ----------- before --------------------- $ time ./src/sort < nums.list >/dev/null real 0m8.644s user 0m8.307s sys 0m0.292s $ time ./src/sort -g < nums.list >/dev/null real 0m11.046s user 0m10.652s sys 0m0.295s $ time ./src/sort -n < nums.list >/dev/null real 0m4.909s user 0m4.567s sys 0m0.298s $ time LANG=C ./src/sort < nums.list >/dev/null real 0m1.959s user 0m1.657s sys 0m0.285s ------------ after --------------------- $ time ./src/sort < nums.list >/dev/null real 0m8.686s user 0m8.300s sys 0m0.232s $ time ./src/sort -g < nums.list >/dev/null real 0m10.196s user 0m9.850s sys 0m0.221s $ time ./src/sort -n < nums.list >/dev/null real 0m2.958s user 0m2.664s sys 0m0.221s $ time LANG=C ./src/sort < nums.list >/dev/null real 0m1.985s user 0m1.750s sys 0m0.217s After copying the text to the stack as mentioned above there is a significant performance drop: $ time ./src/sort -n < nums.list >/dev/null real 0m4.086s user 0m3.848s sys 0m0.218s cheers, Pádraig. ------------=_1278982861-19756-1-- From unknown Mon Jun 23 23:52:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort. Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Tue, 13 Jul 2010 09:15:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6600 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: 6600@debbugs.gnu.org Cc: Gene Auyeung , Paul Eggert , Chen Guo Received: via spool by 6600-submit@debbugs.gnu.org id=B6600.127901249232252 (code B ref 6600); Tue, 13 Jul 2010 09:15:02 +0000 Received: (at 6600) by debbugs.gnu.org; 13 Jul 2010 09:14:52 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OYbZj-0008O9-UR for submit@debbugs.gnu.org; Tue, 13 Jul 2010 05:14:52 -0400 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1OYbZh-0008O4-05 for 6600@debbugs.gnu.org; Tue, 13 Jul 2010 05:14:50 -0400 Received: (qmail 78612 invoked from network); 13 Jul 2010 09:14:53 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 13 Jul 2010 09:14:53 -0000 Message-ID: <4C3C2E44.9050009@draigBrady.com> Date: Tue, 13 Jul 2010 10:13:40 +0100 From: =?UTF-8?Q?P=C3=A1draig?= Brady User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> <4C37C7D9.2030909@draigBrady.com> <4C37E0E2.3060205@cs.ucla.edu> <4C383D5F.4090703@draigBrady.com> <4C3BBA60.9020201@draigBrady.com> In-Reply-To: <4C3BBA60.9020201@draigBrady.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.8 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.8 (--) Here's a small cleanup I missed. Alternatively one could make heap() return NULL rather than aborting, but since it already used xmalloc, I'm tending to this.. commit cec33eb226df63f406f7eb70cd46d960ee02a060 Author: Pádraig Brady Date: Tue Jul 13 08:23:52 2010 +0100 maint: heap.c: simplify heap_alloc * gl/lib/heap.c (heap_alloc): Use the fact that the xalloc routines will not return NULL. Also remove the redundant temporary variables. diff --git a/gl/lib/heap.c b/gl/lib/heap.c index a37224f..f148434 100644 --- a/gl/lib/heap.c +++ b/gl/lib/heap.c @@ -36,22 +36,12 @@ static void heapify_up (void **, size_t, struct heap * heap_alloc (int (*compare)(const void *, const void *), size_t n_reserve) { - struct heap *heap; - void *xmalloc_ret = xmalloc (sizeof *heap); - heap = (struct heap *) xmalloc_ret; - if (!heap) - return NULL; + struct heap *heap = xmalloc (sizeof *heap); - if (n_reserve <= 0) + if (n_reserve == 0) n_reserve = 1; - xmalloc_ret = xmalloc (n_reserve * sizeof *(heap->array)); - heap->array = (void **) xmalloc_ret; - if (!heap->array) - { - free (heap); - return NULL; - } + heap->array = xmalloc (n_reserve * sizeof *(heap->array)); heap->array[0] = NULL; heap->capacity = n_reserve; @@ -84,8 +74,7 @@ heap_insert (struct heap *heap, void *item) if (heap->capacity - 1 <= heap->count) { size_t new_size = (2 + heap->count) * sizeof *(heap->array); - void *realloc_ret = xrealloc (heap->array, new_size); - heap->array = (void **) realloc_ret; + heap->array = xrealloc (heap->array, new_size); From unknown Mon Jun 23 23:52:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort. Resent-From: Jim Meyering Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Tue, 13 Jul 2010 14:36:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6600 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: =?UTF-8?Q?P=C3=A1draig?= Brady Cc: Gene Auyeung , 6600@debbugs.gnu.org, Paul Eggert , Chen Guo Received: via spool by 6600-submit@debbugs.gnu.org id=B6600.127903172910972 (code B ref 6600); Tue, 13 Jul 2010 14:36:02 +0000 Received: (at 6600) by debbugs.gnu.org; 13 Jul 2010 14:35:29 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OYga0-0002qv-QY for submit@debbugs.gnu.org; Tue, 13 Jul 2010 10:35:29 -0400 Received: from smtp1-g21.free.fr ([212.27.42.1]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OYgZy-0002ql-6V for 6600@debbugs.gnu.org; Tue, 13 Jul 2010 10:35:27 -0400 Received: from mx.meyering.net (unknown [82.230.74.64]) by smtp1-g21.free.fr (Postfix) with ESMTP id 5D78A94012A; Tue, 13 Jul 2010 16:35:26 +0200 (CEST) Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000) id E677EDEBA; Tue, 13 Jul 2010 16:35:24 +0200 (CEST) From: Jim Meyering In-Reply-To: <4C3C2E44.9050009@draigBrady.com> =?UTF-8?Q?("P=C3=A1draig?= Brady"'s message of "Tue, 13 Jul 2010 10:13:40 +0100") References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> <4C37C7D9.2030909@draigBrady.com> <4C37E0E2.3060205@cs.ucla.edu> <4C383D5F.4090703@draigBrady.com> <4C3BBA60.9020201@draigBrady.com> <4C3C2E44.9050009@draigBrady.com> Date: Tue, 13 Jul 2010 16:35:24 +0200 Message-ID: <87hbk34fgz.fsf@meyering.net> Lines: 83 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -3.3 (---) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -3.3 (---) P=C3=A1draig Brady wrote: > Here's a small cleanup I missed. > Alternatively one could make heap() return NULL rather than aborting, > but since it already used xmalloc, I'm tending to this.. > > commit cec33eb226df63f406f7eb70cd46d960ee02a060 > Author: P=C3=A1draig Brady > Date: Tue Jul 13 08:23:52 2010 +0100 > > maint: heap.c: simplify heap_alloc > > * gl/lib/heap.c (heap_alloc): Use the fact that the xalloc > routines will not return NULL. Also remove the redundant > temporary variables. > > diff --git a/gl/lib/heap.c b/gl/lib/heap.c > index a37224f..f148434 100644 > --- a/gl/lib/heap.c > +++ b/gl/lib/heap.c > @@ -36,22 +36,12 @@ static void heapify_up (void **, size_t, > struct heap * > heap_alloc (int (*compare)(const void *, const void *), size_t n_reserve) > { > - struct heap *heap; > - void *xmalloc_ret =3D xmalloc (sizeof *heap); > - heap =3D (struct heap *) xmalloc_ret; > - if (!heap) > - return NULL; > + struct heap *heap =3D xmalloc (sizeof *heap); > > - if (n_reserve <=3D 0) > + if (n_reserve =3D=3D 0) > n_reserve =3D 1; > > - xmalloc_ret =3D xmalloc (n_reserve * sizeof *(heap->array)); > - heap->array =3D (void **) xmalloc_ret; > - if (!heap->array) > - { > - free (heap); > - return NULL; > - } > + heap->array =3D xmalloc (n_reserve * sizeof *(heap->array)); > > heap->array[0] =3D NULL; > heap->capacity =3D n_reserve; > @@ -84,8 +74,7 @@ heap_insert (struct heap *heap, void *item) > if (heap->capacity - 1 <=3D heap->count) > { > size_t new_size =3D (2 + heap->count) * sizeof *(heap->array); > - void *realloc_ret =3D xrealloc (heap->array, new_size); > - heap->array =3D (void **) realloc_ret; > + heap->array =3D xrealloc (heap->array, new_size); Thanks. That looks good. Please push. I noticed that heap_insert's reallocation was awkward and inefficient. Using x2nrealloc rather than xrealloc makes the code cleaner as well as more efficient in the face of a growing heap, and also handles integer overflow. diff --git a/gl/lib/heap.c b/gl/lib/heap.c index a37224f..12a7767 100644 --- a/gl/lib/heap.c +++ b/gl/lib/heap.c @@ -82,15 +82,8 @@ int heap_insert (struct heap *heap, void *item) { if (heap->capacity - 1 <=3D heap->count) - { - size_t new_size =3D (2 + heap->count) * sizeof *(heap->array); - void *realloc_ret =3D xrealloc (heap->array, new_size); - heap->array =3D (void **) realloc_ret; - heap->capacity =3D (2 + heap->count); - - if (!heap->array) - return -1; - } + heap->array =3D x2nrealloc (heap->array, &heap->capacity, + sizeof *(heap->array)); heap->array[++heap->count] =3D item; heapify_up (heap->array, heap->count, heap->compare); From unknown Mon Jun 23 23:52:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort. Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Tue, 13 Jul 2010 15:11:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6600 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: Jim Meyering Cc: Gene Auyeung , 6600@debbugs.gnu.org, Paul Eggert , Chen Guo Received: via spool by 6600-submit@debbugs.gnu.org id=B6600.127903382011938 (code B ref 6600); Tue, 13 Jul 2010 15:11:01 +0000 Received: (at 6600) by debbugs.gnu.org; 13 Jul 2010 15:10:20 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OYh7k-00036V-BH for submit@debbugs.gnu.org; Tue, 13 Jul 2010 11:10:20 -0400 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1OYh7h-00036O-Dk for 6600@debbugs.gnu.org; Tue, 13 Jul 2010 11:10:18 -0400 Received: (qmail 50470 invoked from network); 13 Jul 2010 15:10:22 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 13 Jul 2010 15:10:22 -0000 Message-ID: <4C3C8194.9020008@draigBrady.com> Date: Tue, 13 Jul 2010 16:09:08 +0100 From: =?UTF-8?Q?P=C3=A1draig?= Brady User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> <4C37C7D9.2030909@draigBrady.com> <4C37E0E2.3060205@cs.ucla.edu> <4C383D5F.4090703@draigBrady.com> <4C3BBA60.9020201@draigBrady.com> <4C3C2E44.9050009@draigBrady.com> <87hbk34fgz.fsf@meyering.net> In-Reply-To: <87hbk34fgz.fsf@meyering.net> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.8 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.8 (--) On 13/07/10 15:35, Jim Meyering wrote: > I noticed that heap_insert's reallocation was awkward and inefficient. > Using x2nrealloc rather than xrealloc makes the code > cleaner as well as more efficient in the face of a growing > heap, and also handles integer overflow. > > diff --git a/gl/lib/heap.c b/gl/lib/heap.c > index a37224f..12a7767 100644 > --- a/gl/lib/heap.c > +++ b/gl/lib/heap.c > @@ -82,15 +82,8 @@ int > heap_insert (struct heap *heap, void *item) > { > if (heap->capacity - 1 <= heap->count) > - { > - size_t new_size = (2 + heap->count) * sizeof *(heap->array); > - void *realloc_ret = xrealloc (heap->array, new_size); > - heap->array = (void **) realloc_ret; > - heap->capacity = (2 + heap->count); > - > - if (!heap->array) > - return -1; > - } > + heap->array = x2nrealloc (heap->array, &heap->capacity, > + sizeof *(heap->array)); > > heap->array[++heap->count] = item; > heapify_up (heap->array, heap->count, heap->compare); Much cleaner and increases with n *= 1.5 rather than n += 2 Testing here shows no change in performance. Do you want me to roll that into my patch? cheers, Pádraig. From unknown Mon Jun 23 23:52:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort. Resent-From: Jim Meyering Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Tue, 13 Jul 2010 15:17:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6600 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: =?UTF-8?Q?P=C3=A1draig?= Brady Cc: Gene Auyeung , 6600@debbugs.gnu.org, Paul Eggert , Chen Guo Received: via spool by 6600-submit@debbugs.gnu.org id=B6600.127903418412112 (code B ref 6600); Tue, 13 Jul 2010 15:17:02 +0000 Received: (at 6600) by debbugs.gnu.org; 13 Jul 2010 15:16:24 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OYhDc-00039J-2F for submit@debbugs.gnu.org; Tue, 13 Jul 2010 11:16:24 -0400 Received: from smtp1-g21.free.fr ([212.27.42.1]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OYhDY-00039E-US for 6600@debbugs.gnu.org; Tue, 13 Jul 2010 11:16:22 -0400 Received: from mx.meyering.net (unknown [82.230.74.64]) by smtp1-g21.free.fr (Postfix) with ESMTP id 5A9AC940051; Tue, 13 Jul 2010 17:16:21 +0200 (CEST) Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000) id 2EE87E12C; Tue, 13 Jul 2010 17:16:20 +0200 (CEST) From: Jim Meyering In-Reply-To: <4C3C8194.9020008@draigBrady.com> =?UTF-8?Q?("P=C3=A1draig?= Brady"'s message of "Tue, 13 Jul 2010 16:09:08 +0100") References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> <4C37C7D9.2030909@draigBrady.com> <4C37E0E2.3060205@cs.ucla.edu> <4C383D5F.4090703@draigBrady.com> <4C3BBA60.9020201@draigBrady.com> <4C3C2E44.9050009@draigBrady.com> <87hbk34fgz.fsf@meyering.net> <4C3C8194.9020008@draigBrady.com> Date: Tue, 13 Jul 2010 17:16:20 +0200 Message-ID: <87aapv4dkr.fsf@meyering.net> Lines: 38 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -3.3 (---) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -3.3 (---) P=C3=A1draig Brady wrote: > On 13/07/10 15:35, Jim Meyering wrote: >> I noticed that heap_insert's reallocation was awkward and inefficient. >> Using x2nrealloc rather than xrealloc makes the code >> cleaner as well as more efficient in the face of a growing >> heap, and also handles integer overflow. >> >> diff --git a/gl/lib/heap.c b/gl/lib/heap.c >> index a37224f..12a7767 100644 >> --- a/gl/lib/heap.c >> +++ b/gl/lib/heap.c >> @@ -82,15 +82,8 @@ int >> heap_insert (struct heap *heap, void *item) >> { >> if (heap->capacity - 1 <=3D heap->count) >> - { >> - size_t new_size =3D (2 + heap->count) * sizeof *(heap->array); >> - void *realloc_ret =3D xrealloc (heap->array, new_size); >> - heap->array =3D (void **) realloc_ret; >> - heap->capacity =3D (2 + heap->count); >> - >> - if (!heap->array) >> - return -1; >> - } >> + heap->array =3D x2nrealloc (heap->array, &heap->capacity, >> + sizeof *(heap->array)); >> >> heap->array[++heap->count] =3D item; >> heapify_up (heap->array, heap->count, heap->compare); > > Much cleaner and increases with n *=3D 1.5 rather than n +=3D 2 > Testing here shows no change in performance. Thanks for the perf. testing. > Do you want me to roll that into my patch? Sure, thanks. From unknown Mon Jun 23 23:52:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort. Resent-From: Chen Guo Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Wed, 14 Jul 2010 03:15:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6600 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: Jim Meyering Cc: Gene Auyeung , 6600@debbugs.gnu.org, Paul Eggert , =?UTF-8?Q?P=C3=A1draig?= Brady Received: via spool by 6600-submit@debbugs.gnu.org id=B6600.127907727113514 (code B ref 6600); Wed, 14 Jul 2010 03:15:02 +0000 Received: (at 6600) by debbugs.gnu.org; 14 Jul 2010 03:14:31 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OYsQY-0003Vr-JQ for submit@debbugs.gnu.org; Tue, 13 Jul 2010 23:14:30 -0400 Received: from mail-px0-f172.google.com ([209.85.212.172]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OYsQX-0003VS-2k for 6600@debbugs.gnu.org; Tue, 13 Jul 2010 23:14:29 -0400 Received: by pxi20 with SMTP id 20so2477168pxi.3 for <6600@debbugs.gnu.org>; Tue, 13 Jul 2010 20:14:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=QVLioVOqoNqbfQlvIbykD4xqldO3cuNQyyrqNNnqA68=; b=AbhGZI0O8z6S9ln84eZ/0TzwONiw0zOywmJDXM8+6aJIguFC1VY2oFOJvKXGeN6nuu QWLkz6JDVKtzou9RGGYpuuaAXs+GudwKiEMG7EyK1FhXnLegipFSPpU5l5PXroTeV//Z 2e4Y9vh6+GuTlFyRNc5bk1wTUP1pSX5JSprOA= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=EauTYLKEORJLSdjb0htvbpPHrxXRCfdEJiQrYtYUyVFS8JFQ6M8eT9KtgHWjsXpu6F DZr+qwpCIfxVwWPUS3O9xUUgF8JfBl/EuPReghH9D9c1i2OHRhMLHA1nqAQGFgzDpIhr 6qOcXBRpjK5Ke1Gznm2rd2OdYVOkJIkOefY0U= MIME-Version: 1.0 Received: by 10.142.134.13 with SMTP id h13mr20461638wfd.119.1279077275341; Tue, 13 Jul 2010 20:14:35 -0700 (PDT) Received: by 10.142.241.3 with HTTP; Tue, 13 Jul 2010 20:14:35 -0700 (PDT) In-Reply-To: <87aapv4dkr.fsf@meyering.net> References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> <4C37C7D9.2030909@draigBrady.com> <4C37E0E2.3060205@cs.ucla.edu> <4C383D5F.4090703@draigBrady.com> <4C3BBA60.9020201@draigBrady.com> <4C3C2E44.9050009@draigBrady.com> <87hbk34fgz.fsf@meyering.net> <4C3C8194.9020008@draigBrady.com> <87aapv4dkr.fsf@meyering.net> Date: Tue, 13 Jul 2010 20:14:35 -0700 Message-ID: From: Chen Guo Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.6 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.6 (--) Thanks a lot for all the hard work reviewing and revising this, P=E1draig. From unknown Mon Jun 23 23:52:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort. Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Thu, 15 Jul 2010 00:09:01 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6600 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: 6600@debbugs.gnu.org Received: via spool by 6600-submit@debbugs.gnu.org id=B6600.127915251115475 (code B ref 6600); Thu, 15 Jul 2010 00:09:01 +0000 Received: (at 6600) by debbugs.gnu.org; 15 Jul 2010 00:08:31 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OZC07-00041Y-4d for submit@debbugs.gnu.org; Wed, 14 Jul 2010 20:08:31 -0400 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1OZC04-00041P-Cv for 6600@debbugs.gnu.org; Wed, 14 Jul 2010 20:08:29 -0400 Received: (qmail 76922 invoked from network); 15 Jul 2010 00:08:36 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 15 Jul 2010 00:08:36 -0000 Message-ID: <4C3E5134.3040700@draigBrady.com> Date: Thu, 15 Jul 2010 01:07:16 +0100 From: =?UTF-8?Q?P=C3=A1draig?= Brady User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> <4C37C7D9.2030909@draigBrady.com> <4C37E0E2.3060205@cs.ucla.edu> <4C383D5F.4090703@draigBrady.com> <4C3BBA60.9020201@draigBrady.com> In-Reply-To: <4C3BBA60.9020201@draigBrady.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.7 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.7 (--) On 13/07/10 01:59, Pádraig Brady wrote: > I've finally applied the patch. > http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commit;h=9face836 > > I made a few comment tweaks and added > some dependencies for the heap module. > > I also removed the xmemcoll0() calls > which are separate to this concurrent functionality. > I will add those back in Chen's name after updating to > the latest gnulib. > > Thanks everyone for their work on this! > > Pádraig. > > > > Here's the xmemcoll0 follow up: From: Chen Guo Date: Wed, 14 Jul 2010 07:41:05 +0100 Subject: [PATCH] sort: speed up default full line sorting Don't write NUL after the comparison buffers on each compare, which increases performance by about 3% for short lines on a pentium-m with gcc-4.4.1 * src/sort.c: (fillbuf): Delimit input items with NUL. (write_bytes): Restore the item delimiter char which was replaced with NUL in fillbuf(). --- src/sort.c | 18 +++++++++++++++--- 1 files changed, 15 insertions(+), 3 deletions(-) diff --git a/src/sort.c b/src/sort.c index 5ea1b34..45cb78f 100644 --- a/src/sort.c +++ b/src/sort.c @@ -1743,13 +1743,17 @@ fillbuf (struct buffer *buf, FILE *fp, char const *file) if (buf->buf == ptrlim) return false; if (ptrlim[-1] != eol) - *ptrlim++ = eol; + *ptrlim++ = '\0'; } } /* Find and record each line in the just-read input. */ while ((p = memchr (ptr, eol, ptrlim - ptr))) { + /* Delimit the line with NUL. This eliminates the need to + temporarily replace the last byte with NUL when calling + xmemcoll(), which increases performance. */ + *p = '\0'; ptr = p + 1; line--; line->text = line_start; @@ -2642,7 +2646,13 @@ compare (const struct line *a, const struct line *b, bool show_debug) else if (blen == 0) diff = 1; else if (hard_LC_COLLATE) - diff = xmemcoll (a->text, alen, b->text, blen); + { + /* Note xmemcoll0 is a performance enhancement as + it will not unconditionally write '\0' after the + passed in buffers, which was seen to give around + a 3% increase in performance for short lines. */ + diff = xmemcoll0 (a->text, alen + 1, b->text, blen + 1); + } else if (! (diff = memcmp (a->text, b->text, MIN (alen, blen)))) diff = alen < blen ? -1 : alen != blen; @@ -2652,9 +2662,11 @@ compare (const struct line *a, const struct line *b, bool show_debug) static void write_bytes (struct line const *line, FILE *fp, char const *output_file) { - char const *buf = line->text; + char *buf = line->text; size_t n_bytes = line->length; + *(buf + n_bytes - 1) = eolchar; + /* Convert TABs to '>' and \0 to \n when -z specified. */ if (debug && fp == stdout) { -- 1.6.2.5 From unknown Mon Jun 23 23:52:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort. Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Thu, 15 Jul 2010 11:13:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6600 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: 6600@debbugs.gnu.org, Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 6600-submit@debbugs.gnu.org id=B6600.12791923582419 (code B ref 6600); Thu, 15 Jul 2010 11:13:02 +0000 Received: (at 6600) by debbugs.gnu.org; 15 Jul 2010 11:12:38 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OZMMn-0000cy-01 for submit@debbugs.gnu.org; Thu, 15 Jul 2010 07:12:38 -0400 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1OZMMl-0000cq-Ho for 6600@debbugs.gnu.org; Thu, 15 Jul 2010 07:12:36 -0400 Received: (qmail 65705 invoked from network); 15 Jul 2010 11:12:45 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 15 Jul 2010 11:12:45 -0000 Message-ID: <4C3EECDB.9030907@draigBrady.com> Date: Thu, 15 Jul 2010 12:11:23 +0100 From: =?UTF-8?Q?P=C3=A1draig?= Brady User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> <4C37C7D9.2030909@draigBrady.com> <4C37E0E2.3060205@cs.ucla.edu> <4C383D5F.4090703@draigBrady.com> <4C3BBA60.9020201@draigBrady.com> <4C3E5134.3040700@draigBrady.com> In-Reply-To: <4C3E5134.3040700@draigBrady.com> X-Enigmail-Version: 1.0.1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.7 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.7 (--) On 15/07/10 01:07, Pádraig Brady wrote: > On 13/07/10 01:59, Pádraig Brady wrote: >> I've finally applied the patch. >> http://git.savannah.gnu.org/gitweb/?p=coreutils.git;a=commit;h=9face836 >> >> I made a few comment tweaks and added >> some dependencies for the heap module. >> >> I also removed the xmemcoll0() calls >> which are separate to this concurrent functionality. >> I will add those back in Chen's name after updating to >> the latest gnulib. >> >> Thanks everyone for their work on this! >> >> Pádraig. > > Here's the xmemcoll0 follow up: And a follow up fix to that which fixes 2 test failures noticed on our integration server: http://hydra.nixos.org/build/486508 commit aadc67dfdb47f28bb8d1fa5e0fe0f52e2a8c51bf Author: Pádraig Brady Date: Thu Jul 15 12:06:04 2010 +0100 sort: fix a bug when sorting unterminated lines * src/sort.c (fillbuf): The previous commit incorrectly terminated the buffer when the last line of input didn't contain a terminating character. diff --git a/src/sort.c b/src/sort.c index 45cb78f..7d31878 100644 --- a/src/sort.c +++ b/src/sort.c @@ -1743,7 +1743,7 @@ fillbuf (struct buffer *buf, FILE *fp, char const *file) if (buf->buf == ptrlim) return false; if (ptrlim[-1] != eol) - *ptrlim++ = '\0'; + *ptrlim++ = eol; } } From unknown Mon Jun 23 23:52:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort. Resent-From: Jim Meyering Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Thu, 15 Jul 2010 14:09:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6600 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: =?UTF-8?Q?P=C3=A1draig?= Brady Cc: 6600@debbugs.gnu.org, Ludovic =?UTF-8?Q?Court=C3=A8s?= Received: via spool by 6600-submit@debbugs.gnu.org id=B6600.12792029089539 (code B ref 6600); Thu, 15 Jul 2010 14:09:02 +0000 Received: (at 6600) by debbugs.gnu.org; 15 Jul 2010 14:08:28 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OZP6y-0002To-2k for submit@debbugs.gnu.org; Thu, 15 Jul 2010 10:08:28 -0400 Received: from smtp1-g21.free.fr ([212.27.42.1]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1OZP6u-0002Td-PJ for 6600@debbugs.gnu.org; Thu, 15 Jul 2010 10:08:26 -0400 Received: from mx.meyering.net (unknown [82.230.74.64]) by smtp1-g21.free.fr (Postfix) with ESMTP id A7D4894012E; Thu, 15 Jul 2010 16:08:30 +0200 (CEST) Received: by rho.meyering.net (Acme Bit-Twister, from userid 1000) id 47850E267; Thu, 15 Jul 2010 16:08:29 +0200 (CEST) From: Jim Meyering In-Reply-To: <4C3EECDB.9030907@draigBrady.com> =?UTF-8?Q?("P=C3=A1draig?= Brady"'s message of "Thu, 15 Jul 2010 12:11:23 +0100") References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> <4C37C7D9.2030909@draigBrady.com> <4C37E0E2.3060205@cs.ucla.edu> <4C383D5F.4090703@draigBrady.com> <4C3BBA60.9020201@draigBrady.com> <4C3E5134.3040700@draigBrady.com> <4C3EECDB.9030907@draigBrady.com> Date: Thu, 15 Jul 2010 16:08:29 +0200 Message-ID: <87oce8u9b6.fsf_-_@meyering.net> Lines: 51 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -3.3 (---) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -3.3 (---) P=C3=A1draig Brady wrote: > On 15/07/10 01:07, P=C3=A1draig Brady wrote: >> On 13/07/10 01:59, P=C3=A1draig Brady wrote: >>> I've finally applied the patch. >>> http://git.savannah.gnu.org/gitweb/?p=3Dcoreutils.git;a=3Dcommit;h=3D9f= ace836 >>> >>> I made a few comment tweaks and added >>> some dependencies for the heap module. >>> >>> I also removed the xmemcoll0() calls >>> which are separate to this concurrent functionality. >>> I will add those back in Chen's name after updating to >>> the latest gnulib. >>> >>> Thanks everyone for their work on this! >>> >>> P=C3=A1draig. >> >> Here's the xmemcoll0 follow up: > > And a follow up fix to that which > fixes 2 test failures noticed on our integration server: > http://hydra.nixos.org/build/486508 > > commit aadc67dfdb47f28bb8d1fa5e0fe0f52e2a8c51bf > Author: P=C3=A1draig Brady > Date: Thu Jul 15 12:06:04 2010 +0100 > > sort: fix a bug when sorting unterminated lines > > * src/sort.c (fillbuf): The previous commit incorrectly > terminated the buffer when the last line of input > didn't contain a terminating character. > > diff --git a/src/sort.c b/src/sort.c > index 45cb78f..7d31878 100644 > --- a/src/sort.c > +++ b/src/sort.c > @@ -1743,7 +1743,7 @@ fillbuf (struct buffer *buf, FILE *fp, char const *= file) > if (buf->buf =3D=3D ptrlim) > return false; > if (ptrlim[-1] !=3D eol) > - *ptrlim++ =3D '\0'; > + *ptrlim++ =3D eol; Thanks. FYI, here's a simple demo of that bug: $ printf a |src/sort src/sort: memory exhausted From unknown Mon Jun 23 23:52:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort. Resent-From: Chen Guo Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Wed, 21 Jul 2010 08:14:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6600 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: 6600@debbugs.gnu.org Received: via spool by 6600-submit@debbugs.gnu.org id=B6600.12797000085494 (code B ref 6600); Wed, 21 Jul 2010 08:14:02 +0000 Received: (at 6600) by debbugs.gnu.org; 21 Jul 2010 08:13:28 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1ObUQg-0001QZ-PX for submit@debbugs.gnu.org; Wed, 21 Jul 2010 04:13:27 -0400 Received: from mail-pv0-f172.google.com ([74.125.83.172]) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1ObUQf-0001QU-DG for 6600@debbugs.gnu.org; Wed, 21 Jul 2010 04:13:25 -0400 Received: by pvd12 with SMTP id 12so2590403pvd.3 for <6600@debbugs.gnu.org>; Wed, 21 Jul 2010 01:13:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=74TACI6lPpY2ym3KSrkY7zBWztoghKjmizLn97t+OC8=; b=d5/e/vPaZICl9DnLilPZci1gwT2KxjhGFsTmU7DlifHvXDXgWr4i4s0S0wngyAzik6 5x4fFfsqvzP0H0Jdls+udDQg5DKtohGELn1oEk44AEdWDy8NZRMEd0OVQSgC1TfgIm6G Q0sgsCduV3jh2OtaU5Dk5x21z/w8rtpC5YakU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=wkAggI1WM3VD7/tZLVU8UBY49/OGCw7hi9AV5wvG5AiAK6YidLQSC5G1mXxENTWvUc 0gg/sFnTGwzB5ZGtkci1GzWLywMcMOLUEje3y4Wd5qXAqskF7Ee8srCaIOMYH86TwKS6 Sc8lbX6pb8+6Licp+plmoTHkY7HUtIZcA0z7c= MIME-Version: 1.0 Received: by 10.142.157.8 with SMTP id f8mr11575685wfe.85.1279700030841; Wed, 21 Jul 2010 01:13:50 -0700 (PDT) Received: by 10.142.216.2 with HTTP; Wed, 21 Jul 2010 01:13:50 -0700 (PDT) In-Reply-To: <87oce8u9b6.fsf_-_@meyering.net> References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> <4C37C7D9.2030909@draigBrady.com> <4C37E0E2.3060205@cs.ucla.edu> <4C383D5F.4090703@draigBrady.com> <4C3BBA60.9020201@draigBrady.com> <4C3E5134.3040700@draigBrady.com> <4C3EECDB.9030907@draigBrady.com> <87oce8u9b6.fsf_-_@meyering.net> Date: Wed, 21 Jul 2010 01:13:50 -0700 Message-ID: From: Chen Guo Content-Type: text/plain; charset=ISO-8859-1 X-Spam-Score: -3.0 (---) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -3.3 (---) Hi all, So the gcc compile farm just got a 24 core donation from AMD (2x12 at 1.5 GHz), so I couldn't help but run some results. As a way of isolating sort time versus I/O time, I installed a timer output, such that sort prints to stderr the elapsed time during sort()'s call to sortlines(). There are other people using the machine, so the timings wont be the most accurate, but I figure you guys may still be interested. On 24 threads, the pure sort runs in about 15% of the single threaded time, while with I/O that figure's 17.5%. $ for i in 1 2 4 8 12 16 24; do echo "T=$i: "; /usr/bin/time -f "%e" sort --parallel=$i 1M > /dev/null; echo ""; done T=1: Sort without I/O: 11.074463 11.38 T=2: Sort without I/O: 5.672202 5.96 T=4: Sort without I/O: 3.453562 3.71 T=8: Sort without I/O: 2.351003 2.65 T=12: Sort without I/O: 2.100730 2.39 T=16: Sort without I/O: 1.821079 2.09 T=24: Sort without I/O: 1.674106 1.99 From unknown Mon Jun 23 23:52:43 2025 X-Loop: help-debbugs@gnu.org Subject: bug#6600: [PATCH] sort: add --threads option to parallelize internal sort. Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady Original-Sender: debbugs-submit-bounces@debbugs.gnu.org Resent-To: owner@debbugs.gnu.org Resent-CC: bug-coreutils@gnu.org Resent-Date: Wed, 21 Jul 2010 08:59:02 +0000 Resent-Message-ID: Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: followup 6600 X-GNU-PR-Package: coreutils X-GNU-PR-Keywords: patch To: Chen Guo Cc: 6600@debbugs.gnu.org Received: via spool by 6600-submit@debbugs.gnu.org id=B6600.12797026946652 (code B ref 6600); Wed, 21 Jul 2010 08:59:02 +0000 Received: (at 6600) by debbugs.gnu.org; 21 Jul 2010 08:58:14 +0000 Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.69) (envelope-from ) id 1ObV81-0001jF-0Q for submit@debbugs.gnu.org; Wed, 21 Jul 2010 04:58:13 -0400 Received: from mail1.slb.deg.dub.stisp.net ([84.203.253.98]) by debbugs.gnu.org with smtp (Exim 4.69) (envelope-from ) id 1ObV7y-0001j6-8N for 6600@debbugs.gnu.org; Wed, 21 Jul 2010 04:58:11 -0400 Received: (qmail 93967 invoked from network); 21 Jul 2010 08:58:35 -0000 Received: from unknown (HELO ?192.168.2.25?) (84.203.137.218) by mail1.slb.deg.dub.stisp.net with SMTP; 21 Jul 2010 08:58:35 -0000 Message-ID: <4C46B6A4.4020301@draigBrady.com> Date: Wed, 21 Jul 2010 09:58:12 +0100 From: =?UTF-8?Q?P=C3=A1draig?= Brady User-Agent: Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.8) Gecko/20100227 Thunderbird/3.0.3 MIME-Version: 1.0 References: <362522.89643.qm@web180012.mail.gq1.yahoo.com> <4B94CAE0.6000106@draigBrady.com> <535984.88146.qm@web180006.mail.gq1.yahoo.com> <4C37C7D9.2030909@draigBrady.com> <4C37E0E2.3060205@cs.ucla.edu> <4C383D5F.4090703@draigBrady.com> <4C3BBA60.9020201@draigBrady.com> <4C3E5134.3040700@draigBrady.com> <4C3EECDB.9030907@draigBrady.com> <87oce8u9b6.fsf_-_@meyering.net> In-Reply-To: X-Enigmail-Version: 1.0.1 Content-Type: multipart/mixed; boundary="------------020600060901050801070203" X-Spam-Score: -2.4 (--) X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.11 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: debbugs-submit-bounces@debbugs.gnu.org Errors-To: debbugs-submit-bounces@debbugs.gnu.org X-Spam-Score: -2.4 (--) This is a multi-part message in MIME format. --------------020600060901050801070203 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit On 21/07/10 09:13, Chen Guo wrote: > Hi all, > So the gcc compile farm just got a 24 core donation > from AMD (2x12 at 1.5 GHz), so I couldn't help but run > some results. > As a way of isolating sort time versus I/O time, I installed > a timer output, such that sort prints to stderr the elapsed > time during sort()'s call to sortlines(). > There are other people using the machine, so the > timings wont be the most accurate, but I figure you guys > may still be interested. On 24 threads, the pure sort runs > in about 15% of the single threaded time, while with I/O > that figure's 17.5%. > > $ for i in 1 2 4 8 12 16 24; do echo "T=$i: "; /usr/bin/time -f "%e" > sort --parallel=$i 1M > /dev/null; echo ""; done Cool! I had intended to do this at the weekend on the gcc niagra (32 processor) machine but my DSL died. I also intended to plot the cumulative CPU used, and also with using mutexes rather than spinlocks. Reformatting your results as: 1 11.074463 11.38 2 5.672202 5.96 4 3.453562 3.71 8 2.351003 2.65 12 2.100730 2.39 16 1.821079 2.09 24 1.674106 1.99 And using this gnuplot script: set term pngcairo font 'Sans,10' size 640,480 set output 'sort-amd-24way.png' set title "Multicore sort time on 12x2 AMD" set ylabel "time (seconds)" set xlabel "processors" set xtics (2,4,6,8,12,16,24,32) set xrange [0:32] plot "amd" using 1:2 with lines title "no I/O", \ "amd" using 1:3 with lines title "with I/O" Gives the attached plot. cheers, Pádraig. --------------020600060901050801070203 Content-Type: image/png; name="sort-amd-24way.png" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="sort-amd-24way.png" iVBORw0KGgoAAAANSUhEUgAAAoAAAAHgCAYAAAA10dzkAAAABmJLR0QA/wD/AP+gvaeTAAAg AElEQVR4nOzdeVzVVf7H8ddlExBwg7ugmSJupablBqVZ1tjkNLZppeWomWM5LZZmZWXm2JjL 5LTbtO/aT8s0R80yLRNHLcMyEUHDFNcUIdn5/v5gvGmAAn7v/V7ufT953IfwXT/3jozvzvme c2yGYRiIiIiISMAIsroAEREREfEuBUARERGRAKMAKCIiIhJgFABFREREAowCoIiIiEiAUQAU ERERCTAKgCIiIiIBRgFQREREJMAoAIqIiIgEGAVAERERkQCjACgiIiISYBQARURERAKMAqCI iIhIgFEAFBEREQkwCoAiIiIiAUYBUERERCTAKACKiIiIBBgFQBEREZEAowAoEoBSUlKw2Wzk 5eWZcpzosxKRukUBUKSOOB4wGjVqREFBwUn7MjIyCAoKwmazVdhXk2v//twLLriA7Oxs6tev f0a1+7qq3n9NjvWXz2rUqFEkJycTExNT6fscP34855xzDmFhYcTFxTFy5EhycnKqff3anv/G G29gs9mYOnVqhX01/d04frzNZiMoKIiGDRvSo0cPpk6dqgAvAUMBUKSOiYmJYf78+Sdte+WV VzjrrLNMv1doaChOpxObzWb6tUtKSigrKzP9ulbV4MnPyptKSkoYOHAgkydPrnR/amoqU6ZM Ydu2bSxevJgNGzYwbNiwal+/tue/8cYbdO3alTfffLPKY2r6u5GRkcGePXv473//y9ixY1mw YAHdu3fnl19+qfb7EamzDBGpE9auXWsAxqOPPmpccskl7u0lJSVGfHy8MWnSJAMw8vPz3cfm 5uZWOL+y/UCF14nnHD+urKzMmDVrlpGYmGiEhoYaTqfTmDhxovseOTk5xuDBg4369esbTZo0 MR544AGjtLT0pGstWrTI6NChgxEcHGzs2bPHMAzDePbZZ422bdsaYWFhRrNmzYzJkye7z6vM K6+8YjRv3twIDg424uPjjX/+859nVENV778y1fmsjn//6aefGp07dzbCw8ON5ORkIzMz0/j6 66+NCy64wKhXr57Ro0cPIzMz86Tr1/SzqM77XbFihXHeeecZYWFhRs+ePY2MjIwqr3fciX9f TmXJkiVGSEiIUVpaauzevdto1KiR8frrr7v3z54923C5XMahQ4dOe35Vdu7caYSEhBjp6elG gwYNjLVr11Zaa3V+N048/sTfD8MwjPz8fOPcc881xo4de8r3LOIPFABF6ojj/2hlZGQYYWFh 7uCwcOFCo3nz5saaNWtqHQA/+eQTAzB27txpZGdnG9nZ2Sedc/y4Rx991IiNjTXmzZtnZGVl GRs2bDBeeeUV9z0GDx5sdOzY0di0aZOxcuVKIzY21pg1a9ZJ10pKSjLWrVtnpKWlGfn5+cbU qVONc88911i2bJmxa9cuY8WKFUarVq2Mxx9/vNLPYc+ePUZwcLAxc+ZM46effjLWr19vfPzx x2dUQ1XvvzLV+ayOf5+cnGx8/fXXRlpamtGnTx+jW7duxsUXX2ysXbvW2LZtm9GnTx+jf//+ 7mvX9LOo7vvt3bu3sW7dOmPr1q1Gr169jCuuuKLK6x1X3QD4zjvvGE2aNHH//PbbbxsNGjQw du3aZWzdutWIiIgwFi9eXO3zKzNlyhR3zbfddpsxevToSmutzu/Gicf/PgAahmE8//zzRsuW LU9Zj4g/UAAUqSNO/Efr2muvNR5++GHDMAzjqquuMiZNmnTKgHfi+ZXtr+of+xOPy8/PNyIj I4333nuv0vqOHj1qhISEGMuWLXNvmz59utG2bduTrvX555+79xcWFhrR0dHGxo0bT7rWggUL DJfLVel9Nm/ebABGWlqaKTWc6v1Xpjqf1fHvv/rqK/f+jz/+2ACMdevWnbQtIiKi1p9Fdd/v 119/7d6/cOFCIzw8vNbv80S5ublGmzZtjEmTJp20/brrrjMuv/xyo0ePHsaIESNqfP7vtW7d 2pg7d65hGIaxZs0ao1GjRkZBQUGFWqvzu/H7439v5cqVRnBwsFFWVnbKmkTqOj0DKFIHjRw5 ktdff52ff/6ZZcuWMWLECI/fc9u2bRw7doxLL7200v0ZGRmUlJRw/vnnu7d17dqV9PT0k56z 69y5s/v7tLQ0cnNz6d27N1FRUe7XkCFDyM7Opri4uMJ92rdvT8+ePenZsyfDhg1jwYIF7uvX pgZP6tixo/v7uLg4ADp06HDStvz8fAoLC2v1WVT3/Z577rnu7+12OwUFBRQWFp7ReysuLmbQ oEE0a9aMiRMnnrTvhRdeYP369ezevZunnnqqxuefaO3atRw8eJABAwYAkJycTFxcHB9//HGl x5/p74ZhGDU6XqSuCrG6ABGpuX79+gEwdOhQ+vTpQ/PmzdmzZ497f2UDEawecHFcaGio+/vj Na1cuZJGjRpVODYkpOL/RQUHB/Pll1+yfPlyli1bxqhRo/jggw947733alWDJwUFVfxv7Mq2 GYZRq8/iTOo4k6BTWlrK4MGDOXToEJ999lmFz/OHH34gLy+PkpISDh06RExMTI3OP9Ebb7zB 4cOHTxpdXVZWxptvvsnAgQMrHH+6343T+fHHHzn77LPr/GAekdNRC6BIHRQUFMTw4cNZuXIl I0eOrLD/+D+4Bw4ccG/bvHlzldcLDg4GTh0S27RpQ2RkJJ9//nml+1u1akVISAjffPONe9uG DRto06ZNpQEEoG3btkRFRZGVlUViYmKFV1X/CIeEhHDllVfyr3/9i9dee4158+ZRVlZWqxqq +/5rc2xN1OazqO37PRNlZWUMGzaMrVu38p///IeoqKiT9ufl5TF8+HAmTZrEn//8Z4YPH35S 2Dzd+ScqLCxk7ty5vP7662zatMn9WrZsGUuXLmX//v0Vzjnd78apFBQU8Pzzz3P11VfX6DyR ukgtgCJ11COPPMJdd91F48aNK+xr06YNLpeLxx57jMmTJ5OWlsasWbOqvFbz5s2x2WwsXLiQ yy67DJvNRmxs7EnHhIeHM3bsWO666y5CQkLo0aMH+/fv57vvvmPEiBFER0czaNAgxo8fj9Pp 5PDhw0yfPp0HH3ywyvuGh4fz8MMPM3r0aPLz8+nVqxfFxcVs3LiR9PR0Hn744QrnbNiwga+/ /po//OEPhIaGsnDhQs4++2yCgoJqVUN1339tjq2J2nwWtX2/p7Jp0yYA0tPTgfJpW8LCwtzd 5qNHj2bFihUsWbKEoqIi9u7dC4DT6QTgvvvuIzY2lgceeICjR4/SoUMHnn76ae6+++5qnX+i hQsXAjB48OCTWgk7dOhAmzZteOeddxg7dmyF8071u3Gi/fv3k5eXR25uLhs3bmTGjBmUlZWd sktaxG9Y+wiiiFTXqR5cP3H/8QfdV65cabRp08YICwszevfubbz77runHCQyffp0Iz4+3ggK CjrlNDBPPvmk0bJlSyMkJMRwuVzuB+4No3pTklRW/yuvvOKepqR+/fpG9+7dTxpdfKKtW7ca l156qREdHe2e1mTDhg1nXENl778qp/usTjcI51TbavJZ1Ob9nm5wB5VMc3Pi51HV/vz8fGPZ smVGeHi48cMPP7iPX7x4sREZGeketHOq83/vyiuvNIYMGVJpnRMnTjTOO++8Kt/niaoaBAIY NpvNiImJMbp162ZMmTKlymuI+BubYeiJVxEREZFAomcARURERAKMAqCIiIhIgFEAFBEREQkw CoAiIiIiAUYBUERERCTAKACKiIiIBBgFQBEREZEAowAoIiIiEmAUAEVEREQCjAKgiIiISIBR ABQREREJMAqAIiIiIgFGAVBEREQkwCgAioiIiAQYBUARERGRAFOnA+CoUaNITk4mJiYGm81G QUHBSfvHjx/POeecQ1hYGHFxcYwcOZKcnByLqhURERHxDXU6AJaUlDBw4EAmT55c6f7U1FSm TJnCtm3bWLx4MRs2bGDYsGHeLVJERETEx9gMwzCsLuJMpaSkkJSURH5+PuHh4VUe95///Ic/ //nPFBYWEhRUp7OviIiISK0FVAo6fPgwDRo0UPgTERGRgBYwSSgvL4/Jkyfzt7/9zepSRERE RCwVYnUB3lBcXMygQYNo1qwZEydOrNY5NpvNw1WJiIiIVJ+ZT+35fQAsLS1l8ODBHDp0iM8+ +4zQ0NBqn+sLj0fabDbV8Tu+Uouv1OFL9JlUpM9ETkd/R6Q6zG6Y8usAWFZWxrBhw9i6dSur Vq0iKirK6pJERERELFenA+CmTZsASE9PB8qnfQkLC6Nz584AjB49mhUrVrBkyRKKiorYu3cv AE6n05qCRURERHxAnZ4Gpqrm0ONvqar9p5su5vi5vvDRqI6KfKUWX6nDl+gzqUifiZyO/o5I dZj996ROtwCe7oPQL5SIiIhIRQEzDYyIiIiIlFMAFBEREQkwdfoZQE/SMxkiIiLiK8zOJWoB FBEREQkwCoAiIiLiUSkpKdhsNvLy8k7a3rt3b+bNmwdAVlYWw4cPx+VyERYWRkJCAhMmTCA3 N9eKkv2eAqCIiIh43eHDh1m/fj39+vUjMzOTrl27kpOTw4IFC9i+fTtz5szhiy++oE+fPhw7 dszqcv2OAqCIiIi4HW+t++yzz+jcuTP16tUjKSmJzMxM9zFHjx5lyJAhREVFERsby4MPPkhZ WVmN7rN06VJ69uxJgwYNuOeee+jatSvz588nKSmJ5s2bc/nll7Ny5Upyc3OZPn262W8z4CkA ioiISAWPP/44L730EqmpqYSGhjJmzBj3vttvv53NmzezZs0a/u///o+XX36Z2bNn1+j6n3zy Cf379yc3N5clS5Ywbty4Cgs4REZGMmbMGN5//31T3pP8RgFQREREKpg2bRrdu3enbdu2jBs3 ji+++AKA3Nxc5s2bx8yZMznvvPPo06cP999/Py+99FK1r11aWsrSpUvp378/mZmZlJaW0qlT p0qP7dSpE9u3b9fMHCZTABQREZEKzj33XPf3drudgoICCgsLycjIoKSkhPPPP9+9v2vXrqSn p1e7GzglJYWYmBjat29vXsE2m++/fIgCoIiIiFQQFFQxIpjVCrd48WL69+8PQEJCAsHBwaSm plZ6bGpqKomJiRW6hyspzvdfPkQBUERERKqtVatWhISE8M0337i3bdiwgTZt2lQaGitz/Pk/ gOjoaK644gpmzpxZIWAeO3aM5557jhtuuMG8NyCAAqCIiIjUQHR0NIMGDWL8+PGkpqayatUq pk+fzm233Vat87OyssjMzOSSSy5xb5s9ezbr16/nuuuuIyUlhV27drFixQouvfRSoqKimDBh gqfeTsBSABQREZEaeeGFF+jQoQPJyclcd911jBw5knvuuada537yySf07duXevXqubclJiay fv16oqOjGTBgAK1atWLkyJH07t2bVatWERkZ6am3ErC0FnAVtBawiIiI+fr378+AAQMYNWqU 1aXUKVoLuI7YxCZ60cvqMkRERHzKxRdfzIABA6wuI+CpBbAKZ5q0f+ZnkkhiF7tMrEpEREQC kdktgAqAVTjTD7qIIqKJppBCE6sSERGRQKQu4DoijDBiiOEAB6wuRUREROQkCoAeZMfOfvZb XYaIiIjISRQAPciFi73stboMERGR00pJScFms5GXl2fKceLbFAA9yIFDAVBEROqECy64gOzs bOrXrw/8FvQKCgrO+NpVhcbevXszb948oHyC6OHDh+NyuQgLCyMhIYEJEyaQm5t7xveXihQA PciJk33ss7oMERGR0woNDcXpdJ5+zV2THD58mPXr19OvXz8yMzPp2rUrOTk5LFiwgO3btzNn zhy++OIL+vTpw7Fjx7xSUyCp0wFw1KhRJCcnExMTU+l/pZxuv6e5cJFNtlfvKSIictyHH35I QkKC++c33ngDm83G8uXLASguLqZ+/fqsXbu2QitdUlISABEREdhstgrBcN26dXTu3Jl69eqR lJREZmZmjWpbunQpPXv2pEGDBtxzzz107dqV+fPnk5SURPPmzbn88stZuXIlubm5TJ8+/Uw+ BqlEnQ6AJSUlDBw4kMmTJ9dqv6dpEIiIiFjpoosuYseOHezevRuAL7/8kiZNmvDll18C8M03 32AYBhdccEGFcz/55BMAdu7cSXZ2NtnZJzdoPP7447z00kukpqYSGhrKmDFjalTbJ598Qv/+ /cnNzWXJkiWMGzeuQsiMjIxkzJgxvP/++zW6tpxeiNUFnIlXX30VKH+2oDb7PSorC9fGFLKv UQugiIhYIy4ujnbt2rF69WpuuukmVq9ezd13382KFSuA8kDYo0cPwsLCKpzbuHFjABwOB+Hh 4RX2T5s2je7duwMwbtw4brjhhmrXVVpaytKlS5k4cSKZmZmUlpbSqVOnSo/t1KkT27dvxzAM r3VPB4I63QLo0woKcLy2RM8AioiIpXr16sWXX37Jvn372LdvH3fccQcbNmygqKiIL7/8kl69 ards6bnnnuv+3m63U1BQQGFh9RY/SElJISYmhvbt29fq3pWx1YEvX6IA6ClOJ84fDmkUsIiI WKpXr16sXr2a1atXk5ycTJMmTUhMTGTdunWsWbOm1gEwKKhihKjuShWLFy+mf//+ACQkJBAc HExqamqlx6amppKYmHja1j+jDnz5EgXAUzj+0OvvX9USE0PsvlKOcIQSSjxbqIiISBV69erF li1b+Oijj+jdu7d725w5czhy5Ih7sMfvBQcHA1BWVmZ6Tcef/wOIjo7miiuuYObMmRUC5LFj x3juuedq1L1cl1WVOzzR9a0AeAqGYVT6qq4gZzyNSxtwkIMerFJERKRqLVq0oGnTpsydO9fd 2terVy/ef/99unTpQlRUVKXnNW/eHJvNxsKFCzlw4AAHD5rzb1lWVhaZmZlccskl7m2zZ89m /fr1XHfddaSkpLBr1y5WrFjBpZdeSlRUFBMmTDDl3r6uqtxh5hrAxykAepLdTnxBY/awx+pK REQkgPXq1YvQ0FD3oI1evXpRWlp6yu5fh8PBk08+ybhx43A6ncTFxZlSyyeffELfvn2pV6+e e1tiYiLr168nOjqaAQMG0KpVK0aOHEnv3r1ZtWoVkZGRptxbfmMzPBErvWTTpk0AbN68maFD h7Ju3TrCwsLo3Llztfafis1mO/PEfd11/PH5HdzlmMof+eOZXUtERMQP9O/fnwEDBjBq1Cir S6lTTMklJ6jT08B06dLlpJ979OgB/PYQ6un2e5zDgetwNnsdGggiIiICcPHFFzNgwACrywh4 dToAni7IWd646XDgOBDE3nYKgCIiIgD333+/1SUIegbQs+LjcWSXaS5AERER8SkKgJ5ktxP/ U7HWAxYRERGfogDoSS4X9sw8rQcsIiIiPqVOPwPo8xwOXGlHyfax2b9FREQksCkAepLTiWPL IfZxzOpKRERERNzUBexJ9erRsCiSAiOfAgqsrkZEREQEUAD0PIcDZ0kse9FUMCIiIuIbFAA9 LT4ee360BoKIiIiIz1AA9DS7nfijUVoPWERERHyGAqCnOZ3YD4eqBVBERER8hgKgpzmduPYF aTJoERER8RkKgJ7mcODYU6pBICIiIuIzFAA9LT4ex858rQcsIiIiPkMB0NPsdlwZx9QFLCIi Ij5DK4F4msuFY+th9lNqdSUiIiIiANgMw9BCtZWw2WyY8tGUlpLfJJImR4I5piXhREREpBZM yyX/oy5gTwsOJiK8EaFGCEc5anU1IiIiIgqAXhEfj7OosUYCi4iIiE9QAPQGux3nsRgFQBER EfEJCoDe4HJhz6mn1UBERETEJygAeoPDQfyheloPWERERHyCAqA3OJ3Y9xlqARQRERGfoADo DU4nrl0lmgxaREREfEKdDoCjRo0iOTmZmJgYbDYbBQUFJ+0vKipi9OjRxMTE4HA4mDZtmjWF Ohw4firQIBARERHxCXV6JZCSkhIGDhzIwIEDuffeeyvsf/zxx1mxYgWrVq3i4MGDXH/99SQm JnL99dd7t9D4eBzbcthHqHfvKyIiIlIJv1gJJCUlhaSkJPLz8wkPDwfAMAycTiczZsxg6NCh ANx///1s2rSJ5cuXn/aaps64ffgwuy9qSfcf6rOb3eZcU0RERAKGVgKppl27drF//36Sk5Pd 25KSkvjmm2+8X0yjRtizCjjIQQzqfN4WERGROs5vA+CBAwcAiIuLc2+Li4vj0KFDlJWVeb2e 0CZOGpRGcZCDXr+3iIiIyIn8NgCawWazVfqqFYcDR1Ej9rHP3CJFRETEL1SVO2qdPU7BbwPg 8Za/4y2Bx79v3LgxQUHVe9uGYVT6qpX4eJx5URoJLCIiIpWqKnd4YriG3wbAs846C7vdTkpK intbSkoKF1xwgTUF2e04j4QrAIqIiIjl6vQ0MJs2bQIgPT0dgNTUVMLCwujcuTM2m43bbruN xx9/nHPPPZdffvmFOXPm8PLLL1tTrMuF/eBm9rfWaiAiIiJirTodALt06XLSzz169ABwN5U+ +uijHDhwgN69exMREcGECRO8PwfgcQ4H8dk2rQcsIiIilqvTAfB0feJhYWHMmTOHOXPmeKmi U3A4sO8q5DutBywiIiIW89tnAH1OfDyujGNaD1hEREQspwDoLXY7ju25mgZGRERELFenu4Dr FJcLxw8H2Uuh1ZWIiIhIgFMLoLdERBCXE8YRjlBMsdXViIiISABTAPQim8NJbElDDnDg9AeL iIiIeIgCoDc5ncTnN9JAEBEREbGUAqA3ORw48uprIIiIiIhYSgHQm+Ljcf4SpuXgRERExFIK gN5kt+M8EKwAKCIiIpZSAPQmpxN7din7tRqIiIiIWEgB0JucTuJ3Fmk9YBEREbGUAqA3ORzY d/yqFkARERGxlFYC8ab4eFxbDmsSGBEREbGUAqA3xcXhSDvCPoqsrkREREQCmLqAvSkkhAZB jSg0CiigwOpqREREJEApAHqbw4GrOFYDQURERMQyCoDe5nBgz4/WQBARERGxjAKgt8XHE59T X+sBi4iIiGUUAL3Nbsd+OFTrAYuIiIhlFAC9zeXCtdem5eBERETEMgqA3uZw4NxdqgAoIiIi llEA9DanE3tWgQaBiIiIiGUUAL3N6SQ+LVfTwIiIiIhl/DoAbt++ncsuu4zw8HCaNWvG008/ bXVJ5dPAbDuiFkARERGxjN8uBWcYBgMGDKBVq1Zs2rSJX3/9lZEjR5KQkMCf/vQn6wqLjcW1 /VeyybeuBhEREQloNsMwDKuL8IStW7fSvn17MjMzadmyJQCLFi3ihRdeYMmSJac932az4bGP 5qyzaPhTDj8F7aIBDTxzDxEREfEbZucSv+0CLiwsBCA8PNy9LTw8nPXr11tV0m+cThxFjTQX oIiIiFjCbwNg+/btiYuL4/HHH6egoIAjR44wY8YMDh48SFlZmbXFOZ24fo3RQBARERGxhN8G wLCwMN577z0WLlxIdHQ0CQkJdOzYEZvNVu1r2Gy2Sl9nzOHAfjRcA0FERETErarcYUr2+B2/ DYAAffv25eeffyYzM5M9e/bQuXNnmjVrRlBQ9d62YRiVvs6Yy0X8gVCtBywiIiJuVeUOT4xJ 8NtRwMcFBQVx1llnAbBw4UL69etncUWUtwDuM/QMoIiIiFjCrwPgokWLcLlcxMbG8vbbb/Pp p5+yadMmq8sqXw84q5gvtRyciIiIWMCvA2BmZiYjRozg6NGjdO3alRUrVrinhLGUw4EzpUDr AYuIiIgl/HYewDPl0XkA09PZeH9f/vqhnQ1s8Mw9RERExG+YnUv8ugXQZ7lcxG8+xB5KrK5E REREApBaAKvg0RZAoKRhFJFHiiikEBvmD+8WERER/6GVQPxESJyLRqUxHOCA1aWIiIhIgFEA tIrDgaOwoaaCEREREa9TALSK04kjr74CoIiIiHidAqBVnE5ch8O1GoiIiIh4nQKgVex27AeD 1AIoIiIiXqcAaJWmTYnfbagFUERERLxOAdAqdjv2n4vUAigiIiJepwBoFacTZ+YxLQcnIiIi XqeVQKzidOL68Qh79T+BiIiIeJnSh1UcDuxph9lPmdWViIiISIBRF7BVwsKIK4jmCEcoptjq akRERCSAKABayeHAXtKY/ey3uhIREREJIAqAVnK5cB1roKlgRERExKsUAK3kcGg5OBEREfE6 BUArORw4fglVABQRERGvUgC0ksuFa1+QuoBFRETEqxQAreRwYM8u1SAQERER8SoFQCu5XMTv KGQPe6yuRERERAKIAqCVHA7sO37VM4AiIiLiVVoJxEpOJ86tR9irHC4iIiJepABopdhYnNuO spciqysRERGRAOLXTU9r1qwhOTmZ8PBwoqOj6d+/P5mZmVaX9ZugIGJCm1BiFJNPvtXViIiI SIDw2wB49OhRrrzySjp06EBqaipfffUVZWVlXHvttVaXdrKmTYkvitVAEBEREfEav+0CzsjI 4OjRozz++OM4nU4AJkyYwKWXXkpZWRlBQT6Sfe127PnH2F9vP61oZXU1IiIiEgB8JAWZ75xz zqFVq1a8+uqrFBcXk5eXx5tvvslVV13lO+EPwOnElROpyaBFRETEa3woCZmrXr16fPnll8yd O5eIiAhiYmLYt28f8+bNs7q0kzmdOA6FaCoYERER8Rq/DYAAI0eOpFmzZvz3v/9l9erV/Pzz z0yePLna59tstkpfpnI4cOy3KQCKiIgEuKpyh+nZAz9+BjAtLY0lS5awY8cOWrRoAcDUqVO5 4YYbmDp1arU+TMMwPFwl5esBZxWzUV3AIiIiAe1UucPsEOi3LYBFReVz6wUHB7u3BQcHU1xc bFVJlXM4sGcVaD1gERER8Rq/DYDt27enWbNmjB07lm3btrFp0yYmTZrE5Zdf7pGm1FpzuYhP y9U0MCIiIuI1fhsAQ0JCWLRoEfv376dTp0706tWLs846i1dffdXq0k7mcGBPO6xnAEVERMRr bIZXHnSre2w2m3eeAQQKGkXQ8LBBAQVeuZ+IiIjULWbnEr9tAaxLwhu5iNv8cLEAACAASURB VCirx2EOW12KiIiIBIAzDoD5+fm8++673HTTTbRq1YrIyEgiIyNJTExk8ODBvP/+++Tna53b U3I4cBQ21EAQERER8YozCoDTpk2jRYsWfPDBB1x11VV8+umn5OTkkJOTw/Lly+nfvz/vv/8+ LVq0YNq0aWbV7H9cLuLzYjQQRERERLzijOYBXLt2LV9++SVt2rSpsC8hIYGEhASGDBlCeno6 48ePP5Nb+TeHA/vRXeyPUwugiIiIeN4ZBcCFCxdW67jWrVvz0Ucfncmt/JvTietACNmtNBm0 iIiIeJ4GgfgCpxNHdpmmghERERGvMDUA/vvf/2bkyJEALFq0CJfLRVxcHMuWLTPzNv7Hbsex u0QBUERERLzC1LWAp06dyurVqwG4++67efXVVykuLmbcuHH069fPzFv5l6ZNca36lWytBywi IiJeYGoA3L17Nw6Hg127dpGbm8sf//hHCgsLuf766828jf+x27Fn5GoaGBEREfEKUwNg69at Wbx4Mbt37yY5ORmAjIwMWrVqZeZt/I/TSfzmQ+yhzOpKREREJACYGgCnT5/OsGHDiIyMZP78 +QBMmTKFSZMmmXkb/xMeTuyvEfzCL5RRRpDG5oiIiIgHaS3gKnhzLWAA2rXD8cNBUoN/wIHD e/cVERERn6e1gP2Vy4UzvwF72Wt1JSIiIuLnzqgL2GazVftYNTSehsOBI+8Q+6M0EEREREQ8 64xaAA3DOOk1ffp0rrzySrZt20ZxcTFpaWlcccUVTJ8+3ax6/ZfLhetwuNYDFhEREY8z9RlA h8PB5s2bsdvt7m379u3jvPPOY+/eutW16fVnAKdN4/7kj4jrfR3j0brJIiIi8huffgYwNze3 QrewzWYjNzfXzNv4J4cD1+4yTQYtIiIiHmdqABw4cCC33nor27dvp7S0lO3btzNixAgGDRpk 5m38k8uFI6tQy8GJiIiIx5kaAOfMmUPbtm3p06cP9erVo0+fPpxzzjm8+OKLZt7GPzkcOHYc UwAUERERj9M8gFXw+jOAe/ey5YaODFxl5wd+8N59RURExOeZnUtMXQlEzkBcHPb0HK0GLCIi Ih5nahfwt99+S7du3QgNDcVms530ktMIDia2rDFHjRyKKLK6GhEREfFjpgbAESNGcNVVV5GT k1NhjkCpBqcTe0lj9qsdUERERDzI1C7g77//njVr1hAZGWnmZWutspbHiIgIjh07ZkE11WC3 48zPZ2/oXprRzOpqRERExE+ZGgCbNm3KkSNHfCYAZmefPKferbfeSnR0tEXVVEPTpjhz9pMd o7kARURExHNM7QKeMGEC9957L7/88ouZl601p9PpfkVGRvL5559z8803W11W1ex2HIfD1AUs IiIiHmVqC+Add9wBwNy5cyvss/o5wPnz5xMVFcUVV1xhaR2n5HTi2hek9YBFRETEo0wNgFaH vFN55513uOGGGwgJ8eGZb5xO7LuL2aYWQBEREfEgU7uAfdWePXtYuXIlQ4YMqdF5v5/KxuNT 2jgcxP9UrPWARUREAlBVucMT2cPUAJifn899991H06ZNCQ0NpVmzZowfP56CggIzb1Nj7777 Li1btiQpKalG5/1+KhuPT2njcuFIP6rl4ERERAJQVbnDE9nD1AD4t7/9jS1btrBy5Ury8/P5 /PPP2bx5M3fddZeZt6mxd955p8atf5ZwOHCkHVEAFBEREY8ydS3gmJgYtm/fjt1ud2/bt28f bdq0IScnx6zb1MiWLVs499xz2bZtG61bt672eV5fC/h/8mLDcR0MJZdcr99bREREfJPZucTj zwBavQzc22+/Tffu3WsU/qwUVd9BmVHKMXx0smoRERGp80wNgAMHDmTEiBFs376d0tJS0tPT GT58OIMGDTLzNjXyxBNPsG7dOsvuX2MOB/GFTTQVjIiIiHiMqQHw2WefpU2bNlx88cXUq1eP Pn360L59e5555hkzb+PfXC7sx6I0GbSIiIh4jKnPAPoTq54BZNQorntoI0NaTORarvX+/UVE RMTn1LlnAKWGXC6cB0M0F6CIiIh4jKkBcNasWYwdO/akbWPHjuWpp54y8zb+zeHAsQ91AYuI iIjHmNoF3KhRI3bs2EHDhg3d244cOUJCQgK//PKLWbfxCsu6gBcs4N/7p/Lf0efzb/7t/fuL iIiIz/HpLuDg4GDy8vJO2paXl0dQkHqaq83pxL7zmFoARURExGNMTWZDhw7l1ltvZdu2bZSU lLBt2zZuvfVW/vKXv5h5G/9mtxO/LU/PAIqIiIjHmBoAn3zySbp27Urfvn0JDw+nb9++dO/e nSeffNLM2/i3pk1x/HBQy8GJiIiIx2gamCpY9gwgUBgXQ4MDRRRQYMn9RURExLf49DOAYo56 jZzULw3nF+rWwBkRERGpG0wNgLm5udxxxx04HA73wI8bb7yRefPmmXkb/+d0Yi9soIEgIiIi 4hGmBsCRI0eSn59Pamqqu5ly4sSJPPHEE2bexv85HMTnxWg9YBEREfGIEDMvtmjRIvbt20d0 dLR7W7t27diyZYuZt/F/Lhf2I5nst6sFUERERMxnagtg/fr1yc3NPWnbzp07adq0qZm38X8O B86DIexlr9WViIiIiB8yNQAOHz6cESNGkJaWBsDu3bu58847uf322828jf9zuXDuKVMAFBER EY8wNQA+8cQTJCUl0a9fP0JCQkhOTubiiy9m/PjxZt7G/zkcOH4u1lyAIiIi4hGaB7AKVs4D yIYNLHtzCP98ugXLWGZNDSIiIuIzzM4lpg4CEZO4XNjTDrOfSKsrERERET9kahfwv//9b0aO HAmUjwh2uVzExcWxbJlasWrEbsf14xGtBywiIiIeYWoXcIsWLVi9ejXNmzcnISGB5557juLi YiZOnMjmzZvNuo1XWNoFDJS67ERm53CMYwQTbFkdIiIiYj2zc4mpATA0NJS8vDz279/P+eef z4EDBygsLCQ6OpqioiKzbuMVVgdAOnbE+W0234X8gAOHdXWIiIiI5Xz6GcDWrVuzePFidu/e TXJyMgAZGRm0atXKzNsEBqcTV34+2dHZCoAiIiJiKlMD4PTp0xk2bBiRkZHMnz8fgClTpjBp 0iQzbxMYnE7sefvYH63VQERERMRcpg4C+dOf/sTBgwfJysqiW7duALz33nvceOONZt6mRhYs WECHDh0ICwujefPmLF682LJaasRuJ/6XcK0HLCIiIqbz62lgli9fzpgxY3j++efp2rUre/bs ITKyjkyt0rQp9v2w/1y1AIqIiIi5zqgF8JprriEjI+O0x6Wnp3P11Vefya1qZeLEicyYMYNr rrmGs846ix49etCxY0ev11ErdruWgxMRERGPOKMA2K1bN5KSkhg4cCDvvPMOGRkZHDt2jOLi YjIzM3nnnXe4+uqrufDCC+nRo4dZNVfLoUOH2LBhA4WFhZxzzjk4HA5Gjx5NQUGBV+uoNacT 50+FCoAiIiJiujPqAn7ooYe4++67mT9/PgsXLuSRRx4hOzsbm82G0+mka9euXH/99bz77rte 73rNysoC4I033uC1114jNzeXESNGEB4ezuzZs71aS604nTgyf9V6wCIiImI6v10L+JtvvuGC Cy5g/fr1dO3aFYDXX3+d8ePHc+DAgdOeb7PZqtznlY/swAF+vKYd133lYAtbPH8/ERERsdSp sgeYmz/8dhCI3W4HyucmPC4xMZGDBw9SWlpKcPDpV9ewNBvHxmLPyFX7n4iISIA4Ve44XTis KVOngfElzZo1w+VykZmZ6d62Y8cO4uLiqhX+LGez0STYTolRrBZAERERMZXfBkCA22+/nbvv vpstW7awbt06Jk+ezF/+8hery6o+l4vxe4YwmzrwzKKIiIjUGX4dAB988EHOP/98LrzwQv7w hz/Qt29f/v73v1tdVvU5HIze0pv5zOcAp39uUURERKQ6/HYQyJkye9HlWrn1VrjwQkaNSMGB gylMsbYeERERsYTZucTUFsDc3FzuuOMOHA4HQUHll77xxhuZN2+embcJHC4X7NnDeMbzEi+R T77VFYmIiIgfMDUAjhw5kvz8fFJTU90pdeLEiTzxxBNm3iZwOBywfz+taU0SSbzN21ZXJCIi In7A1C7gyMhI9u3bR3R0tLupsri4mPr161NUVGTWbbzCJ7qA/+//YO5c+OADvuALxjCG7/ke G+YOBRcRERHf5tNdwPXr1yc3N/ekbTt37qRp06Zm3iZwOBywr3wmwD70oR71+IRPLC5KRERE 6jpTA+Dw4cMZMWIEaWlpAOzevZs777yT22+/3czbBA6nE/b+thbwOMYxi1kWFiQiIiL+wNQA +MQTT5CUlES/fv0ICQkhOTmZiy++mPHjx5t5m8DhdMLOne4fBzGITDL5lm+tq0lERETqPE0D UwWfeAYQoF07ePFF6NMHgJnMZDObeYM3rK1LREREvMbsXKIAWAWfCYD//jd8+CEsWQLAEY7Q ilakkkpT9GyliIhIIPDpQSDfffcdPXv2JDQ0FJvNdtJLaunmm+G772BL+XrADWnIELQ8nIiI iNSeqQFw6NChXH311eTk5GAYxkkvqaWICLj9dpj12+CPcYzjdV4njzwLCxMREZG6ytQu4NDQ UI4ePUpERIRZl7SMz3QBAxw8CG3bwvffl68OAtzADVzERdzJnRYXJyIiIp7m013AZ599NocO HTLzkgIQGwtDhsCzz7o3jWUsT/M0ZZRZWJiIiIjURaYGwIkTJ3LvvfcqBHrCXXfBnDmQV97t 25OexBLLfOZbXJiIiIjUNaYGwNatW7N06VJiY2M1CMRsiYnQty+89pp70/3czz/5p4VFiYiI SF1kagAcPXo0Dz30EL/++qsGgXjC2LHlg0FKSgAYwAAOcpAUUiwuTEREROoSUweBhIWFkZOT o0EgntSnD9xxBwwaBMAzPMNXfMVc5lpbl4iIiHiMTw8CiY+P1/N/njZ2LEyf7v5xOMP5nM/J JNPCokRERKQuMTUA3n///YwdO1Yh0JOuugp+/RW++AKAKKK4lVs1MbSIiIhUm6ldwKca7OGT 3amn4LNdwFC+PNzHH8OiRQBkk00HOpBBBg1paHFxIiIiYjaf7gL+/cAPDQLxkJtvhvXr3cvD uXDxJ/7Ey7xscWEiIiJSF5jaAuhPfLoFEODvf4esLHjpJQC+5Vuu5moyyCCEEIuLExERETOZ nUvOOACeWJC6gL3o4MHyuQF//NG9PNwlXMJwhjOUoRYXJyIiImbyuS7gE4tRF7AXxcbC0KHw /PPuTROYoMEgIiIiclqmPgPoS1JSUiqsRmKz2Xj//fetLs08d90FL7zgXh6uH/0opJAv+MLa ukRERMSnmfqwWFXNk1Z2p2ZkZBAZGen+uWFDPxolm5gIl1xSvjzcnXdiw8Y93MNsZtOHPlZX JyIiIj7K4y2AaWlpREVFefo2VbLb7TidTvcrPDzcslo84r77YPZs9/JwN3Mza1nLj/xocWEi IiLiq0wJgMe7V0/8/virZ8+ePPDAA2bcplY6dOhA48aN6d69O2+++aZldXhMz54QHw8LFgAQ QQR3cAdP8ZTFhYmIiIivMn0iaF8Z8JGZmcmKFSvo3r07wcHBfPrppzz88MPMmDGDMWPGnPZ8 X3ovp7VwITzxBKxbB8BBDtKWtqSRRiyxFhcnIiIiZ8rnpoGpS2bNmsW//vUvsrKyTntsnZrS pqwM2rUrnxOwTx8ARjGK5jTnYR62tjYRERGpllNlDzA3f/jtKODKdOvWjV27dlFWVlat4+vM lDZBQTB+fPmzgP9zD/fwAi9QQIGFhYmIiEh1eXM6vYAKgN9++y0ul4ugID982zffDF9/7V4e 7hzOoROdeIu3LC5MREREfI0fJqFys2fP5pNPPmHHjh3s2LGDF198kYkTJzJ27FirS/OMiIjy eQFPaAU8PjG0gQ+2WoqIiIhl/HbR2KNHj3LnnXfy888/ExQURLt27Zg1axZ//etfrS7Nc0aP hjZtYPJkcLnoQx/qUY/lLKcf/ayuTkRERHxEQA0CqYk6NQr4RGPGQOPGMGUKAG/yJu/yLktZ anFhIiIiUlsaBewldTYAbt8OF14IGRkQFUURRbSkJYtZTBe6WF2diIiI1ILZucRvnwEMWImJ 5QHwtdcACCOMu7mb2cw+zYkiIiISKNQCWIU62wIIkJICt9wCW7dCcDBHOEIrWvE93+PCZXV1 IiIiUkNqAZTT69kT4uJg/nwAGtKQIQzheZ63uDARERHxBWoBrEKdbgGE8uXhpk2DtWsB2M52 kkkmk0yiiLK4OBEREakJtQBK9Vx1FRw8CKtWAZBIIhdxEa/xmsWFiYiIiNXUAliFOt8CCDBn DvznP/DRRwCkkMIt3EIaaQQp+4uIiNQZagGU6hs6tLwLeOtWAHrSk1hiWcQiiwsTERERKykA +rOICLjjDvjnP92bxjKWp3jKwqJERETEauoCroJfdAFD+XOAbdvCli3gcFBCCa1oxVzm0pOe VlcnIiIi1aAuYKmZ2FgYOBCefRaAEEIYxzi1AoqIiAQwtQBWwW9aAOG35eF27oSICPLIoyUt 2chGmtPc6upERETkNNQCKDWXmAhJSfDyywBEEcUwhvEMz1hcmIiIiFhBLYBV8KsWQPhtebi0 NAgKIossutCFDDJoSEOrqxMREZFTUAug1E7PnuXPA/5vebjmNOcKruBlXra4MBEREfE2tQBW we9aAAE+/BCmT3cvD/ct33I1V5NBBiGEWFyciIiIVEUtgFJ7AwaUTwvz1VcAdKELCSSwgAUW FyYiIiLepAAYSIKCYOxYmDnTveke7mEWsywsSkRERLxNATDQDB9e3gWcng7AVVzFYQ7zBV9Y W5eIiIh4jQJgoImIgL/+FWbMACCIIMYzntnMtrgwERER8RYNAqmCXw4COe748nBpaRAbSz75 tKAFa1hDIolWVyciIiK/o0EgcuZiY+G66+Bf/wIggghGMYqnedriwkRERMQb1AJYBb9uAYTy ZwAvusi9PFw22bSnPdvZTiyxVlcnIiIiJ1ALoJijdevy5eFeew0AFy4GMpAXedHiwkRERMTT AqIFMCcnh86dO7Nz505yc3OJioo67Tl+3wIIsGYNDBvmXh5uC1u4nMvZwQ7CCLO6OhEREfkf tQDWwpgxY2jXrp3VZfieCy8sfx7wo48AOIdz6EhH3ud9iwsTERERT/L7APjee++xbds2Jk6c aHUpvum++9xTwgCMZSyzmIWBn7d+ioiIBDC/DoBZWVncd999vPXWW4SEaK3bSl17bfm0MP9b H/gP/AEDg+Ust7gwERER8RS/DYBlZWXccsstPPTQQ7Rt29bqcnxXUBDcfTc8+SQANmyMYxxP 8ZTFhYmIiIin+G0AnDFjBuHh4YwZM6bW17DZbJW+/M6tt5a3AG7fDsCN3MhmNrOFLRYXJiIi Ejiqyh2eyB5+GwBXrlzJihUrCA0NJSQkhIsuugiAhg0b8vTT1Zvw2DCMSl9+JyICRo2CmTMB CCOM27ldrYAiIiJeVFXu8ET28NtpYH766Sdyc3PdP6empjJkyBD++9//kpCQQJMmTU55fkBM A3Oi/fuhfXv38nAHOUgiifzIj7hwWV2diIhIQDM7l/htAPy9lJQUkpKSNA/gqYwaBfHx8Nhj ANzFXTSgAVOYYm1dIiIiAU7zAIrn3HsvvPAC5OcD5QHwJV4in3yLCxMREREzBUwA7NmzJ4Zh VKv1L2C1a1e+PNybbwKQSCJJJPE2b1tcmIiIiJgpYLqAayogu4ABVq8uHxX8v+XhvuALRjGK rWwlKHD+e0FERMSnqAtYPKt37/Ll4RYtAqAPfWhIQxaxyOLCRERExCwKgFLR2LEwbZr7x3GM YzazLSxIREREzKQAKBVdd1358nApKQBcy7Wkk86rvGpxYSIiImIGBUCpKDgY7roLpk8HIIQQ 3uANZjKTS7iEtay1uEARERE5ExoEUoWAHQRy3K+/QkICrFkDiYkAlFLKW7zFYzxGJzrxd/5O JzpZXKiIiIj/0yAQ8Y769csnhv7nP92bgglmGMPYxjYu53L+yB8ZwhC2s93CQkVERKSmFACl anfcAXPnlj8PeIIwwriTO0kjjQ50IIkk/spf+ZmfLSpUREREakIBUKrmcpUPCHnhhUp3RxHF gzxIOuk0oQld6MI4xnGQg5UeLyIiIr5BAVBO7Z574Jln3MvDVaYhDXmCJ9jMZooooh3teIzH yCXXi4WKiIhIdSkAyqmdcw4kJ8Pbp18OzomTp3majWxkJztpTWtmMUtrCYuIiPgYBUA5vXvu gRkzoKysWoefzdm8zuusZCUppNCGNrzES5RQ4uFCRUREpDoUAOX0+vSBmJjyNYLz8qp9Wnva 8wEf8CEfMp/5tKc97/AOZVQvSIqIiIhnKABK9Xz0EQQFQatW8PzzUFRU7VO70pVlLOMVXuF5 nqcLXbS2sIiIiIU0EXQVAn4i6Kp8/z2MHw/bt8M//lE+Sthmq9ElFrOYh3mYSCJ5gifoQx/P 1CoiIuInzM4lCoBVUAA8jZUrYdw4CAuDJ5+E3r1rdHoZZcxlLpOYRAtaMJWpdKObh4oVERGp 27QSiPiGSy6BDRvK1wweNgyuvhq2bKn26UEEcRM3sYUtDGQg13Ed13M9W6j+NURERKR2FACl 9mw2uOkm2Lq1vAWwb9/y5eOys6t9iRBCuI3bSCONC7mQS7iEv/AXdrLTc3WLiIgEOAVAOXNh YXDvvfDjj9C4MZx3Hjz8cI1GDEcQwVjGkkEGCSTQjW7cyZ1kU/0wKSIiItWjACjmadgQpk0r 7xreswcSE8tHDBcXV/sSUUQxiUn8yI9EEEFHOvIgD3KYwx4sXEREJLAoAIr5mjeHV1+F5cth 0aLy1UQWLIAaPLwaSyzTmc53fMdhDtOWtkxlKnlUv1VRREREKqdRwFXQKGATff55+YjhevXK VxS56KIaXyKDDCYxic/5nAd4gNGMJowwDxQrIiLiezQKWOqeSy+FjRvLRwzfcgtce22NRgwD tKIVb/M2y1jGZ3xGa1rzGq9peTkREZFa8NsAePToUf7whz/gcrkICQkhNjaWwYMHs3v3bqtL C0zHRwynpcGFF5aPGB49ukYjhgE60pGFLGQuc3mbt+lIRz7gAwzUWisiIlJdfhsAbTYb1157 LYsXL2bnzp18/PHH7NmzhwEDBlhdWmALC4P77isfMRwTUz5i+NFHazRiGKAnPfmMz3iGZ5jJ TLrRjaUs9VDRIiIi/iWgngFcuXIll156KUVFRYSGhp7yWD0D6CVZWeUBcPny8qljRo2CkJAa XcLA4CM+4hEeoTGN+Qf/4EIu9FDBIiIi3qdnAGvpyJEjvP7661x55ZWnDX/iRc2bw+uvw5Il v40Y/vDDGo0YtmHjGq4hlVRGMpJbuIU/8Sc2sclzdYuIiNRhfh8Ar7nmGurXr0+jRo3Iy8vj ww8/tLokqUznzvCf/5TPGzhlCiQnw5o1NbpEEEEMZShb2cqVXEl/+nMjN7KNbR4qWkREpG7y +wD4/PPPs3HjRubNm8emTZt44IEHqn2uzWar9CUedNll5RNJ33knDB5cPmL4xx9rdIkwwriD O0gnnS50oRe9uI3byCLLQ0WLiIicuapyhyeyR0A9A7hw4UIGDRrEsWPHCA4OPuWxegbQBxQV wTPPwPTp5UHw0UfB5arxZY5whFnM4kVe5GZu5iEeIo44DxQsIiLiGXoG0ARqxasjjo8YTkuD qKjyEcOPPVbjEcMNacgUpvADP2Bg0J72PMqj5JDjmbpFRER8nN8GwJUrV/LWW2+Rnp7Ozz// zJIlSxg/fjw33HADQUF++7b9U8OG5SuIbNgAO3ZAu3blzwqW1GwSaDt2ZjObb/iG3eymLW2Z znTyyfdQ4SIiIr7Jb5NQWFgYzz33HF26dKFly5b87W9/49prr+Wll16yujSprebN4Y03ykcL L1oEHTqUjxiu6WVoziu8wipWsZGNJJLIi7xIEUUeKFpERMT3BNQzgDWhZwDrgE8/hQkTIDwc Zs2CpKRaXeYbvuERHmErW3mMxxjCEIL897+NRESkDjI7lygAVkEBsI4oK4N334WJE+GCC+Af /4C2bWt1qa/4igd5kCMc4e/8nT/zZ2zoeVEREbGeAqCXKADWMQUF8NxzMG0aDBpUvqpILUYM AyxhCQ/zMGGEMZWp9KWvycWKiIjUjEYBi1QmPLx8xHB6OkREQKdOMHlyjUcMA1zJlWxkI/dy L2MYw2VcxjrWeaBoERERaygAin9p2BBmzoSNG2H79vIRwy+8UOMRwzZsDGIQ3/M9N3ETN3AD V3M13/O9hwoXERHxHgVA8U/Nm8Nbb5WPFv7oI+jYsVYjhkMI4VZuJY00LuESLudybuEWMsn0 QNEiIiLeoWcAq6BnAP3MsmXw4IPl3cOzZkHPnrW6TB55PMVTPMuzXMZl/Jk/04IWJJCg1UVE RMRjNAjESxQA/VBZGbzzDjzyCJx/Pjz5JLRuXatL/cIvPM/zpJLKTnaSSSYFFJBAgjsQtqSl +/sWtCCaaJPfkIiIBAoFQC9RAPRj+fnlK4kcHzH86KPgcJzxZXPJJZNMdyA8/ueO/33Vp/5J gfDEP5vTnDDCTHhzIiLijxQAvUQBMAAcPgxTp8Lrr8Pdd8PYseVrDnvIPva5w+Dxr+MhcTe7 ceCoEAxb/u8rnnjNSSgiEsAUAL1EATCAZGWVPx+4alV59/Ctt0JIiFdLKKWUXeyq0Hp4/M9f +IUWtKjQrXz8zyY08Wq9IiLiXQqAXqIAGIC+/Rbuvx/27IG//x2uucbqitwKKKjQenjil4FR ZfdyS1oSQYTVb0FERM6AAqCXKAAGsKVLy1sEo6Jgxoxajxj2psMcrtCtfOKfDWno7k7+/ddZ nEUI3m3xFBGRmlEA9BIFwABXVlY+j+CkSeVrDD/5JCQmWl1Vre1hutJQTgAAE8xJREFUT5Wt h3vZiwtXpa2HLWiBi9otqSciIuZRAPQSBUABykcMP/ssTJ9ePmJ40iSw262uylTFFJNFVpUB MZfcKlsPW9KSBjSw+i2IiPg9BUAvUQCUkxw6BP/4B7z2Wvlo4Xvu8eiIYV/yK7/+f3t3H9vE fcdx/OPYcZznpIWWbGnSEJLQqNWg7UqK1P3Rba3GtKGhpduKpqKtSkFMQ92Urd2YVqhUqUxt 1UlD2x+027p2GpWgWxl7BlHagaZQsvSfBPMQAl0IHctjyYMd//bH1Y5N7DhA4rv43i/LinNn 3319OR8fvvfgaZe0ib/lKjcWBpdqqZZpmeo/utE9BIC5QQDMEAIgkjp71jo+8K23pNZW6XOf k6qrpbw8uyuzzX/131gYPK3TOhF3G9OY6lWvOtWpXvVqUEPsMZ1DAJg9AmCGEAAxo+PHrQtI d3ZK585JN95oBcFbb526V1dPDQsE7K3XJgMaiIXBLnUp+NHthE6oQAVqUEMsINapTg1q0DIt U57cG6gBIBkCYIYQADFrkYh16ZizZ6Xu7sR7T481/IYbpofC+KCY777LtLyv93VCJxJCYZe6 1K1ufUwfi4XC5Voe6xpWqUpeee0uHQAyjgCYIQRAzBljpN5e6cyZqVAYHxLPnpXKyhJDYVXV 1OOaGlcFxLDC6la3ggqqS12xDmJQQV3URS3V0oRdydHO4c26/q/zAwCnIgBmCAEQGdXbOz0U xncUS0sTQ2G0cxh9XFhoV+UZNarRWLcwvmsYVFATmoidfBK9RUNiiUrsLh0ArgsBMEMIgHCU 3t7poTD6+5kz1hnJyXYx19RYv7vgjOX/6X8JXcP4oFis4ljXMBoK61WvZVomv/x2lw4AaREA M4QAiAWlry955zB6z8+fvou5pmbq9+JiuyrPiHM6l3CsYfTWo57Y8YbRs5SjncMqVSlHOXaX DgCSCIAZQwBEVvngg+knp3R3Tx2XGAhM38VcUzM1rDQ7L9kSUkjd6k7YlRx/vGH8cYbxu5Zv UnZdDByA8xEAr0Jra6v++Mc/6uTJkyotLdXatWv13HPPqXQW/5gRAOEqH3ww/eSUaEexu1vy +RJPTonvHlZVSeXltpU+Xy7rcsJZyvEnpEQUSXptwzrVqVjZ3U0FYA8C4FV48MEH1dLSorvu ukt9fX167LHHVFNTo71796Z9LQEQiHPp0vRdzNHu4dmzksczFQhra63vTV661PpZXW0FyCxy SZcUVFCd6kzoGgYVVJnKErqGlaqUV14Vq1g++eSRR2UqkyT55VehrBN4ClUYOx6xXNkXqAFc HwLgdfjTn/6kL37xixofH1dOzszH9hAAgavQ3z8VCk+dsu4nT0qnT1vXSKysTAyFtbVTQTGL LpJtZHRe5xMuXXNe5yVJwxpWWGEZGQ1oQJI0rnFd1mVJ0ohGFFJIktSvfklSrnJVJOsEngIV xC6QXapS5ShHXnljZzgHFFC+rMsFlahEXnmVo5zYN67kKU8FKpAkFalIucqVNBU2U4XRMpXJ I4988sW6m/nKV0DZ83cDFoK5ziXZ9d/yNPr7+1VaWpo2/AG4SuXl1n3lyunjJiascBgfCg8c sH6ePi0tWpQ6HJaVZfytXA+PPLrlo9un9enrnl5IIY1oRJK1S3pc45Ksb1gxMgorrGENS7Iu kTOmMUnSkIY0qUlFFNGgBiVZYTMaLHvUMy2MTmhCH+pDSdb3P09oImFeqWqJhs1UYTRd5zNZ GL2a4BsfRpMFXwDJuaYDODIyorvuuktf+9rX9NRTT6V9Ph1AIAMiEen8+cRwePLkVBfR55se CpculerqpCVL7K4ekgY1qIgimtSkhjQkSRrTmEY1Kmmq83llGJ2p8xkfNpOF0asJvsk6n7PZ DZ+u85muC5ss+KbqwgKzwS7gaxAKhbR27VqNj4/rz3/+s3Jz03/oPB5PynEuWGSAM1y8mBgK 48PhyEhiOIwPibfcknXHHeLaJAujs+l8Jguj8Z3PdME3VRc2WfAtV7kO6ZDu0B3zuzDgeDNl D2lu80fWB8DJyUl99atfVU9Pj/7xj3+oaJYXxKUDCDjc0ND0jmG0i3jhgnV2crLOYU1NVh13 iIWvX/2x3ddAKnQAr0IkEtEjjzyi9vZ2HTp0SDfccMOsX0sABBaw8fGpE1LiA+KpU9bxiDfd ND0cRruJWXrNQwALGwHwKrS0tOjNN9/U/v37VVFRERu+ZBbHDhEAgSw1OWkdd5isc3jypNUd vDIURn/efLPd1QNwKQLgVUi1L310dFSBNLuACICAS/X1JXYO48Ph2FjqM5YrKyUvu/AAzA8C YIYQAAFMMziY+ozlixenLoR9ZdewvNwKh6WlUm6uNMtjkQEgigCYIQRAAFdlbMw67jAYTAyH H3xgXSh7ctI6cWViQvrwQ2tXc36+FQZzc61w6PVa1z70+aTiYikvTyoosO55edYwn28qUJaU SH6/VFhoTSsQSD49AAseATBDCIAA5tXoqBUaR0akUMjqLobD1s9QyBo+NmY978MPreA4NGQF yfhAOT4uXb5s3cfHpeFhazoDA9ZzBgenuo7R0FlYaAXHkhIrJM4UKKOhs6xsehcz1fQAzDkC YIYQAAFkjVSBcnDQCokDA1ZoHB6eOVCGw4ldzGQh1pjZdTGTBcqZupippge4BAEwQwiAAHAN jEkeKGfqYqYKlPFdzFQhdjZdzPhA6fPNrouZbHqAjQiAGUIABIAFID5QzqaLmSpQxncxU3VF Z9PFvDJQzvbYzqoqLlCOGREAM4QACABIcGWgnE0XM1mgTHZs5xtvSI2Ndr9DOBgBMEMIgAAA wCnmOpfkzNmUAAAAsCAQAAEAAFyGAAgAAOAyBEAAAACXIQACAAC4DAEQAADAZQiAAAAALkMA BAAAcBkCIAAAgMsQAAEAAFyGAAgAAOAyBEAAAACXIQACAAC4DAEQAADAZQiADufxeOwuQZJz 6pCcU4tT6nASlsl0LBOkwzoCOxAAAQAAXCarA2BLS4tWr16tkpISeTwejY2N2V0SAACA7bI6 AIbDYTU3N2vbtm12lwIAAOAYHmOMsbuI+Xb06FHde++9Gh0dVSAQmNVrPB6PnLBoqGM6p9Ti lDqchGUyHcsE6bCOYDbmej3J6g4gAAAApiMAAgAAuIzP7gKczCmn5lPHdE6pxSl1OAnLZDqW CdJhHUGmEQBT4HgMAACQrdgFDAAA4DJZ3QFsb2+XJAWDQUlSR0eH/H6/VqxYYWdZAAAAtsrq y8CkOqYii98yAABAWlndASToAQAATMcxgAAAAC5DAAQAAHAZAiAAAIDLEAABAABchgAIAADg MgRAAAAAlyEAOlhra6saGxvl9/u1ePFiPfrooxocHLS1psHBQdXU1Mjj8WhkZMSWGvbs2aPb b79dfr9fVVVV2rdvny11vPPOO1q9erUCgYCKi4v1+c9/XqdPn7alFju0tLRo9erVKikpkcfj 0djYWMJ4J66/8y3dMpGcs/7CPrP9bDhhe4vMGxoa0gMPPKCKigr5fD4tWrRIDz/8sN5///3Y c+Zi+0oAdLCOjg49/fTTOnHihPbt26e2tjZt2LDB1po2b96s5cuX2zb/v/71r9q8ebOefvpp nTp1Sq+//rqqq6szXsfQ0JDWrFmj22+/XR0dHXr77bcViUS0bt26jNdil3A4rObmZm3bti3p eCeuv/Mt3TJxyvoLe832s2H39hb28Hg8Wrdunfbt26fu7m794Q9/0H/+8x+tXbs29pw52b4a LBj79+83Pp/PTE5O2jL/1157zXzyk580hw8fNpLM8PBwxmu4++67zSuvvJLx+V7p3XffNZJM b29vbNjBgweNx+Ox7e9jlyNHjhhJZnR0dMbn2b3+ZlKqZeKU9RfOkuyz4YTtLZzjwIEDRpKZ mJhIOv5atq90ABeQ/v5+lZaWKicn83+2np4effe739Urr7win8+eL5C5dOmS2traND4+rsbG Rt18883auHFj0t1s862xsVG1tbV66aWXFAqFNDIyol//+tf6whe+YMvfZyGwc/11Aietv3CW Kz8bTtjewjkGBgb0y1/+UmvWrFFubm7S51zL9tWdW+IFaGRkRNu2bdO3vvWtjM87Eono61// un7wgx+ooaEh4/OP6unpkST96le/0ssvv6xXX31V+/fv1xNPPJHxWvLy8nT48GH97ne/U35+ vkpKStTX16fdu3dnvJaFwM711ymctP7COa78bDhlewv7felLX1JhYaHKy8s1MjKivXv3Jn3e tW5fCYALQCgU0kMPPaTKykr98Ic/zPj8f/KTnygQCGjz5s0Zn3c889F3Oz///PNatWqVPvOZ z2j79u169dVXbann0UcfVWVlpf71r3/prbfe0vnz51Me++Vmdq+/TuG09Rf2S/bZcMr2Fvbb uXOnjh07pt27d6u9vT3pfxava/s6N3unMV/C4bD58pe/bO655x7bjgF58MEHTU5OjvF6vbG7 JOP1es2LL76YsTrOnTtnJJmBgYHYsOjxMeFwOGN1GGNMZ2enkWTOnDkTG/bmm2+agoICE4lE MlqL3WY6BtAJ668dki0TJ62/sF+qz4ZTtrdwljfeeMP4/f6EbcX1bl85uMDBIpGINmzYoM7O Th06dEhFRUW21PGLX/xCw8PDsd87Ojq0fv16HTlyREuXLs1YHZWVlaqoqNDp06e1cuVKSdKZ M2e0ePFieb3ejNUhSRMTE5KUMF+v16tQKJTROpzMKeuvUzhp/YW9ZvpsOGV7C2fyeDyS5mb7 SgB0sI0bN+rvf/+79u/fr4mJCV24cEGStGTJkozWceVlKqLXo7rtttsy/o/6pk2btGXLFv38 5z/X8PCwtm3bpkceeSSjNUjWe6+srNTjjz+uZ555RpcvX9aPf/xjffazn419QLNde3u7JCkY DEqy/qHy+/1asWKFJOesv5mUbpk4Zf2FvWb6bDhpewt7HDx4UOfPn1dTU5Py8/PV0dGh1tZW feUrX4md5DEn29e5bFFibklKek93uY35Ft29ZccuvVAoZLZs2WLKyspMSUmJaWlpMWNjYxmv wxhjjh8/bu677z6Tl5dnioqKzLp168yFCxdsqcUOqdbPdOPtXn/nU7pl4qT1F/a5ms+Gndtb 2OPtt982q1atMoWFhcbn85mamhrz/e9/P2H9mIvtq+ejCQEAAMAlOAsYAADAZQiAAAAALkMA BAAAcBkCIAAAgMsQAAEAAFyGAAgAAOAyBEAAAACXIQACAAC4DAEQAADAZQiAAAAALkMABAAA cBkCIAAAgMsQAAEAAFyGAAgAAOAyBEAAAACXIQACcLVwOKxIJGJ3GXMm294PgPlBAASQVY4e PSqPx6M9e/aotrZWhYWF2rRpkyYnJxPG79u3T3fccYcCgYD6+vo0NDSk9evXq6ioSIsWLdKT Tz6ZEKSMMXr++edVV1cnv9+viooKbd26NWHeP/vZz7R8+XLl5eXplltu0fbt2xOm8dJLL6m6 ulo+n08f//jH9cILL8xq3Ey1pXo/M00PAHx2FwAA8+GFF17Q3r17NTAwoObmZtXU1Oh73/te bPwzzzyjXbt2qaysTOXl5frmN7+p9957T++88476+/vV3NysxYsX6zvf+Y4k6amnntLOnTu1 c+dONTU16eLFi/r3v/+dML3XXntNP/3pT9XY2Kiuri499thj8ng8+tGPfqTe3l61tLTo2Wef VXNzsy5evKje3l5JmnGcJG3atGnG2q58P2NjYzNODwBkACCLHDlyxEgyf/vb32LDnnvuOdPQ 0JAw/sCBA7HxQ0NDxufzmb/85S+xYTt27Ii9ZnR01BQUFJjf/va3Sec5Pj5uiouLzbFjxxKG 79mzx1RUVBhjjHnvvfeMJNPV1TXt9TONS1dbsvcz0/QAwBhj2AUMICutXLky9vjOO+9UMBhM 2B27YsWK2ONTp04pHA7rzjvvjA27++67Y685ceKELl++rPvvvz/pvLq6ujQ8PKxPfepTKioq it3Xr1+v3t5ehUIh3XbbbWpqalJTU5M2bNigPXv2xOqZaVy62pK9n5mmBwASxwACcKnc3Nw5 m1Y0XB08eFDt7e2xe0dHh4LBoHw+n7xerw4fPqzf/OY3Ki0tVUtLi9avXy9JM467lvczF9MD kN0IgACyUnt7e+zx8ePHVVdXp5yc5Ju82tpa+Xw+vfvuu7FhbW1tqq+vV05Ojurr61VQUKAD Bw4kfX1DQ4OKiorU09OjZcuWTbt7PB5Jks/n05o1a/Tiiy/q5Zdf1u7du2PhMdW4dLWlMtO8 AICTQABkpe3bt+umm27S4OCgduzYoS1btqR8bnFxsR566CG1trZqyZIl6u/v144dO/Tkk09K kgKBgB5//HF9+9vfls/n06pVq2IngXzjG99QIBDQ1q1btXHjRo2Ojuq+++5TKBTSsWPHFAwG tXXrVrW1temf//ynHnjgAeXm5ur3v/+9qqurlZOTM+O4dLUlM9P0AEASJ4EAyC7RkyJef/11 c+utt5qCggLT0tJiQqFQwvjh4eGE1w0ODpqHH37YFBYWmhtvvNE88cQTZnJyMjY+EomYZ599 1tTU1Bifz2cqKirM1q1bE6axa9cu84lPfML4/X5TWFho7rnnHrNr1y5jjDGdnZ3m/vvvN8XF xcbv95umpibT1taWdly62pK9n3TTAwCPMcbYnEEBYM4cPXpU9957r4aHh1VUVGR3OQDgSOwP AAAAcBkCIAAAgMuwCxgAAMBl6AACAAC4DAEQAADAZQiAAAAALvN/HZta+PA/3FgAAAAASUVO RK5CYII= --------------020600060901050801070203--