GNU bug report logs - #32234
Cuirass: The SQLite built in busy handler might block the Fibers scheduler

Previous Next

Full log

View this message in rfc822 format

From: help-debbugs <at> gnu.org (GNU bug Tracking System)
To: Clément Lassieur <clement <at> lassieur.org>
Cc: tracker <at> debbugs.gnu.org
Subject: bug#32234: closed (Cuirass: The SQLite built in busy handler
 might block the Fibers scheduler)
Date: Mon, 27 Aug 2018 13:48:03 +0000

[Message part 1 (text/plain, inline)]

Your message dated Mon, 27 Aug 2018 15:47:27 +0200
with message-id <878t4rc45s.fsf <at> lassieur.org>
and subject line Re: bug#32234: [PATCH 2/2] database: Serialize all database accesses in a thread.
has caused the debbugs.gnu.org bug report #32234,
regarding Cuirass: The SQLite built in busy handler might block the Fibers scheduler
to be marked as done.

(If you believe you have received this mail in error, please contact
help-debbugs <at> gnu.org.)


-- 
32234: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=32234
GNU Bug Tracking System
Contact help-debbugs <at> gnu.org with problems

[Message part 2 (message/rfc822, inline)]

From: Clément Lassieur <clement <at> lassieur.org>
To: bug-guix <at> gnu.org
Subject: Cuirass: The SQLite built in busy handler might block the Fibers
 scheduler
Date: Sat, 21 Jul 2018 11:57:57 +0200

Hi,

I'm trying to understand why Berlin web API times out[1].

I think the SQlite built in busy handler may block the Fibers scheduler.
We use "PRAGMA busy_timeout = 30000;", which is an alternative to
calling sqlite3_busy_timeout(), whose description[2] is:

    This routine sets a busy handler that sleeps for a specified amount
    of time when a table is locked. The handler will sleep multiple
    times until at least "ms" milliseconds of sleeping have
    accumulated. After at least "ms" milliseconds of sleeping, the
    handler returns 0 which causes sqlite3_step() to return SQLITE_BUSY.

To me this sounds like non-cooperative and non-resumable code.

A solution would be to set a custom handler through
sqlite3_busy_handler[3] that would be Fibers compatible, i.e. it would
let the scheduler schedule other fibers instead of just sleeping, using
Fibers 'sleep' procedure[4].

WDYT?
Clément

[1]: https://bugs.gnu.org/32233.
[2]: https://www.sqlite.org/c3ref/busy_timeout.html
[3]: https://www.sqlite.org/c3ref/busy_handler.html
[4]: https://github.com/wingo/fibers/wiki/Manual#user-content-index-sleep

[Message part 3 (message/rfc822, inline)]

From: Clément Lassieur <clement <at> lassieur.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 32234-done <at> debbugs.gnu.org
Subject: Re: bug#32234: [PATCH 2/2] database: Serialize all database accesses
 in a thread.
Date: Mon, 27 Aug 2018 15:47:27 +0200

Ludovic Courtès <ludo <at> gnu.org> writes:

> Hi Clément,
>
> Clément Lassieur <clement <at> lassieur.org> skribis:
>
>> Ludovic Courtès <ludo <at> gnu.org> writes:
>>
>>> Excellent, thanks for working on this!  This looks great to me, and I
>>> think the pros outweigh the cons.  Did you check on a big database how
>>> well it performs?
>>
>> Yes, I didn't see any difference.  When I use Berlin's database, it
>> works well but crashes quickly for another reason (lack of disk space I
>> think, and /tmp being tmpfs).
>
> Sounds good (not that it crashes, but that you didn’t see any
> difference ;-)).
>
>>> I think I find it nicer to keep the ‘db’ parameter everywhere (except
>>> that it’s now a channel instead of an actual database) rather than using
>>> this global variable.
>>>
>>> WDYT?
>>
>> That 'db' parameter made sense before, because there were different
>> database connections: one per fiber.  But now that there is only one
>> global channel accessible from everywhere, I can't find any use for a
>> 'db-channel' parameter.
>>
>> Also, using two differents channels for the same database would be a
>> bug, it would break the serialization mechanism.
>>
>> And I don't think using several databases (with one channel per
>> database) would make sense either.
>
> These are all good points, indeed.  I’m mildly reluctant to the global
> parameter, but if you prefer it that way, I don’t mind; the end result
> matters more than this tiny issue anyway!
>
> So: LGTM.
>
> Thank you!
>
> Ludo’.

Ok, pushed!

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.