Subject: bug#32234: closed (Re: bug#32234: [PATCH 2/2] database: Serialize all database accesses in a thread.)

From: [email-masked] (GNU bug Tracking System)
Date: Mon, 27 Aug 2018 13:48:03 +0000

Your bug report

#32234: Cuirass: The SQLite built in busy handler might block the Fibers scheduler

which was filed against the guix package, has been closed.

The explanation is attached below, along with your original report.
If you require more details, please reply to [email-masked].

-- 
32234: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=32234
GNU Bug Tracking System
Contact [email-masked] with problems

Subject: Re: bug#32234: [PATCH 2/2] database: Serialize all database accesses in a thread.
From: Clément Lassieur <[email-masked]>
To: Ludovic Courtès <[email-masked]>
Cc: [email-masked]
Date: Mon, 27 Aug 2018 15:47:27 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

Ludovic Courtès <[email-masked]> writes:

> Hi Clément,
>
> Clément Lassieur <[email-masked]> skribis:
>
>> Ludovic Courtès <[email-masked]> writes:
>>
>>> Excellent, thanks for working on this!  This looks great to me, and I
>>> think the pros outweigh the cons.  Did you check on a big database how
>>> well it performs?
>>
>> Yes, I didn't see any difference.  When I use Berlin's database, it
>> works well but crashes quickly for another reason (lack of disk space I
>> think, and /tmp being tmpfs).
>
> Sounds good (not that it crashes, but that you didn’t see any
> difference ;-)).
>
>>> I think I find it nicer to keep the ‘db’ parameter everywhere (except
>>> that it’s now a channel instead of an actual database) rather than using
>>> this global variable.
>>>
>>> WDYT?
>>
>> That 'db' parameter made sense before, because there were different
>> database connections: one per fiber.  But now that there is only one
>> global channel accessible from everywhere, I can't find any use for a
>> 'db-channel' parameter.
>>
>> Also, using two differents channels for the same database would be a
>> bug, it would break the serialization mechanism.
>>
>> And I don't think using several databases (with one channel per
>> database) would make sense either.
>
> These are all good points, indeed.  I’m mildly reluctant to the global
> parameter, but if you prefer it that way, I don’t mind; the end result
> matters more than this tiny issue anyway!
>
> So: LGTM.
>
> Thank you!
>
> Ludo’.

Ok, pushed!

Subject: Cuirass: The SQLite built in busy handler might block the Fibers scheduler
From: Clément Lassieur <[email-masked]>
To: [email-masked]
Date: Sat, 21 Jul 2018 11:57:57 +0200
List-Id: <debbugs-submit.debbugs.gnu.org>

Hi,

I'm trying to understand why Berlin web API times out[1].

I think the SQlite built in busy handler may block the Fibers scheduler.
We use "PRAGMA busy_timeout = 30000;", which is an alternative to
calling sqlite3_busy_timeout(), whose description[2] is:

    This routine sets a busy handler that sleeps for a specified amount
    of time when a table is locked. The handler will sleep multiple
    times until at least "ms" milliseconds of sleeping have
    accumulated. After at least "ms" milliseconds of sleeping, the
    handler returns 0 which causes sqlite3_step() to return SQLITE_BUSY.

To me this sounds like non-cooperative and non-resumable code.

A solution would be to set a custom handler through
sqlite3_busy_handler[3] that would be Fibers compatible, i.e. it would
let the scheduler schedule other fibers instead of just sleeping, using
Fibers 'sleep' procedure[4].

WDYT?
Clément

[1]: https://bugs.gnu.org/32233.
[2]: https://www.sqlite.org/c3ref/busy_timeout.html
[3]: https://www.sqlite.org/c3ref/busy_handler.html
[4]: https://github.com/wingo/fibers/wiki/Manual#user-content-index-sleep