GNU bug report logs - #32234
Cuirass: The SQLite built in busy handler might block the Fibers scheduler

Previous Next

Full log

View this message in rfc822 format

From: ludo <at> gnu.org (Ludovic Courtès)
To: Clément Lassieur <clement <at> lassieur.org>
Cc: 32234 <at> debbugs.gnu.org
Subject: bug#32234: Cuirass: The SQLite built in busy handler might block the Fibers scheduler
Date: Mon, 23 Jul 2018 22:11:07 +0200

Clément Lassieur <clement <at> lassieur.org> skribis:

> Ludovic Courtès <ludo <at> gnu.org> writes:
>
>> Clément Lassieur <clement <at> lassieur.org> skribis:
>>
>>> Ludovic Courtès <ludo <at> gnu.org> writes:
>>
>> [...]
>>
>>>> Perhaps instead we need to set the timeout to a small value and handle
>>>> SQLITE_BUSY at the call site in our code.  We could define a macro that
>>>> automatically retries upon SQLITE_BUSY.
>>>
>>> That would limit the issue to the first timeout span: for that short
>>> time the scheduler would be blocked.  I think a timeout of 0 would be
>>> better.
>>
>> Yes, 0 is an acceptable “small value.”  ;-)  Perhaps 100ms would be
>> acceptable if the situation is rare enough, dunno.
>>
>>> Another solution would be to serialize all the database accesses as we
>>> do already with the url handler, and stop using the SQLITE
>>> multithreading features.  It would probably make the code simpler
>>> because we would use the same paradigm everywhere, and we would avoid
>>> looping until SQLITE isn't busy at each request.
>>
>> In essence we’d introduce a “database server” running as a fiber, and
>> everyone would talk to that server.
>
> Why not as a thread?  There would be only one thread dedicated to SQLITE
> (thus adding a constant overhead), and that would prevent the Scheduler
> from being bloqued by long SQLite queries.

Right, if we want to have a big timeout, it should be a thread, not a
fiber.  Sorry for the confusion.

>> I considered doing that before but then though sqlite would probably be
>> able to do better than this, but I don’t know.
>
> And it's hard to test...
>
>> What’s a bit annoying with switching to a database server model is that
>> we’d need to adapt every call site.
>>
>> Thoughts?
>
> I'm a tiny bit in favor of switching to a database server model because
> it's more consistent overall (we already use it for the url handler) and
> the scheduler wouldn't be bloqued by long SQLite queries.
>
> It's annoying to adapt every call site, but it would be done once and
> for all. :-)

Indeed.  If you want to take that route, that’s fine with me!

Thank you,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.