GNU bug report logs -
#33210
Cuirass: Use a SQLite in single-thread mode
Previous Next
Full log
Message #41 received at 33210 <at> debbugs.gnu.org (full text, mbox):
Hi Danny,
Danny Milosavljevic <dannym <at> scratchpost.org> writes:
> Yeah, right now users can query something using the web interface while
> a build is updating (or running) at the cost of the returned data being
> potentially very strange.
This is quite unlikely.
> The "one-line fix" would make it worse in that users cannot query while
> a build is running, making them wait until the build is done (approx.
> 30 min) before the query succeeds. The upside is that the returned
> data is consistent at all times. This is how DBMSes did it in the 90s,
> too.
No, there are no SQL requests during the build. There are some before
the build, and some after the build. But they are all short.
> What I'd like to eventually have is the proper fix where users can query
> while a build is running, *and* the build doesn't have to wait either.
> This works just fine using two transactions with WAL mode of sqlite,
> which means it uses MVCC in order to keep both "world views", one for the
> querier and one for the builder (easily extended to an arbitrary
> number of queriers and builders at once by just having more transactions)
> while they are both using "the world".
It's already the case, because all the queries are very short.
>> is an essential difference however: if we take care of the scheduling,
>> we won't have SQLITE_BUSY blocking the Fibers scheduler all the time.
>> And blocking the Fibers scheduler has an impact on all other possibly
>> unrelated Fibers clients.
>
> Right. I just wanted to make sure we understand the possible implications here.
>
> In the end I'm not sure we even need multithreading even for my scenario -
> maybe (probably) just having an extra sqlite_open would be enough, threads
> or not. On the other hand there are shared caches etc and this change here
> could cause some very tricky problems then.
I don't understand this. Could you explain why we would need an extra
sqlite_open, what change are you talking about, and why there could be
tricky problems with shared caches?
> I have to say I liked the external evaluator much more since then all
> this complexity would be contained in the external program and it would
> just magically work without special-casing any of this stuff.
The evaluator is still external, I'm not sure what you are talking
about.
>> When guile-sqlite3 handles SQLITE_BUSY
>> correctly, I'll be happy to switch back to a multi-threading SQLite.
>> While waiting for it, I believe our users want a fast Cuirass, and I'd
>> like the time spent in the Fibers scheduler to be balanced by removing
>> the SQLite now useless mutexes.
>
> That makes sense.
>
> It's difficult for guile-sqlite3 to handle SQLITE_BUSY correctly since
> sqlite also uses SQLITE_BUSY to indicate errors that you are supposed to
> fail on.
[...]
> Hmmmmmmmm. I think that can be done.
Cool!
Cheers,
Clément
This bug report was last modified 6 years and 217 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.