Subject: Re: bug#59346: Adding sqlite-backup

From: Andrew Hyatt <[email-masked]>
Date: Fri, 18 Nov 2022 23:05:20 -0300

On Fri, Nov 18, 2022 at 9:17 AM Eli Zaretskii <[email-masked]> wrote:

> > From: Andrew Hyatt <[email-masked]>
> > Date: Fri, 18 Nov 2022 09:05:58 -0300
> > Cc: [email-masked]
> >
> >  Thanks, but I'm not sure Emacs should support DB administration
> >  functions of sqlite.  Aren't there utilities out there which can be
> >  used for this?  Why should we have this built-in in Emacs? why not
> >  simply use shell-command or somesuch to invoke the necessary external
> >  program?
> >
> > Good questions. Let me respond to your two questions: why does emacs
> need to be involved in backups,
> > and why can't it control backups some other way?
> >
> > Emacs already has a file backup facility, but if emacs is primarily
> working with data in sqlite, there is no
> > mechanism for backups, which seems scary to me.  I'd like modules that
> rely on sqlite for data to be able to
> > back up their data, because the user would like to have some security,
> knowing if something goes wrong
> > they can always restore a recent backup.
>
> A database is not like a normal file, and so the fact that Emacs has
> backups doesn't seem to be a reason good enough to extend the backups
> to DB operations.  We don't bother with this when we send email or do
> other operations.  We also have auto-save and file-locks for normal
> file, but not for DBs.  And a DB is not a file, it is a (large)
> collection of tables and records, and Emacs deals with at most a
> single table at a time, AFAIU.

> More generally, DB administration is outside of the Emacs scope, and
> how to do that properly is outside our expertise.  There's more to it
> than just backing up the DB.
>
> So I don't think we should extend the Emacs sqlite3 support in this
> direction.
>

I disagree, but I understand this is just a difference of opinion, so I'm
happy to let this patch sit here until there's more demand for this sort of
thing.

But if you are interested in furthering the conversation, which has a lot
to do with what I'm talking about in my upcoming (and already recorded)
Emacs conference talk, I don't think the distinction you are making between
files and databases is important.  Let's say I wanted to store, use and
otherwise manipulate contact information with emacs.  Right now that sort
of thing happens with a file with emacs-readable lists inside, but it's
really more natural to do it with a database.  The fact that it's done via
a file now seems historical, especially now that we have sqlite.  I expect
that sqlite will be used for this sort of thing more often.  However, if
there's no database option, then maybe not, since it won't be seen as
reliable as the filesystem.  I don't think it is reasonable to expect users
to figure out how to do this, even if the info is easy to find.  I'd like
us to be able to encourage the use of sqlite for data, and give the user a
good experience when sqlite is the backend.

>
> > But why can't we do this by invoking backup via the binary?  Certainly a
> possibility, and in fact that's how I
> > would implement it if the sqlite library was instead emacsql-sqlite.
> However, I have no idea how to locate the
> > binary; that isn't part of the current built-in sqlite implementation.
> For good reason, too, it's a built-in
> > implementation, including a variable holding the executable path seems
> odd.  I could add one, but this
> > solution seems better to me, since it doubles down on the built-in
> sqlite, and doesn't provide an alternative
> > mechanism for doing things with sqlite.
>
> This SO article:
>
>
> https://stackoverflow.com/questions/25675314/how-to-backup-sqlite-database
>
> (which is the first hit I get if I "how to backup sqlite database"
> into the browser search box) says that you can backup the DB by using
> the sqlite3 executable.  I think it is reasonable to expect users who
> want to backup their DB to have this executable and use it for that
> purpose (and other purposes, as they need).  I do have this executable
> here, FWIW.
>
> So I think these needs should be fulfilled "by other means", not by
> Emacs Lisp programs.
>