Subject: bug#10125: 24.0.91; package.el (org): Macros in tar packages & order of byte compilation

From: Jambunathan K <[email-masked]>
Date: Thu, 24 Nov 2011 22:52:26 +0530

> Org Mode files certainly have no dependency rules in lisp/Makefile.in.

The Makefile - in devel repo of Orgmode - does define rules. Read on ...

> So the question why the problem does not happen while compiling Org in
> Emacs remains.

I believe the way the files are compiled makes a substantial difference.

When compiling with makefiles:

The compilation happens with *minimal* emacs and in batch mode.

--8<---------------cut here---------------start------------->8---
BATCH=$(EMACS) -batch -q -no-site-file -eval                             	
  "(setq load-path (cons (expand-file-name \"./lisp/\") 
  (cons \"$(lispdir)\" load-path)))"
--8<---------------cut here---------------end--------------->8---

As can be seen above, any (require 'something) of macro files in the
compiled elisp file has to be loaded from the development version
itself.

Furthermore there are dependencies like this in the Makefile:

--8<---------------cut here---------------start------------->8---
lisp/org.elc:		lisp/org-macs.el lisp/org-compat.el lisp/org-faces.el
lisp/org-agenda.elc:	lisp/org.el
--8<---------------cut here---------------end--------------->8---

(I believe removing the dependencies from the Makefiles will still do
the right thing because of the require directives in the compiled files
will load the development version and not the system version)

When compiling with package manager, the compilation happens from within
a running Emacs session and very likely the "old" Org files are already
loaded in to the runtime "inadvertently" by the user either by looking
at the org agenda for the day or may be by just viewing an Org file or
by the plain old (require 'org-whatever) out of habit in .emacs. 

While reporting macro issues, users never say whether they were already
running Org when they were trying to fetch and compile a new Org. They
think it is immaterial. I believe it matters

If "old" org and hence "old" org-macs is already loaded in the
environment when the package is installed, any subsequent (require
'something) will be essentially no-ops. (Can you confirm this?)

What ideally should happen is that during package compilation, a require
should *forcibly* load from the compiled package and not merely check
for availability of a feature symbol.

>> 2. While building from ELPA, the compilation order seems to be
>> alphabetical. So the files get compiled bass ackwards. For example,
>> org-macs.el gets compiled after org-agenda.el.
>> 
>> In summary, there needs to be a way to specify the order in which files
>> are compiled in a multifile tar.
>
> This was discussed several time, in the context of recompiling
> multiple Lisp files while building Emacs, and the decision till now
> was to ignore the issue.  While at least in principle one could write
> a Lisp program that would analyze the various `require' and `load'
> calls (possibly as side effect of byte compilation, like GCC does),
> and generate Makefile rules with correct prerequisites, this is a
> non-trivial project.

I have tried giving an explanation in the earlier paragraph.

> One simple band-aid is to remove all the *.elc files before
> byte-compiling after resync.  This prolongs the compilation, but the
> results are predictably correct.

We are talking of "automatic" compilation by package manager. What you
say applies to hand compilation via makefiles.

>> A supplementary question to (2):
>> 
>> During the package compilation, when encountering (require
>> 'some-org-library-with-macros) does the library get loaded from *within*
>> the tarball or from the *emacs core*.
>
> Whichever is found first along load-path, I think.  See `openp'.

What does package manager do?
--