GNU bug report logs - #48463
gnu: Add j.

Previous Next

Full log

View this message in rfc822 format

From: Liliana Marie Prikler <liliana.prikler <at> gmail.com>
To: elaexuotee <at> wilsonb.com
Cc: Maxime Devos <maximedevos <at> telenet.be>, 48463 <at> debbugs.gnu.org
Subject: [bug#48463] gnu: Add j.
Date: Tue, 18 Jan 2022 09:04:42 +0100

Hi,

Am Dienstag, dem 18.01.2022 um 13:01 +0900 schrieb
elaexuotee <at> wilsonb.com:
> 
> 
> > The (guix cpu) code is not meant to be used at runtime as the
> > comment to ijconsole implies it wants it to be used.  Rather, you
> > should use it at compile time to bake in the correct tuning
> > parameters, and it should ideally also support "cross-tuning"
> > (though in that case you probably need to disable the tests because
> > your CPU won't make sense of the code).
> 
> Hrm... Mind if I ask about the rationale? We could go back to
> scraping /proc/cpuinfo, but I suspect that's not addressing the issue
> you have in mind.
> 
> > ijconsole not only fails that, but it also fails at an even more
> > basic task in Guix' launcher scripts, which is actually knowing the
> > thing you launch. (We hardcode everything, period.)
> 
> Okay. I see the logic in that.
> 
> The original patch indeed hard-coded paths in the ijconsole script,
> but that required having ijconsole be a wrapper package. I thought I
> was being clever by eliminating a "needless" wrapper, but it sounds
> like we can just bring it back.
As far as I can see, you still need a wrapper to provide a symlink
stripped of the version.  I think you can combine the CPU detection
code with that (see below), so you can have a wrapper procedure, which
strips the version LOGICAL OR detects the CPU.

> > Now I could just disable everything AVX-related in J and push
> > something that works on x86 and amd64, but since you do claim that
> > AVX is important to J, there are also other options one could take
> > here.  One is to implement tuning for this package the correct way,
> > though since it reinvents build systems, that's probably going to
> > be a hard one. The other would be to define package variants (e.g.
> > jsoftware-j-with-avx) and use those -- there ought to be an old
> > blog post on Guix HPC detailing the rationale behind doing that.
> 
> Not including AVX would be a hard hit.
> 
> Morally, the ijconsole script just wraps the libj*.so libs into a
> "fat binary".  From what I understand, fat binaries are (were?) an
> accepted solution in the HPC world. Heck, the libj*.so total up to a
> whopping 15Mb, which feels more like slim that fat, anyway.
> 
> Questions floating around in my head:
> 
> - What is wrong with detecting CPU features in a wrapper vs in the
> binary?
> - If a wrapper is theoretically okay, how can we fix the current one?
> - Is such a "proper fix" massively more subtle and bug-prone than the
> naive solution? Why?
FWIW I don't quite think that fat binaries are the issue here, but that
building AVX blows up the check phase in the way we currently do.  It's
a similar issue w.r.t. check? for cross-compiling.  In my opinion only
the base feature set can be checked unless the CPU supports the same
features as the target.

I think if we want to do fat binaries, the solution would be to build
all three variants from make-j and tests only the base variants, and
then have a non-substitutable wrapper package, that takes that as an
input and simply provides a wrapper tuned to the current CPU features.
If you want, you can then rerun the correct tests in the wrapper
package.  The package returned by make-j could itself also provide
three binaries just in case someone doesn't want to generate the
wrapper package, but knows they can run ijconsole-avx2 just fine. 
WDYT?

> > WDYT?  Is AVX worth the trouble?
> 
> Personally, I really do not like the jsoftware-j-with-* variants
> workaround.  If you say that tuning support is the only other
> acceptable solution, then I'll bite the bullet and see what is
> possible.
> 
> P.S. Just throwing this out there, but if you'd like, I'd be willing
> to do a mob session together. It seems like we are in compatible
> timezones.
I'm in UTC+1 and have a day job from 8am to 4pm (plus delta and bus
routes), which normally make me unable to reply from roughly 6:30am to
8pm.  I'm also a little shy when it comes to letting strangers hear my
voice and experiencing healthy vaccine side effects atm.

Cheers

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.