Thanks, Pip! Really appreciate you putting so much effort into this. See
below for what you asked for.

George

It's a good idea to use "gcore" to generate a core file from gdb, just
> in case the session is destroyed.


Done.

Can you check whether eln files were created just before the crash?
> Over here, the relevant command would be
>
> $ ls -Strl ~/.emacs.d/eln-cache/*/*
>
> and looking at the last few lines to see whether any files have the
> creation date in the right range (after you started using eshell but
> before the crash happened).
>

Nothing in the directory (
/u/panagopo/.config/emacs/.local/cache/eln/30.1-1ed0c1e8 for me) that is on
the same day of the crash. In fact, eshell isn't there at all, which
probably makes sense as I am using the built-in version, so it was already
natively-compiled.

  So we first should inspect the memory around 0x196922b0 to find out
> whether it looks like a valid vector block, and whether the bad word was
> in this block or in the string object marked just before.  I'd suggest
> running
>
>     x/32gx 0x196922b0
>
> to look at the memory following the pointer, and
>
>     x/32gx 0x19692200
>
> to get some idea of whether it might be the middle of a vectorlike.


(gdb)   x/32gx 0x196922b0
0x196922b0: 0xc00000001f000005 0x0000000000000406
0x196922c0: 0x000015554efc8fac 0x000000000ffdf6e1
0x196922d0: 0x0000000000000016 0x000015554efc8f74
0x196922e0: 0xc00000001200a000 0x00001555386f8c00
0x196922f0: 0x0000000000000000 0x0000000074e43510
0x19692300: 0x0000000000000000 0x0000000000000000
0x19692310: 0x000000000000000b 0x00000000556ee565
0x19692320: 0x000000007a4401e0 0x0000000000000000
0x19692330: 0x000000005d1f2243 0xc000000003005000
0x19692340: 0x00000000134314f0 0x0000000000000000
0x19692350: 0x000000001965e118 0x00000000000026ba
0x19692360: 0x00000000000026ba 0x8000000000000004
0x19692370: 0x0000000000000f26 0x0000000000000c4e
0x19692380: 0x0000000000000c02 0x0000000000000d16
0x19692390: 0xc000000003005000 0x000015554ef1fbb0
0x196923a0: 0x00000000012d6144 0x0000000019e14120
(gdb)  x/32gx 0x19692200
0x19692200: 0x0000000000000000 0x0000000000019082
0x19692210: 0x0000000000019102 0x000000000000004e
0x19692220: 0x0000000000000000 0x0000000000000192
0x19692230: 0x0000000000000002 0x0000000000000000
0x19692240: 0x400000002300000d 0x0000000000000000
0x19692250: 0x000000000544a480 0x00000000019e1500
0x19692260: 0x0000000000000000 0x000000000000b640
0x19692270: 0x0000000000000000 0x0000000000019082
0x19692280: 0x0000000000019102 0x000000000000004e
0x19692290: 0x0000000000000000 0x0000000000000192
0x196922a0: 0x0000000000000002 0x0000000000000000
0x196922b0: 0xc00000001f000005 0x0000000000000406
0x196922c0: 0x000015554efc8fac 0x000000000ffdf6e1
0x196922d0: 0x0000000000000016 0x000015554efc8f74
0x196922e0: 0xc00000001200a000 0x00001555386f8c00
0x196922f0: 0x0000000000000000 0x0000000074e43510


You could try running
>
>     p mem_find (0x196922b5)
>     p *$
>
> to see where the vector block (if it is one) was supposed to start.
> (There's a slight chance this will crash). It may be worth it to print
> the entire vector block, which should be doable by running
>
>     x/512gx $.start
>
> after that command.
>
>

 (gdb) p mem_find (0x196922b5)
[LWP 3311069 exited]
$7 = (struct mem_node *) 0x18f97010
(gdb) p *$
$8 = {left = 0x18f96fd0, right = 0x18f97050, parent = 0x18f96f90, start =
0x196915a0, end = 0x19692598, color = MEM_BLACK, type =
MEM_TYPE_VECTOR_BLOCK}
(gdb) x/512gx $.start
0x196915a0: 0xc000000021000081 0x0000068000000003
0x196915b0: 0x000000000000158a 0x000000000000158e
0x196915c0: 0x0000000000001592 0x0000000000001596
0x196915d0: 0x000000000000159a 0x000000000000159e
0x196915e0: 0x00000000000015a2 0x00000000000015a6
0x196915f0: 0x00000000000015aa 0x00000000000015ae
0x19691600: 0x00000000000015b2 0x00000000000015b6
0x19691610: 0x00000000000015ba 0x00000000000015be
0x19691620: 0x00000000000015c2 0x00000000000015c6
0x19691630: 0x00000000000015ca 0x00000000000015ce
0x19691640: 0x00000000000015d2 0x00000000000015d6
0x19691650: 0x00000000000015da 0x00000000000015de
0x19691660: 0x00000000000015e2 0x00000000000015e6
0x19691670: 0x00000000000015ea 0x00000000000015ee
0x19691680: 0x00000000000015f2 0x00000000000015f6
0x19691690: 0x00000000000015fa 0x00000000000015fe
0x196916a0: 0x0000000000001602 0x0000000000001606
0x196916b0: 0x000000000000160a 0x000000000000160e
0x196916c0: 0x0000000000001612 0x0000000000001616
0x196916d0: 0x000000000000161a 0x000000000000161e
0x196916e0: 0x0000000000001622 0x0000000000001626
0x196916f0: 0x000000000000162a 0x000000000000162e
0x19691700: 0x0000000000001632 0x0000000000001636
0x19691710: 0x000000000000163a 0x000000000000163e
0x19691720: 0x0000000000001642 0x0000000000001646
0x19691730: 0x000000000000164a 0x000000000000164e
0x19691740: 0x0000000000001652 0x0000000000001656
0x19691750: 0x000000000000165a 0x000000000000165e
0x19691760: 0x0000000000001662 0x0000000000001666
0x19691770: 0x000000000000166a 0x000000000000166e
0x19691780: 0x0000000000001672 0x0000000000001676
0x19691790: 0x000000000000167a 0x000000000000167e
0x196917a0: 0x0000000000001682 0x0000000000001686
0x196917b0: 0x000000000000168a 0x000000000000168e
0x196917c0: 0x0000000000001692 0x0000000000001696
0x196917d0: 0x000000000000169a 0x000000000000169e
0x196917e0: 0x00000000000016a2 0x00000000000016a6
0x196917f0: 0x00000000000016aa 0x00000000000016ae
0x19691800: 0x00000000000016b2 0x00000000000016b6
0x19691810: 0x00000000000016ba 0x00000000000016be
0x19691820: 0x00000000000016c2 0x00000000000016c6
0x19691830: 0x00000000000016ca 0x00000000000016ce
0x19691840: 0x00000000000016d2 0x00000000000016d6
0x19691850: 0x00000000000016da 0x00000000000016de
0x19691860: 0x00000000000016e2 0x00000000000016e6
0x19691870: 0x00000000000016ea 0x00000000000016ee
0x19691880: 0x00000000000016f2 0x00000000000016f6
0x19691890: 0x00000000000016fa 0x00000000000016fe
0x196918a0: 0x0000000000001702 0x0000000000001706
0x196918b0: 0x000000000000170a 0x000000000000170e
0x196918c0: 0x0000000000001712 0x0000000000001716
0x196918d0: 0x000000000000171a 0x000000000000171e
0x196918e0: 0x0000000000001722 0x0000000000001726
--Type <RET> for more, q to quit, c to continue without paging--c
0x196918f0: 0x000000000000172a 0x000000000000172e
0x19691900: 0x0000000000001732 0x0000000000001736
0x19691910: 0x000000000000173a 0x000000000000173e
0x19691920: 0x0000000000001742 0x0000000000001746
0x19691930: 0x000000000000174a 0x000000000000174e
0x19691940: 0x0000000000001752 0x0000000000001756
0x19691950: 0x000000000000175a 0x000000000000175e
0x19691960: 0x0000000000001762 0x0000000000001766
0x19691970: 0x000000000000176a 0x000000000000176e
0x19691980: 0x0000000000001772 0x0000000000001776
0x19691990: 0x000000000000177a 0x000000000000177e
0x196919a0: 0x0000000000001782 0x0000000000001786
0x196919b0: 0xc000000021000081 0x0000070000000003
0x196919c0: 0x000000000000178a 0x000000000000178e
0x196919d0: 0x0000000000001792 0x0000000000001796
0x196919e0: 0x000000000000179a 0x000000000000179e
0x196919f0: 0x00000000000017a2 0x00000000000017a6
0x19691a00: 0x00000000000017aa 0x00000000000017ae
0x19691a10: 0x00000000000017b2 0x00000000000017b6
0x19691a20: 0x00000000000017ba 0x00000000000017be
0x19691a30: 0x00000000000017c2 0x00000000000017c6
0x19691a40: 0x00000000000017ca 0x00000000000017ce
0x19691a50: 0x00000000000017d2 0x00000000000017d6
0x19691a60: 0x00000000000017da 0x00000000000017de
0x19691a70: 0x00000000000017e2 0x00000000000017e6
0x19691a80: 0x00000000000017ea 0x00000000000017ee
0x19691a90: 0x00000000000017f2 0x00000000000017f6
0x19691aa0: 0x00000000000017fa 0x00000000000017fe
0x19691ab0: 0x0000000000001802 0x0000000000001806
0x19691ac0: 0x000000000000180a 0x000000000000180e
0x19691ad0: 0x0000000000001812 0x0000000000001816
0x19691ae0: 0x000000000000181a 0x000000000000181e
0x19691af0: 0x0000000000001822 0x0000000000001826
0x19691b00: 0x000000000000182a 0x000000000000182e
0x19691b10: 0x0000000000001832 0x0000000000001836
0x19691b20: 0x000000000000183a 0x000000000000183e
0x19691b30: 0x0000000000001842 0x0000000000001846
0x19691b40: 0x000000000000184a 0x000000000000184e
0x19691b50: 0x0000000000001852 0x0000000000001856
0x19691b60: 0x000000000000185a 0x000000000000185e
0x19691b70: 0x0000000000001862 0x0000000000001866
0x19691b80: 0x000000000000186a 0x000000000000186e
0x19691b90: 0x0000000000001872 0x0000000000001876
0x19691ba0: 0x000000000000187a 0x000000000000187e
0x19691bb0: 0x0000000000001882 0x0000000000001886
0x19691bc0: 0x000000000000188a 0x000000000000188e
0x19691bd0: 0x0000000000001892 0x0000000000001896
0x19691be0: 0x000000000000189a 0x000000000000189e
0x19691bf0: 0x00000000000018a2 0x00000000000018a6
0x19691c00: 0x00000000000018aa 0x00000000000018ae
0x19691c10: 0x00000000000018b2 0x00000000000018b6
0x19691c20: 0x00000000000018ba 0x00000000000018be
0x19691c30: 0x00000000000018c2 0x00000000000018c6
0x19691c40: 0x00000000000018ca 0x00000000000018ce
0x19691c50: 0x00000000000018d2 0x00000000000018d6
0x19691c60: 0x00000000000018da 0x00000000000018de
0x19691c70: 0x00000000000018e2 0x00000000000018e6
0x19691c80: 0x00000000000018ea 0x00000000000018ee
0x19691c90: 0x00000000000018f2 0x00000000000018f6
0x19691ca0: 0x00000000000018fa 0x00000000000018fe
0x19691cb0: 0x0000000000001902 0x0000000000001906
0x19691cc0: 0x000000000000190a 0x000000000000190e
0x19691cd0: 0x0000000000001912 0x0000000000001916
0x19691ce0: 0x000000000000191a 0x000000000000191e
0x19691cf0: 0x0000000000001922 0x0000000000001926
0x19691d00: 0x000000000000192a 0x000000000000192e
0x19691d10: 0x0000000000001932 0x0000000000001936
0x19691d20: 0x000000000000193a 0x000000000000193e
0x19691d30: 0x0000000000001942 0x0000000000001946
0x19691d40: 0x000000000000194a 0x000000000000194e
0x19691d50: 0x0000000000001952 0x0000000000001956
0x19691d60: 0x000000000000195a 0x000000000000195e
0x19691d70: 0x0000000000001962 0x0000000000001966
0x19691d80: 0x000000000000196a 0x000000000000196e
0x19691d90: 0x0000000000001972 0x0000000000001976
0x19691da0: 0x000000000000197a 0x000000000000197e
0x19691db0: 0x0000000000001982 0x0000000000001986
0x19691dc0: 0xc000000021000081 0x0000078000000003
0x19691dd0: 0x000000000000198a 0x000000000000198e
0x19691de0: 0x0000000000001992 0x0000000000001996
0x19691df0: 0x000000000000199a 0x000000000000199e
0x19691e00: 0x00000000000019a2 0x00000000000019a6
0x19691e10: 0x00000000000019aa 0x00000000000019ae
0x19691e20: 0x00000000000019b2 0x00000000000019b6
0x19691e30: 0x00000000000019ba 0x00000000000019be
0x19691e40: 0x00000000000019c2 0x00000000000019c6
0x19691e50: 0x00000000000019ca 0x00000000000019ce
0x19691e60: 0x00000000000019d2 0x00000000000019d6
0x19691e70: 0x00000000000019da 0x00000000000019de
0x19691e80: 0x00000000000019e2 0x00000000000019e6
0x19691e90: 0x00000000000019ea 0x00000000000019ee
0x19691ea0: 0x00000000000019f2 0x00000000000019f6
0x19691eb0: 0x00000000000019fa 0x00000000000019fe
0x19691ec0: 0x0000000000001a02 0x0000000000001a06
0x19691ed0: 0x0000000000001a0a 0x0000000000001a0e
0x19691ee0: 0x0000000000001a12 0x0000000000001a16
0x19691ef0: 0x0000000000001a1a 0x0000000000001a1e
0x19691f00: 0x0000000000001a22 0x0000000000001a26
0x19691f10: 0x0000000000001a2a 0x0000000000001a2e
0x19691f20: 0x0000000000001a32 0x0000000000001a36
0x19691f30: 0x0000000000001a3a 0x0000000000001a3e
0x19691f40: 0x0000000000001a42 0x0000000000001a46
0x19691f50: 0x0000000000001a4a 0x0000000000001a4e
0x19691f60: 0x0000000000001a52 0x0000000000001a56
0x19691f70: 0x0000000000001a5a 0x0000000000001a5e
0x19691f80: 0x0000000000001a62 0x0000000000001a66
0x19691f90: 0x0000000000001a6a 0x0000000000001a6e
0x19691fa0: 0x0000000000001a72 0x0000000000001a76
0x19691fb0: 0x0000000000001a7a 0x0000000000001a7e
0x19691fc0: 0x0000000000001a82 0x0000000000001a86
0x19691fd0: 0x0000000000001a8a 0x0000000000001a8e
0x19691fe0: 0x0000000000001a92 0x0000000000001a96
0x19691ff0: 0x0000000000001a9a 0x0000000000001a9e
0x19692000: 0x0000000000001aa2 0x0000000000001aa6
0x19692010: 0x0000000000001aaa 0x0000000000001aae
0x19692020: 0x0000000000001ab2 0x0000000000001ab6
0x19692030: 0x0000000000001aba 0x0000000000001abe
0x19692040: 0x0000000000001ac2 0x0000000000001ac6
0x19692050: 0x0000000000001aca 0x0000000000001ace
0x19692060: 0x0000000000001ad2 0x0000000000001ad6
0x19692070: 0x0000000000001ada 0x0000000000001ade
0x19692080: 0x0000000000001ae2 0x0000000000001ae6
0x19692090: 0x0000000000001aea 0x0000000000001aee
0x196920a0: 0x0000000000001af2 0x0000000000001af6
0x196920b0: 0x0000000000001afa 0x0000000000001afe
0x196920c0: 0x0000000000001b02 0x0000000000001b06
0x196920d0: 0x0000000000001b0a 0x0000000000001b0e
0x196920e0: 0x0000000000001b12 0x0000000000001b16
0x196920f0: 0x0000000000001b1a 0x0000000000001b1e
0x19692100: 0x0000000000001b22 0x0000000000001b26
0x19692110: 0x0000000000001b2a 0x0000000000001b2e
0x19692120: 0x0000000000001b32 0x0000000000001b36
0x19692130: 0x0000000000001b3a 0x0000000000001b3e
0x19692140: 0x0000000000001b42 0x0000000000001b46
0x19692150: 0x0000000000001b4a 0x0000000000001b4e
0x19692160: 0x0000000000001b52 0x0000000000001b56
0x19692170: 0x0000000000001b5a 0x0000000000001b5e
0x19692180: 0x0000000000001b62 0x0000000000001b66
0x19692190: 0x0000000000001b6a 0x0000000000001b6e
0x196921a0: 0x0000000000001b72 0x0000000000001b76
0x196921b0: 0x0000000000001b7a 0x0000000000001b7e
0x196921c0: 0x0000000000001b82 0x0000000000001b86
0x196921d0: 0x400000002300000d 0x0000000000000000
0x196921e0: 0x000000000544a480 0x00000000019e1500
0x196921f0: 0x0000000000000000 0x000000000000b640
0x19692200: 0x0000000000000000 0x0000000000019082
0x19692210: 0x0000000000019102 0x000000000000004e
0x19692220: 0x0000000000000000 0x0000000000000192
0x19692230: 0x0000000000000002 0x0000000000000000
0x19692240: 0x400000002300000d 0x0000000000000000
0x19692250: 0x000000000544a480 0x00000000019e1500
0x19692260: 0x0000000000000000 0x000000000000b640
0x19692270: 0x0000000000000000 0x0000000000019082
0x19692280: 0x0000000000019102 0x000000000000004e
0x19692290: 0x0000000000000000 0x0000000000000192
0x196922a0: 0x0000000000000002 0x0000000000000000
0x196922b0: 0xc00000001f000005 0x0000000000000406
0x196922c0: 0x000015554efc8fac 0x000000000ffdf6e1
0x196922d0: 0x0000000000000016 0x000015554efc8f74
0x196922e0: 0xc00000001200a000 0x00001555386f8c00
0x196922f0: 0x0000000000000000 0x0000000074e43510
0x19692300: 0x0000000000000000 0x0000000000000000
0x19692310: 0x000000000000000b 0x00000000556ee565
0x19692320: 0x000000007a4401e0 0x0000000000000000
0x19692330: 0x000000005d1f2243 0xc000000003005000
0x19692340: 0x00000000134314f0 0x0000000000000000
0x19692350: 0x000000001965e118 0x00000000000026ba
0x19692360: 0x00000000000026ba 0x8000000000000004
0x19692370: 0x0000000000000f26 0x0000000000000c4e
0x19692380: 0x0000000000000c02 0x0000000000000d16
0x19692390: 0xc000000003005000 0x000015554ef1fbb0
0x196923a0: 0x00000000012d6144 0x0000000019e14120
0x196923b0: 0x0000000000000094 0x0000000000000098
0x196923c0: 0x8000000000000001 0x00000000000018ea
0x196923d0: 0xc00000001a003007 0x000000001cfbfe44
0x196923e0: 0x000000001cf21683 0x000000001969242d
0x196923f0: 0x0000000019692475 0x00000000444a49c5
0x19692400: 0x000000001cfc4b65 0x000000001969c275
0x19692410: 0x000015552436e9e0 0x0000000000000000
0x19692420: 0x000000001cfc02a0 0xc00000000e008000
0x19692430: 0x000000001cfd6320 0x000000001cfd6190
0x19692440: 0x000000001cfd5b80 0x00000000006e9960
0x19692450: 0x000000001cfd59f0 0x0000001c0000001c
0x19692460: 0x0000400700000060 0x0000000000000000
0x19692470: 0xc00000000e008000 0x000000001cfd6cd0
0x19692480: 0x000000001cfd6b40 0x000000001cfd6530
0x19692490: 0x00000000006e98e0 0x000000001cfd4bd0
0x196924a0: 0x0000001c0000001c 0x0000400700000060
0x196924b0: 0x0000000000000000 0xc00000001f000005
0x196924c0: 0x0000000000000606 0x000000001cfc0004
0x196924d0: 0x0000000005d42ca5 0x000000000000000e
0x196924e0: 0x000000001cfbffe4 0x800000000000000e
0x196924f0: 0x000015554de5d450 0x000015554de5d4a0
0x19692500: 0x000015554df89888 0x000015554df91ec8
0x19692510: 0x000015554dfd0818 0x0000000000003e10
0x19692520: 0x000015554de35288 0x000015554de35258
0x19692530: 0x0000000000000006 0x0000000000003de0
0x19692540: 0x000000001968c50d 0x00000000196924bd
0x19692550: 0x0000000000000000 0x000015554de34540
0x19692560: 0xc00000001f000004 0x0000000000000002
0x19692570: 0x000000001e807624 0x00000000028d4475
0x19692580: 0x0000000000000006 0x8000000000000001
0x19692590: 0x000015554e3190c8 0x0000000019690590

This means that 0x15554efc8fac is most likely the string containing the
> bytecode; could you please run
>
>    p *(struct Lisp_String *)0x15554efc8fa8
>
> to see whether this is true, and whether any data is still there?
>
> It would be interesting to know whether we were marking the string or
> the bytecode object when the crash happened; please run
>
>    p mark_stk
>    p mark_stk.stack[0]
>    p mark_stk.stack[1]
>    p mark_stk.stack[2]
>
> so we see how deep in the mark stack we were at the time.
>

(gdb) p *(struct Lisp_String *)0x15554efc8fa8
$9 = {u = {s = {size = 8, size_byte = -3, intervals = 0x0, data =
0x15554f6b661a "\300"...}, next = 0x8, gcaligned = 8 '\b'}}
(gdb)    p mark_stk
$10 = {stack = 0x155539361010, size = 12288, sp = 1}
(gdb)    p mark_stk.stack[0]
$11 = {n = 2, u = {value = 0x196922d0, values = 0x196922d0}}
(gdb)    p mark_stk.stack[1]
$12 = {n = 0, u = {value = 0x6851e793, values = 0x6851e793}}
(gdb)    p mark_stk.stack[2]
$13 = {n = 0, u = {value = 0x160c2d5, values = 0x160c2d5}}