Eli's suggestion looks good. I tweaked his patch slightly to avoid some calls and installed the attached into the emacs-26 branch to implement it. Please give it a try.