I figured out this issue, it was due to my custom font configuration in ~/.config/fontconfig/fonts.conf'. When deleting that file, emojis display and I can also set the face- and frame-fonts to emoji fonts. Probably it's just broken, but I think it's still possible that it's formally correct, but the hinting and aliasing rules that it specifies cause some bug down the line. Feel free to close this, though if anyone is interested, I attached the file.

I also tried applying the suggested patch from bug#57066, but saw no difference: Without the custom fonts.conf the emoji display works well whether the patch is applied or not . With the custom fonts.conf the patch doesn't fix the issue.

Thanks so much for the help Eli! I'm glad to have this finally solved as I'm seeing more and more emacs users and packages starting to use unicode emoji.

Cheers, Michael