chester's blog

technology, travel, comics, books, math, web, software and random thoughts

A workaround to fix the Firefox emoji keyboard shortcut on macOS Sonoma

18 Feb 2024

macOS 14 (Sonoma) broke the “Emoji & Symbols” keyboard shortcuts (fn/🌐+e or control+cmd+space) on Firefox: instead of opening, the emoji picker briefly flashes and disappears:

The "Emoji & Symbols" panel briefly flashes and disappears

If you use the “Edit… Emoji & Symbols” menu, the picker works - but it’s annoying to reach out for the mouse whenever you need an emoji or special character! I thought such an inconvenient bug would be fixed quickly on a minor Firefox or macOS update, but months passed and the bug was still there.

Between annoyed and curious, I dug the source code a bit and wrote a patch that fixes it, and also a secondary problem with the fn/🌐+e shortcut (introduced in Monterey as a replacement/alternative for control+cmd+space): it works sometimes, but when it does, it also writes the letter “e” where the cursor is, which is equally irksome.

For reasons that I will explain below, Mozilla did not accept the fix. They are working on another solution, but it will take a while to be released. Since many people have the problem right now, I decided to share some details about my fix here, alongside with instructions for applying the patch to the official Firefox source code or downloading my patched version - which I rebranded “EmojiFox” to avoid confusion and respect Mozilla’s trademarks/license.

Finding and fixing the bug

(skip to the next section if you are not interested in the programming details and just want a workaround for the bug)

One thing that caught my attention was that this bug also happened with Chrome after the macOS Sonoma update. They quickly produced a fix - a surprisingly short one:

     if (is_a_system_shortcut_event) {
       [[NSApp mainMenu] performKeyEquivalent:theEvent];
+
+      // Behavior changed in macOS Sonoma - now it's important we early-out
+      // rather than allow the code to reach
+      // _hostHelper->ForwardKeyboardEventWithCommands(). Go with the existing
+      // behavior for prior versions because we know it works for them.
+      if (base::mac::MacOSVersion() >= 14'00'00) {
+        _currentKeyDownCode.reset();
+        _host->EndKeyboardEvent();
+        return;
+      }

     } else {
       [self interpretKeyEvents:@[ theEvent ]];
     }

It gives some hints on the root cause: the app is forwarding the system shortcut event, instead of stopping it. It caused no harm before Sonoma, but after it, it results in the shortcut triggering twice, and the second trigger closes the picker. You can check it by pressing the shortcut twice in quick succession on any other (non-buggy) app: it cause the exact behavior of the bug!

With that in mind, I downloaded the Firefox source code and started poking at it. It’s a quite large (but well documented) codebase, but I didn’t need to understand it all - just had to find a suitable place to stop the event processing. Finding one, I produced a proof-of-concept, which crudely detected the key combinations and abruptly stopped processing:

if (((anEvent.modifierFlags & NSEventModifierFlagControl) &&
     (anEvent.modifierFlags & NSEventModifierFlagCommand) &&
     anEvent.keyCode == 49) ||
    ((anEvent.modifierFlags & NSEventModifierFlagFunction) &&
     anEvent.keyCode == 14)) {
  return;
}
break;

That worked, but wouldn’t be much useful because users can redefine the emoji keyboard shortcut, so the only way to figure out whether an event is the shortcut I want to prevent is to go through the menus and find one that matches the event keys and triggers the action of opening the emoji picker. Chrome already had code for that, so I had to add something equivalent to Firefox:

// Determines whether the key event matches the shortcut assigned to the Emoji &
// Symbols menu item, so we can avoid dispatching it (and closing the picker).
//
// It works by looking for a second-level menu item that triggers the picker AND
// matches the shortcut, skipping any top-level menus that don't contain picker
// triggers (avoiding potentially long menus such as Bookmarks).
//
// It handles fn-E (the standard shortcut), ^⌘Space (which appears in a hidden
// menu when the standard is not redefined) and custom shortcuts (created via
// System Settings > Keyboard > Keyboard Shortcuts > App Shortcuts), save for a
// few complex key combos that send incorrect modifiers.
static bool IsKeyEventEmojiAndSymbols(NSEvent* event, NSMenu* menu) {
  SEL targetAction = @selector(orderFrontCharacterPalette:);
  for (NSMenuItem* topLevelItem in menu.itemArray) {
    if (topLevelItem.hasSubmenu &&
        [topLevelItem.submenu indexOfItemWithTarget:nil
                                          andAction:targetAction] != -1) {
      for (NSMenuItem* item in topLevelItem.submenu.itemArray) {
        if (item.action == targetAction) {
          NSString* itemCharacters = [[item keyEquivalent] lowercaseString];
          NSUInteger itemModifiers = [item keyEquivalentModifierMask];
          NSString* eventCharacters =
              [[event charactersIgnoringModifiers] lowercaseString];
          NSUInteger eventModifiers =
              [event modifierFlags] &
              NSEventModifierFlagDeviceIndependentFlagsMask;

          if ([itemCharacters isEqualToString:eventCharacters] &&
              itemModifiers == eventModifiers) {
            return true;
          }
        }
      }
    }
  }
  return false;
}

(a nice bonus of doing that was discovering how Apple managed to replace control+cmd+space with fn/🌐+e on the menu, yet the old shortcut still worked: they keep two menu items for “Emoji & Symbols”, but the one linked to the control+cmd+space shortcut is hidden)

With that in place, the fix is as simple as Chrome’s:

// Early exit if Emoji & Symbols shortcut is pressed; fixes bug 1855346
// (first seen on macOS Sonoma) and bug 1833923 (first seen on Monterey)
if ((nsCocoaFeatures::OnMontereyOrLater()) &&
    IsKeyEventEmojiAndSymbols(aNativeEvent, [NSApp mainMenu])) {
  return currentKeyEvent->IsDefaultPrevented();
}

It turns out that doing this on the right place fixes not only the “flashing” bug, but the “e” one as well, because the later is also being processed twice, but fn/🌐 being such a special key on the Mac, sometimes the second occurrence loses the fn, acting as the e key being pressed right after the fn/🌐+e. That’s why I only apply the check to Monterey or later (after confirming that the “e” bug was introduced in Monterey, and the “flashing” one in Sonoma).

"Person with a hammer hitting a keyboard"

Of course it took me quite some time and learning to get there, and the true heroes are the Mozilla developers who kindly gave me feedback and pointed me towards the right direction at every step. At the end, we had a patch that fully fixed both problems, addressing all performance and compatibility concerns with the traversal.

However, those same Mozilla developers reasoned it would be better to prevent the event from trickling down at all instead of catching it on the TextInputHandler (and catch any system shortcut events, not just the emoji picker one, which would avoid the need to traverse menus altogether), and wrote a different patch in that direction.

I was happy with that: I learned a lot, helped raising awareness and researching towards the cleanest solution, and - most important - the bug would soon be fixed for good. But a couple months passed, Firefox got a few major version updates, yet the keyboard shortcut was still broken!

So I asked around, and it seems the cleaner patch fixes the “flashing” issue, but not the “e” bug. They are actively working on a second patch for that, and will release both together - which is technically the best approach, but will take a while to be available for Firefox + Mac users.

What are my options?

If you are affected by this bug, you can:

  1. Wait for the official fix. This is the simplest and safest option, but considering the release calendar, my best guess is that a fix won’t come before Firefox 125 (due April 16).

  2. Apply my patch to the Firefox source code and build it. This is also very safe: you don’t need to trust anyone but Mozilla, since you are using their code (which you already trust as a Firefox user) and my patch (which is public and you can review). But it requires familiarity with the command line and a lot of time to compile the browser.

  3. Download and install EmojiFox, that is, Firefox Nightly/Unofficial with my patch applied. The downsides: you have to trust me and it won’t auto-update - you should throw it away as soon as the official fix is released.

Applying the patch

First step is to download and build Firefox on macOS by following the official instructions. Just keep in mind that:

  • You should not select artifact builds when asked, as they are based on pre-built binaries and you need to build the binaries yourself.
  • Before running ./mach build, you should add ac_add_options --enable-release to your .mozconfig file (and comment out any debug-related options, or pretty much anything but this one)

./mach build takes a few hours, even on a beefy machine. Once you are running Nightly, confirm the bug is still there, then apply the patch with:

curl -L https://phabricator.services.mozilla.com/D193328?download=true | patch

and ./mach build again (don’t worry, it will be much faster, since it only recompiles the files that changed), then ./mach run again. You should see the bug fixed now:

"Emoji & Symbols" shortuct now opens the full or contextual panel

You are not done yet - even though there is a Nightly.app that you can copy, it will be bound to assets that will disappear once you clean up your building environment. So now run:

./mach package

That will generate a .dmg file in your obj-x86_64-apple-darwin23.1.0/dist folder (or obj-arm64/dist if you are using an Apple Silicon Mac). That .dmg contains a Nightly.app that you can copy to your Applications folder and use as your main browser.

Downloading EmojiFox

I have been using Firefox 121 with this path since December, and recently re-applied it to the nightly build of Firefox 124. You can download this patched version (which I rebranded as “EmojiFox”) as long as you keep in mind that:

  • I (Chester) do not represent Mozilla, and this is not a Mozilla/Firefox official release.
  • This software will not auto-update, and I don’t intend to release new versions (by the time it gets old, we should have the proper fix on official Firefox). It’s a workaround that you should throw away and go back to default Firefox as soon as a fixed version is released. Always keep your browser updated!
  • Neither I, nor Mozilla, nor anyone is responsible for any damage caused by the patched version.
  • Per Mozilla Public License terms, my source code changes are available here (raw diff).

Open the file and drag EmojiFox to your Applications folder, as usual. But when you run the app, it will say it’s from an unidentified developer (I currently don’t do enough macOS development to justify the yearly USD 99 for an Apple Developer Program membership that would allow me to sign it).

Dialog saying EmojiFox cannot be opened because the developer cannot be verified, with no option to override

The trick here is to hold control while clicking the app icon, then select “Open” from the context menu. It will also show a warning, but now there will be an Open button that will open the app. You only need to do this once - from now on, it will open normally.

Aslo a dialog saying EmojiFox cannot be opened because the developer cannot be verified, with but now it has an Open button

Verify that both the control+cmd+space and fn/🌐+e shortcuts work as expected.

"Emoji & Symbols" shortuct on EmojiFox

Use your existing profile

Regardless of whether you built or downloaded your fixed browser, you will want to bring your existing bookmarks, history, extensions, etc. to it, and the easy way to do that is to configure it to use your existing profile (instead of the one it just created).

To do so, ensure the original Firefox is closed, and type about:profiles at the new browser’s address bar. Click Set as default profile on your Firefox profile, restart the browser and you should see your bookmarks, history, extensions, etc.

(if you have multiple profiles and don’t know which is the right one, just try Launch profile in new browser on any candidates)

Final considerations

The only issue I had with this build so far: unlocking the 1Password extension doesn’t work with Touch ID (I suppose it’s because it isn’t an official release); you can still unlock with your password, and/or use Touch ID on the 1Password application. It’s a minor trade-off for people who, like myself, type emoji much more often than log on to websites.

If you try any of these solutions, please let me know in the comments below whether it worked for you or not. And let’s 🤞 for Mozilla to release the official fix soon (they are working on it) so we don’t need these workarounds anymore!