Clickable Area on UI buttons get shifted when starting game in aspect ratio that differs from canvas reference resolution

I've disabled the resolution dialogue, as I am letting the user change the resolution in game. I am then saving the users selected resolution with playerprefs.

My canvas scaler's reference resolution, as well as my playersettings default resolution, is set to 1920 x 1080, which is a 16:9 aspect ratio. As an example, when I try to set the resolution to 1680 x 1050, a 16:10 aspect ratio, it all loads fine and looks like it should, but all the UI buttons clickable area shifts to the left, off the actual object/sprite.

However, the area only shifts when it first loads the game. The buttons also work fine when fullscreen is not enabled. It starts working as it should if I manage to find the clickable area on the buttons to change the resolution, then change it back.
For example:

-The game loads the playerprefs and sets resolution to 1680 x 1050

-The clickable area has shifted

-Finding the buttons, I change it to 1920 x 1080

-The buttons work perfectly

• then change it back to 1680 x 1050

-The buttons work perfectly

This might be related to how the canvas scaler component works, as I came across this in the documentation:

If the current screen resolution has a different aspect ratio than the reference resolution, scaling each axis individually to fit the screen would result in non-uniform scaling, which is generally undesirable. Instead of this, the ReferenceResolution component will make the Canvas resolution deviate from the reference resolution in order to respect the aspect ratio of the screen. It is possible to control how this deviation should behave using the Screen Match Mode setting.

But I guess I don't fully understand what this is telling me, assuming it's even relevant to my problem. From everything I've seen and read, my UI should be behaving normally. I've changed the settings of Screen Match Mode and other settings, but no results so far.

Here are my canvas scaler settings:

Ui Scale Mode: Scale with screen size

Reference Resolution: 1920 x 1080

Screen Match Mode: Match Width Or Height

Match: 0.5

Reference Pixels... : 100

Any help would be greatly appreciated