Rotating an object in a specific direction to a specific point

I have a ring menu. I got the separate icons to disperse evenly by calculating the degrees of space between each icon based on the number of icons. That part works fine. I need it so when I hit up or down the menu parent will rotate on the X axis to the next/previous icon.

I already did the work to figure out what the next/previous icon's index is, including looping (if I hit down from the first icon it will go to the last icon and get the proper index). The problem is the rotation.

I can get it to go to the proper icon just fine (I think with a different version of what I posted), but it just finds the shortest way to get there basically. I know the theory behind it--I can just divide my current X rotation by 360 to find out which 'interval' of degrees I've done, then just add or subtract from there so that it will go up/down (depending on which input was pressed) to that icon. The problem is that rotations work super funky. Even the Euler angles start putting stuff into the Y and Z.

It's just very frustrating because I've tried SO many different methods, and I KNOW the theory behind what I need to do, but the way rotations work are just making something so simple into something impossible for me to do.

I've considered using RotateToward because that basically does what I need it to do...the problem again is that I can't really tell it when to stop based on the X rotation value (which I know) because the X value isn't consistent because of how rotations work.

Please help.

 public IEnumerator CycleSkillsDown (int i) {
         float r = 0;
         bool looped = false;

         if (i == skillIcons.Count-1) {  // went around from 0
             r = -skillsCycleDegrees;
             looped = true;
         } else {
             r = (float)i * skillsCycleDegrees;
         }

         Quaternion rot = Quaternion.AngleAxis (r, Vector3.right);
         float t = 0;
         Quaternion startRot = skillRotator.rotation;

         while (t < 1) {
             t += Time.deltaTime * 3;
             skillRotator.transform.rotation = Quaternion.Lerp (startRot, rot, t);
             yield return new WaitForEndOfFrame ();
         }
     }