Editor: How to display properties in inspector based on enum in same script ?

First of all you should understand that a custom inspector can only change how the properties are displayed. They all still need to exist in the actual class. In your case you can reuse certain variables since only one "type" is valid at a time.

For example:

 public class MyTest : MonoBehavioud
 {
     public PlayerType type;
     public string objName;  // shared between type X, Y and Z
     public GameObject addon; // shared between type X and Y
     public ZBehaviour zType;
 }

In the custom inspector you then can show different GUI depending on the set type:

 [CustomEditor(typeof(MyTest))]
 public class MyTestEditor : Editor
 {
     MyTest mTest;
     void OnEnable()
     {
         mTest = (MyTest)target;
     }
     public override void OnInspectorGUI()
     {
         mTest.type = (PlayerType)EditorGUILayout.EnumPopup("PlayerType", mTest.type);
         switch(mTest.type)
         {
             case PlayerType.X:
                 {
                     mTest.objName = EditorGUILayout.TextField("Name", mTest.objName);
                     mTest.addon = (GameObject)EditorGUILayout.ObjectField("Addon", mTest.addon, typeof(GameObject), true);
                     break;
                 }
             case PlayerType.Y:
                 {
                     mTest.objName = EditorGUILayout.TextField("Name", mTest.objName);
                     mTest.addon = (GameObject)EditorGUILayout.ObjectField("Addon", mTest.addon, typeof(GameObject), true);
                     break;
                 }
             case PlayerType.Z:
                 {
                     mTest.objName = EditorGUILayout.TextField("Name", mTest.objName);
                     mTest.zType = (ZBehaviour)EditorGUILayout.EnumPopup("ZType", mTest.zType);
                     break;
                 }
         }
     }
 }

Note that you could use seperate variables for your name and addon variables for each type but it wouldn't make much sense if you want to switch between the different types and only one type is in use at a time.

ps: Keep in mind that a MonoBehaviour already has a "name" property so i renamed the variable to "objName".