Question by dabigmahmoud · Oct 12, 2021 at 01:03 PM · c#3d ai

ML Agents Ends Episode Every Couple Of Seconds For No Reason.

Hello, I'm not good at coding but I decided to learn ml agents for fun, I was playing around with it and made a simple project, when testing the project for some reason the episode ends every couple of seconds resetting my agent to the start so it doesn't learn anything.

the idea of this project is that the agent has to move around and grab the sphere that gives it the most rewards, there are four spheres and the bigger the sphere the more rewards it gives. the agents moves around looking for rewards but it resets it's position cause the episode ends for some strange reason.

here is some code

Collecting Observations (The Sphere Scale)

 public override void CollectObservations(VectorSensor sensor)
     {
 
         sensor.AddObservation(transform.localPosition);
 
         foreach(Transform sphereLoc in scoreSpheres)
     {

         Transform sphere = sphereLoc.GetChild(0);

         sensor.AddObservation(sphere.localScale);
         sensor.AddObservation(sphere.localPosition);
     }
 
     }

OnEpisodeBegin (resetting the agent's position)

 public override void OnEpisodeBegin()
     {
 
         transform.localPosition = agentStart.localPosition;
         scoreSpawner.ResetScore();
 
     }

Giving Rewards or Penalties

 private void OnCollisionEnter(Collision collision)
     {
 
         if (collision.transform.tag == "Wall")
         {
 
             SetReward(-1);
             EndEpisode();
 
         }
 
         if (collision.transform.tag == "Score")
         {
             Score scoreSphere = collision.gameObject.GetComponent<Score>();
 
             SetReward(scoreSphere.score);
             EndEpisode();
 
         }
 
     }

And the rest of the code is just the movement but I don't believe that's relevant . Thanks in advance for any answers :D

EDIT:

I made some changes, first I fixed it so it gets the scale and position of the spheres and not their parents, that did not fix the issue.

also I added log messages to EndEpisode functions like user @Captain_Pineapple suggested and the strange thing is that it never hits the debug messages, but it only hits the debug message on OnEpisodeBeing which means the it ends the episode somewhere else however the EndEpisode functions are only called in the OnCollisionEnter so I have no idea how the episode ends.

Add comment · Show 1