koobas.hobune.stream
Not getting observations in python with ML agents
I've been playing around with creating Unity Machine Learning environments to use with my python neural net code. Using basic state info and actions I've had good success. However, I am now trying to use camera observations as state information for my ml code. I added a camera to the scene, added it to the agent, in the inspector, under observations. I also added a resolution to my brain object for that agent. I run the environment from python using env = UnityEnvironment(file_name=filename, worker_id=0). However when I use env.brains["Brain"] to get the info from my brain object it returns "Number of observations (per agent): 0". Also the "observations" list from the BrainInfo object seems to be empty.
There isn't much documentation on how to use observations for external python brains so I was wondering what am I doing wrong? What I would like is to have "observations" return a matrix of pixel data from the camera. I would just render the camera to a texture then sample it for the matrix but that hardly seems like a fast option, and I'd prefer to use the built in method for observations if that is what it is there for.
Hey @alphaspec,
I am just now looking at trying to do the same thing, getting Python into the ml-agent mix, and even trying to leverage a interior python-interpreter client with a websocket connection to an exterior python server aka [https://hackernoon.com/user-interfaces-with-unity3d-and-python-eb2e7744518a?gi=72251ef6af26].
Perhaps an answer can be found in the Issues for the project ? https://github.com/Unity-Technologies/ml-agents/issues
-ll3v3ll-
