koobas.hobune.stream
Fast Game Simulation for Reinforcement Learning
Hi,
First of all let me explain that I am aware that Unity offers some libraries related with RL, but they are oriented to agents, while the approach that I am looking for is completely different.
I am developing a strategic AI for a RTS. This strategic AI considers a bunch of parameters to weight its goals and send orders to the units. I would like to optimize this weighting by using Neural Networks with Reinforcement Learning. However, to achieve this I need to make thousands of simulations of the game and Unity is not designing thinking in something like that.
However, I wonder how can I speed up the game to work much faster without breaking it. I have though about increasing a lot the TimeScale but it has some incoveniences. Because it makes actions go faster, it results in less FPS per action, with can be a huge loss in precision. If for example a unit is moving towards a position it is not the same to do it in 500 frames than in 5, because in the middle of the path there might be some considerations.
To speedup I can only think about to prescind of everything related to rendering, but it is not enough.
Has anyone tried to do anything like this with Unity?
This is the approach that algorithms like AlphaGo follow, but as you know simulating Go is way easier. Yes, there is this thing of AlphaStar, so I wonder how many programmers where creating the simulation world.
Thanks!
as you mentioned, visual rendering is typically by far the highest drain on a machines proccesing power. by simply turning the renderes OFF on all the game objects in your scene you could come up with much more preficiency. your coding is another thing, i dont know how complex your game is or what proficiency shortcuts you could take for training. hopefully you have considered running multiple game copies running on a CPU with lots of cores since the video card ability wouldnt matter without renering.
also if you have a user base consider getting background input from them from a server.
keep us informed on your findings. i assume this is a new topic that probobly doesnt have many googleable answers yet. good luck to you
I believe there is also potential to set the project up as a server side application in order to ignore the visuals
Yes, you need to build your project with Server Build option. Completely turns off all rendering overhead.
A couple options come to mind. You could have multiple simulations and train the network in parallel. You could also move the frame specific logic into FixedUpdate and crank the timescale.
