koobas.hobune.stream
Visual Part Of A Neural Network
What would be the best way of taking a certain camera in the scene, down sampling it by a certain amount, then somehow reading it in another script as inputs for a neural network? I would like this to be colored as to distinguish objects. I have already made the neural network part, the inputs, outputs and hidden layers. Its just I am using either ray casts or just plain distance and angles to tell the neural network where it is. I am wanting to go a step further to use cameras as input. Its just I am not good with anything of the sort. Would anyone care to help?
Thank You! Aiden
Add a camera as a child of your main camera. This new camera should then follow wherever your main camera goes. Set the target texture of this camera to a RenderTexture (Learn more about those here: https://docs.unity3d.com/ScriptReference/RenderTexture.html). The data in RenderTexture will then be the output of the camera / input into your NN. Good luck! I'd like to hear how it goes.
Image processing can be quite complicated and slow as you're dealing with a lot of information. It's not really clear what you want to do exactly, but general object detection is extremely difficult. You may want to watch this computerphile video about convolutional neural networks. If you're not familiar with kernal convolutions you may want to watch this and this as well.
It covers most of the basics. However concrete implementation depends on your needs and resolutions. Some kernal convolutions could be done in a shader to improve performance. However the more complex the system gets the slower it gets.
