Can't find a way to downscale a 560x560 Texture2D / Image to a 28x28 resolution without losing lots of pixels

Question

Over the past days I've developed a Doodle recognition application with a neural network library I wrote a while back, everything Is running fine now and I only need to be able to draw on top of a texture big enough for me to draw comfortably (580x580 works just fine for me), and then get the pixel data to feedforward it through my neural net library, the problem comes when I try to downscale the image because I lose too much data, the resulting image is just a bunch of points following the trace I draw on the bigger image and I don't know how to fix it.

I just want to be able to downscale the texture in a pixel-perfect way since I don't care about blockiness because it's the nature of the training data I used for the ML model.

Here's the script I've been using to downscale my image with no luck: https://pastebin.com/qkkhWs2J

I used @petersvp script

And here's the code triggered by a button once I finish my Doodle on the main texture:

 public void GetDrawingData(){
 
         ScalableTex = new Texture2D(28,28);
         ScalableTex = TextureScale.scaled(DrawnTex,28,28,FilterMode.Point);
         ScalableTex.Apply();
         
         GetResult(ScalableTex.GetPixels().ColorToFloat());
 
         targetSprite.GetComponent<SpriteRenderer>().sprite = Sprite.Create(ScalableTex,new Rect(0,0,ScalableTex.width,ScalableTex.height),new Vector2(0.5f,0.5f));
 
         byte[] bytes = ScalableTex.EncodeToPNG();
         File.WriteAllBytes(Application.dataPath + "/../a.png", bytes);
     }

alt text

Answer 1

The reason this is occurring is because you are only sampling the full resolution texture once per low-res pixel. As a result, most of the pixels get skipped and you end up with an image that might be passable for some textures, but in your case not at all. Instead what you want to do is keep all of the data for every high resolution pixel and combine them all into the nearest low-res pixel in some way.

To do this 'accurately' you would need to find all of the full-res pixels that are covered by the area of the low-res pixel and average them (be conservative; full-res pixels should be shared if there isn't a 1-1 match). This requires a lot of manual work however - an optimised way to do this is to progressively downsample, i.e. rather than going straight from full-res -> low-res you can instead go from full-res -> half-res -> quarter-res -> ... -> desired-res:

 public Texture2D FilteredDownscale (Texture2D a_Source, int a_NewWidth, int a_NewHeight)
 {
     // Keep the last active RT
     RenderTexture _LastActiveRT = RenderTexture.active;
 
     // Start by halving the source dimensions
     int _Width = a_Source.width / 2;
     int _Height = a_Source.height / 2;
 
     // Cap to the target dimensions.
     // This could be done with Mathf.Max() but that wouldn't take into account aspect ratio.
     if (_Width < a_NewWidth || _Height < a_NewHeight)
     {
         _Width = a_NewWidth;
         _Height = a_NewHeight;
     }
 
     // Create a temporary downscaled RT
     RenderTexture _Tmp1 = RenderTexture.GetTemporary (_Width, _Height, 0, RenderTextureFormat.ARGB32);
 
     // Copy the source into our temporary RT
     Graphics.Blit (a_Source, _Tmp1);
 
     // Loop until our target dimensions have been reached
     while (_Width > a_NewWidth && _Height > a_NewHeight)
     {
         // Keep halving our current dimensions
         _Width /= 2;
         _Height /= 2;
 
         // And match our target dimensions once small enough
         if (_Width < a_NewWidth || _Height < a_NewHeight)
         {
             _Width = a_NewWidth;
             _Height = a_NewHeight;
         }
 
         // Downscale again into a smaller RT
         RenderTexture _Tmp2 = RenderTexture.GetTemporary (_Width, _Height, 0, RenderTextureFormat.ARGB32);
         Graphics.Blit (_Tmp1, _Tmp2);
 
         // Swap our temporary RTs and release the oldest one
         (_Tmp1, _Tmp2) = (_Tmp2, _Tmp1);
         RenderTexture.ReleaseTemporary (_Tmp2);
     }
 
     // At this point _Tmp1 should hold our fully downscaled image,
     // so set it as the active RT
     RenderTexture.active = _Tmp1;
 
     // Create a new texture of the desired dimensions and copy our data into it
     Texture2D _Tex = new Texture2D (a_NewWidth, a_NewHeight);
     _Tex.ReadPixels (new Rect (0, 0, a_NewWidth, a_NewHeight), 0, 0);
     _Tex.Apply ();
 
     // Reset the active RT and release our last temporary copy
     RenderTexture.active = _LastActiveRT;
     RenderTexture.ReleaseTemporary (_Tmp1);
 
     return _Tex;
 }

The reason this works is because sampling at exactly half resolution with bilinear filtering will cause every low-res texture sample to land directly inbetween all 4 full-res neighbouring pixels - which are then averaged directly by the bilinear filtering.

Can't find a way to downscale a 560x560 Texture2D / Image to a 28x28 resolution without losing lots of pixels

1 Reply

Your answer

Follow this Question

Related Questions