koobas.hobune.stream
How can I check how many pixels on a texture are transparent without tanking performance?
I'm trying to measure how many pixels I erased from a texture, but each time I do, Unity seemingly buckles under the weight of the calculations. I'm looking for a faster way to accomplish the following:
int alphaPixels = 0;
// Convert texture to Texture2D, so that we can count its alpha pixels with Texture2D.GetPixel().
tex2D = new Texture2D(texWidth, texHeight);
tex2D.ReadPixels(new Rect(0, 0, texWidth, texHeight), 0, 0);
tex2D.Apply();
// Check all pixels along the texture's x axis.
colors = tex2D.GetPixels32();
foreach (Color color in colors)
{
if (color.a < 1) alphaPixels++;
}
print(alphaPixels + " / " + totalPixels + " pixels cleaned on " + erasableCanvas.name);
This is not called every frame or anything outrageous like that. As for multithreading, since I am a total newbie at that, I guess I am now looking for a non-main-thread alternative to Texture2D.Apply and Texture2D.GetPixels32?
I think I managed to save a little performance by replacing the double for loop with this:
colors = tex2D.GetPixels();
foreach (Color color in colors)
{
if (color.a < 1) alphaPixels++;
}
But there is still quite a noticeable frame rate hiccup every time the above code is called.
Still looking for help on this - maybe it's a better question for the forums, since folks will probably have different solutions?
Here is a simple script which might help you
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.Component$$anonymous$$odel;
using System;
using UnityEditor;
public class Worker : $$anonymous$$onoBehaviour {
// Use this for initialization
void Start () {
}
void Update () {
if (Input.Get$$anonymous$$eyDown($$anonymous$$eyCode.Space)) {
for (int i = 0; i < 200; i++) {
BackgroundWorker worker = new BackgroundWorker ();
string response = null;
worker.DoWork += (sender, e) => {
response = Task();
};
worker.RunWorkerCompleted += (sender, e) => {
CompletedTask(response);
};
worker.RunWorkerAsync (response);
}
}
}
public void CompletedTask(string Response){
//Debug.LogError (Response);
}
string Task(){
return (1000 * 600).ToString ();
}
}
Hello, I'm Dane's coworker! We've put together a combination of your solutions. Sorry for the late response!
Summary:
"CheckPixels" is called every second when a trigger is held (not every update)
We refresh the readTexture only when the calculation starts (GetPixels32 is expensive)
We don't need to use "readTexture.Apply()" because we don't need to render this texture. It's only used for calculations.
In our project, textures can be converted to RenderTextures. If they were already Texture2D or Texture3D, we could skip this part entirely.
We spread the calculation out over several frames, only checking one "part" per frame. This smooths out those spikes.
SKIP_PIXEL is used to check every Nth pixel when checking parts. Approximations are acceptable, and it's MUCH faster than checking every pixel.
Our last issue is garbage collection. Some textures are 2048x2048, making our color array over 4 million long. Ideally, we'd get a pointer to the texture's first pixel and iterate to the end of the texture's array (hoping to avoid memory allocation for a new array). Unfortunately, I couldn't find a way to get that first pixel's actual memory location. I'm unfamiliar with shaders and native code plugins. If I find a solution to this, I'll be sure to update this answer.
Here's the core of the calculation, which is called on the delay:
private void RefreshReadTexture()
{
//VERSION 3 (current winner) : We don't need to apply because we don't need to bother the GPU
RenderTexture.active = meshRenderer.material.mainTexture as RenderTexture;
readTexture.ReadPixels(sourceRect, 0, 0); // This operation is expensive.
RenderTexture.active = null;
pixelsInReadTexture = readTexture.GetPixels32(); // This operation is expensive and causes a GC spike every so often
}
private IEnumerator CheckPixelsForTransparency()
{
isCalculating = true;
int testedPixels = 0;
for (int part = 0; part < parts.Length; part++)
{
numTransparentPixels -= parts[part]; //prevents counting transparent pixels multiple times
parts[part] = 0; // /\
for (int i = part * pixPerPart; i < (part + 1) * pixPerPart; i += SKIP_PIXELS)
{
testedPixels++;
if (pixelsInReadTexture[i].a < 255) { parts[part]++; }
}
numTransparentPixels += parts[part];
yield return null;
}
percentCleaned = (float)numTransparentPixels / (float)testedPixels * 100;
isCalculating = false;
}
Great! And thank you for posting your solution! I didn't know that it is not necessary to apply if you don't modify the texture (makes sense, though).
@Harinezumi and @toddisarockstar, I would've bashed my head against this a lot longer without your help. Thank you!
Some additional notes: I mentioned "CheckPixels" but I didn't include it!
if (!isCalculating) //if the coroutine is not currently running...
{
//$$anonymous$$oved these two lines here to save a Ninja jump
RefreshReadTexture();
this.StartCoroutineAsync(CheckPixelsForTransparency());
}
print(name + ": " + percentCleaned + "% clean");
isClean = CheckIsClean();
We used ThreadNinja (thus the "StartCoroutineAsync") for multithreading, though I don't see much of a difference.
Lastly, GetPixels is best if called OnPostRender(), but I haven't transferred it over yet.
you can run your code on multiple threads to gain additional performance.
I'm super new to threading, and since all the hardcore engineers say to be careful with it, I have to ask: is the above code thread-safe and/or possible to be used in a thread besides the main thread? Also, the method which I am using to check textures (in Update) has raycasting, GetComponent, and Unity's Renderer class. I would just use the thread for the calculation part, right? Would I have to call that behavior from Update and pass in a Texture2D, as well as a GameObject? This is all uncharted territory for me, and the information I have found about threading so far kind of assumes a higher level of program$$anonymous$$g knowledge than I have. I am using Unity 2017.3.
I changed my code to not use any Unity API except Texture2D.GetPixel, so that seems to be the one remaining challenge before the code can be run in another thread. Any suggestions on how I could accomplish that? Calling that line for every pixel on a texture is what's really costing me, I'm sure.
Are you sure that this is the code that causes a performance problem? Have you profiled it? It does not seem that heavy (I did more complex calculations in every frame, and didn't even notice it).
One minor improvement could be that instead of using Texture2D.GetPixel(), you could use Texture2D.GetPixels32(), which only accesses the texture memory once (which is slow), and returns all the pixels in one array (it returns Color32[]). Note that Color32 stores color components as bytes, you would need to check against color32.a < 255.
As unity3d-Yesgnome mentioned, you could easily make this run in parallel with multi-threading, because you only read from the texture, not write. To do this, you would need to create a function that receives a part of the pixels array (possibly with an index offset), and does on the part what your double-loop does - counts the transparent pixels. When its done, it would return the count - look into Futures how that could be done easiest.
However, unless you have really large textures, the overhead of creating the threads might negate the benefits of parallelizing this.
UPDATE: following the discussion, I put together this script to count the non-opaque pixels of a RenderTexture over numParts frames. That means that changes in the RenderTexture will not be immediately counted. With this script, on a RenderTexture of size 2048x2048 I achieved 11 FPS when using only 1 part (basically counting all pixels), and 25 FPS when dividing into at least 32 parts. Higher number of parts didn't improve much, but make the updating slower. According to the profiler, the most time consuming part was Gfx.ReadbackImage with 5.47 ms.
Usage: attach the script to any game object, and assign to it the RenderTexture. Set the numParts to a power of 2, but don't modify it once the algorithm started (that's because initialization is done in Start(). You could do initialization from OnValidate() to solve this). The script:
using UnityEngine;
public class PixelCounter : MonoBehaviour {
[SerializeField] private RenderTexture renderTexture = null;
[SerializeField] private int numParts = 1;
private int texWidth, texHeight;
private int numPixels;
private int numPixelsInPart;
private Texture2D readTexture;
private Color32[] colors;
private int currentPart = 0;
private int[] numTransparentPixelsInParts;
private int numTransparentPixels;
private void Start () {
numTransparentPixelsInParts = new int[numParts];
texWidth = renderTexture.width;
texHeight = renderTexture.height;
numPixels = texWidth * texHeight;
numPixelsInPart = numPixels / numParts;
readTexture = new Texture2D(texWidth, texHeight);
}
private void Update () {
UpdateColors();
UpdateCounter();
Debug.Log("Num transparent pixels: " + numTransparentPixels);
currentPart = (currentPart + 1) % numParts;
}
private void UpdateColors () {
UpdateReadTexture();
colors = readTexture.GetPixels32();
}
private void UpdateReadTexture () {
RenderTexture.active = renderTexture;
readTexture.ReadPixels(new Rect(0, 0, texWidth, texHeight), 0, 0);
readTexture.Apply();
RenderTexture.active = null;
}
private void UpdateCounter () {
numTransparentPixels -= numTransparentPixelsInParts[currentPart];
numTransparentPixelsInParts[currentPart] = 0;
CountTransparentPixels();
numTransparentPixels += numTransparentPixelsInParts[currentPart];
}
private void CountTransparentPixels () {
int startPixelIndex = currentPart * numPixelsInPart;
int endPixelIndex = (currentPart + 1) * numPixelsInPart;
for (int i = startPixelIndex; i < endPixelIndex; ++i) {
if (colors[i].a < 255) { numTransparentPixelsInParts[currentPart]++; }
}
}
}
There is a big hiccup when I call the above code in Update, like so:
/// <summary>
/// This raycast checks the alpha value of any texture with an ink canvas.
/// </summary>
private void UpdateAlphaCheckerRaycast()
{
RaycastHit hit;
Ray ray = new Ray(pressureWasher.Origin.transform.position, pressureWasher.Origin.transform.forward);
// Our raycast hits objects in the Default layer (I don't know why Default is listed as 1 and not 0) and ignores triggers.
if (Physics.Raycast(ray, out hit, pressureWasher.DropletRange, 1, QueryTriggerInteraction.Ignore))
{
InkCanvas inkCanvas = hit.transform.GetComponent<InkCanvas>();
if (inkCanvas != null && canCheckTransparency)
{
erasableCanvas = inkCanvas;
tex = erasableCanvas.GetComponent<Renderer>().material.mainTexture;
texWidth = tex.width;
texHeight = tex.height;
// Convert texture to Texture2D, so that we can count its alpha pixels with Texture2D.GetPixel().
tex2D = new Texture2D(texWidth, texHeight);
tex2D.ReadPixels(new Rect(0, 0, texWidth, texHeight), 0, 0);
tex2D.Apply();
// ---------- This is the code I pasted in my original post, now more threadable ----------
canCheckTransparency = false;
int totalPixels = 0, alphaPixels = 0;
// Check all pixels along the texture's x axis.
for (int x = 0; x < texWidth; x++)
// Check all pixels along the texture's y axis.
for (int y = 0; y < texHeight; y++)
{
// totalPixels represents each pixel on a texture.
totalPixels++;
// If any pixel is clear, make note of it.
if (tex2D.GetPixel(x, y).a < 1) alphaPixels++;
}
print(alphaPixels + " / " + totalPixels + " pixels cleaned on " + erasableCanvas.name);
// --------------------------------------------------------------------------------
Invoke("SetCanCheckTransparency", coolDownTime);
}
}
}
For this reply, I clumped it all into one method and deleted the references to a package I was using to help me with multithreading.
God, that formatting is horrifying... Sorry, it might be more readable if you copy it into Visual Studio.
I replaced Texture2D.GetPixel() with Texture2D.GetPixels32() and it helped noticeably! I would still love some more help with perfor$$anonymous$$g this in a different thread - I'm not sure I understand how to "create a function that receives a part of the pixels array" or even what an index offset is in this context. Sorry, thanks for bearing with me!
I was going to write that maybe the problem is the creation of a new Texture2D every frame, but then I quickly tested it, and even the simples counting algorithm greatly drops the FPS (for me, from ~80 to ~20)! Here is what I wrote:
public int CountTransparentPixels (Texture2D texture) {
int numTransparentPixels = 0;
Color32[] pixels = texture.GetPixels32();
for (int i = 0; i < pixels.Length; ++i) {
if (pixels[i].a < 255) { numTransparentPixels++; }
}
return numTransparentPixels;
}
As you can see, it can't get simpler than that, and I'm pretty sure that not even multi-threading helps if you update the texture every frame.
So what are you trying to achieve? $$anonymous$$aybe there is a better solution...
About Future<T>s, they are a multi-threading concept, a kind of task that you start with a function delegate as parameter, and at some future point returns a result of type T. Here is a good article about an introduction to them. Unfortunately, I've just realised that they are .NET 4.0, so probably not available in Unity :(
I'm trying to measure how many pixels I painted alpha onto - it's a pressure-washing simulator, and when the "dirty" texture's number of transparent pixels is high enough, the surface is declared "clean". I'm not updating this number every frame; ins$$anonymous$$d, I have a cool-down period of one second between each time I count the transparent pixels. I am using the Ink Painter plugin to paint transparency, found here: https://github.com/EsProgram/InkPainter
Can't wait to try it out! I will accept as soon as I get a chance to test. I wish I'd seen this yesterday afternoon, because I found another (admittedly less elegant) solution to the problem. I have this script, applied to an erasable object:
public class CleanlinessThreshold_DW : $$anonymous$$onoBehaviour
{
public float CleanlinessThresholdPercent { get { return cleanlinessThresholdPercent; } }
public List<Color32> Colors { get { return colors; } }
public int OpaquePixels { get { return opaquePixels; } set { opaquePixels = value; } }
public int AlphaPixels { get { return alphaPixels; } set { alphaPixels = value; } }
[SerializeField]
private float cleanlinessThresholdPercent = 75;
private List<Color32> colors = new List<Color32>();
private int opaquePixels = 0, alphaPixels = 0;
private void Start()
{
Texture tex = GetComponent<Renderer>().materials[0].mainTexture;
int texWidth = tex.width, texHeight = tex.height;
// Convert Texture to Texture2D, so that we can count its alpha pixels with Texture2D.GetPixels32().
Texture2D tex2D = new Texture2D(texWidth, texHeight);
colors = tex2D.GetPixels32().ToList();
opaquePixels = colors.Count;
}
}
And this code, called in a script applied to the eraser (I only included the part I felt was relevant, for ease of reading, but I can post more if anybody would find it helpful):
for (int i = 0; i < brushSize; ++i)
{
if (cleanlinessThreshold.OpaquePixels > 0)
{
cleanlinessThreshold.AlphaPixels++;
cleanlinessThreshold.OpaquePixels--;
}
}
float percentCleaned = ((float)cleanlinessThreshold.AlphaPixels / (float)cleanlinessThreshold.Colors.Count) * 100;
bool hasCleanedSurface = percentCleaned > cleanlinessThreshold.CleanlinessThresholdPercent && !CleanedObjects.Contains(cleanableSurface);
It seems to work smoothly, since the for loop only contains super-simple calculations - the only drawback seems to be that the threshold has to be pretty low for most surfaces, sometimes as low as 10%. @Harinezumi what do you think?
Also, not sure why I wanted Colors to be a list... I think that's just leftovers from a different idea.
Ugh, I was super super dumb - I didn't include any kind of check for whether an area is transparent or not, so the above code I posted is useless. I can currently just point the pressure washer at a single point, and after long enough, the whole surface would be declared clean.
@Harinezumi $$anonymous$$aybe my implementation is wrong, but I am able to get a texture to read as completely transparent by simply pointing the pressure washer's raycast at one spot and waiting. This is confusing, since I don't see how the values can represent anything other than the texture's alpha pixels in this code.
EDIT: I don't fully understand the UpdateCounter method - for instance, why are these lines necessary...
numTransparentPixels -= numTransparentPixelsInParts[currentPart];
numTransparentPixelsInParts[currentPart] = 0;
...if we are just going to do numTransparentPixels += numTransparentPixelsInParts[currentPart]; after counting the transparent pixels anyway?
numTransparentPixels represents the total number of transparent pixels, meaning the sum of all elements in 'numTransparentPixelsInParts`. So I subtract the value in the current part, which is the previous value, I count how many are transparent now, and then I add that to the total, numTransparentPixels.
Basically I'm saving sum$$anonymous$$g up all of numTransparentPixelsInParts every time a part is updated by "negating" the current parts contribution to the sum before adding the updated value. In pseudo-code: sum -= contribution; /* update contribution */ sum += contribution;
I was thinking about a better method of calculating the transparent pixels, and I'm pretty sure that using the brush (its position, size, and contribution to transparency) would be a lot more efficient.
Can you explain how the brush works, what kind of texture it has?
Thanks for the clarification, it makes a little more sense to me now. I don't know if you've got InkPainter handy, but if so, these are my inspector fields for the Brush class: 
In any case, I send out raycasts looking for meshes with InkPainter's InkCanvas script applied to them (additionally, one dedicated raycast also looks for my CleanlinessThreshold_DW script, which I am using to measure pixel transparency), and if a raycast finds an InkCanvas, it calls its Paint method. I haven't touched InkPainter's functionality, but what it does is paint the brush's color onto the main texture of the first element in the InkCanvas object's renderer's materials array, in the shape of the brush's texture, and over an area deter$$anonymous$$ed by the brush's scale.
how accurate does your count need to be? could you evenly take samples of the texture to get a close estimate?
it would GREATLY speed things up if you could afford some slight inaccuracy.
public static int EstimateTransparentPix(Texture2D t){
int SkipPixels = 10;//<smaller number here is more accurate and slow
int tested = 0;
int tran =0;
int x = 0;
while (x<t.width) {
int y = 0;
while(y<t.height){
tested ++;
if(t.GetPixel(x,y).a<.5f){tran++;}
y+=SkipPixels;
}
x+=SkipPixels;
}
float percent = (float)tran / (float)tested;
print (" i have sampled "+tested+" pixels in a grid pattern");
print ((percent*10)+"% had alpha below .5");
tran = Mathf.RoundToInt(t.width * t.height * percent);
print (" so i estimate " + tran + " pixels in your texture are transparent");
return tran;
}
I totally agree that estimates are the way to go - I think I will integrate this into my existing code, once I get texture measurement to work. It seems that no matter how much alpha I paint onto a surface, the only print statements that ever come back are these same numbers:
$$anonymous$$aybe the problem is the Texture2D I am feeding in - here is my code for that, credit to @Harinezumi:
private void InitializeValues()
{
texWidth = renderTexture.width;
texHeight = renderTexture.height;
readTexture = new Texture2D(texWidth, texHeight);
}
private void UpdateReadTexture()
{
RenderTexture.active = renderTexture;
readTexture.ReadPixels(new Rect(0, 0, texWidth, texHeight), 0, 0);
readTexture.Apply();
RenderTexture.active = null;
PixelCounter_DW.EstimateTransparentPix(readTexture);
}
In the above code, renderTexture is assigned in an editor field - I simply created a blank 2048x2048 RenderTexture and dragged it into the inspector field for every cleanable surface in the scene. @Harinezumi was this your intended usage? I have a nagging suspicion that this is not the implementation you intended in the first line of your own PixelCounter.cs... $$anonymous$$y only experience with RenderTextures is using them as mirrors.
