koobas.hobune.stream
HLSL bit shifting
I have a sprite that I am trying to enable palette swapping on. I've written a tool to iterate over the color values in the sprite and encode the colors palette index within the first(?) two bits in each color value:
ex: pixel index is 21 and the original color value was #a22633 (162, 38, 51)
1. convert 21 to 6 bits: 010101
2. separate those bits into three 2 bit segments: 01 01 01
3. drop the last two bits off each color value and replace them with the bits from each segment
R: 162 => 161 10100010 => 10100001
G: 38 => 37 100110 => 100101
B: 51 => 49 110011 => 110001
The next part is where I'm struggling. I want to be able to read these values in and reconstruct the index inside my fragment shader, but I'm having some difficulty with reading the bits.
I've added the following function to my shader
int CheckBit(float f, float bit, int setValue)
{
float sample = f % bit;
if (sample == 0) return 0;
if (bit == 2 && sample == 1.0) return setValue;
if (sample >= bit/2.0) return setValue;
return 0;
}
and I'm using it like this:
float4 UnlitFragment(Varyings i) : SV_Target
{
float4 mainTex = SAMPLE_TEXTURE2D(_MainTex, sampler_MainTex, i.uv);
float4 pack = mainTex * 255.0f;
int index = 0;
index += CheckBit(pack.r, 2, 16);
index += CheckBit(pack.r, 4, 32);
index += CheckBit(pack.g, 2, 4);
// etc ...
// use the index value to sample color value from our palette
float2 puv = float2(index / 64.0, 0); // palette is a 64x1 sprite
float4 palette = SAMPLE_TEXTURE2D(_PaletteTex, sampler_PaletteTex, puv);
palette.a = mainTex.a;
palette.rgb *= mainTex.a;
return palette;
}
This doesn't appear to be generating the correct index values. I think my bit reading code is off, but I've tried a lot of other methods and none seem to work correctly. I feel like I'm missing something basic.
So if I read correctly, you are just trying to reconstruct the index from the packed texture? Assuming you are targeting SM4.0+ and have already correctly packed the index the way you say (2 bits into each colour channel), then you can use bitwise ops to mask and shift like so to reconstruct;
// Assuming the 6 bits are packed evenly and in segment order of BGR;
uint4 p = (uint4)pack;
// Mask the first 2 bits (3 = 0011) of each channel and shift/combine to get the index
int index = (p.r & 3u) | ((p.g & 3u) << 2u) | ((p.b & 3u) << 4u);
// Invert the mask to remove the packed index from the normal colour info
p.rgb &= ~3u;
This gets me a lot closer than I was before so thank you! it seems the index is still off, would I need to do anything to the color values to make sure they were correct for unpacking? Currently I sample the texture and then multiply the float4 by 255 to get the integer values, but the color indexes are still off.
Well, of course you want to set the texture filtering mode to "point", otherwise you get interpolation between pixels. Also keep in $$anonymous$$d that all vertex attributes are interpolated across each rendered triangle. So since your code is in the fragment shader, if you want to get the same value across the whole triangle you have to make sure each vertex has the same uv coordinate, or if you look at it the other way round, every texel in the texture that is mapped onto the triangle has to have the same bits across the triangle.
