- Home /
Custom Datatypes for optimization?
Situation: I have a three dimensional volume filled with evenly placed nodes. At each node I want to store a few values ranging from 0 to 100. I could use a byte for each value, since using 101/256 values of a byte doesn't seem like much, however over a very large number of nodes it feels inefficient.
Proposal: Say I want 101 values (0-100) and currently the smallest datatype I can declare has 256. Is there a way to create, pass, and utilize my own custom datatype with 7 bits instead of 8? Or would this have more of an impact on performance over a large scale than simply using a byte?
I figure a 7 bit datatype, holding 127 values and more fit for my needs, would actually be more optimal;
1,000,000 node values x 8 bits = 1000 kB
1,000,000 node values x 7 bits = 875 kB
Would the above hold true if I manage to implement a 7 bit datatype and how exactly could one go about doing that? I only ask if there is indeed an accepted method to such madness.
Pardon my need for such efficiency, but I'm an engineer.
well depending on your use case you can or can't optimize for cpu and or memory usage. Let's say you don't need to ever access a singular node value but you only need to access or modify 9 at a time. Then you could combine the 9 values into one 64-bit long. Then you could modify 9 values with one addition or subtraction. This would be faster and you would've saved 8 bits.
It really depends on what you're doing with your data. Also if you can modify your data in parallel then you might think about using the gpu to do your calculations.
Answer by HarshadK · Jun 22, 2015 at 09:09 AM
I'm not an expert on this subject matter but as long as I understand having a 7-bit data type makes no reduction in storage size at processor level.
Now consider this Intel 64 and IA-32 Architectures, the storage size for the fundamental data types is in order of 8-bits. So at processor level your 7-bit data type will actually be stored as an 8-bit data type itself providing so you have no storage space gain. And that is why the smallest size of a data type is 8-bit.
Notice how the size of boolean also is 8-bit from this MSDN page aboutData Types (C# vs. Java)
Also read this article on Data Types and Addressing Modes for further details.
Ahh, it makes sense to me now. I was under the impression the smallest you could process in at that level was a bit; now I know that is not the case. Thank you for the reference material!
Quick clarification: processors indeed cannot operate on anything smaller than a byte (and technically, nothing smaller than the processor's word size), but bit packing will reduce storage size in RA$$anonymous$$ and on disk.
Answer by zach-r-d · Jun 22, 2015 at 09:03 AM
Unfortunately, there is no unit of storage in C# smaller than a byte; and specifying field lengths in bits when declaring structs doesn't seem to be possible either, like it is in C/C++. However, if the tradeoff of time for space is truly worth it, it is possible to store many such values in an array of bytes (or uints or ulongs), using bit shifting and bitwise-AND to reconstruct values that cross boundaries.
For example, eight 7-bit values can be neatly stored in an array of 7 bytes:
11111112 22222233 33333444 44445555 55566666 66777777 78888888
and to access value 3, it would be:
byte value3 = ((arr[1] & 3) << 5) | ((arr[2] & 248) >> 3);
Again, this is a significant tradeoff of time for space; I count 7 operations (including two memory accesses) required just to read one value. The general form will have even more. But sometimes 12.5% space savings is worth it, especially if the application isn't CPU bound in the first place!
It'd definitely be an interesting way to go about things, but unfortunately I was looking to decrease CPU usage ins$$anonymous$$d of storage volume. However the method you provided may deem useful for me in the future for other things, thanks!
Ah, in that case, it's worth remembering that saving CPU time is most easily achieved by using *more* space. The space-time tradeoff comes up all the time in program$$anonymous$$g. In this case, using one byte to store a seven-bit value seems like a good balance indeed.
Answer by YoungDeveloper · Jun 22, 2015 at 08:32 AM
There's a BitArray class, but it will need at least 8 bytes just for the instance. For that 7 bits to work you would actually need a primitive, but i am not sure is it even possible. I believe good old byte is your best bet.
Your answer
![](https://koobas.hobune.stream/wayback/20220613201247im_/https://answers.unity.com/themes/thub/images/avi.jpg)
Follow this Question
Related Questions
Store multiple types of data in an Array? 4 Answers
What data format should I store data in? 0 Answers
FindGameObjectsWithTag alternative for better perfomance 1 Answer
Receive Array data from webservice 0 Answers
Development of the shop? 1 Answer