- Home /
Fastest Way to Read in Data
Hi guys,
So I have a game that needs to read in a 1,600 kb text file and then a 2,400 kb text file. Decreasing either file is not an option, but I can recreate the files as binary files or excel files since they just contain data that my game needs.
My questions is whether changing the file types would result in the data being read in faster because at the moment it is taking quite a while to read in these text files line by line. Based upon other examples on this site, I believe I have made reading in the text as fast as possible, but I was hoping y'all might have some other suggestions for formats to try.
Thanks
Answer by colinday · Oct 19, 2016 at 11:27 PM
Binary files are much faster than deserializing text files, in most cases just for the simple fact that they are usually smaller because you're specifying what you really need for each piece of data.
Checkout BinaryReader and BinaryWriter.
Creating a format where you can write just bits instead of bytes will make your data a lot smaller as well and therefore load faster. As an example, one generic way of thinking about that is reading and writing 7 bit encoded ints so small numbers take up less space than large ones.
private BinaryWriter m_writer;
private BinaryReader m_reader;
protected void write7BitEncodedInt(int value)
{
// Write out an int 7 bits at a time. The high bit of the byte,
// when on, tells reader to continue reading more bytes.
uint v = (uint) value; // support negative numbers
while (v >= 0x80)
{
m_writer.Write((byte) (v | 0x80));
v >>= 7;
}
m_writer.Write((byte)v);
}
protected int read7BitEncodedInt()
{
// Read out an int 7 bits at a time. The high bit
// of the byte when on means to continue reading more bytes.
int count = 0;
int shift = 0;
byte b;
do
{
b = m_reader.ReadByte();
count |= (b & 0x7F) << shift;
shift += 7;
} while ((b & 0x80) != 0);
return count;
}
You don't have to take it that far though ... just reading and writing ints, bools, etc with the BinaryReader and BinaryWriter will get you loads of performance improvement over parsing text.
Thank you, I just finished writing the code I needed to read the data in from a binary file and it sped my code up enormously. Definitely a lot faster than reading it in through a text file
Your answer
Follow this Question
Related Questions
How to make GUI Text appear after a certain amount of time 2 Answers
Why is reading some text files with JS so darn slow?! (title changed to reflect answers) 1 Answer
UI Text: Wrap and Expand to keep aspect ratio 2 Answers
Set generated text size inside layout group 0 Answers
Unity Asset Store and Text Serialisation 0 Answers