Wayback Machinekoobas.hobune.stream
May JUN Jul
Previous capture 13 Next capture
2021 2022 2023
1 capture
13 Jun 22 - 13 Jun 22
sparklines
Close Help
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
  • Asset Store
  • Get Unity

UNITY ACCOUNT

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account
  • Blog
  • Forums
  • Answers
  • Evangelists
  • User Groups
  • Beta Program
  • Advisory Panel

Navigation

  • Home
  • Products
  • Solutions
  • Made with Unity
  • Learning
  • Support & Services
  • Community
    • Blog
    • Forums
    • Answers
    • Evangelists
    • User Groups
    • Beta Program
    • Advisory Panel

Unity account

You need a Unity Account to shop in the Online and Asset Stores, participate in the Unity Community and manage your license portfolio. Login Create account

Language

  • Chinese
  • Spanish
  • Japanese
  • Korean
  • Portuguese
  • Ask a question
  • Spaces
    • Default
    • Help Room
    • META
    • Moderators
    • Topics
    • Questions
    • Users
    • Badges
  • Home /
avatar image
1
Question by rule62 · Feb 28, 2011 at 04:48 PM · texttimefileslowread

Why is reading some text files with JS so darn slow?! (title changed to reflect answers)

Here's the relevant code:

var textAsset : TextAsset;

var lineCounter = 0;

function Start() {

 readHugeFile();

}

function readHugeFile() {

 var startTimer = Time.realtimeSinceStartup;

 if (textAsset == null) return;
 reader = new StringReader(textAsset.text);
 line = reader.ReadLine();
 while (line != null)
 {
     lineCounter++;
     line = reader.ReadLine();
 }

 var timeExpired = Time.realtimeSinceStartup - startTimer;

 print("Time Expired:");
 print(timeExpired);

 print("Number of Lines:");
 print(lineCounter);

}

Ok, by points.

  1. I'm not interested in alternatives to reading these files. Alternative/smarter code would be super welcome!

  2. This code is to darn SLOW when the number of lines grows.

  3. This thing scales weirdly. The length of time taken isn't linear with the number of lines in the file.

Some random sampling of text files of different sizes.

(Edits after modification suggested)

Number of Lines Lines Read / second | *Lines Read / second

57,012 251 518,000 54,818 245 550,000 41,239 359 412,000 38,138 385 545,000 32,305 447 538,000 25,908 551 518,000 21,098 681 527,000 17,668 803 589,000 16,038 891 535,000 13,118 1161 656,000 11,616 1263 581,000 2,293 6551 573,000

WTH?

Thanks for any answers, pointers, or directions for better code. Am I doing something silly here?

*Notes The large variance in the number in the far right column are a result of variances in actual line lengths in the files, rounding error, and poor use of significant digits by myself.

With the largest file tested here, the suggested change of code represents a 2000X speed increase!

Comment
Add comment · Show 3
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image Bunny83 · Feb 28, 2011 at 05:43 PM 0
Share

Never worked with StringReader. How long does it take to execute just the line new StringReader(textAsset.text);? I guess the StringReader doesn't work very efficient on the input data internally. Looks really strange.

avatar image Bunny83 · Feb 28, 2011 at 05:45 PM 0
Share

Just found this article: http://lists.ximian.com/pipermail/mono-bugs/2009-November/095088.html

avatar image rule62 · Feb 28, 2011 at 09:23 PM 0
Share

Thanks Bunny! Indeed, your link explains the strange scaling. Super helpful. I hate just going "wth? I don't get it."

1 Reply

· Add your reply
  • Sort: 
avatar image
2
Best Answer

Answer by Eric5h5 · Feb 28, 2011 at 05:32 PM

How about

if (textAsset == null) return;
var lines = textAsset.text.Split("\n"[0]);
var lineCounter = lines.Length;

Might have to be "\r" if that's how lines are delineated.

Comment
Add comment · Show 4 · Share
10 |3000 characters needed characters left characters exceeded
▼
  • Viewable by all users
  • Viewable by moderators
  • Viewable by moderators and the original poster
  • Advanced visibility
Viewable by all users
avatar image rule62 · Feb 28, 2011 at 09:25 PM 0
Share

Well "Slap my hind with a melon rind"!

Check this out! Editing table in post with new data.

avatar image rule62 · Feb 28, 2011 at 09:36 PM 0
Share

Eric, this seems like a whopper of a small easily-made error. Should this be brought to anyone's attention? This teensy code-change just saved a project that I was about to give up on.

avatar image Eric5h5 · Mar 01, 2011 at 01:09 AM 0
Share

@rule62: Well, it depends on what you're doing. Using StringReader with ReadLine may be correct for certain cases. I wasn't sure if my suggestion would be appropriate for what you're doing (since I don't know what that actually is), which is why I phrased my answer as a question. :) Glad it worked for you though.

avatar image Eric5h5 · Mar 01, 2011 at 01:14 AM 0
Share

After reading Bunny83's link, it sounds like a bug report would be in order. Seems like that should have been fixed given the version of $$anonymous$$ono Unity uses?

Your answer

Hint: You can notify a user about this post by typing @username

Up to 2 attachments (including images) can be used with a maximum of 524.3 kB each and 1.0 MB total.

Follow this Question

Answers Answers and Comments

No one has followed this question yet.

Related Questions

Best way to manage stats in a text file 2 Answers

Move a Camera with a Data from a Text file after Build. 1 Answer

I cant read the file i wrote, unless i reload it in VS. 1 Answer

Trying to download and show contents of text file 1 Answer

Help with reading a text file? 1 Answer


Enterprise
Social Q&A

Social
Subscribe on YouTube social-youtube Follow on LinkedIn social-linkedin Follow on Twitter social-twitter Follow on Facebook social-facebook Follow on Instagram social-instagram

Footer

  • Purchase
    • Products
    • Subscription
    • Asset Store
    • Unity Gear
    • Resellers
  • Education
    • Students
    • Educators
    • Certification
    • Learn
    • Center of Excellence
  • Download
    • Unity
    • Beta Program
  • Unity Labs
    • Labs
    • Publications
  • Resources
    • Learn platform
    • Community
    • Documentation
    • Unity QA
    • FAQ
    • Services Status
    • Connect
  • About Unity
    • About Us
    • Blog
    • Events
    • Careers
    • Contact
    • Press
    • Partners
    • Affiliates
    • Security
Copyright © 2020 Unity Technologies
  • Legal
  • Privacy Policy
  • Cookies
  • Do Not Sell My Personal Information
  • Cookies Settings
"Unity", Unity logos, and other Unity trademarks are trademarks or registered trademarks of Unity Technologies or its affiliates in the U.S. and elsewhere (more info here). Other names or brands are trademarks of their respective owners.
  • Anonymous
  • Sign in
  • Create
  • Ask a question
  • Spaces
  • Default
  • Help Room
  • META
  • Moderators
  • Explore
  • Topics
  • Questions
  • Users
  • Badges