koobas.hobune.stream
Is it just me, or does debug log crash unity on large iterations?
So I'm working on a procedural texture generator, and I was trying to get some output from my code to see where one of my math equations was not coming out as intended. I added a debug line in the iteration, which in turn caused unity to crash. This is not the first time I've experience such a thing, and from my experiences, it seems to only occur when you attempt to add a Debug.Log call to an iteration of around or above 1000,000 equations. I understand that the Debug.Log was never meant to handle over a million entries, but I'd of though Unity would of found a better way of handling the debug log overflow. So is it just me? Is my computer just so terrible, that it can't handle the logs? Or does unity really crash every time the log gets overflowed?
It's just really, really slow. When I have large or nested loops, I'll keep a logString string variable, and concatenate to that, rather than call Debug.Log everytime. Then, at the end of the loop, I'll call Debug.Log(logString) ONCE, (or every 300 loop iterations using a counter, or something like that, to just make it called less often.)
Same thing here. And when I quit play mode during the debug.log spam, Unity, Visual Studio and Chrome crashed (Chrome was open, playing a song on YouTube). Windows whined about lack of RA$$anonymous$$ (and I have 8 GB of it!)
Like Glurth alreads said Debug.Log in general is quite slow and shouldn't called too often. And yes, there's a "magic limit" at which the performance drops even more drastically. That most likely is related to the fact that each debug log call contains a lot information (log text, context, stacktrace) and the addition to the console is handled in a seperate thread. If you produce too many log entries in a short time span Unity most likely runs out of buffer space and the execution stalls.
The editor usually don't crash, it just freezes (though that could be for several seconds or even minutes). If you have a case where you constantly adding more logs Unity becomes almost unusable because when another frame is "finished" the entries just stack up. Also keep in mind that holding those log entries in memory can make you easily run out of memory.
Since the datastructures grow and grow in size the more entries there are the more time it takes to allocate / re-allocate those structures which slows it down even further.
Also like Glurth said, it's usually a good idea if you need to log a lot information to use a StringBuilder and log a single string at the end. The console window limits the text of a single log entry to 64k (that's a limit of the TextArea). Though the logfile should have the full log entry. Another advantage is that the data is grouped without the stacktrace overhead when you read it in the editor log file and it's easier to copy and paste blocks of logged data.
StringBuilder sb = new StringBuilder();
for(int i = 0; i < 1000000; i++)
sb.Append("Some log text #" + i + "\n");
Debug.Log("results: \n" +sb.ToString());
The StringBuilder class is located in the System.Text namespace. It's basically like a "List" of chars which dynamically expand itself to minimize garbage generation. \n is the line-feed character which makes a new line in a string.
ps: The log overhead is much smaller when you execute the code from a build. Inside the editor the live-feedback in the console is what seems to cause most problems. At runtime the log is not held in memory but directly dumped to the logfile. Though don't forget to remove those logs when you ship your product. For example the game Creativerse has some chunk generation "bug" and the log file receives thousands / millions of error messages. After playing for a few hours the logfile can grow into hundreds of MB.
$$anonymous$$ake sure you upvote Glurth comment as well ^^.
Thanks for the info, I actually had not thought of using a string builder for this purpose. It makes total sense :) I still feel like Unity should of made the Debug.Log just dump after a specific point of overload. Thought I'm sure I could just code an extension to handle such things.
That's probably not as trivial as you might think as the logging system has to work across threads and also from native code. The allocation of the stacktrace and the log entry itself also takes some time. The actual logging class is a native C++ class.
The Log method simply isn't designed for that much data. Well, they could have implemented a safety cap like they did with profiler frames. I once had a quite complex simulation going on where the default max frames was not enough to capture one single game frame (a lot method calls). So i boosted the max frame count but the profiler pretty much crashed as the profile has grown into several hundreds GB!! Of course windows started to swap the memory to disk since my system only has 12GB ram. That pretty much killed my whole OS ^^.
$$anonymous$$eep in $$anonymous$$d if you have say 1$$anonymous$$ log entries in a single frame that each entry has several hundreds bytes. So 1$$anonymous$$ entries quickly requires 1G of memory. If you do this every frame you basically need another gig of ram each frame. As soon as you run low on memory other things play into this like memory fragmentation. It's simply something you want to avoid in general ^^.
if you are going to code your own extension for this (a good idea), let me bring to your attention: the conditional attribute https://msdn.microsoft.com/en-us/library/aa664622(v=vs.71).aspx
The Debug function uses this too, I think, not sure. Basically prevents functions from executing (but not from being defined), unless certain flags, like say... DEBUG, are set. (Even prevents parameters from being accessed/executed!!) (went though a lot of headaches making my own debug stuff until someone pointed out this attribute to me.)
Yes, this attribute is quite useful, though Unity's Log method doesn't use this since logging works in a build as well. Windows builds will write all logs into the output_log.txt inside the _Data folder. See logfiles to figure out where the logs can be viewed on your target platform.
