koobas.hobune.stream
What is the maximum amount of objects that can be stored in an Array or List without causing any error?
I am planning to create a replay system for my game, and I decided (after reading a few forums) that storing the position and rotation of the players every frame is what I should do. Then when I view the replay, I just make the position of the object and rotation update according to the array. Now I decided to use a class to store the transforms of several moving parts of the car, such as the tires, the player's bones (armature), and the car itself. Basically the object will have references to a few transforms, and an array of many transforms (about 30). Now this has gotten me wondering if there would be any errors or performance problems storing several (1 per every frame) objects inside a huge List<>. Is this an alright approach or is there something else that I should do?
According to https://docs.microsoft.com/en-us/dotnet/api/system.array
The array size is limited to a total of 4 billion elements, and to a maximum index of 0X7FEFFFFF in any given dimension (0X7FFFFFC7 for byte arrays and arrays of single-byte structures).
.NET Framework only: By default, the maximum size of an Array is 2 gigabytes (GB). In a 64-bit environment, you can avoid the size restriction by setting the enabled attribute of the gcAllowVeryLargeObjects configuration element to true in the run-time environment.
The maximum index is the max value of an int32, both the array and list use an int32 for indexing. (I don't see how you can get the the 4 billion elements when int32 goes to 2 billion) So you don't need to worry about that part; Even if you do 60 objects per second, it would take over a year running non-stop to hit 2 billion.
The max memory size of 2GB would be your limiting factor. You would need to find out the size of one of your objects is to find how much you can hold.
Since this is the first answer that appears on Google search, there's one additional detail to take into account when dealing with huge lists: Never make them visible in the Inspector. The maximum length of a list visible in the Inspector depends on the PC that runs the engine editor/client, but you'll usually be stuck around 200K or 300K before the engine hang up or just gets unusable when you select the element that display the list in the inspector. This is due to how the engine display the lists/arrays in the editor. I haven't looked much in the latest 2022 version, but up to 2021, the engine calculate all elements in a public list/array and keep track of its UI/Editor related data as a whole.)
If you're managing huge amount of elements in a list, keep it either set it to private or use the [HideInInspector]. You should create some tools/functions that prints or Debug any desired/tested value within the List.
There are other solutions besides hiding the array in the inspector. The issue is that it would still draw / layout all elements, even through they are not visible. One solution is to implement a "virtual" scroll view. This has some limitations. Specifically each element should have the same height. In that case we can only selectively show the elements of the visible area and fill the rest of the space with a single placeholder. I have an example implementation in my UVViewer which can show a full list of all trianges and vertices they are made up. Note the two GUILayout.Space, one before and one after the list. They essentially represent all the space of the elements that are not visible. So we simply calculate the start and end index and just draw those elements. So this even works with meshes with millions of triangles.
If the elements are more complex and can change size, it's not that simple anymore. Though with a custom property drawer we can essentially force a certain element height. For variable height elements there are solutions as well, but they would require a lot of extra book keeping and overhead to get a smooth scrolling experience. Otherwise it scrolls worse than Excel or LibreOffice Calc.
Maybe a "paged" list / array view may be an alternative. So only showing 10/50/100 elements and provide buttons to cycle through the pages.
