koobas.hobune.stream
C# - Reading BIG data files into Unity (>4gb)
Hi everybody,
I am facing quite a difficult, yet interesting challenge: I have a big data file (>4gb) that contains data on a linebase (every line is a position-logging event that I need to read). For every second, there are 15.000 position logging events. The idea is to read this data in realtime (15.000 lines/second) and visualising the x,y,z vector (every line in the file represents a vector) in 3-dimensional space.
For clearification, the file looks like this:
10629342490369879,26406,-6869,1070,1166235,7161971,-5706,3485,-7435,-2410,-8454,-4765
10629347605342274,26406,-6869,1070,1166235,7161971,-5706,3485,-7435,-2410,-8454,-4765
10629352720314738,26406,-6869,1070,1166235,7161971,-5706,3485,-7435,-2410,-8454,-4765
10629357835286658,26406,-6869,1070,1166235,7161971,-5706,3485,-7435,-2410,-8454,-4765
10629362950259427,26406,-6869,1070,1166235,7161971,-5706,3485,-7435,-2410,-8454,-4765
Every line has a timestamp (first integer on the left, i.e. 10629342490369879,26406) which is in pico-seconds.
How do I go about this? Threading? StreamReader in buffers?
