java - how to handle large amount of float data? -
we have binary file contains large amount of float data (about 80mb). need process in our java application. data medical scanner. 1 file contains data 1 rotation. 1 rotation contains 960 views. 1 view contains 16 rows , 1 rows contains 1344 cells. numbers (their relationship) fixed.
we need read floats our application code structure reflect above structure rotation-view-row-cell.
what doing using float[] hold data cells , using arraylist rotation, view , row hold data.
i have 2 questions:
- how populate cell data (read floats our float[]) quickly?
- do have better idea hold data?
for data loading:
datainputstream should work well. make sure wrap underlying fileinputstream in bufferedinputstream, otherwise run risk of doing i/o operations every float can kill performance.
several options holding data:
- the (very marginally) memory-efficient way store entire array in on large float[], , calculate offsets needed. bit ugly use, might make sense if doing lot of calculations or processing loops on entire set.
- the "oop" way have separate objects rotation, view, row , cell. having each cell separate object pretty wasteful, might blow memory limits.
- you use nested arraylists float[1344] represent lowest level data cells in each row. understand you're doing - in fact think it's pretty choice. overhead of arraylists won't compared overall data size.
- a final option use float[rotationnum][rownum][cellnum] represent each rotation. bit more efficient arraylists, arrays less nice manipulate. seems pretty option if, say, array sizes fixed. i'd choose option myself.
Comments
Post a Comment