java - how to handle large amount of float data? -
we have binary file contains large amount of float
data (about 80mb). need process in our java application. data medical scanner. 1 file contains data 1 rotation
. 1 rotation
contains 960 views
. 1 view
contains 16 rows
, 1 rows
contains 1344 cells
. numbers (their relationship) fixed.
we need read floats our application code structure reflect above structure rotation-view-row-cell
.
what doing using float[]
hold data cells
, using arraylist
rotation
, view
, row
hold data.
i have 2 questions:
- how populate cell data (read floats our float[]) quickly?
- do have better idea hold data?
for data loading:
datainputstream should work well. make sure wrap underlying fileinputstream in bufferedinputstream, otherwise run risk of doing i/o operations every float can kill performance.
several options holding data:
- the (very marginally) memory-efficient way store entire array in on large float[], , calculate offsets needed. bit ugly use, might make sense if doing lot of calculations or processing loops on entire set.
- the "oop" way have separate objects rotation, view, row , cell. having each cell separate object pretty wasteful, might blow memory limits.
- you use nested arraylists float[1344] represent lowest level data cells in each row. understand you're doing - in fact think it's pretty choice. overhead of arraylists won't compared overall data size.
- a final option use float[rotationnum][rownum][cellnum] represent each rotation. bit more efficient arraylists, arrays less nice manipulate. seems pretty option if, say, array sizes fixed. i'd choose option myself.
Comments
Post a Comment