How to write lists "one by one" to a binary file in python? -

i have piece of code generates quite large lists in each iteration. save memory want write each list binary file in each iteration after list has been generated. have tried text files(even setting parameter "wb" in linux). "wb" seems not have effect file written in binary or text format. moreover, written file huge , don't want this. sure if can write these lists in binary format file smaller. thanks

since mentioned need compressibility, i'd suggest using pickle gzip module compress output. can write , read lists 1 @ time, here's example of how:

import gzip, pickle  output = gzip.open('pickled.gz', 'wb', compresslevel=9)  x in range(10):      output.write(pickle.dumps(range(10)) + '\n\n') output.close() 

and use generator yield lists 1 @ time:

def unpickler(input):     partial = []     line in input:         partial.append(line)         if line == '\n':             obj = ''.join(partial)             partial = []             yield pickle.loads(obj)  input = gzip.open('pickled.gz', 'rb') l in unpickler(input):     print l  [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]