William R. Wing (Bill Wing) wrote:
> In the middle of a longer program that reads and plots data from a log
> file, I have added the following five lines (rtt_data is fully qualified
> file name):
>
> wd = open(rtt_data, 'w')
> stat = wd.write(str(i))
> stat = wd.writelines(str(x_dates[:i]))
> stat = wd.writelines(str(y_rtt[:i]))
> wd.close()
>
> The value of i is unknown before I have read through the input log file,
> but is typically in the neighborhood of 2500. x_dates is a list of time
> stamps from the date2num method, that is values of the form
> 734716.72445602, day number plus decimal fraction of a day. y_rtt is a
> list of three- or four-digit floating point numbers. The x_dates and
> y_rtt lists are complete and plot correctly using matplotlib. Reading and
> parsing the input log file and extracting the data I need is time
> consuming, so I decided to save the data for further analysis without the
> overhead of reading and parsing it every time.
>
> Much to my surprise, when I looked at the output file, it only contained
> 160 characters. Catting produces:
>
> StraylightPro:Logs wrw$ cat RTT_monitor.dat
> 2354[ 734716.72185185 734716.72233796 734716.72445602 ...,
> 734737.4440162
> 734737.45097222 734737.45766204][ 240. 28.5 73.3 ..., 28.4
> 27.4 26.4]
>
> Clearly I'm missing something fundamental about using the writelines
> method, and I'm sure it will be a DUH moment for me, but I'd sure
> appreciate someone telling me how to get that data all written out. I
> certainly don't insist on writelines, but I would like the file to be
> human-readable.
When you apply str() to a numpy array big arrays are helpfully truncated,
probably because the functionality is meant to be used in the interactive
interpreter rather than to write to a file.
The default value is 1000 entries. One way to get the desired output is to
increase the threshold:
>>> numpy.set_printoptions(threshold=4)
>>> print numpy.arange(10)
[0 1 2 ..., 7 8 9]
>>> numpy.set_printoptions(threshold=10)
>>> print numpy.arange(10)
[0 1 2 3 4 5 6 7 8 9]
Also, in
file.writelines(some_str)
writelines iterates over the characters of the some_string, so you should
instead write the above as
file.write(some_str)
Your code will become
assert numpy.get_printoptions()["threshold"] >= i
wd.write(str(x_dates[:i]))
If you intended to write one array entry at a time with writelines() here's
how to do that:
wd.write("[")
wd.writelines("%s " % x for x in x_dates[:i])
wd.write("]\n")
numpy.savetxt() may also suit your needs.
--
http://mail.python.org/mailman/listinfo/python-list
