On Friday, 12 April 2013 09:26:21 UTC+1, Cameron Simpson wrote:
>
> | > > Question: is the size of the incomplete file a round number? (Like
>
> | > > a multiple of a decent sized power of 2>)
>
> [...]
>
> | Source (correct one) is 47,970 bytes. Target after copy of 45,056
>
> | bytes. I've tried changing what gets written to change the file
>
> | size. It is usually this sort of difference.
>
>
>
> 45046 is exactly 11 * 4096. I'd say your I/O is using 4KB blocks,
>
> and the last partial block (to make it up to 47970) didn't get
>
> written (at the OS level).
>
>
>
> Earlier you wrote:
>
> | I have created a file in temp space, then use the function
>
> | "shutil.copyfile(fn,loc+fname)" from "fn" to "loc+fname".
>
> and:
>
> | Yes, there is a close function call before the copy is launched. No other
> writes.
>
> | Does Python wait for file close command to complete before proceeding?
>
>
>
> Please show us the exact code used to make the temp file.
>
>
>
> I would guess the temp file has not been closed (or flushed) before
>
> the call to copyfile.
>
>
>
> If you're copying data to a tempfile, it will only have complete
>
> buffers (i.e. multiples of 4096 bytes) in it until the final flush
>
> or close.
>
>
>
> So I'm imagining something like:
>
>
>
> tfp = open(tempfilename, "w")
>
> ... lots of tfp.write() ...
>
> shutil.copyfile(tempfilename, newfilename)
>
>
>
> Note above no flush or close of tfp. So the final incomplete I/O
>
> buffer is still in Python's memory; it hasn't been actually written
>
> to the temp file because the buffer has not been filled, and the file
>
> has not been closed.
>
>
>
> Anyway, can you show us the relevant bits of code involved?
>
>
>
> Cheers,
>
> --
>
> Cameron Simpson <c...@zip.com.au>
>
>
>
> Processes are like potatoes. - NCR device driver manual
Thanks for the observation.
Code (simplified but results in same flaw) (which a close, far as I can tell).
def CreateSpeakerList1():
import shutil
import filecmp
import os.path
t=get_template('speaker_list.html')
fn=TEMP_DIR+SOC_SPEAKER_LIST
fn=tempfile.gettempdir()+"/"+SOC_SPEAKER_LIST
f=open(fn,'w')
speaker_list=Speaker.objects.order_by('status__order','targetmtg__date')
print " Creating " + SOC_SPEAKER_LIST + " ..."
html=(smart_str(t.render(Context(
{
'css_include_file':CSS_INCLUDE_FILE,
'css_link':False,
'title': ORG_NAME+" Speaker List",
'speaker_list': speaker_list,
}))))
f.write(html)
f.close
print " Wrote "+fn
shutil.copyfile(fn,SOC_GENERAL_OUTPUT_FOLDER+SOC_SPEAKER_LIST)
print "Filecompare
:",filecmp.cmp(fn,SOC_GENERAL_OUTPUT_FOLDER+SOC_SPEAKER_LIST)
print "Statinfo :"+fn+":\n", os.stat(fn)
print "Statinfo :"+SOC_GENERAL_OUTPUT_FOLDER+SOC_SPEAKER_LIST+"\n",
os.stat(SOC_GENERAL_OUTPUT_FOLDER+SOC_SPEAKER_LIST)
return
Output on latest run:
Creating speakers.htm ...
Wrote /var/folders/p_/n5lktj2n0r938_46jyqb52g40000gn/T/speakers.htm
Filecompare : True
Statinfo :/var/folders/p_/n5lktj2n0r938_46jyqb52g40000gn/T/speakers.htm:
posix.stat_result(st_mode=33188, st_ino=32332374, st_dev=16777218L, st_nlink=1,
st_uid=501, st_gid=20, st_size=45056, st_atime=1365758139, st_mtime=1365758139,
st_ctime=1365758139)
Statinfo :/Users/rmschne/Documents/ScottishOilClub/Output/speakers.htm
posix.stat_result(st_mode=33188, st_ino=32143886, st_dev=16777218L, st_nlink=1,
st_uid=501, st_gid=20, st_size=45056, st_atime=1365758029, st_mtime=1365758139,
st_ctime=1365758139)
--
http://mail.python.org/mailman/listinfo/python-list
