how to set nth bit of large binary file.

2016-04-25 Thread srinivas devaki
I use aria2c to download files, aria2c has this feature of allocating the memory to file before downloading the file and then it will download using multiple connections so filling the data into this file concurrently. So i wonder how to do it. I found a way to do that from here http://stackoverfl

Re: find and replace string in binary file

2014-03-05 Thread Emile van Sebille
On 3/5/2014 1:59 AM, loial wrote: Unfortunately I have to use python 2.6 for this Did you try it? Emile -- https://mail.python.org/mailman/listinfo/python-list

Re: find and replace string in binary file

2014-03-05 Thread Mark Lawrence
On 05/03/2014 09:59, loial wrote: I'm pleased to see that you have answers. In return would you please read and action this https://wiki.python.org/moin/GoogleGroupsPython to prevent us seeing double line spacing, thanks. -- My fellow Pythonistas, ask not what our language can do for you, as

Re:find and replace string in binary file

2014-03-05 Thread Dave Angel
loial Wrote in message: > How do I read a binary file, find/identify a character string and replace it > with another character string and write out to another file? > > Its the finding of the string in a binary file that I am not clear on. > > Any help appreciated >

Re: find and replace string in binary file

2014-03-05 Thread loial
> > >> > > >>> How do I read a binary file, find/identify a character string and replace > > >>> it with another character string and write out to another file? > > >>> > > >>> Its the finding of the string in a binary

Re: find and replace string in binary file

2014-03-04 Thread emile
On 03/04/2014 02:44 PM, Chris Angelico wrote: On Wed, Mar 5, 2014 at 12:18 AM, Peter Otten <__pete...@web.de> wrote: loial wrote: How do I read a binary file, find/identify a character string and replace it with another character string and write out to another file? Its the finding

Re: find and replace string in binary file

2014-03-04 Thread Chris Angelico
On Wed, Mar 5, 2014 at 12:18 AM, Peter Otten <__pete...@web.de> wrote: > loial wrote: > >> How do I read a binary file, find/identify a character string and replace >> it with another character string and write out to another file? >> >> Its the finding of the s

Re: find and replace string in binary file

2014-03-04 Thread Peter Otten
loial wrote: > How do I read a binary file, find/identify a character string and replace > it with another character string and write out to another file? > > Its the finding of the string in a binary file that I am not clear on. That's not possible. You have to convert either

Re: find and replace string in binary file

2014-03-04 Thread MRAB
On 2014-03-04 12:27, loial wrote: How do I read a binary file, find/identify a character string and replace it with another character string and write out to another file? Its the finding of the string in a binary file that I am not clear on. Any help appreciated Read it in chunks and search

find and replace string in binary file

2014-03-04 Thread loial
How do I read a binary file, find/identify a character string and replace it with another character string and write out to another file? Its the finding of the string in a binary file that I am not clear on. Any help appreciated -- https://mail.python.org/mailman/listinfo/python-list

Re: writing fortran equivalent binary file using python

2013-11-14 Thread Antoon Pardon
Op 14-11-13 01:53, Sudheer Joseph schreef: > Hi, > I need to write a binary file exactly as written by fortran code > below to be read by another code which is part of a model which is not > advisable to edit.I would like to use python for this purpose as python has > m

Re: writing fortran equivalent binary file using python

2013-11-14 Thread Sudheer Joseph
Thank you, But it wont allow to write it in unformatted way so that the fortran code can read with open(11,file="input.bin") read(11) IWI,JWI,XFIN,YFIN,DXIN,DYIN,NREC,WDAY with best regards, sudheer On Thu, Nov 14, 2013 at 7:48 PM, Oscar Benjamin wrote: > On 14 November

Re: writing fortran equivalent binary file using python

2013-11-14 Thread Oscar Benjamin
On 14 November 2013 00:53, Sudheer Joseph wrote: > My trial code with Python (data is read from file here) > > from netCDF4 import Dataset as nc > import numpy as np > XFIN=0.0,YFIN=-90.0,NREC=1461,DXIN=0.5;DYIN=0.5 > TITLE="NCMRWF 6HOURLY FORCING MKS" > nf=nc('ncmrwf_uv.nc') > ncv=nf.variables.ke

Re: writing fortran equivalent binary file using python

2013-11-13 Thread David Froger
(So never use the buildit open and write of Python and Fortran, but always the NetCDF library). Hope it helps, David Quoting Sudheer Joseph (2013-11-14 01:53:42) > Hi, > I need to write a binary file exactly as written by fortran code > below to be read by another code which is pa

writing fortran equivalent binary file using python

2013-11-13 Thread Sudheer Joseph
Hi, I need to write a binary file exactly as written by fortran code below to be read by another code which is part of a model which is not advisable to edit.I would like to use python for this purpose as python has mode flexibility and easy coding methods. character(40) :: TITLE

How can I wrap a binary file-like object with an IO stream?

2013-03-21 Thread Will McGugan
Hi, I am looking in to improving Python 3.X support for PyFilesystem (https:// code.google.com/p/pyfilesystem/). There is provisional Python 3 support in there, but a stumbling block is that I would like the open method to work like io.open in Py3 -- specifically returning text mode streams tha

Re: parse binary file

2012-01-29 Thread Steven D'Aprano
On Sun, 29 Jan 2012 15:50:49 +0100, Aaron wrote: > On 01/29/2012 03:04 PM, Andrea Crotti wrote: >> On 01/29/2012 07:51 AM, contro opinion wrote: >>> please download the attachment ,and put in c:\test.data >>> >>> >> Your program should never use hard-coded path, and actually I think the >> majori

Re: parse binary file

2012-01-29 Thread Aaron
On 01/29/2012 03:04 PM, Andrea Crotti wrote: On 01/29/2012 07:51 AM, contro opinion wrote: please download the attachment ,and put in c:\test.data Your program should never use hard-coded path, and actually I think the majority here is not using windows. But I also think that the majority o

Re: parse binary file

2012-01-29 Thread Andrea Crotti
On 01/29/2012 07:51 AM, contro opinion wrote: please download the attachment ,and put in c:\test.data Your program should never use hard-coded path, and actually I think the majority here is not using windows. -- http://mail.python.org/mailman/listinfo/python-list

Re: parse binary file

2012-01-29 Thread Steven D'Aprano
On Sun, 29 Jan 2012 13:39:25 +0100, Peter Otten wrote: > contro opinion wrote: > >> please download the attachment ,and put in c:\test.data > > Your data didn't make it through. Since this is a text-only newsgroup, it won't. >> and run the folloeing code: >> >> from struct import unpack >>

Re: parse binary file

2012-01-29 Thread Peter Otten
contro opinion wrote: > please download the attachment ,and put in c:\test.data Your data didn't make it through. > and run the folloeing code: > > from struct import unpack > file_obj = open('c:\\test.data', 'r') Open the file in binary mode to disable CRNL-to-NL translation which will corr

parse binary file

2012-01-28 Thread contro opinion
please download the attachment ,and put in c:\test.data and run the folloeing code: from struct import unpack file_obj = open('c:\\test.data', 'r') day = file_obj.read(40) while day: parsed = list(unpack('LLL', day[:28])) print parsed day = file_obj.read(4

Re: urllib2 request with binary file as payload

2011-05-12 Thread Michiel Sikma
> > 2.6 is expecting a string, according to the above. No mention of file. > Moreover it expects the data to be urlencoded. 2.7.1 docs say the same > thing. Are you sure you have shown the code that worked with 2.6? > > > -- > http://mail.python.org/mailman/listinfo/python-list > Yes, in fact I si

Re: urllib2 request with binary file as payload

2011-05-11 Thread John Machin
On Thu, May 12, 2011 10:20 am, Michiel Sikma wrote: > Hi there, > I made a small script implementing a part of Youtube's API that allows > you to upload videos. It's pretty straightforward and uses urllib2. > The script was written for Python 2.6, but the server I'm going to use > it on only has 2.

urllib2 request with binary file as payload

2011-05-11 Thread Michiel Sikma
Hi there, I made a small script implementing a part of Youtube's API that allows you to upload videos. It's pretty straightforward and uses urllib2. The script was written for Python 2.6, but the server I'm going to use it on only has 2.5 (and I can't update it right now, unfortunately). It seems t

Re: Re: Re: Re: How to save a binary file?

2010-10-09 Thread hidura
Finally i had problems to save the files what are encoded i can't encode the string to save the file, any ideas? On Oct 6, 2010 3:15pm, hid...@gmail.com wrote: Ppl thanyou, for all your help finally i did it! thanks, another thing to who i can send a propose code, i fixed the little probl

Re: Re: Re: How to save a binary file?

2010-10-06 Thread hidura
Ppl thanyou, for all your help finally i did it! thanks, another thing to who i can send a propose code, i fixed the little problem of the wsig.input in Python 3 i will tested in the next months but i want to share the code with the community, how i can do that? On Oct 6, 2010 3:13pm, h

Re: Re: How to save a binary file?

2010-10-06 Thread hidura
Ppl thanyou, for all your help finally i did it! thanks, another thing to i have to send a propouse code, i can fixed the litle problem of the wsig.input in Python 3 i will tested in the next months but i want to share the code with the comunnity, how i can do that? On Oct 6, 2010 1:45p

Re: How to save a binary file?

2010-10-06 Thread MRAB
On 06/10/2010 15:25, hid...@gmail.com wrote: When you put the 'wb' extension you have to pass a Encode the string Python does not accept a string on a wb file, Python3 [snip] You are using Python 3 and type(str) returns ""? Binary data in Python 3 should be an instance of the 'bytes' class, no

Re: Re: Re: Re: How to save a binary file?

2010-10-06 Thread hidura
quot; jo...@lophus.org> wrote: > > >> > > On 10/05/2010 11:11 PM, hid...@gmail.com wrote: > > >> > > Hello, how i can save a binary file, i read in the manual in the IO > > >> area > > >> > > but doesn' t show how to save it

Re: Re: Re: How to save a binary file?

2010-10-06 Thread hidura
> wrote: >> > > On 10/05/2010 11:11 PM, hid...@gmail.com wrote: >> > > Hello, how i can save a binary file, i read in the manual in the IO >> area >> > > but doesn' t show how to save it. >> >> > > Here is the co

Re: Re: How to save a binary file?

2010-10-05 Thread Chris Rebert
>  > On Oct 5, 2010 6:18pm, "Jonas H." jo...@lophus.org> wrote: >>  > > On 10/05/2010 11:11 PM, hid...@gmail.com wrote: >>  > > Hello, how i can save a binary file, i read in the manual in the IO >> area >>  > > but doesn' t show how t

Re: Re: How to save a binary file?

2010-10-05 Thread hidura
binary code here is: [snip] Sorry for the last send. > On Oct 5, 2010 6:18pm, "Jonas H." jo...@lophus.org> wrote: > > On 10/05/2010 11:11 PM, hid...@gmail.com wrote: > > > > > > Hello, how i can save a binary file, i read in the manual in t

Re: How to save a binary file?

2010-10-05 Thread MRAB
..@lophus.org> wrote: > > On 10/05/2010 11:11 PM, hid...@gmail.com wrote: > > > > > > Hello, how i can save a binary file, i read in the manual in the IO area > > > > but doesn' t show how to save it. > > > > Here is the code what i am using:

Re: Re: Re: How to save a binary file?

2010-10-05 Thread hidura
\\x11\\x83\\x8aU\\x90\\xf4#&\\x9e\\xc3\\x8c?D\\x89\\x1eBC\\x17\\xe7\\xe5B Sorry for the last send. On Oct 5, 2010 6:18pm, "Jonas H." jo...@lophus.org> wrote: > On 10/05/2010 11:11 PM, hid...@gmail.com wrote: > > > Hello, how i can save a binary file, i read in th

Re: How to save a binary file?

2010-10-05 Thread Jonas H.
On 10/05/2010 11:11 PM, hid...@gmail.com wrote: Hello, how i can save a binary file, i read in the manual in the IO area but doesn' t show how to save it. Here is the code what i am using: s = open('/home/hidura/test.jpeg', 'wb') s.write(str.encode(formFields[5])) s

How to save a binary file?

2010-10-05 Thread hidura
Hello, how i can save a binary file, i read in the manual in the IO area but doesn' t show how to save it. Here is the code what i am using: s = open('/home/hidura/test.jpeg', 'wb') s.write(str.encode(formFields[5])) s.close() If needs the binary code i could upload. -

Re: writing numbers in binary file

2010-05-31 Thread Dave Angel
eskandari wrote: On May 31, 12:30 pm, MRAB wrote: eskandari wrote: Hi, I am a newbie in python. I have an data.pickle file which is serialized form of an "array of strings", I want to write their offsets in another binary file, so an C++ program can read and analyse them.

Re: writing numbers in binary file

2010-05-31 Thread Terry Reedy
On 5/31/2010 12:43 PM, eskandari wrote: On May 31, 12:30 pm, MRAB wrote: eskandari wrote: Use the 'struct' module to convert the int to a bytestring, and remember to open the file as a binary file. Thanks alot, I have an question, if I do so, Will the second program (C++ prog

Re: writing numbers in binary file

2010-05-31 Thread eskandari
On May 31, 12:30 pm, MRAB wrote: > eskandari wrote: > > Hi, > > I am a newbie in python. I have an data.pickle file which is > > serialized form of an "array of strings", I want to write their > > offsets in another binary file, so an C++ program can read and

Re: writing numbers in binary file

2010-05-31 Thread Tim Chase
On 05/31/2010 10:56 AM, eskandari wrote: But when I try to write offset (number) in binary file, it raise exception below in line "offsetfile.write(offset)" "TypeError: argument 1 must be string or read-only buffer, not int" I search the internet, find that all suggest

Re: writing numbers in binary file

2010-05-31 Thread MRAB
eskandari wrote: Hi, I am a newbie in python. I have an data.pickle file which is serialized form of an "array of strings", I want to write their offsets in another binary file, so an C++ program can read and analyse them. But when I try to write offset (number) in binary file

writing numbers in binary file

2010-05-31 Thread eskandari
Hi, I am a newbie in python. I have an data.pickle file which is serialized form of an "array of strings", I want to write their offsets in another binary file, so an C++ program can read and analyse them. But when I try to write offset (number) in binary file, it raise exception bel

Binary file email attachment problem

2010-05-09 Thread Alan Harris-Reid
Hi there, Using Python 3.1.2 I am having a problem sending binary attachment files (jpeg, pdf, etc.) - MIMEText attachments work fine. The code in question is as follows... for file in self.attachments: part = MIMEBase('application', "octet-stream") part.set_payload(open(file,"rb").read()

Re: [Python3] Reading a binary file and wrtiting the bytes verbatim?in an utf-8 file

2010-04-25 Thread fab
> Another possibility is to open the file in binary mode and do the > encoding yourself when writing text. This might actually be a better > solution, since I'm not sure RTF uses utf-8 by default. Yes, thanks for this suggestion, it seems the best to me. Actually RTF is not UTF-8 encoded, it's 8

Re: [Python3] Reading a binary file and wrtiting the bytes verbatim in an utf-8 file

2010-04-24 Thread Stefan Behnel
Antoine Pitrou, 25.04.2010 02:16: Another possibility is to open the file in binary mode and do the encoding yourself when writing text. This might actually be a better solution, since I'm not sure RTF uses utf-8 by default. That's a lot cleaner as it doesn't use two interfaces to write to the

Re: [Python3] Reading a binary file and wrtiting the bytes verbatim in an utf-8 file

2010-04-24 Thread Antoine Pitrou
Hello, > I have to read the contents of a binary file (a PNG file exactly), and > dump it into an RTF file. > > The RTF-file has been opened with codecs.open in utf-8 mode. You should use the built-in open() function. codecs.open() is outdated in Python 3. > As I expected, th

Re: [Python3] Reading a binary file and wrtiting the bytes verbatim ?in an utf-8 file

2010-04-23 Thread fab
Thanks, I'll try this. > I have no idea how you'd go about reading the contents of such a file > in a sensible way. The purpose is to embed PNG pictures in an RTF file that will be read by OpenOffice. It seems that OpenOffice reads RTF in 8-bit, so it should be ok. The RTF is produced from a TeX

Re: [Python3] Reading a binary file and wrtiting the bytes verbatim in an utf-8 file

2010-04-23 Thread Chris Rebert
On Fri, Apr 23, 2010 at 9:48 AM, Chris Rebert wrote: > On Fri, Apr 23, 2010 at 9:22 AM,   wrote: >> I have to read the contents of a binary file (a PNG file exactly), and >> dump it into an RTF file. >> how can I tell python to dump the bytes as they are, without >>

Re: [Python3] Reading a binary file and wrtiting the bytes verbatim in an utf-8 file

2010-04-23 Thread Chris Rebert
On Fri, Apr 23, 2010 at 9:22 AM, wrote: > I have to read the contents of a binary file (a PNG file exactly), and > dump it into an RTF file. > > The RTF-file has been opened with codecs.open in utf-8 mode. > > As I expected, the utf-8 decoder You mean encoder. > chokes on

[Python3] Reading a binary file and wrtiting the bytes verbatim in an utf-8 file

2010-04-23 Thread fab
Hello. I have to read the contents of a binary file (a PNG file exactly), and dump it into an RTF file. The RTF-file has been opened with codecs.open in utf-8 mode. As I expected, the utf-8 decoder chokes on some combinations of bits; how can I tell python to dump the bytes as they are, without

Re: Read header and data from a binary file [LONG]

2009-09-25 Thread Gabriel Genellina
En Tue, 22 Sep 2009 18:18:16 -0300, Jose Rafael Pacheco escribió: Hello, I want to read from a binary file called myaudio.dat Then I've tried the next code: import struct name = "myaudio.dat" f = open(name,'rb') f.seek(0) chain = "< 4s 4s I 4s I 20s I I i

Re: Read header and data from a binary file [LONG]

2009-09-23 Thread Ishwor
> char is 1 bytes long on Python (as per struct modules' definition) Also, this is also another option for you to use instead of built-in struct. http://www.sis.nl/python/xstruct/xstruct.shtml -- Regards, Ishwor Gurung -- http://mail.python.org/mailman/listinfo/python-list

Re: Read header and data from a binary file [LONG]

2009-09-23 Thread Ishwor
x27;Jul 13 11:57:41 1994', 'identNOTE': 68, 'max_cA': -44076, > 'cid': 'DS16', 'hcid32': 32, 'identifier': 'FORM', 'clength': 300126} > > So far when I run f.tell() >>>f.tell() > 136L tell( ) giv

Read header and data from a binary file [LONG]

2009-09-23 Thread Jose Rafael Pacheco
Hello, I want to read from a binary file called myaudio.dat Then I've tried the next code: import struct name = "myaudio.dat" f = open(name,'rb') f.seek(0) chain = "< 4s 4s I 4s I 20s I I i 4s I 67s s 4s I" s = f.read(4*1+4*1+4*1+4*1+4*1+20*1+4*1+4*1+4*1+4*1

Re: Read header and data from a binary file

2009-09-22 Thread MRAB
Jose Rafael Pacheco wrote: Hello, I want to read from a binary file called myaudio.dat Then I've tried the next code: import struct name = "myaudio.dat" f = open(name,'rb') f.seek(0) chain = "< 4s 4s I 4s I 20s I I i 4s I 67s s 4s I" s = f.read(4*1+4*1+4*1

Re: Read header and data from a binary file

2009-09-22 Thread Simon Forman
On Tue, Sep 22, 2009 at 4:30 PM, Jose Rafael Pacheco wrote: > Hello, > > I want to read from a binary file called myaudio.dat > Then I've tried the next code: > > import struct > name = "myaudio.dat" > f = open(name,'rb') > f.seek(0) Don

Read header and data from a binary file

2009-09-22 Thread Jose Rafael Pacheco
Hello, I want to read from a binary file called myaudio.dat Then I've tried the next code: import struct name = "myaudio.dat" f = open(name,'rb') f.seek(0) chain = "< 4s 4s I 4s I 20s I I i 4s I 67s s 4s I" s = f.read(4*1+4*1+4*1+4*1+4*1+20*1+4*1+4*1+4*1+4*1

Re: Convert raw binary file to ascii

2009-07-27 Thread Jan Kaliszewski
Hello Friends, It's my first post to python-list, so first let me introduce myself... * my name is Jan Kaliszewski, * country -- Poland, * occupation -- composer (studied in F. Chopin Academy of Music @Warsaw) and programmer (currently in Record System company,

Re: Convert raw binary file to ascii

2009-07-27 Thread Dave Angel
binary, so I went digging into Python to see if there were any modules to help. I found one I think might do what I want it to do - the binascii module. Can anyone describe to me how to convert a raw binary file to an ascii file using this module. I've tried? Boy, I've tried.

Re: Convert raw binary file to ascii

2009-07-27 Thread r2
a hex editor to see text strings in the binary code. I > >> > don't see a way to save these ascii representations of the binary, so > >> > I went digging into Python to see if there were any modules to help. > > >> > I found one I think might do what I

Re: Convert raw binary file to ascii

2009-07-27 Thread Peter Otten
way to save these ascii representations of the binary, so >> > I went digging into Python to see if there were any modules to help. >> >> > I found one I think might do what I want it to do - the binascii >> > module. Can anyone describe to me how to convert a raw

Re: Convert raw binary file to ascii

2009-07-27 Thread r2
On Jul 27, 10:11 am, Grant Edwards wrote: > On 2009-07-27, r2 wrote: > > > I have a memory dump from a machine I am trying to analyze. I can view > > the file in a hex editor to see text strings in the binary code. I > > don't see a way to save these ascii representations of the binary, > > $ str

Re: Convert raw binary file to ascii

2009-07-27 Thread r2
ntations of the binary, so > > I went digging into Python to see if there were any modules to help. > > > I found one I think might do what I want it to do - the binascii > > module. Can anyone describe to me how to convert a raw binary file to > > an ascii file using th

Re: Convert raw binary file to ascii

2009-07-27 Thread Grant Edwards
On 2009-07-27, r2 wrote: > I have a memory dump from a machine I am trying to analyze. I can view > the file in a hex editor to see text strings in the binary code. I > don't see a way to save these ascii representations of the binary, $ strings memdump.binary >memdump.strings $ hexdump -C memd

Re: Convert raw binary file to ascii

2009-07-27 Thread Peter Otten
y modules to help. > > I found one I think might do what I want it to do - the binascii > module. Can anyone describe to me how to convert a raw binary file to > an ascii file using this module. I've tried? Boy, I've tried. That won't work because a text editor does

Convert raw binary file to ascii

2009-07-27 Thread r2
think might do what I want it to do - the binascii module. Can anyone describe to me how to convert a raw binary file to an ascii file using this module. I've tried? Boy, I've tried. Am I correct in assuming I can get the converted binary to ascii text I see in a hex editor using this mod

Re: binary file compare...

2009-04-18 Thread Piet van Oostrum
> Adam Olsen (AO) wrote: >AO> The Wayback Machine has 150 billion pages, so 2**37. Google's index >AO> is a bit larger at over a trillion pages, so 2**40. A little closer >AO> than I'd like, but that's still 56294995000 to 1 odds of having >AO> *any* collisions between *any* of the file

Re: binary file compare...

2009-04-17 Thread Steven D'Aprano
On Fri, 17 Apr 2009 11:19:31 -0700, Adam Olsen wrote: > Actually, *cryptographic* hashes handle that just fine. Even for files > with just a 1 bit change the output is totally different. This is known > as the Avalanche Effect. Otherwise they'd be vulnerable to attacks. > > Which isn't to say

Re: binary file compare...

2009-04-17 Thread Lawrence D'Oliveiro
In message , Nigel Rantor wrote: > Adam Olsen wrote: > >> The chance of *accidentally* producing a collision, although >> technically possible, is so extraordinarily rare that it's completely >> overshadowed by the risk of a hardware or software failure producing >> an incorrect result. > > Not

Re: binary file compare...

2009-04-17 Thread Adam Olsen
On Apr 17, 9:59 am, SpreadTooThin wrote: > You know this is just insane.  I'd be satisfied with a CRC16 or > something in the situation i'm in. > I have two large files, one local and one remote.  Transferring every > byte across the internet to be sure that the two files are identical > is just n

Re: binary file compare...

2009-04-17 Thread Adam Olsen
On Apr 17, 9:59 am, norseman wrote: > The more complicated the math the harder it is to keep a higher form of > math from checking (or improperly displacing) a lower one.  Which, of > course, breaks the rules.  Commonly called improper thinking. A number > of math teasers make use of that. Of cou

Re: binary file compare...

2009-04-17 Thread Adam Olsen
On Apr 17, 5:30 am, Tim Wintle wrote: > On Thu, 2009-04-16 at 21:44 -0700, Adam Olsen wrote: > > The Wayback Machine has 150 billion pages, so 2**37.  Google's index > > is a bit larger at over a trillion pages, so 2**40.  A little closer > > than I'd like, but that's still 56294995000 to 1 od

Re: binary file compare...

2009-04-17 Thread SpreadTooThin
On Apr 17, 4:54 am, Nigel Rantor wrote: > Adam Olsen wrote: > > On Apr 16, 11:15 am, SpreadTooThin wrote: > >> And yes he is right CRCs hashing all have a probability of saying that > >> the files are identical when in fact they are not. > > > Here's the bottom line.  It is either: > > > A) Sever

Re: binary file compare...

2009-04-17 Thread norseman
Adam Olsen wrote: On Apr 16, 11:15 am, SpreadTooThin wrote: And yes he is right CRCs hashing all have a probability of saying that the files are identical when in fact they are not. Here's the bottom line. It is either: A) Several hundred years of mathematics and cryptography are wrong. The

Re: binary file compare...

2009-04-17 Thread Tim Wintle
On Thu, 2009-04-16 at 21:44 -0700, Adam Olsen wrote: > The Wayback Machine has 150 billion pages, so 2**37. Google's index > is a bit larger at over a trillion pages, so 2**40. A little closer > than I'd like, but that's still 56294995000 to 1 odds of having > *any* collisions between *any* o

Re: binary file compare...

2009-04-17 Thread Nigel Rantor
Adam Olsen wrote: On Apr 16, 11:15 am, SpreadTooThin wrote: And yes he is right CRCs hashing all have a probability of saying that the files are identical when in fact they are not. Here's the bottom line. It is either: A) Several hundred years of mathematics and cryptography are wrong. The

Re: binary file compare...

2009-04-17 Thread Nigel Rantor
Adam Olsen wrote: On Apr 16, 4:27 pm, "Rhodri James" wrote: On Thu, 16 Apr 2009 10:44:06 +0100, Adam Olsen wrote: On Apr 16, 3:16 am, Nigel Rantor wrote: Okay, before I tell you about the empirical, real-world evidence I have could you please accept that hashes collide and that no matter ho

Re: binary file compare...

2009-04-16 Thread Adam Olsen
On Apr 16, 4:27 pm, "Rhodri James" wrote: > On Thu, 16 Apr 2009 10:44:06 +0100, Adam Olsen wrote: > > On Apr 16, 3:16 am, Nigel Rantor wrote: > >> Okay, before I tell you about the empirical, real-world evidence I have > >> could you please accept that hashes collide and that no matter how many

Re: binary file compare...

2009-04-16 Thread Adam Olsen
On Apr 16, 11:15 am, SpreadTooThin wrote: > And yes he is right CRCs hashing all have a probability of saying that > the files are identical when in fact they are not. Here's the bottom line. It is either: A) Several hundred years of mathematics and cryptography are wrong. The birthday problem

Re: binary file compare...

2009-04-16 Thread Rhodri James
On Thu, 16 Apr 2009 10:44:06 +0100, Adam Olsen wrote: On Apr 16, 3:16 am, Nigel Rantor wrote: Okay, before I tell you about the empirical, real-world evidence I have could you please accept that hashes collide and that no matter how many samples you use the probability of finding two files th

Re: binary file compare...

2009-04-16 Thread Adam Olsen
On Apr 16, 8:59 am, Grant Edwards wrote: > On 2009-04-16, Adam Olsen wrote: > > I'm afraid you will need to back up your claims with real files. > > Although MD5 is a smaller, older hash (128 bits, so you only need > > 2**64 files to find collisions), > > You don't need quite that many to have a

Re: binary file compare...

2009-04-16 Thread SpreadTooThin
On Apr 16, 3:16 am, Nigel Rantor wrote: > Adam Olsen wrote: > > On Apr 15, 12:56 pm, Nigel Rantor wrote: > >> Adam Olsen wrote: > >>> The chance of *accidentally* producing a collision, although > >>> technically possible, is so extraordinarily rare that it's completely > >>> overshadowed by the

Re: binary file compare...

2009-04-16 Thread Grant Edwards
On 2009-04-16, Adam Olsen wrote: > The chance of *accidentally* producing a collision, although > technically possible, is so extraordinarily rare that it's > completely overshadowed by the risk of a hardware or software > failure producing an incorrect result. Not when

Re: binary file compare...

2009-04-16 Thread Nigel Rantor
Adam Olsen wrote: On Apr 16, 3:16 am, Nigel Rantor wrote: Adam Olsen wrote: On Apr 15, 12:56 pm, Nigel Rantor wrote: Adam Olsen wrote: The chance of *accidentally* producing a collision, although technically possible, is so extraordinarily rare that it's completely overshadowed by the risk

Re: binary file compare...

2009-04-16 Thread Adam Olsen
On Apr 16, 3:16 am, Nigel Rantor wrote: > Adam Olsen wrote: > > On Apr 15, 12:56 pm, Nigel Rantor wrote: > >> Adam Olsen wrote: > >>> The chance of *accidentally* producing a collision, although > >>> technically possible, is so extraordinarily rare that it's completely > >>> overshadowed by the

Re: binary file compare...

2009-04-16 Thread Nigel Rantor
Adam Olsen wrote: On Apr 15, 12:56 pm, Nigel Rantor wrote: Adam Olsen wrote: The chance of *accidentally* producing a collision, although technically possible, is so extraordinarily rare that it's completely overshadowed by the risk of a hardware or software failure producing an incorrect resu

Re: binary file compare...

2009-04-16 Thread Adam Olsen
On Apr 15, 12:56 pm, Nigel Rantor wrote: > Adam Olsen wrote: > > The chance of *accidentally* producing a collision, although > > technically possible, is so extraordinarily rare that it's completely > > overshadowed by the risk of a hardware or software failure producing > > an incorrect result.

Re: binary file compare...

2009-04-15 Thread Nigel Rantor
Adam Olsen wrote: The chance of *accidentally* producing a collision, although technically possible, is so extraordinarily rare that it's completely overshadowed by the risk of a hardware or software failure producing an incorrect result. Not when you're using them to compare lots of files. Tr

Re: binary file compare...

2009-04-15 Thread Adam Olsen
On Apr 15, 11:04 am, Nigel Rantor wrote: > The fact that two md5 hashes are equal does not mean that the sources > they were generated from are equal. To do that you must still perform a > byte-by-byte comparison which is much less work for the processor than > generating an md5 or sha hash. > > I

Re: binary file compare...

2009-04-15 Thread SpreadTooThin
On Apr 15, 8:04 am, Grant Edwards wrote: > On 2009-04-15, Martin wrote: > > > > > Hi, > > > On Mon, Apr 13, 2009 at 10:03 PM, Grant Edwards wrote: > >> On 2009-04-13, SpreadTooThin wrote: > > >>> I want to compare two binary files and see if they are the same. > >>> I see the filecmp.cmp functi

Re: binary file compare...

2009-04-15 Thread Nigel Rantor
Grant Edwards wrote: We all rail against premature optimization, but using a checksum instead of a direct comparison is premature unoptimization. ;) And more than that, will provide false positives for some inputs. So, basically it's a worse-than-useless approach for determining if two files

Re: binary file compare...

2009-04-15 Thread Nigel Rantor
Martin wrote: On Wed, Apr 15, 2009 at 11:03 AM, Steven D'Aprano wrote: The checksum does look at every byte in each file. Checksumming isn't a way to avoid looking at each byte of the two files, it is a way of mapping all the bytes to a single number. My understanding of the original question

Re: binary file compare...

2009-04-15 Thread Grant Edwards
On 2009-04-15, Martin wrote: > On Wed, Apr 15, 2009 at 11:03 AM, Steven D'Aprano > I'd still say rather burn CPU cycles than development hours (if I got > the question right), _Hours_? Calling the file compare module takes _one_line_of_code_. Implementing a file compare from scratch takes abo

Re: binary file compare...

2009-04-15 Thread Grant Edwards
On 2009-04-15, Martin wrote: > Hi, > > On Mon, Apr 13, 2009 at 10:03 PM, Grant Edwards wrote: >> On 2009-04-13, SpreadTooThin wrote: >> >>> I want to compare two binary files and see if they are the same. >>> I see the filecmp.cmp function but I don't get a warm fuzzy feeling >>> that it is doin

Re: binary file compare...

2009-04-15 Thread Martin
On Wed, Apr 15, 2009 at 11:03 AM, Steven D'Aprano wrote: > The checksum does look at every byte in each file. Checksumming isn't a > way to avoid looking at each byte of the two files, it is a way of > mapping all the bytes to a single number. My understanding of the original question was a way t

Re: binary file compare...

2009-04-15 Thread Steven D'Aprano
On Wed, 15 Apr 2009 07:54:20 +0200, Martin wrote: >> Perhaps I'm being dim, but how else are you going to decide if two >> files are the same unless you compare the bytes in the files? > > I'd say checksums, just about every download relies on checksums to > verify you do have indeed the same fil

Re: binary file compare...

2009-04-14 Thread Martin
Hi, On Mon, Apr 13, 2009 at 10:03 PM, Grant Edwards wrote: > On 2009-04-13, SpreadTooThin wrote: > >> I want to compare two binary files and see if they are the same. >> I see the filecmp.cmp function but I don't get a warm fuzzy feeling >> that it is doing a byte by byte comparison of two files

Re: binary file compare...

2009-04-14 Thread Adam Olsen
On Apr 13, 8:39 pm, Grant Edwards wrote: > On 2009-04-13, Peter Otten <__pete...@web.de> wrote: > > > But there's a cache. A change of file contents may go > > undetected as long as the file stats don't change: > > Good point.  You can fool it if you force the stats to their > old values after you

binary file compare...

2009-04-14 Thread SpreadTooThin
I want to compare two binary files and see if they are the same. I see the filecmp.cmp function but I don't get a warm fuzzy feeling that it is doing a byte by byte comparison of two files to see if they are they same. What should I be using if not filecmp.cmp? -- http://mail.python.org/mailman/li

  1   2   3   >