binary conversion issues
I'm using python's struct and binascii modules to write some values from my parser to binary floats. This works great for all of my binary files except one. For some reason this file is saving to 836 (stated by my command shell) bytes instead of 832 like it should. It sounds like an issue with whatever's writing that particular file out but I've checked it 100 times. Also python can read in the 836 byte file, find it has a size of 832 bytes and convert back each value perfectly. However, when I read in the file in C++ it finds a file of size 836 and in the middle of the data starts spitting out junk. Has anyone run into an issue like this or have an idea of what it could be? -- http://mail.python.org/mailman/listinfo/python-list
Re: binary conversion issues
You guys are awesome! I had to set the 'b' flag when writing the
binaries.
file_obj = open('filename.bin', 'wb')
instead of just using 'w'
It worked fine for all of the other 10 binary files I made, but had a
small error with one of them. In that one file's case it wrote out an
extra 4 bytes in the middle somewhere. Strange.
Thanx for your help.
Grant Edwards wrote:
> On 2006-08-08, godavemon <[EMAIL PROTECTED]> wrote:
>
> > I'm using python's struct and binascii modules to write some values
> > from my parser to binary floats. This works great for all of my binary
> > files except one. For some reason this file is saving to 836 (stated
> > by my command shell) bytes instead of 832 like it should.
>
> I'm just making a wild-ass guess since you didn't provide any
> sample code or data, but it sounds like cr/lf translation
> problem to me. Make sure you open the files in binary mode
> using the "b" flag.
>
> --
> Grant Edwards grante Yow! Is this "BIKINI
> at BEACH"?
>visi.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: newb question: file searching
Hey, I've done similar things.
You can use system comands with the following
import os
os.system('command here')
You'd maybe want to do something like
dir_name = 'mydirectory'
import os
os.system('ls ' +dir_name + ' > lsoutput.tmp')
fin = open('lsoutput.tmp', 'r')
file_list = fin.readlines()
fin.close()
Now you have a list of all the files in dir_name stored in file_list.
Then you'll have to parse the input with string methods. They're easy
in python. Here's the list of them:
http://docs.python.org/lib/string-methods.html
There is probably a better way to get the data from an os.system
command but i haven't figured it out. Instead what i'm doing is
writing the stdio output to a file and reading in the data. It works
fine. Put it in your tmp dir if you're in linux.
[EMAIL PROTECTED] wrote:
> I'm new at Python and I need a little advice. Part of the script I'm
> trying to write needs to be aware of all the files of a certain
> extension in the script's path and all sub-directories. Can someone
> set me on the right path to what modules and calls to use to do that?
> You'd think that it would be a fairly simple proposition, but I can't
> find examples anywhere. Thanks.
--
http://mail.python.org/mailman/listinfo/python-list
Re: newb question: file searching
Hey, I've done similar things.
You can use system comands with the following
import os
os.system('command here')
You'd maybe want to do something like
dir_name = 'mydirectory'
import os
os.system('ls ' +dir_name + ' > lsoutput.tmp')
fin = open('lsoutput.tmp', 'r')
file_list = fin.readlines()
fin.close()
Now you have a list of all the files in dir_name stored in file_list.
Then you'll have to parse the input with string methods. They're easy
in python. Here's the list of them:
http://docs.python.org/lib/string-methods.html
There is probably a better way to get the data from an os.system
command but i haven't figured it out. Instead what i'm doing is
writing the stdio output to a file and reading in the data. It works
fine. Put it in your tmp dir if you're in linux.
[EMAIL PROTECTED] wrote:
> I'm new at Python and I need a little advice. Part of the script I'm
> trying to write needs to be aware of all the files of a certain
> extension in the script's path and all sub-directories. Can someone
> set me on the right path to what modules and calls to use to do that?
> You'd think that it would be a fairly simple proposition, but I can't
> find examples anywhere. Thanks.
--
http://mail.python.org/mailman/listinfo/python-list
convert floats to their 4 byte representation
I need to take floats and dump out their 4 byte hex representation.
This is easy with ints with the built in hex function or even better
for my purpose
def hex( number, size ):
s = "%"+str(size) + "X"
return (s % number).replace(' ', '0')
but I haven't been able to find anything for floats. Any help would be
great.
--
http://mail.python.org/mailman/listinfo/python-list
Re: convert floats to their 4 byte representation
I've been a member for a while but I had no idea how helpful this form
is. I had a one hour meeting and when I came back there were 4
replies. Thanks for your help!
Scott David Daniels wrote:
> godavemon wrote:
> > I need to take floats and dump out their 4 byte hex representation.
> > This is easy with ints with the built in hex function or even better
> > for my purpose
> >
> > def hex( number, size ):
> > s = "%"+str(size) + "X"
> > return (s % number).replace(' ', '0')
>
> This should be a trifle nicer:
> def hexx(number, size):# don't shadow the "hex" builtin
> s = "%0s" + str(size) + "X"
> return s % number
>
> > but I haven't been able to find anything for floats. Any help would be
> > great.
>
> Getting to float bytes is tougher.
> First, python Floats are C Doubles, so you probably mean 8=byte hex.
>
> import array
>
> def eights(number, swap=False):
> data = array.array('d', [number])
> if swap:
> data.byteswap()
> return ' '.join(hexx(ord(char), 2) for char in data.tostring())
>
> def fours(number, swap=False):
> data = array.array('f', [number])
> if swap:
> data.byteswap()
> return ' '.join(hexx(ord(char), 2) for char in data.tostring())
>
> --Scott David Daniels
> [EMAIL PROTECTED]
--
http://mail.python.org/mailman/listinfo/python-list
path error
I'm on an intel macbook using OS X 10.4 and for some reason my path is
being interpreted incorrectly. See the example:
[EMAIL PROTECTED] pwd
/Users/dave/til/jared <- dirname = jared
[EMAIL PROTECTED] python
...
>>> import os
>>> os.path.abspath('')
'/Users/dave/til/Jared' <- dirname = Jared
python is capitalizing my jared directory. It has worked for a long
time but suddenly doesn't. I'm working on the project over SVN, could
it be some kind of incompatibility with someone who submitted using
windows or something? Any solutions for this?
--
http://mail.python.org/mailman/listinfo/python-list
Re: path error
On Jul 20, 11:42 am, godavemon <[EMAIL PROTECTED]> wrote:
> I'm on an intel macbook using OS X 10.4 and for some reason my path is
> being interpreted incorrectly. See the example:
>
> [EMAIL PROTECTED] pwd
> /Users/dave/til/jared <- dirname = jared
> [EMAIL PROTECTED] python
> ...>>> import os
> >>> os.path.abspath('')
>
> '/Users/dave/til/Jared' <- dirname = Jared
>
> python is capitalizing my jared directory. It has worked for a long
> time but suddenly doesn't. I'm working on the project over SVN, could
> it be some kind of incompatibility with someone who submitted using
> windows or something? Any solutions for this?
Also my python version is
Python 2.4.4 (#1, Oct 18 2006, 10:34:39)
[GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
--
http://mail.python.org/mailman/listinfo/python-list
Re: path error
On Jul 20, 11:45 am, godavemon <[EMAIL PROTECTED]> wrote:
> On Jul 20, 11:42 am, godavemon <[EMAIL PROTECTED]> wrote:
>
> > I'm on an intel macbook using OS X 10.4 and for some reason my path is
> > being interpreted incorrectly. See the example:
>
> > [EMAIL PROTECTED] pwd
> > /Users/dave/til/jared <- dirname = jared
> > [EMAIL PROTECTED] python
> > ...>>> import os
> > >>> os.path.abspath('')
>
> > '/Users/dave/til/Jared' <- dirname = Jared
>
> > python is capitalizing my jared directory. It has worked for a long
> > time but suddenly doesn't. I'm working on the project over SVN, could
> > it be some kind of incompatibility with someone who submitted using
> > windows or something? Any solutions for this?
>
> Also my python version is
>
> Python 2.4.4 (#1, Oct 18 2006, 10:34:39)
> [GCC 4.0.1 (Apple Computer, Inc. build 5341)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
I deleted the directory and updated. It fixed the problem. Still an
odd one, but not a problem anymore.
--
http://mail.python.org/mailman/listinfo/python-list
Re: MySQL-python Error
Hopefully you've found it by now and didn't have a frustrating christmas :). Get the source from sourceforge and then follow the instructions here. http://www.davidcramer.net/code/57/mysqldb-on-leopard.html Worked perfectly for me on OSX 10.5, python 2.5. Was frustrating to find. Good luck! On Dec 23, 3:09 pm, Steve Ametjan <[EMAIL PROTECTED]> wrote: > I've been trying to get MySQL-python to install on Leopard for the > past couple of days, and I keep running into relatively the same > error. I'm hoping that someone on this list will be able to help me > out in solving the issue. I'd like to get this solved so I can > continue developing with Django using MySQL since that's what my web > server uses as well. I'd hate to have to develop using a different > database engine on my local machine. > > Here's what happens when I try to do an easy_install: > > W8743145X91:~ stevea$ sudo easy_install MySQL-python > Searching for MySQL-python > Readinghttp://pypi.python.org/simple/MySQL-python/ > Readinghttp://sourceforge.net/projects/mysql-python > Readinghttp://sourceforge.net/projects/mysql-python/ > Best match: MySQL-python 1.2.2 > Downloadinghttp://osdn.dl.sourceforge.net/sourceforge/mysql-python/MySQL-python-... > Processing MySQL-python-1.2.2.tar.gz > Running MySQL-python-1.2.2/setup.py -q bdist_egg --dist-dir /tmp/ > easy_install-DYH0yq/MySQL-python-1.2.2/egg-dist-tmp-zpJkox > In file included from _mysql.c:35: > /usr/include/mysql/my_config.h:1030:2: warning: #warning defining > SIZEOF_CHARP = 4 > /usr/include/mysql/my_config.h:1044:2: warning: #warning defining > SIZEOF_LONG = 4 > /usr/include/mysql/my_config.h:1151:1: warning: "WORDS_BIGENDIAN" > redefined > In file included from /System/Library/Frameworks/Python.framework/ > Versions/2.5/include/python2.5/Python.h:8, > from pymemcompat.h:10, > from _mysql.c:29: > /System/Library/Frameworks/Python.framework/Versions/2.5/include/ > python2.5/pyconfig.h:928:1: warning: this is the location of the > previous definition > In file included from /usr/include/mysql/mysql.h:43, > from _mysql.c:40: > /usr/include/sys/types.h:92: error: duplicate 'unsigned' > /usr/include/sys/types.h:92: error: two or more data types in > declaration specifiers > In file included from _mysql.c:35: > /usr/include/mysql/my_config.h:1030:2: warning: #warning defining > SIZEOF_CHARP = 4 > /usr/include/mysql/my_config.h:1044:2: warning: #warning defining > SIZEOF_LONG = 4 > In file included from /usr/include/mysql/mysql.h:43, > from _mysql.c:40: > /usr/include/sys/types.h:92: error: duplicate 'unsigned' > /usr/include/sys/types.h:92: error: two or more data types in > declaration specifiers > lipo: can't open input file: /var/tmp//ccg4YkGM.out (No such file or > directory) > error: Setup script exited with error: command 'gcc' failed with exit > status 1 -- http://mail.python.org/mailman/listinfo/python-list
Retrieve Custom 404 page.
I'm using urllib2 to pull pages for a custom version of a web proxy and am having issues with 404 errors. Urllib2 does a great job of letting me know that a 404 happened with the following code. import urllib2 url = 'http://cnn.com/asfsdafsadfasdf/' try: page = urllib2.urlopen( url ) except urllib2.URLError, e: print e returns: HTTP Error 404: Not Found However, upon finding the error I would like to load the domain's custom 404 message instead of returning my own. I've checked the error information and searched for some sort of web standard but cannot find anything. Does anyone know if it is possible to retrieve a url or the page information for the custom 404 page like this? http://cnn.com/asdfasdfadsf -- http://mail.python.org/mailman/listinfo/python-list
Re: Retrieve Custom 404 page.
Perfect! Thanks! On Nov 17, 4:16 pm, Albert Hopkins <[EMAIL PROTECTED]> wrote: > On Mon, 2008-11-17 at 13:59 -0800, godavemon wrote: > > I'm using urllib2 to pull pages for a custom version of a web proxy > > and am having issues with 404 errors. Urllib2 does a great job of > > letting me know that a 404 happened with the following code. > > > import urllib2 > > url = 'http://cnn.com/asfsdafsadfasdf/' > > try: > > page = urllib2.urlopen( url ) > > except urllib2.URLError, e: > > print e > > > returns: HTTP Error 404: Not Found > >From the urllib2 docs: HTTPError is also a valid HTTP response, so you > > can treat an HTTP error as an exceptional event or a valid response: > > import urllib2 > url = 'http://cnn.com/asfsdafsadfasdf/' > try: > page = urllib2.urlopen(url) > except urllib2.URLError, e: > print e.read() > > > > >http://cnn.com/asdfasdfadsf > > -- http://mail.python.org/mailman/listinfo/python-list
Hamming Distance
I need to calculate the Hamming Distance of two integers. The hamming distance is the number of bits in two integers that don't match. I thought there'd be a function in math or scipy but i haven't been able to find one. This is my function but it seems like there should be a faster way. I do this computation many times and speed up is important. def hamdist( a, b , bits = 32): def _hamdist( x, bits): if bits: return (x & 1) + _hamdist(x >> 1, bits-1) return x & 1 return _hamdist( a ^ b, bits) Another alternative would be to convert the XOR to a binary string and count the # of 1's. Which would be fastest? Are there better alternatives? Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Hamming Distance
Awesome! Thanks a lot. On Jun 19, 5:00 pm, Mensanator <[EMAIL PROTECTED]> wrote: > On Jun 19, 6:27 pm, godavemon <[EMAIL PROTECTED]> wrote: > > > > > I need to calculate the Hamming Distance of two integers. The hamming > > distance is the number of bits in two integers that don't match. I > > thought there'd be a function in math or scipy but i haven't been able > > to find one. This is my function but it seems like there should be a > > faster way. I do this computation many times and speed up is > > important. > > > def hamdist( a, b , bits = 32): > > def _hamdist( x, bits): > > if bits: > > return (x & 1) + _hamdist(x >> 1, bits-1) > > return x & 1 > > return _hamdist( a ^ b, bits) > > > Another alternative would be to convert the XOR to a binary string and > > count the # of 1's. > > > Which would be fastest? Are there better alternatives? > > Yep, use the gmpy module. > > >>> import gmpy > >>> help(gmpy.hamdist) > > Help on built-in function hamdist in module gmpy: > hamdist(...) > hamdist(x,y): returns the Hamming distance (number of bit- > positions > where the bits differ) between x and y. x and y must be mpz, or > else > get coerced to mpz. > > >>> gmpy.hamdist(15,6) > 2 > >>> gmpy.hamdist(2**177149,10**53330) > 61877 > >>> gmpy.hamdist(2**177149-1,10**53330) > > 115285 > > > > > Thanks! -- http://mail.python.org/mailman/listinfo/python-list
Re: Hamming Distance
Great thanks! On Jun 19, 5:37 pm, Matimus <[EMAIL PROTECTED]> wrote: > On Jun 19, 4:27 pm, godavemon <[EMAIL PROTECTED]> wrote: > > > > > I need to calculate the Hamming Distance of two integers. The hamming > > distance is the number of bits in two integers that don't match. I > > thought there'd be a function in math or scipy but i haven't been able > > to find one. This is my function but it seems like there should be a > > faster way. I do this computation many times and speed up is > > important. > > > def hamdist( a, b , bits = 32): > > def _hamdist( x, bits): > > if bits: > > return (x & 1) + _hamdist(x >> 1, bits-1) > > return x & 1 > > return _hamdist( a ^ b, bits) > > > Another alternative would be to convert the XOR to a binary string and > > count the # of 1's. > > > Which would be fastest? Are there better alternatives? > > > Thanks! > > I see no good reason to use recursion for this type of thing. Here are > some of my attempts: > > [code] > from math import log > > def yours(a, b , bits = 32): > def _hamdist( x, bits): > if bits: > return (x & 1) + _hamdist(x >> 1, bits-1) > return x & 1 > return _hamdist(a ^ b, bits) > > def simple(a, b, bits=32): > x = a ^ b > return sum((x >> i & 1) for i in xrange(bits)) > > def lazy(a, b, bits=32): > x = (a ^ b) & ((1 << bits) - 1) > tot = 0 > while x: > tot += x & 1 > x >>= 1 > return tot > > def fancy(a, b, bits=32): > x = (a ^ b) & ((1 << bits) - 1) > tot = 0 > while x: > tot += 1 > x ^= 1 << int(log(x, 2)) > return tot > > test_vals = ( > ((0x, 0), 32), > ((0,0), 0), > ((1,0), 1), > ((0x8000, 0), 1), > ((0x, 0), 16) > ) > > def test(f): > test_vals = ( > ((0x, 0), 32), # ALL > ((0,0), 0), # None > ((1,0), 1), # First > ((0x8000, 0), 1), # Last > ((0x, 0), 16), # Every Other > ((0x, 0), 16), # First Half > ((0x, 0), 16), # Last Half > ) > for i, (args, exp) in enumerate(test_vals): > if f(*args) != exp: > return 0 > return 1 > > if __name__ == "__main__": > for f in (yours, simple, lazy, fancy): > if not test(f): > print "%s failed"%f.__name__ > [/code] > > The python module `timeit` is handy for testing speed: > > python -mtimeit -s"from hamdist import *" "test(yours)" > 1 loops, best of 3: 95.1 usec per loop > > python -mtimeit -s"from hamdist import *" "test(simple)" > 1 loops, best of 3: 65.3 usec per loop > > python -mtimeit -s"from hamdist import *" "test(lazy)" > 1 loops, best of 3: 59.8 usec per loop > > python -mtimeit -s"from hamdist import *" "test(fancy)" > 1 loops, best of 3: 77.2 usec per loop > > Even the ridiculous `fancy` version beat the recursive version. > > Matt -- http://mail.python.org/mailman/listinfo/python-list
