Re: An asyncio example
On Sat, Jul 4, 2015 at 8:45 AM, Adam Bartoš wrote: > On Fri, Jul 3, 2015 at 7:38 PM, Adam Bartoš wrote: >>> Ian Kelly: >>> >> 2) In the blocked situaction even KeyboardInterrupt doesn't break the loop >>> >> is that desired behavior? And why? >>> > >>> > I don't think so. When I tried this locally (using Python 3.4.0, so >>> > replacing "async def" with "def" and "await" with "yield from" and >>> > "loop.create_task" with "asyncio.async") pressing Ctrl-C did interrupt >>> > the loop >>> > >>> Ok, I'll try to get more information and I'll eventually raise an issue. >>> >>> I tried it on Python 3.4.1 and Ctrl-C doesn't interrupt the loop. Maybe >>> it has something to do with the fact I'm on Windows. >> >> >> Please try with 3.4.3, which has further asyncio changes. >> > > I originally tried with 3.5.0b2. > > This is a minimal example: import asyncio async def wait(): await asyncio.sleep(5) loop = asyncio.get_event_loop() loop.run_until_complete(wait()) Ctrl-C doesn't interrupt the waiting, instead KeyboardInterrupt occurs after those five seconds. It's 3.5.0b2 on Windows. Is it a bug? -- https://mail.python.org/mailman/listinfo/python-list
Re: An asyncio example
>
> On Fri, Jul 3, 2015 at 9:14 AM, Marko Rauhamaa
> wrote:
> >
> >>> 1) is there a way to close just one direction of the connection?
> >>
> >> No. SOCK_STREAM sockets are always bidirectional.
> >
> > socket.shutdown(socket.SHUT_WR) does the trick.
> >
> > I think the asyncio.StreamWriter.write_eof() is the high-level
> > equivalent.
>
> I stand corrected. And you're also correct about write_eof: it causes
> shutdown(SHUT_WR) to be called on the underlying socket once the
> buffer has been written.
>
>
> https://hg.python.org/cpython/file/34460219c0e0/Lib/asyncio/selector_events.py#l737
>
> That said, just replacing the writer.close() with writer.write_eof()
> in the OP's code doesn't seem to work; the server response comes back
> empty.
>
> This works:
>
> >>> sock1, sock2 = socket.socketpair()
> >>> sock1.send(b'REQUEST')
> 7
> >>> sock1.shutdown(socket.SHUT_WR)
> >>> sock2.recv(100)
> b'REQUEST'
> >>> sock2.send(b'RESPONSE')
> 8
> >>> sock1.recv(100)
> b'RESPONSE'
>
> And this works:
>
> import asyncio
> import socket
>
> def server(sock):
> request = yield from asyncio.get_event_loop().sock_recv(sock, 100)
> print("got request {!r}".format(request))
> yield from asyncio.get_event_loop().sock_sendall(sock, b'RESPONSE')
>
> def client(sock):
> yield from asyncio.get_event_loop().sock_sendall(sock, b'REQUEST')
> sock.shutdown(socket.SHUT_WR)
> response = yield from asyncio.get_event_loop().sock_recv(sock, 100)
> print("got response {!r}".format(response))
> asyncio.get_event_loop().stop()
>
> def connect():
> clientsock, serversock = socket.socketpair()
> clientsock.setblocking(False)
> serversock.setblocking(False)
> asyncio.async(client(clientsock))
> asyncio.async(server(serversock))
>
> connect()
> asyncio.get_event_loop().run_forever()
>
> I'm wondering whether there might be a bug in the higher-level
> transport code that interferes with reading after calling write_eof.
There is a bug. See http://bugs.python.org/issue24539 .
--
https://mail.python.org/mailman/listinfo/python-list
Re: calculating entropy of image or alternative?
Am 04.07.15 um 03:17 schrieb telmo bacile: Hi list, I found a code that calculates entropy of images with python that can be used for classifying interesting images from uninteresting ones. Interesting images has more structured patterns while uninsteresting are more noisy or completely homogeneous. I was thinking this code (entropy of image) can be used for measuring the level of disorder of a group of points in the image. For example: Imagine that we have 3 images, each image has 6 dots, the first one has very ordered dots , the second one have dots a little bit disordered and the third one has very dissordered dots. Then entropy of each image should measure the level of dissorganization of the dots. Your question is not a Python question, it is about image processing. Therefore x-post and fup to comp.dsp. Since you seem to be using the mailing list, you can access comp.dsp via Google Groups: https://groups.google.com/forum/#!forum/comp.dsp But the wierd thing is that when i experimented with this i got resuts without sense. The result i get is that the image with intermedium dissorder has less entropy that the very ordered image . Do anybody have an idea why im getting this result? Your model is too simple. Entropy by itself is not defined on bytes, but on a stream of symbols. Your program simply takes each pixel as a symbol. Therefore, the order is not considered at all! You could randomly shuffle all pixels, and still get the exact same result. Only the relative frequencies of the color is respected. In order to improve on that, and to take into account the distribution you would need to build a model. For example, you could predict each pixel value from the previous line and then send it to the encoder resp. the entropy calculation. This is what PNG does, for instance. As a rough estimate, you could compress your image using PNG and compare the file size. Maybe im misunderstanding something. Is it possible to use entropy of image to measure the level of dissorganization of a group of points in an image? If not , is there is another method for doing this? In principle yes Christian thanks in advance T. here is the code: import math from PIL import Image imageFile = 'int2.jpg' print imageFile im = Image.open(imageFile) rgbHistogram = im.histogram() print 'Snannon Entropy for Red, Green, Blue:' for rgb in range(3): totalPixels = sum(rgbHistogram[rgb * 256 : (rgb + 1) * 256]) ent = 0.0 for col in range(rgb * 256, (rgb + 1) * 256): freq = float(rgbHistogram[col]) / totalPixels if freq > 0: ent = ent + freq * math.log(freq, 2) ent = -ent print ent -- https://mail.python.org/mailman/listinfo/python-list
Re: An asyncio example
On Sat, Jul 4, 2015 at 1:07 PM, Adam Bartoš wrote:
> On Fri, Jul 3, 2015 at 9:14 AM, Marko Rauhamaa
>> wrote:
>> >
>> >>> 1) is there a way to close just one direction of the connection?
>> >>
>> >> No. SOCK_STREAM sockets are always bidirectional.
>> >
>> > socket.shutdown(socket.SHUT_WR) does the trick.
>> >
>> > I think the asyncio.StreamWriter.write_eof() is the high-level
>> > equivalent.
>>
>> I stand corrected. And you're also correct about write_eof: it causes
>> shutdown(SHUT_WR) to be called on the underlying socket once the
>> buffer has been written.
>>
>>
>> https://hg.python.org/cpython/file/34460219c0e0/Lib/asyncio/selector_events.py#l737
>>
>> That said, just replacing the writer.close() with writer.write_eof()
>> in the OP's code doesn't seem to work; the server response comes back
>> empty.
>>
>> This works:
>>
>> >>> sock1, sock2 = socket.socketpair()
>> >>> sock1.send(b'REQUEST')
>> 7
>> >>> sock1.shutdown(socket.SHUT_WR)
>> >>> sock2.recv(100)
>> b'REQUEST'
>> >>> sock2.send(b'RESPONSE')
>> 8
>> >>> sock1.recv(100)
>> b'RESPONSE'
>>
>> And this works:
>>
>> import asyncio
>> import socket
>>
>> def server(sock):
>> request = yield from asyncio.get_event_loop().sock_recv(sock, 100)
>> print("got request {!r}".format(request))
>> yield from asyncio.get_event_loop().sock_sendall(sock, b'RESPONSE')
>>
>> def client(sock):
>> yield from asyncio.get_event_loop().sock_sendall(sock, b'REQUEST')
>> sock.shutdown(socket.SHUT_WR)
>> response = yield from asyncio.get_event_loop().sock_recv(sock, 100)
>> print("got response {!r}".format(response))
>> asyncio.get_event_loop().stop()
>>
>> def connect():
>> clientsock, serversock = socket.socketpair()
>> clientsock.setblocking(False)
>> serversock.setblocking(False)
>> asyncio.async(client(clientsock))
>> asyncio.async(server(serversock))
>>
>> connect()
>> asyncio.get_event_loop().run_forever()
>>
>> I'm wondering whether there might be a bug in the higher-level
>> transport code that interferes with reading after calling write_eof.
>
>
> There is a bug. See http://bugs.python.org/issue24539 .
>
>
The following code also works.
import asyncio
import socket
async def server(reader, writer):
print("server is starting")
print("server is going to receive a request")
request = (await reader.read()).decode()
print("server received request {!r}".format(request))
response = "RESPONSE"
writer.write(response.encode())
print("server sent response {!r}".format(response))
writer.write_eof()
print("server sent EOF")
print("server is finishing")
async def client(reader, writer):
print("client is starting")
request = "REQUEST"
writer.write(request.encode())
print("client sent request {!r}".format(request))
writer.write_eof()
print("client sent EOF")
print("client is going to receive a response")
response = (await reader.read()).decode()
print("client received response {!r}".format(response))
print("client is finishing")
class FixedStreamReaderProtocol(asyncio.StreamReaderProtocol):
def eof_received(self):
super().eof_received()
return True # keep alive
async def open_connection(*, loop, sock):
reader = asyncio.StreamReader(loop=loop)
protocol = FixedStreamReaderProtocol(reader, loop=loop)
transport, _ = await loop.create_connection(lambda: protocol,
sock=sock)
writer = asyncio.StreamWriter(transport, protocol, reader, loop)
return reader, writer
async def connect(loop):
serversock, clientsock = socket.socketpair()
reader, writer = await open_connection(sock=serversock, loop=loop)
server_coro = server(reader, writer)
reader, writer = await open_connection(sock=clientsock, loop=loop)
client_coro = client(reader, writer)
server_task = loop.create_task(server_coro)
client_task = loop.create_task(client_coro)
return server_task, client_task
loop = asyncio.get_event_loop()
server_task, client_task = loop.run_until_complete(connect(loop))
loop.run_until_complete(server_task)
loop.run_until_complete(client_task)
--
https://mail.python.org/mailman/listinfo/python-list
Re: installing libraries like numpy scipy matplotlib
PIYUSH KUMAR wrote: > I have never used linux in my life.. only windows based computing.. So I > have problems in installing third party libraries in python. It depends. One question is if there's already a ready-for-use package for the third party library you want to install. If that is the case then the next question is which distro you're using - there are different package-management systems. Since you mentioned Pyke: if you got Ubuntu or Debian there's a package for it you can simply install it using the command sudo apt-get install python-pyke (The 'sudo' bit is for temporarily assuming the permissions to install the package, you're going to be asked for your password. 'apt-get' is the program for installing and de- installing packages. And 'python-pyke' is the name of the package._ If you also want the documentation installed add 'python-pyke-doc'. If you have some other distribution there might be a different program for installing new packages. And there's basically always also some program with a graphical user interface, wrapped around that and which shows you which packages exist (thousands). If there's no package for what you want you need to download the sources and install them yourself. For the details there usually is a file named README.txt and/or INSTALL.txt (or similar). Often all you need to use is the three commands ./configure make sudo make install If what you want to install depends on further software you also need to install that before creating and installing of what you need will succeed. So, again check if a ready-made package for that dependencies available or download, compile and install that. This can, in some cases, become a bit of work if A need B, B in turn need C and D, D needs E etc.;-) > So can somebody just explain me how many softwares or other python packages > I have to install before installing any of these. Sorry, but I don't understand this sentence. But in case for what you want to install there's a package for your distro you don't need to worry at all - all dependencies will get installed automatically (so if you want to install package A and that needs package B, B will installed automatically before A is installed). > I also had problems installing PYKE last month. What were these problems? Best regards, Jens -- \ Jens Thoms Toerring ___ j...@toerring.de \__ http://toerring.de -- https://mail.python.org/mailman/listinfo/python-list
Re: Matplotlib X-axis timezone trouble [SOLVED]
On Sat, 04 Jul 2015 07:29:45 +0300, Akira Li <4kir4...@gmail.com> wrote:
> Peter Pearson writes:
>
>> The following code produces a plot with a line running from (9:30, 0) to
>> (10:30, 1), not from (8:30, 0) to (9:30, 1) as I desire.
>>
>> If I use timezone None instead of pacific, the plot is as desired, but
>> of course that doesn't solve the general problem of which this is a
>> much-reduced example.
>>
>> If I use timezone US/Central, I get the same (bad) plot.
>>
>> import matplotlib.pyplot as plt
>> import datetime
>> import pytz
>> pacific = pytz.timezone("US/Pacific")
>> fig = plt.figure()
>> plt.plot([datetime.datetime(2014, 10, 7, 8, 30, tzinfo=pacific),
>> datetime.datetime(2014, 10, 7, 9, 30, tzinfo=pacific)],
>> [0,1], marker="o", color="green")
>> fig.autofmt_xdate()
>> plt.show()
>>
>> Does anybody know why this shift is occurring? Is Matplotlib
>> confused about what timezone to use in labeling the axis? How
>> would I tell it what timezone to use (preferably explicitly in
>> the code, not in matplotlibrc)?
>>
>
> Your pytz usage is incorrect.
>
> Don't pass a pytz tzinfo object to the datetime construtor directly, use
> `.localize()` method instead. Read the note at the very beginning of
> pytz docs http://pytz.sourceforge.net/
Exactly. Thank you.
For newcomers, the denouement of this thread is this:
* Matplotlib had nothing to do with this problem, it was correctly
displaying bad datetime.datetime values.
* Python's datetime.datetime(..., tzinfo=timezone) is unreliable if
timezone has daylight-saving time.
* pytz's creators provide the localize() method specifically to
remedy this problem. If you want to create a datetime object
that has a timezone that might have daylight-saving time,
use localize().
--
To email me, substitute nowhere->runbox, invalid->com.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Should iPython Notebook replace Idle
Jason Swails wrote: > Everything gets swallowed into Python. I can't imagine this ever happening. IPython's successor Jupyter is also an REPL environment for Julia and R, and many other languages will also be supported (e.g. Java and C++). Having this swallowed into Python is probably never going to happen. IIRC, the current release is the last to be named IPython. Sturla -- https://mail.python.org/mailman/listinfo/python-list
Re: An asyncio example
On 7/4/2015 3:04 AM, Adam Bartoš wrote: This is a minimal example: import asyncio async def wait(): await asyncio.sleep(5) loop = asyncio.get_event_loop() loop.run_until_complete(wait()) Ctrl-C doesn't interrupt the waiting, instead KeyboardInterrupt occurs after those five seconds. It's 3.5.0b2 on Windows. Is it a bug? I asked on pydev list and was pointed to https://bugs.python.org/issue23057 (where treated as missing feature ). It is desired that this stop immediately on Windows as on Unix. I suggest you post your minimal example there. User interest in an issue being fixed and willingness to test patches can help motivate. Even more minimal: import asyncio loop = asyncio.get_event_loop() loop.run_forever also not interruptible, -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: installing libraries like numpy scipy matplotlib
On 7/4/2015 10:58 AM, Jens Thoms Toerring wrote: PIYUSH KUMAR wrote: I have never used linux in my life.. only windows based computing.. So I have problems in installing third party libraries in python. The numpy and scipy projects create Windows binararies for all recent releases that are available on pypi.python.org and installed with pip install scipy for instance. Many other ditto. http://www.lfd.uci.edu/~gohlke/pythonlibs/ provides binaries for about a hundred popular packages. pip can access these also (see page header). -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Bug in floating point multiplication
In a message of Fri, 03 Jul 2015 00:52:55 +1000, "Steven D'Aprano" writes: >Despite the title, this is not one of the usual "Why can't Python do >maths?" "bug" reports. > >Can anyone reproduce this behaviour? If so, please reply with the version of >Python and your operating system. Printing sys.version will probably do. > > >x = 1 - 1/2**53 >assert x == 0. >for i in range(1, 100): >if int(i*x) == i: >print(i); break > > >Using Jython and IronPython, the loop runs to completion. That is the >correct behaviour, or so I am lead to believe. Using Python 2.6, 2.7 and >3.3 on Centos and Debian, it prints 2049 and breaks. That should not >happen. If you can reproduce that (for any value of i, not necessarily >2049), please reply. > >See also http://bugs.python.org/issue24546 for more details. > > > >-- >Steven PyPy says: Python 2.7.9 (2.5.1+dfsg-1, Mar 27 2015, 19:45:43) [PyPy 2.5.1 with GCC 4.9.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. x = 1 - 1/2**53 assert x == 0. Traceback (most recent call last): File "", line 1, in AssertionError for i in range(1, 100): if int(i*x) == i: print(i); break 1 So the loop terminates, but there is an Assertion Error. Did you want that? Laura -- https://mail.python.org/mailman/listinfo/python-list
Re: Should iPython Notebook replace Idle
On Sunday, 5 July 2015 05:16:04 UTC+10, Sturla Molden wrote: > Jason Swails wrote: > > > Everything gets swallowed into Python. I can't imagine this ever happening. > > IPython's successor Jupyter is also an REPL environment for Julia and R, > and many other languages will also be supported (e.g. Java and C++). > Having this swallowed into Python is probably never going to happen. IIRC, > the current release is the last to be named IPython. > > Sturla Yeah I listened to the recent podcast talk python I believe and the creators of ipython & notebooks were on there and they were putting further improvements into it as they are using it in education for python and wanted to address some of the pain points so that it would be an improved environment for python. It will probably be that the learning environment for python but just not by default. sayth -- https://mail.python.org/mailman/listinfo/python-list
Can't call file from another - well correctly
I was playing with odo(blaze http://blaze.pydata.org/en/latest/) and wanted to
use it with a current script I have been using on the command line.
So my 2 scripts are below, I will explain here hopefully to keep question
clearer what I have done. Script 2 works for me from the command line as
python clean.py some.csv
i wanted to use my script to fix up a bad csv and then use odo to a dataframe
and hopefully build upon this later.
When I run script 1 I get the error I need more than 1 value to unpack, which
makes sense in that I have Script and Filename.
##Error###
C:\Users\sayth\Repos\Notebooks>python odo_test.py
Traceback (most recent call last):
File "odo_test.py", line 3, in
import clean
File "C:\Users\sayth\Repos\Notebooks\clean.py", line 9, in
SCRIPT, FILENAME = argv
ValueError: need more than 1 value to unpack
But if I change script2 to have just FILENAME = argv I get this error and I am
not sure what to do.
##Error###
C:\Users\sayth\Repos\Notebooks>python odo_test.py
Traceback (most recent call last):
File "odo_test.py", line 3, in
import clean
File "C:\Users\sayth\Repos\Notebooks\clean.py", line 62, in
MY_FILE = out_file_name(FILENAME)
File "C:\Users\sayth\Repos\Notebooks\clean.py", line 15, in out_file_name
file_parts = file_name.split(".",)
AttributeError: 'list' object has no attribute 'split'
What can i do?
##Scripts ###
# Script 1
from odo import odo
import pandas as pd
import clean
print(argv)
myFile = race_table('20150704RHIL0.csv')
odo(myFile, pd.DataFrame)
# Script 2
import csv
import re
from sys import argv
SCRIPT, FILENAME = argv
#FILENAME = argv
def out_file_name(file_name):
"""take an input file and keep the name with appended _clean"""
file_parts = file_name.split(".",)
output_file = file_parts[0] + '_clean.' + file_parts[1]
return output_file
def race_table(text_file):
"""utility to reorganise poorly made csv entry"""
output_table = []
for record in text_file:
if record[0] == 'Meeting':
meeting = record[3]
rail = record[6]
weather = record[7]
track = record[8]
elif record[0] == 'Race':
date = record[13]
race = record[1]
benchmark = record[4]
distance = record[5]
elif record[0] == 'Horse':
number = record[1]
name = record[2]
jockey = record[6]
barrier = record[7]
weight = record[8]
results = record[9]
res_split = re.split('[- ]', results)
starts = res_split[0]
wins = res_split[1]
seconds = res_split[2]
thirds = res_split[3]
try:
prizemoney = res_split[4]
except IndexError:
prizemoney = 0
trainer = record[4]
location = record[5]
b_rating = record[15]
sex = record[16]
print(name, wins, seconds)
output_table.append((meeting, date, rail, weather, track, distance,
benchmark, race, number, name, sex, b_rating,
weight, barrier, starts, wins, seconds,
thirds, prizemoney, trainer, location, jockey
))
return output_table
MY_FILE = out_file_name(FILENAME)
# with open(FILENAME, 'r') as f_in, open(MY_FILE, 'w') as f_out:
# for line in race_table(f_in.readline()):
# new_row = line
with open(FILENAME, 'r') as f_in, open(MY_FILE, 'w') as f_out:
CONTENT = csv.reader(f_in)
# print(content)
FILE_CONTENTS = race_table(CONTENT)
# print new_name
# f_out.write(str(FILE_CONTENTS))
headers = ['MEETING', 'DATE', 'RAIL', 'WEATHER', 'TRACK', 'DISTANCE',
'BENCHMARK', 'RACE', 'NUMBER', 'NAME', 'SEX', 'B_RATING',
'WEIGHT', 'BARRIER', 'STARTS', 'WINS', 'SECONDS', 'THIRDS',
'PRIZEMONEY', 'TRAINER', 'LOCATION', 'JOCKEY']
f_csv = csv.writer(f_out)
f_csv.writerow(headers)
f_csv.writerows(FILE_CONTENTS)
# Old implementation for reference
# input_table = [[item.strip(' "') for item in record.split(',')]
#for record in text_file.splitlines()]
# At this point look at input_table to find the record indices
# identity = string.maketrans("", "")
# print(input_table)
# input_table = [s.translate(identity, ",'") for s
#in input_table]
if __name__ == '__main__':
main()
many thanks for your time.
Sayth
--
https://mail.python.org/mailman/listinfo/python-list
Re: Searching for a usable X509 implementation
In a message of Fri, 03 Jul 2015 17:11:10 -0700, Dennis Jacobfeuerborn writes: >Hi, >I'm trying to implement certificate functionality in a python app but after >fighting with pyOpenSSL and M2Crypto I'm thinking about writing wrapper >functions for the OpenSSL command line tool instead or switching the app to >another language all together. > >Apparently PyOpenSSL has no way to save a public key to a file which is >baffling. M2Crypto has that ability but apparently no usable way to verify a >certificate? PyOpenSSL does, you must have missed it when looking. You are looking for OpenSSL.crypto.dump_certificate(type, cert) Dump the certificate cert into a buffer string encoded with the type type. Laura -- https://mail.python.org/mailman/listinfo/python-list
Re: Can't call file from another - well correctly
On Sunday, 5 July 2015 10:23:17 UTC+10, Sayth Renshaw wrote:
> I was playing with odo(blaze http://blaze.pydata.org/en/latest/) and wanted
> to use it with a current script I have been using on the command line.
>
> So my 2 scripts are below, I will explain here hopefully to keep question
> clearer what I have done. Script 2 works for me from the command line as
>
> python clean.py some.csv
>
> i wanted to use my script to fix up a bad csv and then use odo to a
> dataframe and hopefully build upon this later.
>
> When I run script 1 I get the error I need more than 1 value to unpack, which
> makes sense in that I have Script and Filename.
>
> ##Error###
> C:\Users\sayth\Repos\Notebooks>python odo_test.py
> Traceback (most recent call last):
> File "odo_test.py", line 3, in
> import clean
> File "C:\Users\sayth\Repos\Notebooks\clean.py", line 9, in
> SCRIPT, FILENAME = argv
> ValueError: need more than 1 value to unpack
>
> But if I change script2 to have just FILENAME = argv I get this error and I
> am not sure what to do.
>
> ##Error###
> C:\Users\sayth\Repos\Notebooks>python odo_test.py
> Traceback (most recent call last):
> File "odo_test.py", line 3, in
> import clean
> File "C:\Users\sayth\Repos\Notebooks\clean.py", line 62, in
> MY_FILE = out_file_name(FILENAME)
> File "C:\Users\sayth\Repos\Notebooks\clean.py", line 15, in out_file_name
> file_parts = file_name.split(".",)
> AttributeError: 'list' object has no attribute 'split'
>
> What can i do?
>
> ##Scripts ###
>
> # Script 1
>
> from odo import odo
> import pandas as pd
> import clean
>
> print(argv)
> myFile = race_table('20150704RHIL0.csv')
>
>
> odo(myFile, pd.DataFrame)
>
> # Script 2
>
> import csv
> import re
> from sys import argv
> SCRIPT, FILENAME = argv
> #FILENAME = argv
>
>
> def out_file_name(file_name):
> """take an input file and keep the name with appended _clean"""
> file_parts = file_name.split(".",)
> output_file = file_parts[0] + '_clean.' + file_parts[1]
> return output_file
>
>
> def race_table(text_file):
> """utility to reorganise poorly made csv entry"""
> output_table = []
> for record in text_file:
> if record[0] == 'Meeting':
> meeting = record[3]
> rail = record[6]
> weather = record[7]
> track = record[8]
> elif record[0] == 'Race':
> date = record[13]
> race = record[1]
> benchmark = record[4]
> distance = record[5]
> elif record[0] == 'Horse':
> number = record[1]
> name = record[2]
> jockey = record[6]
> barrier = record[7]
> weight = record[8]
> results = record[9]
> res_split = re.split('[- ]', results)
> starts = res_split[0]
> wins = res_split[1]
> seconds = res_split[2]
> thirds = res_split[3]
> try:
> prizemoney = res_split[4]
> except IndexError:
> prizemoney = 0
> trainer = record[4]
> location = record[5]
> b_rating = record[15]
> sex = record[16]
> print(name, wins, seconds)
> output_table.append((meeting, date, rail, weather, track,
> distance,
> benchmark, race, number, name, sex, b_rating,
> weight, barrier, starts, wins, seconds,
> thirds, prizemoney, trainer, location, jockey
> ))
> return output_table
>
> MY_FILE = out_file_name(FILENAME)
>
> # with open(FILENAME, 'r') as f_in, open(MY_FILE, 'w') as f_out:
> # for line in race_table(f_in.readline()):
> # new_row = line
> with open(FILENAME, 'r') as f_in, open(MY_FILE, 'w') as f_out:
> CONTENT = csv.reader(f_in)
> # print(content)
> FILE_CONTENTS = race_table(CONTENT)
> # print new_name
> # f_out.write(str(FILE_CONTENTS))
> headers = ['MEETING', 'DATE', 'RAIL', 'WEATHER', 'TRACK', 'DISTANCE',
>'BENCHMARK', 'RACE', 'NUMBER', 'NAME', 'SEX', 'B_RATING',
>'WEIGHT', 'BARRIER', 'STARTS', 'WINS', 'SECONDS', 'THIRDS',
>'PRIZEMONEY', 'TRAINER', 'LOCATION', 'JOCKEY']
>
> f_csv = csv.writer(f_out)
> f_csv.writerow(headers)
> f_csv.writerows(FILE_CONTENTS)
>
>
> # Old implementation for reference
> # input_table = [[item.strip(' "') for item in record.split(',')]
> #for record in text_file.splitlines()]
> # At this point look at input_table to find the record indices
> # identity = string.maketrans("", "")
> # print(input_table)
> # input_table = [s.translate(identity, ",'") for s
> #in input_table]
>
> if __name__ == '__main__':
> main()
>
>
> many thanks for your time.
>
> Sayth
Solved it, wel
Re: Searching for a usable X509 implementation
In a message of Sun, 05 Jul 2015 02:27:22 +0200, Laura Creighton writes: >In a message of Fri, 03 Jul 2015 17:11:10 -0700, Dennis Jacobfeuerborn writes: >>Hi, >>I'm trying to implement certificate functionality in a python app but after >>fighting with pyOpenSSL and M2Crypto I'm thinking about writing wrapper >>functions for the OpenSSL command line tool instead or switching the app to >>another language all together. >> >>Apparently PyOpenSSL has no way to save a public key to a file which is >>baffling. M2Crypto has that ability but apparently no usable way to verify a >>certificate? > >PyOpenSSL does, you must have missed it when looking. >You are looking for OpenSSL.crypto.dump_certificate(type, cert) >Dump the certificate cert into a buffer string encoded with the type type. > >Laura Excuse me. I misunderstood your mail. You only want to save the public key, and not a certificate or a certificate request. I don't see a way to do this in PEM or ASN.1 format. For an RSA key in PEM format you can do: from OpenSSL.crypto import _new_mem_buf, _lib, _bio_to_string def dump_rsa_public_key(pkey): bio = _new_mem_buf() result = _lib.PEM_write_bio_RSAPublicKey(bio, _lib.EVP_PKEY_get1_RSA(pkey._ pkey)) # if result == 0: ERROR! Figure out what you want to do here ... return _bio_to_string(bio) There are similar things for other formats and DSA keys. The original version of PyOpenSSL was written by Martin Sjögren, when he was working for me, and we had no need for such a thing at the time, since we just saved full certificates. You are right that it is very odd that nobody else has needed them since then, and this probably should be added to PyOpenSSL. Laura -- https://mail.python.org/mailman/listinfo/python-list
Re: Python 3 resuma a file download
I have a working solution. :)
The function below will download a file securely.
Thank anyone who helped. I wouldn't be able to write this function without your
help.
I hope, someone else will benefit from our united work.
Best regards.
import os
import urllib.request
def Download(rfile, lfile):
lsize = -1
rsize = -2
while True:
try:
if os.path.isfile(lfile):
lsize = os.stat(lfile).st_size
else:
lsize = 0
req = urllib.request.Request(rfile)
rsize = urllib.request.urlopen(req).length
if lsize == rsize:
break
req.add_header('Range', "bytes={}-".format(lsize))
response = urllib.request.urlopen(req)
with open(lfile, 'ab') as out_file:
chunk = response.read(64 * 1024)
if not chunk:
break
out_file.write(chunk)
out_file.flush()
lsize = os.stat(lfile).st_size
prc_dloaded = round(lsize / rsize * 100, 2)
print(prc_dloaded)
if prc_dloaded == 100:
break
except ConnectionResetError as e:
print('Exception ConnectionResetError {0}
%'.format(prc_dloaded))
if lsize == rsize:
retval = True
else:
retval = False
return retval
while not Download('http://video.hrt.hr/2906/otv296.mp4', 'otv296.mp4'):
print('1')
--
https://mail.python.org/mailman/listinfo/python-list
EuroPython 2015 Keynote: Holger Krekel
We are pleased to introduce our next keynote speaker for EuroPython 2015: Holger Krekel. He will be giving a keynote on Wednesday, July 22. About Holger Krekel --- Holger is a prolific Python developer with a strong interest in communication: “Socially this means engaging and co-organizing neighborhoods and technically it means i am interested in distributed systems and thriving to make available and built better communication machines for other people.” He also is a proud father and loves to dance to “electronic swing” music. Python projects --- You will probably know Holger as author of the well-known pytest testing framework and co-founder the PyPy project: “When i discovered Python I was thrilled by its high-level constructs and introspection facilities. I am still thrilled by the idea of dynamically deploying and executing high level programs on the net. In my view, Python and testing are a wonderfully productive combination for writing software. Out of this conviction, I founded and co-developed the PyPy project and maintain the pytest and tox testing tools. I also maintain a number of other projects, among the more popular are execnet for ad-hoc cross-interpreter communication and the py lib. Most of my code you find at bitbucket/hpk42.” The coding culture in almost all his projects consists of test- and documentation-driven development and applying meta programming techniques. The Keynote: Towards a more effective, decentralized web In this talk, Holger will discuss the recent rise of immutable state concepts in languages and network protocols: “The advent of hash-based data structures and replication strategies are shaking the client/server web service paradigm which rests on managing mutable state through HTTP. By contrast, building on git, bittorrent and other content addressed data structures provides for a more secure, efficient decentralized communication topology. There are projects, thoughts and talk to create new web standards to bring such technologies to mass deployment and fuel a new wave of decentralization. What can Python bring to the table?” Enjoy, -- EuroPython 2015 Team http://ep2015.europython.eu/ http://www.europython-society.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: Looking up a dictionary _key_ by key?
In a message of Tue, 23 Jun 2015 18:06:45 -0700, Paul Rubin writes: >Chris Angelico writes: >>> Would I have to do an O(n) search to find my key? >> Iterate over it - it's an iterable view in Py3 - and compare. > >I think the question was whether the O(n) search could be avoided, not >how to do it. I don't see a way to avoid it. There is fundamental >brokenness in having unequal objects compare as equal, and the breakage >messes up the dictionary when those objects are used as keys. > >Solution is to either fix the object equality test, or wrap them in >something (maybe a tuple containing the objects and the distinguishing >fields that are missing from the original object's equality method) that >treats unequal objects as unequal. >-- >https://mail.python.org/mailman/listinfo/python-list This just showed up in my mailbox: Subject: [ANN] pyskiplist-1.0.0 From:Geert Jansen PySkipList is a fast, pure Python implementation of an indexable skiplist. It implements a SkipList data structure that provides an always sorted, list-like data structure for (key, value) pairs. ... more details including timing. For the full text see https://github.com/geertj/pyskiplist It's also available on PyPI. Looks to me as if he's fixed the 0(n) problem Laura -- https://mail.python.org/mailman/listinfo/python-list
