Re: asyncio and blocking - an update
"Frank Millman" wrote in message news:n9hjfp$ad7$1...@ger.gmane.org... However, my concern is not to maximise database performance, but to ensure that in an asynchronous environment, one task does not block the others from responding. My tests simulate a number of tasks running concurrently and trying to access the database. Among other measurements, I track the time that each database access commences. As I expected, tasks run with 'run_in_executor' run sequentially, i.e. the next one only starts when the previous one has finished. This is not because the tasks themselves are sequential, but because 'fetchall()' is (I think) a blocking operation. Conversely, with my approach, all the tasks start within a short time of each other. Because I can process the rows as they are received, it seems to give each task a fairer time allocation. Not to mention that there are very likely to be other non-database tasks running concurrently, and they should also be more responsive. It would be quite difficult to simulate all of this, so I confess that I am relying on gut instinct at the moment. It seems that my gut instinct was correct. Up to now my timing tests have been run independently of my app, but now I have embedded them so that I can run the tests while I am logged in as a user. I run a task every 10 seconds that runs 25 concurrent tasks, each reading a database table of about 2000 rows. Using run_in_executor() and cur.fetchall(), I experience delays of up to 2 seconds while the task is active. Using my approach, the maximum I saw was about a tenth of a second, and that is because I was looking for it - a normal user would not notice. Obviously the task took longer, but I can live with that trade-off. As I mentioned before, I could be using run_in_executor() in a naïve way, and there could be better approaches. But until someone points out a better way, I have nothing else to go on. Frank -- https://mail.python.org/mailman/listinfo/python-list
Re: asyncio and blocking - an update
On Fri, Feb 12, 2016 at 7:17 PM, Frank Millman wrote: > Using run_in_executor() and cur.fetchall(), I experience delays of up to 2 > seconds while the task is active. > > Using my approach, the maximum I saw was about a tenth of a second, and that > is because I was looking for it - a normal user would not notice. Obviously > the task took longer, but I can live with that trade-off. That looks like good, useful data! Cool. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: tarfile : read from a socket?
On 02/11/2016 06:27 PM, Lars Gustäbel wrote:
> On Thu, Feb 11, 2016 at 04:41:43PM +, Ulli Horlacher wrote:
>> sfo = sock.makefile('r')
>> taro = tarfile.open(fileobj=sfo,mode='r|')
>> taro.extractall(path=edir)
> What about using an iterator?
>
> def myiter(tar):
> for t in tar:
> print "extracting", t.name
> yield t
>
> sfo = sock.makefile('r')
> taro = tarfile.open(fileobj=sfo,mode='r|')
> taro.extractall(members=myiter(taro),path=edir)
>
> Cheers,
The tarfile is already an iterator. Just do the following: for ti in
taro: print "extracting", ti.name taro.extract(ti)
--
https://mail.python.org/mailman/listinfo/python-list
Re: tarfile : read from a socket?
On Fri, Feb 12, 2016 at 09:35:40AM +0100, Antoon Pardon wrote:
> On 02/11/2016 06:27 PM, Lars Gustäbel wrote:
> > What about using an iterator?
> >
> > def myiter(tar):
> > for t in tar:
> > print "extracting", t.name
> > yield t
> >
> > sfo = sock.makefile('r')
> > taro = tarfile.open(fileobj=sfo,mode='r|')
> > taro.extractall(members=myiter(taro),path=edir)
> >
> The tarfile is already an iterator. Just do the following: for ti in
> taro: print "extracting", ti.name taro.extract(ti)
The extractall() method does a little bit more than just extract(), i.e.
setting directory mtimes, see
https://docs.python.org/3.5/library/tarfile.html#tarfile.TarFile.extractall
--
Lars Gustäbel
l...@gustaebel.de
--
https://mail.python.org/mailman/listinfo/python-list
Python study
hello thank you very much for this intersting programming languch now I finished study Fundamentals of python in microsift and now im asking about what course should study after that and in web or game or networking? thank you for your time mz -- https://mail.python.org/mailman/listinfo/python-list
working
hello i want to know the company that ican work as freelance python devloper -- https://mail.python.org/mailman/listinfo/python-list
Re: tarfile : read from a socket?
Lars Gustäbel wrote:
> On Fri, Feb 12, 2016 at 09:35:40AM +0100, Antoon Pardon wrote:
> > On 02/11/2016 06:27 PM, Lars Gustäbel wrote:
> > > What about using an iterator?
> > >
> > > def myiter(tar):
> > > for t in tar:
> > > print "extracting", t.name
> > > yield t
> > >
> > > sfo = sock.makefile('r')
> > > taro = tarfile.open(fileobj=sfo,mode='r|')
> > > taro.extractall(members=myiter(taro),path=edir)
> > >
> > The tarfile is already an iterator. Just do the following: for ti in
> > taro: print "extracting", ti.name taro.extract(ti)
>
> The extractall() method does a little bit more than just extract(), i.e.
> setting directory mtimes, see
> https://docs.python.org/3.5/library/tarfile.html#tarfile.TarFile.extractall
This is an important hint!
Thanks!
--
Ullrich Horlacher Server und Virtualisierung
Rechenzentrum IZUS/TIK E-Mail: horlac...@tik.uni-stuttgart.de
Universitaet Stuttgart Tel:++49-711-68565868
Allmandring 30aFax:++49-711-682357
70550 Stuttgart (Germany) WWW:http://www.tik.uni-stuttgart.de/
--
https://mail.python.org/mailman/listinfo/python-list
I cannot open IDLE
I have downloaded python but I cannot open up idle. I really need this issue resolved for I have work do for one of my computer science class. -- https://mail.python.org/mailman/listinfo/python-list
Re: Best programs written completly in Python
On Sunday, August 5, 2007 at 3:14:38 AM UTC-7, Franz Steinhäusler wrote: > Hello NG, > > wWhat are the best programs in your opinion, written entirly > in pyhton, divided into categories like: Maybe such a list of categories should include a) Artificial Intelligence > a) Games > b) Utilities/System > c) Office > d) Web/Newsreader/Mail/Browser > ... > > I don't want to start a long thread, if a site of such > an discussion already exists, a link will be enough. > > Many thanks in advance! > > -- > Franz Steinhaeusler A winner-take-all genuine "killer app" for Python would be a port of the webserver "Ghost" Strong AI artificial intelligence being ported from Forth first into Perl5 and then the newly released Perl6. See http://dl.acm.org/citation.cfm?doid=307824.307853 on Mind.Forth AI; http://aihub.net/artificial-intelligence-lab-projects http://ai.neocities.org/perlmind.txt -- Download URL; http://ai.neocities.org/P6AI_FAQ.html -- Frequently Asked Questions; http://ai.neocities.org/P6AI_man.html -- Perl6 AI User Manual; http://ai.neocities.org/PMPJ.html -- Perl Mind Programming Journal; http://www.amazon.com/dp/B00FKJY1WY shows how the Perl AI reasons. May the best programming language win on the way to the Singularity! Respectfully submitted AiHasBeenSolved -- https://mail.python.org/mailman/listinfo/python-list
Re: I cannot open IDLE
On 11 February 2016 at 18:02, anthony averett wrote: > I have downloaded python but I cannot open up idle. I really need this > issue resolved for I have work do for one of my computer science class. What operating system are you using? Windows? Which version? Have you installed Python and if so where did you get it from? Assuming you're on Windows and used the normal Python installer and that you installed Python (rather than simply downloading the installer) then there should be a menu item for IDLE somewhere in your programs menu. -- Oscar -- https://mail.python.org/mailman/listinfo/python-list
[ANN] MicroPython 1.6
Hello, MicroPython is a lean and efficient Python implementation for microcontrollers, embedded, and mobile systems (which also runs just as fine on desktops, servers, and clouds). https://github.com/micropython/micropython https://github.com/micropython/micropython/releases/tag/v1.6 There're following major changes since 1.5: 1. LwIP module support for embedded TCP/IP networking. 2. IPv6 support in the Unix port. 3. Beta support for persistent bytecode (similar to CPython's .pyc) 4. 64-bit NaN boxing (improved floating-point performance if enabled). 5. Support for new official PyBoards PYBv1.1 and PYBLITEv1.0. 6. Long int constant folding during bytecode compilation (glad that CPython will catch up in that area thanks to FAT Python project). 7. There's a ongoing crowdfunding campaign to fund complete and well-maintained MicroPython port to ubiquitous ESP8266 WiFi SoC, and improve networking and IoT support in MicroPython in general: https://www.kickstarter.com/projects/214379695/micropython-on-the-esp8266-beautifully-easy-iot -- Best regards, Paul mailto:pmis...@gmail.com -- https://mail.python.org/mailman/listinfo/python-list
Re: tarfile : secure extract?
On Thu, Feb 11, 2016, at 18:24, Ulli Horlacher wrote: > A better approach would be to rename such files while extracting. > Is this possible? What happens if you change member.name before extracting? -- https://mail.python.org/mailman/listinfo/python-list
Re: Storing a big amount of path names
Chris Angelico wrote: > Start by coding things in the > simple and obvious way, and then fix problems only when you see them. Is that statement available in 10 foot letters etched into stone? -- Rob Gaddi, Highland Technology -- www.highlandtechnology.com Email address domain is currently out of order. See above to fix. -- https://mail.python.org/mailman/listinfo/python-list
Re: Storing a big amount of path names
On Sat, Feb 13, 2016 at 4:05 AM, Rob Gaddi wrote: > Chris Angelico wrote: > >> Start by coding things in the >> simple and obvious way, and then fix problems only when you see them. > > Is that statement available in 10 foot letters etched into stone? I actually had that built behind my house, at one point. Sadly, the letters sank until they were partly embedded into the ground, and what's left says, in the local language, "Go stick your head in a PHP", so it's lit up only for special celebrations. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: working
On Friday, February 12, 2016 at 1:47:24 AM UTC-8, Mohammed Zakria wrote: > hello > i want to know the company that ican work as freelance python devloper There are some recruiters that read this mailing list and will send unsolicited e-mail about job openings, but they might pass right over you if you're not willing to spend the time to proofread your messages. -- https://mail.python.org/mailman/listinfo/python-list
Re: tarfile : secure extract?
On Thu, Feb 11, 2016 at 11:24:01PM +, Ulli Horlacher wrote: > In https://docs.python.org/2/library/tarfile.html there is a warning: > > Never extract archives from untrusted sources without prior inspection. > It is possible that files are created outside of path, e.g. members that > have absolute filenames starting with "/" or filenames with two dots > "..". > > My program has to extract tar archives from untrusted sources :-} Read the discussion in this issue on why this might be a bad idea: http://bugs.python.org/issue21109 -- Lars Gustäbel l...@gustaebel.de -- https://mail.python.org/mailman/listinfo/python-list
Re: tarfile : secure extract?
Random832 wrote: > On Thu, Feb 11, 2016, at 18:24, Ulli Horlacher wrote: > > A better approach would be to rename such files while extracting. > > Is this possible? > > What happens if you change member.name before extracting? Ohh... such an easy solution! :-) -- Ullrich Horlacher Server und Virtualisierung Rechenzentrum IZUS/TIK E-Mail: horlac...@tik.uni-stuttgart.de Universitaet Stuttgart Tel:++49-711-68565868 Allmandring 30aFax:++49-711-682357 70550 Stuttgart (Germany) WWW:http://www.tik.uni-stuttgart.de/ -- https://mail.python.org/mailman/listinfo/python-list
[no subject]
I have downloaded python and when I click on it, it asks me to repair which I do, it then says successful however when I click on it again it won’t let me on it Sent from Mail for Windows 10 -- https://mail.python.org/mailman/listinfo/python-list
Re: modifying a standard module? (was: Re: tarfile : read from a socket?)
On 11 February 2016 at 17:10, Ulli Horlacher
wrote:
>
> Ulli Horlacher wrote:
> As a hack, I modified the standard library module tarfile.py:
>
> root@diaspora:/usr/lib/python2.7# vv -d
> --- ./.versions/tarfile.py~1~ 2015-06-22 21:59:27.0 +0200
> +++ tarfile.py 2016-02-11 18:01:50.18952 +0100
> @@ -2045,6 +2045,7 @@
> directories.append(tarinfo)
> tarinfo = copy.copy(tarinfo)
> tarinfo.mode = 0700
> +print('untar "%s"' % tarinfo.name)
> self.extract(tarinfo, path)
>
> # Reverse sort directories.
>
>
> This gives me exact the output I want :-)
>
> BUT I want to distribute my program and all others will not see the tar
> extracting information.
>
> Now my question:
>
> How can I substitute the standard module function tarfile.extractall() with
> my own function?
import tarfile
def new_extractall(self, *args, **kwargs):
print("I am a function. Woohoo!")
tarfile.TarFile.extractall = new_extractall
But bear in mind that that will change tarfile.extractall for every
single module that imports it within the same python process. Is that
really what you want?
Is there a reason you can't subclass TarFile as others have suggested?
Perhaps even this is enough:
class NoisyTarFile(TarFile):
"""untested, sorry"""
def extract(self, member, *args, **kwargs):
print('extracting "%s"' % member.name)
super(NoisyTarFile, self).extract(member, *args, **kwargs)
As the very next step after your print in extractall is a call to
extract anyway?
If you must patch the standard library tarfile module then I would
suggest patching it to have an extra, default False, argument to
enable your printing behaviour, so you don't risk messing up anyone
else's use of it.
--
Matt Wheeler
http://funkyh.at
--
https://mail.python.org/mailman/listinfo/python-list
Re: Storing a big amount of path names
On 12/02/2016 17:05, Rob Gaddi wrote: Chris Angelico wrote: Start by coding things in the simple and obvious way, and then fix problems only when you see them. Is that statement available in 10 foot letters etched into stone? Hopefully not as that would be a waste, it should be made more obvious by using a red hot poker to engrave it onto every newbies' forehead. Even then some simply wouldn't take a blind bit of notice. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: (unknown)
On 12/02/2016 20:16, Manas Soni wrote: I have downloaded python and when I click on it, it asks me to repair which I do, it then says successful however when I click on it again it won’t let me on it Sent from Mail for Windows 10 Please search the archives as this has been asked and answered repeatedly over the last few months. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence -- https://mail.python.org/mailman/listinfo/python-list
Re: Storing a big amount of path names
Chris Angelico writes: > I actually had that built behind my house, at one point. Sadly, the > letters sank until they were partly embedded into the ground, and > what's left says, in the local language, "Go stick your head in a > PHP", so it's lit up only for special celebrations. Douglas Adams, you are sorely missed. -- \“The greatest tragedy in mankind's entire history may be the | `\ hijacking of morality by religion.” —Arthur C. Clarke, 1991 | _o__) | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: modifying a standard module?
Matt Wheeler wrote:
> > How can I substitute the standard module function tarfile.extractall() with
> > my own function?
>
> import tarfile
> def new_extractall(self, *args, **kwargs):
> print("I am a function. Woohoo!")
>
> tarfile.TarFile.extractall = new_extractall
This is more easy than I could imagined :-)
It is in my Python notes, now.
> But bear in mind that that will change tarfile.extractall for every
> single module that imports it within the same python process. Is that
> really what you want?
Yes. I have no own modules. Just one program file.
> Is there a reason you can't subclass TarFile as others have suggested?
The reason: I have no ideas on classes :-}
Of course, I should have to learn about, but until now it was not necessary.
The other solutions in this thread are sufficent for me.
Meanwhile I have implemented the iterator function:
taro.extractall(members=itar(taro),path=edir)
def itar(tar):
for ti in tar:
# minimal protection against dangerous file names
# see http://bugs.python.org/issue21109#msg215656
ti.name = subst(r'^(?i)([a-z]:)?(\.\.)?[/\\]','',ti.name)
print('untar "%s"' % ti.name)
yield ti
Perfekt solution for me :-)
Thanks to all.
> If you must patch the standard library tarfile module then I would
> suggest patching it to have an extra, default False, argument to
> enable your printing behaviour, so you don't risk messing up anyone
> else's use of it.
Yes, a good idea.
--
Ullrich Horlacher Server und Virtualisierung
Rechenzentrum IZUS/TIK E-Mail: horlac...@tik.uni-stuttgart.de
Universitaet Stuttgart Tel:++49-711-68565868
Allmandring 30aFax:++49-711-682357
70550 Stuttgart (Germany) WWW:http://www.tik.uni-stuttgart.de/
--
https://mail.python.org/mailman/listinfo/python-list
Re: Storing a big amount of path names
On 12 Feb 2016 21:37, "Mark Lawrence" wrote: > Hopefully not as that would be a waste, it should be made more obvious by using a red hot poker to engrave it onto every newbies' forehead. Even then some simply wouldn't take a blind bit of notice. Yes sorry about that, I think our aim was a little off with a few of the brandings. -- Matt Wheeler http://funkyh.at -- https://mail.python.org/mailman/listinfo/python-list
Re: modifying a standard module?
Ulli Horlacher writes: >> tarfile.TarFile.extractall = new_extractall > > This is more easy than I could imagined :-) It is in my Python notes, > now. This is called "duck punching" or "monkey patching" and sometimes it's necessary, but it's something of an antipattern since the module could change under you between versions, and that sort of thing. If you have to do it, then fine; but if you have a choice, it's preferable to avoid that sort of thing. -- https://mail.python.org/mailman/listinfo/python-list
Re:
On Fri, Feb 12, 2016 at 3:16 PM, Manas Soni wrote: > > I have downloaded python and when I click on it, it asks me to repair > which I do, it then says successful however when I click on it again it > won’t let me on it > Sent from Mail for Windows 10 > > -- > https://mail.python.org/mailman/listinfo/python-list > I don't do windows, so I can't give you an answer. To the group, this has come up so consistantly lately. Maybe a windows python user could explain why this is so confusing to newbies on windows? -- Joel Goldstick http://joelgoldstick.com/stats/birthdays -- https://mail.python.org/mailman/listinfo/python-list
