Re: Performance with and without the garbage collector

2016-05-17 Thread dieter
Steven D'Aprano  writes:
> ...
> Is anyone able to demonstrate a replicable performance impact due to garbage
> collection?

I have had some experience with the performance impacts of garbage collection
-- not completely replicable but rather frequently visible.

Huge Zope instance (with around 2 GB heap usage; millions of objects);
a badly designed (external) component
that created thousands of temporary objects
during searches. What we could observe: occasionally, our Zope got
unresponsive for about 1 minute; a monitoring software typically
told us that at least on of Zope's worker threads was in the external
component mentioned above.
The analysis finally revealed: creating and releasing thousands
of short lived temporary objects frequently called for a
garbage collection. The garbage collection prevented
all other Python work while running -- and this caused outages in
the order of a minute.

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Wanted Python programmer to join team

2016-05-17 Thread Steven D'Aprano
On Tuesday 17 May 2016 16:18, Marko Rauhamaa wrote:

> Steven D'Aprano :
>> Personally, I think that advertising a job position without saying who
>> you are, what you do, and offering at least an indicative salary
>> range, are *astonishingly* rude
> 
> I don't believe they care.
> 
>> (to say nothing of counter-productive).
> 
> Maybe, maybe not.
> 
> 
> I bet the zebras on the savannah consider the lions astonishingly rude
> and their strategy counter-productive. The savannah would be a nicer
> place if the lions ate grass like everybody else.

A strange analogy. Employers and potential employees are not really in a 
predator/prey relationship.

(Employers and *actual* employees sometimes are, but that's a sign of a really 
dysfunctional business culture.)

The problem is that recruiter's best interests do not align neatly with either 
potential employees *or* employers. They're like real estate agents. The 
incentives for a recruiter is to find a barely acceptable hire as quickly as 
possible for the least amount of effort possible. There's no point in doing 
extra work to find the best new hire, if the employer is willing to take a so-
so hire. Since the employer is only seeing potentials that the recruiter passes 
on, the employer has no way of telling what the pool of would-be employees is 
really like.

I'm not saying that all recruiters are unscrupulous or are intentionally 
deceiving the other parties, but the incentives are such that:

- recruiters will take a bit less care to choose the right employee for the 
job;

- they'll take a bit less care worrying about attracting the right people, 
because their relationship with the parties is (on average) quite short;

- they're more likely 


The worst part of this is the vicious circle aspect. The less care recruiters 
put into targeting their positions, the more they get inundated with poor 
quality applicants. This trains applicants to carpet bomb recruiters and 
employers (since they're trained to expect that all job ads are misleading, and 
also because the unemployment office requires them to apply to X positions a 
week, whether X suitable positions exist or not), which gives employers an 
incentive to use recruiters (rather than deal with the carpet bombing of 
applicants). 



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list


testfixtures 4.10.0 Released!

2016-05-17 Thread Chris Withers

Hi All,

I'm pleased to announce the release of testfixtures 4.10.0 featuring the 
following:


- Fixed examples in documentation broken in 4.5.1.

- Add RangeComparison for comparing against values that fall in a
  range.

- Add MockPopen.set_default().

Thanks to Asaf Peleg for the RangeComparison implementation.

The package is on PyPI and a full list of all the links to docs, issue 
trackers and the like can be found here:


https://github.com/Simplistix/testfixtures

Any questions, please do ask on the Testing in Python list or on the 
Simplistix open source mailing list...


cheers,

Chris

--
Simplistix - Content Management, Batch Processing & Python Consulting
   - http://www.simplistix.co.uk
--
https://mail.python.org/mailman/listinfo/python-list


Re: Wanted Python programmer to join team

2016-05-17 Thread Steven D'Aprano
On Tuesday 17 May 2016 17:39, Steven D'Aprano wrote:

> - they're more likely

Er, apparently they're not more likely to do anything specific, just more 
likely.



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list


Quote of the day

2016-05-17 Thread Steven D'Aprano
Overhead in the office today:


"I don't have time to learn an existing library - much faster to make my own 
mistakes!"



-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Quote of the day

2016-05-17 Thread Radek Holý
2016-05-17 9:50 GMT+02:00 Steven D'Aprano <
steve+comp.lang.pyt...@pearwood.info>:

> Overhead in the office today:
>
>
> "I don't have time to learn an existing library - much faster to make my
> own
> mistakes!"
>
>
>
> --
> Steve
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

*THUMBS UP* At least they are aware of that "own mistakes" part... Not like
my employer...
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Wanted Python programmer to join team

2016-05-17 Thread Marko Rauhamaa
Steven D'Aprano :

> On Tuesday 17 May 2016 16:18, Marko Rauhamaa wrote:
>
>> Steven D'Aprano :
>>> Personally, I think that advertising a job position without saying who
>>> you are, what you do, and offering at least an indicative salary
>>> range, are *astonishingly* rude
>> I don't believe they care.
>> 
>>> (to say nothing of counter-productive).
>> Maybe, maybe not.
>> 
>> I bet the zebras on the savannah consider the lions astonishingly rude
>> and their strategy counter-productive. The savannah would be a nicer
>> place if the lions ate grass like everybody else.
>
> A strange analogy. Employers and potential employees are not really in a 
> predator/prey relationship.

The unsavory recruiters are the predators.

> The problem is that recruiter's best interests do not align neatly
> with either potential employees *or* employers. They're like real
> estate agents. The incentives for a recruiter is to find a barely
> acceptable hire as quickly as possible for the least amount of effort
> possible. There's no point in doing extra work to find the best new
> hire, if the employer is willing to take a so- so hire. Since the
> employer is only seeing potentials that the recruiter passes on, the
> employer has no way of telling what the pool of would-be employees is
> really like.

Correct. Also, they may genuinely not care about ethics of any sort.

> I'm not saying that all recruiters are unscrupulous or are intentionally 
> deceiving the other parties,

Nor am I.

> but the incentives are such that:

Different recruiters follow different strategies, methods and practices.
Psychopathy stays in our gene pool over generations because it sometimes
*is* a winning strategy.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Quote of the day

2016-05-17 Thread Marko Rauhamaa
Radek Holý :

> 2016-05-17 9:50 GMT+02:00 Steven D'Aprano <
> steve+comp.lang.pyt...@pearwood.info>:
>
>> Overhead in the office today:
>>
>> "I don't have time to learn an existing library - much faster to make
>> my own mistakes!"
>
> *THUMBS UP* At least they are aware of that "own mistakes" part... Not
> like my employer...

Also:

   With a third party solution I don't need to fix the bugs.

   But with an in-house solution I at least *can* fix the bugs.

The feeling of powerlessness can be crushing when you depend on a
third-party component that is broken with no fix in sight.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Quote of the day

2016-05-17 Thread Sivan Greenberg
But isn't that counter wise to batteries included? :)

On Tue, May 17, 2016 at 11:30 AM, Marko Rauhamaa  wrote:

> Radek Holý :
>
> > 2016-05-17 9:50 GMT+02:00 Steven D'Aprano <
> > steve+comp.lang.pyt...@pearwood.info>:
> >
> >> Overhead in the office today:
> >>
> >> "I don't have time to learn an existing library - much faster to make
> >> my own mistakes!"
> >
> > *THUMBS UP* At least they are aware of that "own mistakes" part... Not
> > like my employer...
>
> Also:
>
>With a third party solution I don't need to fix the bugs.
>
>But with an in-house solution I at least *can* fix the bugs.
>
> The feeling of powerlessness can be crushing when you depend on a
> third-party component that is broken with no fix in sight.
>
>
> Marko
> --
> https://mail.python.org/mailman/listinfo/python-list
>



-- 
Sivan Greenberg
Co founder & CTO
Vitakka Consulting
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: OT: limit number of connections from browser to my server?

2016-05-17 Thread jmp

On 05/16/2016 09:31 PM, Grant Edwards wrote:

On 2016-05-16, jmp  wrote:


Have you considered upgrading the device with a recent CPU ? Or is it
completely out of the picture ?


Not an option.  We have to continue to support devices that are in the
field.  The newer models that are coming out now run at 133MHz instead
of 44MHz, and page load times for https still aren't much better.


That being said, your first idea seems also a good lead, have your
server refuse more than one connection.


Just got to figure out how to try it out without wasting a lot of time
reverse-engineering the web server.  Hence a Python prototype. :)



I don't have time to read the whole thread but if I got it right, the 
main CPU consuming part is the crypto.

Why not drop the https part an support only http ?

Is is a device that needs to be accessed in untrusted networks ? Sorry 
for asking the obvious :o)


jm



--
https://mail.python.org/mailman/listinfo/python-list


Re: Quote of the day

2016-05-17 Thread Paul Rudin
Marko Rauhamaa  writes:

> Radek Holý :
>
>> 2016-05-17 9:50 GMT+02:00 Steven D'Aprano <
>> steve+comp.lang.pyt...@pearwood.info>:
>>
>>> Overhead in the office today:
>>>
>>> "I don't have time to learn an existing library - much faster to make
>>> my own mistakes!"
>>
>> *THUMBS UP* At least they are aware of that "own mistakes" part... Not
>> like my employer...
>
> Also:
>
>With a third party solution I don't need to fix the bugs.
>
>But with an in-house solution I at least *can* fix the bugs.
>
> The feeling of powerlessness can be crushing when you depend on a
> third-party component that is broken with no fix in sight.
>
>

Presumably it depends on whether you have the source for the third party
component...

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Quote of the day

2016-05-17 Thread Cem Karan

On May 17, 2016, at 4:30 AM, Marko Rauhamaa  wrote:

> Radek Holý :
> 
>> 2016-05-17 9:50 GMT+02:00 Steven D'Aprano <
>> steve+comp.lang.pyt...@pearwood.info>:
>> 
>>> Overhead in the office today:
>>> 
>>> "I don't have time to learn an existing library - much faster to make
>>> my own mistakes!"
>> 
>> *THUMBS UP* At least they are aware of that "own mistakes" part... Not
>> like my employer...
> 
> Also:
> 
>   With a third party solution I don't need to fix the bugs.
> 
>   But with an in-house solution I at least *can* fix the bugs.
> 
> The feeling of powerlessness can be crushing when you depend on a
> third-party component that is broken with no fix in sight.

+1000 on this one.  Just downloaded and used a library that came with unit 
tests, which all passed.  When I started using it, I kept getting odd errors.  
Digging into it, I discovered they had commented out the bodies of some of the 
unit tests... glad it was open source, at least I *could* dig into the code and 
figure out what was going on :/

Thanks,
Cem Karan
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Wanted Python programmer to join team

2016-05-17 Thread Larry Martell
On Tue, May 17, 2016 at 1:29 AM, Chris Angelico  wrote:
> On Tue, May 17, 2016 at 3:20 PM, Steven D'Aprano
>  wrote:
>>> And a lot of job postings do come from that sort of really small
>>> business, trying to expand a bit. Plus, some of them want some
>>> anonymity (why, I don't know, but there are plenty of jobs posted
>>> without too much in the way of company details)
>>
>> That probably means the job advert is coming from a recruiter. They don't 
>> want
>> people to contact the company directly, and they want to hide the fact that
>> they are a recruiter.
>
> Ehh, that's possible. Given that I seldom get any responses back from
> them, I can't tell whether they (a) are using a recruiter who hates my
> guts, (b) are using a recruiter who is utterly incompetent, (c) are
> doing the recruitment themselves, and are so utterly flat-out busy
> that they can't fire off a simple email to their applicants, or (d)
> are doing it themselves, and don't consider failed applicants worthy
> of an email. Could be any of the above, for all I can tell. Or (e),
> the job was posted from just outside the Bermuda Triangle.

Recruiters have changed so much in my career. Back in the early 80's
you would have to have an interview with the recruiter before they
would even consider submitting you for a job. They would know your
skill set and they would not present a job you were not qualified for.
They returned phone calls and emails. Nowadays I get emails and calls
for jobs with requirements that are nowhere on my resume (e.g. .NET or
sales) and when I am qualified and interested and I reply, in most
cases I get no response at all.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Quote of the day

2016-05-17 Thread Chris Angelico
On Tue, May 17, 2016 at 7:54 PM, Paul Rudin  wrote:
>> Also:
>>
>>With a third party solution I don't need to fix the bugs.
>>
>>But with an in-house solution I at least *can* fix the bugs.
>>
>> The feeling of powerlessness can be crushing when you depend on a
>> third-party component that is broken with no fix in sight.
>>
>>
>
> Presumably it depends on whether you have the source for the third party
> component...

Yes and no. A student of mine asked me how hard it would be to use in
Python a service that provided Java, .NET, and a couple of other SDKs,
but not Python. Source is available for them, sure, but they're so
massive and complicated that it's utterly impractical. (I had been
hoping the SDKs were basically just offering a friendly API to an
underlying HTTP-based service, but no.) So the options were (1) use
Jython so the Java SDK became usable, or (2) fire off a subprocess
that does the work and pipes it back to your app, or (3) spend about
fifty years porting a gigantic lot of code to a new language. And if
there'd been bugs in any of the code, well, options 1 and 2 mean my
student (who knows Python but none of the SDK languages on offer)
would be completely unable to fix it, source or no source - and with
option 3, it'd make the port virtually impossible.

Having the source available is great. It tells you which projects you
DON'T want to touch.

ChrisA
fully aware that some of his projects will be in other people's "DON'T
want to touch" boxes
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Autotool - compile module for both Python 2 _and_ 3

2016-05-17 Thread Dirk Bächle

Hi Ervin,

On 16.05.2016 11:05, Ervin Hegedüs wrote:

Hi All,


there is a library, which written in C. I'ld like to use it from
Python - from Python 2 _and_ 3.

I can make the autotools* files for Python 2 and Python 3, but
only exclusively. I can't make it for both in same time.

[...]

Is there any "best practice" to solve this problem?

Practical advice or links are welcome!



here's my practical advice: have a look at SCons (http://www.scons.org). It handles variant builds (multiple packages/target) from 
the same source very well, even in parallel.


Best regards,

Dirk

--
https://mail.python.org/mailman/listinfo/python-list


Re: Autotool - compile module for both Python 2 _and_ 3

2016-05-17 Thread Ervin Hegedüs
Hi Dirk,

On Tue, May 17, 2016 at 09:13:02AM +0200, Dirk Bächle wrote:
> Hi Ervin,
> 
> On 16.05.2016 11:05, Ervin Hegedüs wrote:
> >Hi All,
> >
> >
> >there is a library, which written in C. I'ld like to use it from
> >Python - from Python 2 _and_ 3.
> >
> >I can make the autotools* files for Python 2 and Python 3, but
> >only exclusively. I can't make it for both in same time.
> >
> >[...]
> >
> >Is there any "best practice" to solve this problem?
> >
> >Practical advice or links are welcome!
> >
> 
> here's my practical advice: have a look at SCons
> (http://www.scons.org). It handles variant builds (multiple
> packages/target) from the same source very well, even in parallel.

many thanks - I'll check it out soon.


Thanks,


a.
 
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: OT: limit number of connections from browser to my server?

2016-05-17 Thread Rodrigo Bistolfi
Long shot here: Create a JS framework for loading resources in a better way:

1. Load HTTP and your JS core.
2. Load the rest of the resources via JS (maybe using promises for chaining
the requests one after the other)
-- 
https://mail.python.org/mailman/listinfo/python-list


EuroPython 2016 Keynote: Nicholas Tollervey

2016-05-17 Thread M.-A. Lemburg
We are pleased to introduce our first keynote speaker for EuroPython
2016:

  *** Nicholas Tollervey ***


About Nicholas Tollervey


Nicholas is a classically trained musician, philosophy graduate,
teacher, author (for O'Reilly) and freelance programmer.

He was a founding member of the London Python Code Dojo, created and
organized PyCon UK’s education track and is a fellow of the Python
Software Foundation. He also proposed, coordinated and contributed to
the PSF’s partnership with the BBC in the creation of the micro:bit.

He’s just like this biography: concise, honest and full of useful
information.


The Keynote: A Million Children (and MicroPython)
-

The BBC micro:bit is a small programmable device for children. A
million of them have been handed out to the UK’s 11 and 12 years
olds. The Python Software Foundation was a partner in this moon-shot
scheme and, thanks to the efforts of Damien George, MicroPython (a
full reimplementation of Python 3) runs on the device:

https://en.wikipedia.org/wiki/Micro_Bit
http://micropython.org/

In his keynote, Nicholas will tell the story of the project, describe
Python’s role in it and explain how the wider Python community can
become involved. It may involve demonstrations, live coding and
audience participation.


With gravitational regards,
--
EuroPython 2016 Team
http://ep2016.europython.eu/
http://www.europython-society.org/
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Quote of the day

2016-05-17 Thread Marko Rauhamaa
Paul Rudin :

> Marko Rauhamaa  writes:
>> The feeling of powerlessness can be crushing when you depend on a
>> third-party component that is broken with no fix in sight.
>
> Presumably it depends on whether you have the source for the third
> party component...

Just having such an experience. The linux kernel has a critical bug in a
major distribution (who shall be left unnamed here) that has been fixed
in a later kernel version.

Thanks to linux being free software, I managed to pin down the root
cause after more than a month of debugging. I sent a bug report to the
linux vendor and attached a tiny patch. The vendor has graciously agreed
to consider releasing an update in the summer (we are in the process of
verifying the fix).

The problem was first detected in December. A semi-reliable reproduction
was discovered in early February. The root cause and proposed fix was
identified mid-March. A vendor fix will likely come out by the end of
June.

That's a long time to be without a product to sell.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: OT: limit number of connections from browser to my server?

2016-05-17 Thread Grant Edwards
On 2016-05-17, Gregory Ewing  wrote:

> Is there some way you can get more stuff into a single
> html page? For example, use inline css and image data
> instead of delivering them as separate files.

Yes.  That's one option that's still on the table, and that's probably
what the smart money is betting on: use server-side includes for css
and javascript files.  For image data, I'd need to add a
server-side-include-and-base64-encode capability, but that shouldn't
take more than a couple hours.


Then the HTML editing starts...

I've seen examples of including base64 image data in HTML  tags,
but don't know how to do it in CSS for things like backgrounds.

That said, I've got two more things to try to see if I can convince
browsers to behave:

 1) Stall the connection after the TCP connection opens but before the
SSL handshake.

 2) Try listen(0) on my BSD-derived network stack to limit the number
of TCP connections.  IIRC, BSD behaves differently than Linux when
the listen queue overflows.

I doubt either one will help -- and even if they do, it'll probably
all fall apart again when new browser versions come out.

I'm also looking into SSL session resumption.  My SSL stack appears to
support it, but I can't tell if it's being used or not by browsers.

It may be a moot point.  All of the descriptions I've read about SSL
session resumption say it's for low-overhead reopening of a previously
closed connection.  That's not what I need: I need to speed up opening
of a new connection while the first connection is still open.

-- 
Grant Edwards   grant.b.edwardsYow! Mary Tyler Moore's
  at   SEVENTH HUSBAND is wearing
  gmail.commy DACRON TANK TOP in a
   cheap hotel in HONOLULU!

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: OT: limit number of connections from browser to my server?

2016-05-17 Thread Grant Edwards
On 2016-05-17, Marko Rauhamaa  wrote:
> Gregory Ewing :
>
>> Is there some way you can get more stuff into a single
>> html page? For example, use inline css and image data
>> instead of delivering them as separate files.
>
> Better yet, is there some way you could send less stuff?

No.  Somebody else decides what the pages look like and behaves.  I
don't get to toss out logos, background images, customized checkbox
glyphs, etc.  I get to make it work. :)

-- 
Grant Edwards   grant.b.edwardsYow! I have a very good
  at   DENTAL PLAN.  Thank you.
  gmail.com

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: OT: limit number of connections from browser to my server?

2016-05-17 Thread Grant Edwards
On 2016-05-17, Rodrigo Bistolfi  wrote:
> Long shot here: Create a JS framework for loading resources in a better way:
>
> 1. Load HTTP and your JS core.
>
> 2. Load the rest of the resources via JS (maybe using promises for
>chaining the requests one after the other)

I thought about that.  It's probably more work than going the
"single-fetch" route via server-side includes, and probably more
fragile.

-- 
Grant Edwards   grant.b.edwardsYow! An air of FRENCH FRIES
  at   permeates my nostrils!!
  gmail.com

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: OT: limit number of connections from browser to my server?

2016-05-17 Thread Grant Edwards
On 2016-05-17, jmp  wrote:

> I don't have time to read the whole thread but if I got it right,
> the main CPU consuming part is the crypto.

Yep.

> Why not drop the https part an support only http ?

Product spec explicitly states HTTPS only.  I'm told that is not open
for discussion.  The customer is a large, somewhat bureaucratic German
corporation, and they generally mean it when they say something is
non-negotiable.

> Is is a device that needs to be accessed in untrusted networks?
> Sorry for asking the obvious :o)

-- 
Grant Edwards   grant.b.edwardsYow! I'm EMOTIONAL
  at   now because I have
  gmail.comMERCHANDISING CLOUT!!

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Quote of the day

2016-05-17 Thread Paul Rudin
Marko Rauhamaa  writes:

> Paul Rudin :
>
>> Marko Rauhamaa  writes:
>>> The feeling of powerlessness can be crushing when you depend on a
>>> third-party component that is broken with no fix in sight.
>>
>> Presumably it depends on whether you have the source for the third
>> party component...
>
> Just having such an experience. The linux kernel has a critical bug in a
> major distribution (who shall be left unnamed here) that has been fixed
> in a later kernel version.
>
> Thanks to linux being free software, I managed to pin down the root
> cause after more than a month of debugging. I sent a bug report to the
> linux vendor and attached a tiny patch. The vendor has graciously agreed
> to consider releasing an update in the summer (we are in the process of
> verifying the fix).
>
> The problem was first detected in December. A semi-reliable reproduction
> was discovered in early February. The root cause and proposed fix was
> identified mid-March. A vendor fix will likely come out by the end of
> June.
>
> That's a long time to be without a product to sell.
>

But you do have the option of building a kernel incorporating your fix
and using that.

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Design: Idiom for classes and methods that are customizable by the user?

2016-05-17 Thread Dirk Bächle

Hi Michael,

and thanks a lot for chiming in on this topic.

On 13.05.2016 22:33, Michael Selik wrote:



[...]


I share Greg's trepidation when I hear a phrase like that, but the general
idea of a registry of classes or functions and then picking the right one
based on string input is fine.



See my answer to Gregory, addressing his concerns. ;)


How would the API work for custom Taskmasters? It's not so great to require
that the user must explicitly ``add`` their derived class after defining
it. Perhaps that add function could be a decorator?



Our current API doesn't use decorators at all, since it's also aimed at people with no (or only some) knowledge of Python. Until a 
year or so ago, we have been backwards compatible down to Python 1.5.2...and then to Python 2.4 for some time. Only during the last 
two months we made a clear cut and are now seeing Python 2.7 as our floor to further develop from.

So to some degree, decorators were technically not possible to support...and 
would have looked strange to the average user perhaps?

We're now free to add them into the mix for our new Python 2.7.x/3.y codebase, but this would probably require an API discussion 
first. However, I'll keep the "decorator" idea in mind when I get back to putting "all the dots on the i" for the interface.


Best regards,

Dirk

--
https://mail.python.org/mailman/listinfo/python-list


Re: How to create development Python environment on Linux.

2016-05-17 Thread Zachary Ware
On Mon, May 16, 2016 at 4:28 PM,   wrote:
> Thanks Zach, that's a big help. The only reason I want to get a Python 2.7 
> environment working first is because I'll be working on third party code and 
> that's the platform it uses. For any new projects I would use Python 3.

Fair enough :)

> After considering your guidance I think what I will do is install virtualenv 
> using apt-get and then use that to create a dev environment. Is it ok to run 
> get-pip.py in a virtual environment?

Sounds like a plan.  get-pip.py should work fine in a venv, but first
check to see if pip is already there, as Michael mentioned.

> I won't worry about using the latest version of 2.7 for now, since it's only 
> one or two third party open source projects I'll use 2.7 for and they don't 
> need a more recent version.
>
> There are a couple of other things I'm not quite clear on, such as where it 
> would be best to create my new virtual environment (I'm tempted to put it in 
> /usr/local if that means it can be used by all user accounts on my machine), 
> and how I can can control which Python environment is used by the various 
> system and user programs that depend on them, but I expect I can find that 
> information on the web, though I'll make another post here if I do get stuck.

The thing about virtual environments is that you can create as many as
you want wherever you want, and they're all independent of each other.
All you need to do is have a way to consistently recreate the same
environment, which is usually easiest to do with a 'requirements.txt'
file with all of your dependencies' versions pinned.  Then recreating
the venv is as simple as `virtualenv --python=some_python
/path/to/venv && /path/to/venv/bin/pip install -r
/path/to/requirements.txt`.  Then you can do your development wherever
you want and install into somewhere in `/usr/local` when you deploy.
For example, one project I work on creates a venv in
`/usr/local/lib/project_name/`, then creates links to the entry point
scripts (that actually live in `/usr/local/lib/project_name/bin/`) in
`/usr/local/bin/`.

As for controlling which environment is used: all system scripts/apps
should specify some version-specific flavor of /usr/bin/python in
their shebang--if they don't it's an OS bug and should be filed as
such.  For your stuff, the path of least headache is probably to
create a venv for each one, and specify the path to the venv's python
in shebang lines.

Hope this helps,
-- 
Zach
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: OT: limit number of connections from browser to my server?

2016-05-17 Thread Rob Gaddi
Steven D'Aprano wrote:

> On Tue, 17 May 2016 02:52 am, Chris Angelico wrote:
>
>> On Tue, May 17, 2016 at 2:34 AM, Rob Gaddi
>>  wrote:
 The solution might actually be to move all your static files
 elsewhere. Slap 'em up onto github.io or something, and then the
 browser is free to make all the parallel connections it likes; your
 embedded device can just serve the stuff that actually varies
 (presumably the main HTML file). I know that isn't what you asked for,
 but it's something to consider :)

 ChrisA
>>>
>>> Oooof.  Not to be rude, Chris, but your "software guy" is showing.
>>> Grant's got the right of it; if you're shipping a box with an RJ-45 and
>>> a webpage, and you want the customer to be able to always make it
>>> work, then it needs to be a self-contained entity.  The belief that your
>>> external dependancies will always be there is why leftpad was able to
>>> break everything, and why Google just bricked a bunch of people's
>>> expensive Revolv Hubs.
>
> Schadenfreude is a beautiful emotion :-)
>
> "Yes, let's put a critical requirement of our business in the hands of a
> third party with absolutely *no* obligations to us, and no government
> oversight. What could *possibly* go wrong???"
>
>
>> I agree, but I also make no apology for suggesting the option of
>> getting someone else to do some of the work. In this case, it can be
>> rejected for the exact reason you cite (dependencies are a cost, and
>> in this case way too high a cost), and that's fine and correct.
>> Ultimately, if the job gets done, everything else is implementation
>> detail, with consequences - and I know a lot of people who'll
>> willingly sacrifice "reliability in the face of an internet connection
>> outage" in favour of "less than fifteen second response time". 
>
> How can you not serve a web page over your LAN in 15s?
>
> I mean, you could *almost* do it by hand, copying the files onto a USB stick
> and walking them across the room in 15 seconds. Maybe 30.
>

Simple, because embedded web servers running on toy microprocessors are
HARD.  When you're trying to work with whatever browser the customer may
have, you have no control over the number of simultaneous connections 
it will try to make to your box.  If you don't allow those connections
it can cause huge stalls by forcing the TCP layer to time out.  If you
do, in Grant's case, it forces you to perform tons of expensive
public-key crypto on a 40 MHz processor (which, hmmm, external memory
bus, ~40 MHz... Coldfire?).

A lot of things you can take for granted on a compute monster (like a
Chromebook or Atom based laptop) get much more complicated when you're
resource constrained.

-- 
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Design: Idiom for classes and methods that are customizable by the user?

2016-05-17 Thread Michael Selik
On Tue, May 17, 2016 at 11:18 AM Dirk Bächle  wrote:

>
> > It's not so great to require
> > that the user must explicitly ``add`` their derived class after defining
> > it. Perhaps that add function could be a decorator?
>
> Our current API doesn't use decorators at all, since it's also aimed at
> people with no (or only some) knowledge of Python.


I think you'll find that users of decorators do not need any special
knowledge. The Flask framework (http://flask.pocoo.org/) is proof of that.


> decorators...would have looked strange to the average user perhaps?
>

Many beginners are willing to mimic your example code without questioning
what that "@" thingy is doing above the function or class.

Compare that with trying to explain to a C++ user that in Python classes
are actually objects and you can pass them to the ``add`` function to
register them... that's going to take a while.

We're now free to add them into the mix for our new Python 2.7.x/3.y
> codebase, but this would probably require an API discussion
> first.
>

An alternative would be creating a TaskmasterMeta metaclass that registers
the Taskmaster subclass when it's defined. I'm a little reluctant to
recommend that one. I find decorators to be more elegant.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: OT: limit number of connections from browser to my server?

2016-05-17 Thread Grant Edwards
On 2016-05-17, Rob Gaddi  wrote:
>
>> How can you not serve a web page over your LAN in 15s?
>>
>> I mean, you could *almost* do it by hand, copying the files onto a
>> USB stick and walking them across the room in 15 seconds. Maybe 30.
>
> Simple, because embedded web servers running on toy microprocessors are
> HARD.

40MHz with multiple MB of RAM is pretty high-end in my book.  I've
worked on projects where the clock speed was in KHz, the supply
current was measrued in micro-Amps, and there were a few KB of RAM.
Of course you don't try to do TCP and SSL on something like that.

> When you're trying to work with whatever browser the customer may
> have, you have no control over the number of simultaneous connections 
> it will try to make to your box.  If you don't allow those connections
> it can cause huge stalls by forcing the TCP layer to time out.  If you
> do, in Grant's case, it forces you to perform tons of expensive
> public-key crypto on a 40 MHz processor (which, hmmm, external memory
> bus, ~40 MHz... Coldfire?).

The 40MHz one is a Samsung ARM7TDMI.  There's a newer model with a
133MHz Cortex-M3.  For most things it's 2-3 times faster than the
ARM7, but the ARM7 has an I/D cache and the M3 doesn't.  So there are
few highly localized tasks where there's not a lot of difference.

> A lot of things you can take for granted on a compute monster (like
> a Chromebook or Atom based laptop) get much more complicated when
> you're resource constrained.

And that's what keeps me paid. :)

-- 
Grant Edwards   grant.b.edwardsYow! I know th'MAMBO!!
  at   I have a TWO-TONE CHEMISTRY
  gmail.comSET!!

-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Quote of the day

2016-05-17 Thread Michael Torrie
On 05/17/2016 08:27 AM, Paul Rudin wrote:
> Marko Rauhamaa  writes:
>> That's a long time to be without a product to sell.
> 
> But you do have the option of building a kernel incorporating your fix
> and using that.

Sure as an individual end user that may be the best option. But not
necessarily for a business.  The cost of doing that could be
prohibitive.  Sometimes we forget just how costly open source software
can be (really *all* software).  They can either deal with lost revenue
waiting, or they can budget a tremendous amount of money, time, and
effort to support their own kernel which would entail doing updates, QA
testing, etc.  Letting the upstream vendor do all that (their core
business after all) is often the least costly option.  Though it sounds
like they've already spent a lot of money doing QA to identify this bug.

When I did IT professionally, our policy with regards to Linux was to
stick with existing packages from a known set of (mostly) official
channels and to discourage any installing of libraries and frameworks
from source.  Allowing packages to be installed from source was just a
maintenance nightmare.  RPM (or deb or whatever) brings at least a tiny
bit of stability and consistency.
-- 
https://mail.python.org/mailman/listinfo/python-list


Re: Quote of the day

2016-05-17 Thread Marko Rauhamaa
Michael Torrie :

> On 05/17/2016 08:27 AM, Paul Rudin wrote:
>> Marko Rauhamaa  writes:
>>> That's a long time to be without a product to sell.
>> 
>> But you do have the option of building a kernel incorporating your fix
>> and using that.
>
> Sure as an individual end user that may be the best option. But not
> necessarily for a business.

Correct, the answer would be no.

> The cost of doing that could be prohibitive.

Really, the customer would simply refuse to do it. They are not in the
business of building kernels. Also, they would immediately fall out of
any kind of distro support if they improvised with their own kernel.

> When I did IT professionally, our policy with regards to Linux was to
> stick with existing packages from a known set of (mostly) official
> channels and to discourage any installing of libraries and frameworks
> from source. Allowing packages to be installed from source was just a
> maintenance nightmare. RPM (or deb or whatever) brings at least a tiny
> bit of stability and consistency.

Exactly.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list


Python 3.6.0a1 is now available

2016-05-17 Thread Ned Deily
On behalf of the Python development community and the Python 3.6 release
team, I'm happy to announce the availability of Python 3.6.0a1.
3.6.0a1 is the first of four planned alpha releases of Python 3.6,
the next major release of Python.  During the alpha phase, Python 3.6
remains under heavy development: additional features will be added
and existing features may be modified or deleted.  Please keep in mind
that this is a preview release and its use is not recommended for
production environments.

You can find Python 3.6.0a1 here:

https://www.python.org/downloads/release/python-360a1/ 

The next release of Python 3.6 will be 3.6.0a2, currently scheduled for
2016-06-13.

Enjoy!

--Ned

--
  Ned Deily
  n...@python.org -- []

-- 
https://mail.python.org/mailman/listinfo/python-list


Backwards-incompatible changes

2016-05-17 Thread Steven D'Aprano
Every change breaks somebody's workflow.

http://xkcd.com/1172/


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list