Re: Call a shell command from Python
in 767198 20161104 142132 Thomas 'PointedEars' Lahn wrote: >Ben Finney wrote: > >> Note that âsudoâ is specifically designed to be invoked interactively, > >Nonsense. > >> seeking to verify that the current user has credentials to run the >> command. > >NOPASSWD is not the default in sudoers(5), It is on the Raspberry Pi but conclude from that what you >concluded is a bit far-fetched, to say the least. -- https://mail.python.org/mailman/listinfo/python-list
Re: Pre-pep discussion material: in-place equivalents to map and filter
I've been giving your proposal a bit more thought, and while I can't say I'm
really keep on the idea, I have warmed slightly to it.
On Fri, 4 Nov 2016 07:29 am, Arthur Havlicek wrote:
> I understand that, the cost of change is such that it's very unlikely
> something like this ever goes into Python, but I feel like the interest of
> the proposition is being underestimated here, that's why I'm going to
> argue a few points and give a bit more context as needed.
>
>> While mapping and filtering are common operations, I'm not sure mapping
>> and filtering and then reassigning back to the original sequence is
>> especially common.
>
> It depends of your context. On the last 3 months, I stumbled across this
> at least 3 times,
I don't know who you are quoting there. It is considered impolite to quote
people without giving attribution, and makes it harder to respond. But for
what it's worth, I agree with this person.
In my code, it is quite common for me to map back to the same *variable*,
for example I might write something like this:
def process(alist):
alist = [float(x) for x in alist]
...
But that is not the same as *in-place* modification of the list. I would
very rarely modify the list in place, but if I did, I would write it like
this:
alist[:] = [float(x) for x in alist]
In my experience, wanting to modify a list in-place without creating a
temporary version first is unusual. And *needing* to do so (rather than
doing so as premature optimization) is even rarer.
That's my experience. But if you can demonstrate the need for in-place
modifications, that changes the situation somewhat.
> which is 3 times more than I used a lambda or a
> metaclass or a decorator or other fancy language feature that we simply
> avoid whenever possible.
You aren't helping your case by describing "lambda or ... decorator" as
fancy language features to be avoided.
The impression that gives (hopefully this is the wrong impression!) is that
you are a barely competent Python developer, fearful and suspicious of some
rather simple, powerful and widely-used features, stuck in an ancient 1980s
programming paradigm.
I trust that isn't actually the case, but by describing decorators and
lambda as too fancy to use, you're giving a good impression of somebody who
doesn't know what they're doing.
[...]
> The reason I'm being especially impacted by this is because I am
> maintainer of a decent-sized Python application (~50-100K lines of code)
> that extensively uses lists and dictionaries. We value "low level" data
> manipulation and efficiency a lot more than complex, non-obvious
> constructs.
And what do you consider "complex, non-obvious"?
[...]
> Like most Python programmers, I'm not in the case of needing a performance
> boost very bad, but that does not mean I disregard performance entirely.
> The need of performance is not so binary that it either don't matter at
> all or is enough to motivate a rewrite.
Indeed.
But you're arguing for a new feature on the basis that it will boost
performance, without giving any reason to think that it actually will lead
to a performance improvement!
I know that there is a chicken-and-egg problem here. Nobody wants to spend
time writing a new feature if there's no possibly chance for it to be
accepted, but when your sole argument for a new feature is that it will be
more efficient (in both memory and time!), then you need to prove that is
the case!
In other words, in my opinion:
- you would need to prove that there is a widespread need for in-place
modification of lists, where the lists are big enough that using a
temporary list is not practical;
- you would have to demonstrate a good reason to think that this new feature
will actually be more efficient and faster
before this feature would be seriously considered for addition to the
language.
You might think that it is obvious that in-place modification MUST be faster
since it avoids making a temporary copy, but that's not obviously true at
all! Not for a high-level language like Python. Its often the case that
algorithms can exchange space for time (use more memory to save time, or
use more time to save memory). It may be that this is one of the times.
Even when doing (nearly) everything in pure Python, making a new list and
then doing a slice assignment is virtually as fast as writing directly to
the original list:
py> from timeit import Timer
py> setup = "data = list(range(1))"
py> t1 = Timer("""for i, a in enumerate(data):
... data[i] = a+1
... """, setup=setup)
py>
py> t2 = Timer("""tmp = [None]*len(data)
... for i, a in enumerate(data):
... tmp[i] = a+1
... data[:] = tmp
... """, setup=setup)
py>
py> min(t1.repeat(number=1, repeat=7))
36.63875983003527
py>
py> min(t2.repeat(number=1, repeat=7))
37.70047474466264
Taking the plain Python for-loop, modifying the list directly in place, we
see that making a temporary list and then copying it over the original list
is just 3% slower.
Re: pip3 : command not found
On 2016-10-31, Steve D'Aprano wrote: > On Mon, 31 Oct 2016 07:21 pm, Jon Ribbens wrote: >> On 2016-10-31, Ben Finney wrote: >>> Instead, you should invoke the exact Python interpreter you want – and, >>> by extension, the Python environment into which you want packages >>> installed. >>> >>> $ /foo/bar/virtualenv/bin/python3 -m pip install LoremIpsum >> >> I'm slightly curious about that. /foo/bar/virtualenv/bin/python3 >> will just be a symbolic link to /usr/bin/python3, so how does >> invoking the intepreter that way make any difference? > > It doesn't. If you read the rest of Ben's post, or for that matter the > subject line of this thread, you will see he is comparing: > > path/to/python3 -m pip install LoremIpsum > > against: > > pip3 install LoremIpsum No, if you read the rest of Ben's post you will see that that is not what he wrote. Maybe he meant what you are saying, I don't know, but it isn't what he wrote. He clearly implied that you can run Python in the context of a virtualenv by just invoking that virtualenv's local Python without running 'activate' first. I'm curious as to whether this is true or not (how virtualenvs work seems to be very opaque). -- https://mail.python.org/mailman/listinfo/python-list
Re: Pre-pep discussion material: in-place equivalents to map and filter
2016-11-05 9:42 GMT+01:00 Steve D'Aprano : > > I don't know who you are quoting there. It is considered impolite to quote > people without giving attribution, and makes it harder to respond. > My bad. I was unaware of that. This was quoted from Ned Batchelder's mali. 2016-11-05 9:42 GMT+01:00 Steve D'Aprano : > I trust that isn't actually the case, but by describing decorators and > lambda as too fancy to use, you're giving a good impression of somebody who > doesn't know what they're doing. > I was a bit humorous here, making fun about the fact I do not write anything sophisticated. However, the fact that we do avoid them is entirely true ! We do so because these concepts are foreign to the average developer (lambda being a bit apart: it's easily understood, but functional constructs may not, so its a result of us avoiding functional constructs as well). Pick 10 programmers for hire and count how many know how to write a decorator. If you have specified you needed python specialists, you may have 3-4. If not, you are lucky to find even one. And where I live we don't have the luxury of finding competent Python experts by kicking a tree. In our open space, we are 6 devs (+ one ex-dev being my manager), only 2 had previous Python experience: me and the CEO. And the other 4 are doing fine ! Locking ourselves in Python-specific semantics, over time, will make us loose precious dev time. The best code is the one everyone can understand. That is why I point this as "complex, non-obvious", and my manager wouldn't have so kind words (but he is a Java fan, so he doesn't count.) 2016-11-05 9:42 GMT+01:00 Steve D'Aprano : > - you would have to demonstrate a good reason to think that this new > feature > will actually be more efficient and faster > This is easy but time-consuming, I could roll my implementation and showcase a few benchs. I am very confident that is the case. I would have bet that I would need to do it at some point, but I wanted to poll opinions a bit beforehand. 2016-11-05 9:42 GMT+01:00 Steve D'Aprano : > In other words, in my opinion: > > - you would need to prove that there is a widespread need for in-place > modification of lists, where the lists are big enough that using a > temporary list is not practical > However this is tricky, I may very well be an exception. About your bench: I don't really know why you are surprised the for loop is slower. That's the result of comprehension being native while for loop isn't. That does not mean writing to a copy would save time for exchange of memory. In my opinion, the fact that we will win something is guaranteed because we save a copy call and do the exact same operation. There is nothing like cache magic optimization that could happen because the mapping needs to read the first list anyway. Nor we need a temporary buffer to cache operations since they are independent. Really, I am ready to make a serious bet. -- https://mail.python.org/mailman/listinfo/python-list
Re: pip3 : command not found
On Sat, 5 Nov 2016 08:45 pm, Jon Ribbens wrote: > On 2016-10-31, Steve D'Aprano wrote: >> On Mon, 31 Oct 2016 07:21 pm, Jon Ribbens wrote: >>> On 2016-10-31, Ben Finney wrote: Instead, you should invoke the exact Python interpreter you want – and, by extension, the Python environment into which you want packages installed. $ /foo/bar/virtualenv/bin/python3 -m pip install LoremIpsum >>> >>> I'm slightly curious about that. /foo/bar/virtualenv/bin/python3 >>> will just be a symbolic link to /usr/bin/python3, so how does >>> invoking the intepreter that way make any difference? >> >> It doesn't. If you read the rest of Ben's post, or for that matter the >> subject line of this thread, you will see he is comparing: >> >> path/to/python3 -m pip install LoremIpsum >> >> against: >> >> pip3 install LoremIpsum > > No, if you read the rest of Ben's post you will see that that is not > what he wrote. Excuse me, that is what he wrote. As you yourself quoted. The OP said he tried to call the "pip3" command, and got an error that the command was not found. Ben replied: "There is no guarantee that a command named ‘pip3’ will be installed." and then stated: "Instead, you should invoke the exact Python interpreter you want" The fact that (and now I'm quoting *you*, not Ben) /foo/bar/virtualenv/bin/python3 will just be a symbolic link to /usr/bin/python3 is irrelevant. It doesn't matter whether you call python3 via the actual executable file, or the symbolic link. But Ben never suggested that it would make a difference: he was contrasting calling the actual Python interpreter wanted (Python 3, via a virtualenv) with calling a command pip3 (which does not actually exist on the OP's system). Your implied question here: > Maybe he meant what you are saying, I don't know, but > it isn't what he wrote. He clearly implied that you can run Python > in the context of a virtualenv by just invoking that virtualenv's > local Python without running 'activate' first. I'm curious as to > whether this is true or not (how virtualenvs work seems to be very > opaque). is a separate issue from the symbolic link question. Possibly you do have to run `activate` first, I don't know, but either way this was not part of your earlier question. You said nothing about `activate` in your earlier post, and this is the first time you have mentioned it. The bottom line is, I couldn't have answered your question about `activate` because you hadn't asked it yet; you are correct that there's no difference between calling Python via the executable and via a symlink; but Ben never said that there was such a difference. This thread reminds me of a scene from the Marx Bros movie "At The Circus". Alas, Google has let me down as far as the exact quote, but it goes something like this: Groucho Marx, accidentally letting the big secret slip: "The elephants will be here soon." Society dame Margaret Dumont, taken aback: "Elephants? What elephants?" Groucho: "Nobody said anything about elephants!" [shakes head disgustedly] "Elephants! At your age!" I'll let you decide whether I'm Groucho or Dumont. -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: Pre-pep discussion material: in-place equivalents to map and filter
On Sat, Nov 5, 2016 at 9:50 PM, Arthur Havlicek
wrote:
> Pick 10 programmers for hire and count how many know how to write a
> decorator. If you have specified you needed python specialists, you may
> have 3-4. If not, you are lucky to find even one.
By "write a decorator", I presume you mean implement the decorator
function, because anyone who's used Flask will have used "@app.route",
for instance.
But here's the thing. For everyone who writes a decorator function,
there could be dozens who use it. How many people *on this mailing
list* know how to implement namedtuple, vs how how many know how to
use it? How many know how to properly implement a cache, vs how many
can slap "@lru_cache()" on top of a function? For the most utterly
extreme example possible, how many people in the world know how to
encrypt data securely, vs how many know how to put "https:" into their
browsers? When you look at "programmers for hire", you get a small
number of brilliant experts with huge amounts of experience, and the
rest are split between mid-level people looking for a new job, and
entry-level people straight out of college. (Depending on your
environment, the stats could easily be skewed either direction.) Most
entry-level programmers are not going to have experience with writing
decorator functions - but they don't need it. (Not that it's that
hard. They're functions that accept and return functions, that's all.)
> This is easy but time-consuming, I could roll my implementation and
> showcase a few benchs. I am very confident that is the case. I would have
> bet that I would need to do it at some point, but I wanted to poll opinions
> a bit beforehand.
>
> About your bench: I don't really know why you are surprised the for loop is
> slower. That's the result of comprehension being native while for loop
> isn't. That does not mean writing to a copy would save time for exchange of
> memory. In my opinion, the fact that we will win something is guaranteed
> because we save a copy call and do the exact same operation. There is
> nothing like cache magic optimization that could happen because the mapping
> needs to read the first list anyway. Nor we need a temporary buffer to
> cache operations since they are independent. Really, I am ready to make a
> serious bet.
Okay! *Do* make that bet. Gamble the most precious currency there is:
developer time. How much are you prepared to bet? Two hours? Ten
hours? Spend that much time writing the implementation and
benchmarking it. If you're confident that this really is such a win,
the time won't be lost - you'll end up with a viable patch for
inclusion. If you lose the bet, well, that's what betting is like.
Sometimes you lose.
I may sound cynical and critical from the above ("show me"), but in
reality, I would love to see the results of your benchmark. Improving
performance is a good thing. I just want to know that it's actually
going to happen.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: Django Application Model Reference
On Thu, Nov 3, 2016 at 4:44 AM, Dreyton Scott wrote: > On Wednesday, November 2, 2016 at 1:40:35 PM UTC-4, Dreyton Scott wrote: >> Hello. I am currently creating a notification django application that will >> need to be able to "hook" into another django application. What my app needs >> is a way to retrieve all model classes in the connected application. Is >> there a way to do this? > > In a way such that my app (a) is connected to app (b). Have (a) retrieve all > model class instances in the connected app (b). Is that possible? You may have to elaborate on "connected" here. Is it able to import the code of the other app, or is it restricted to HTTP requests and responses? If the latter, there's no significance to them both being written in Django; you'll have to create an explicit endpoint that enumerates the models, probably sending the info as a JSON object. (Caveat: I don't know Django, but I do know HTTP.) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: mentor training python Romania with certification
On Monday, September 5, 2016 at 3:33:55 PM UTC+5:30, neha.agr...@gmail.com wrote: > On Tuesday, August 16, 2016 at 6:45:04 PM UTC+5:30, blue wrote: > > Hi. > > > > I'm from Romania. > > I need to update my skils under python. > > I need to find one mentor ( class, training ) to obtain one certified under > > python language. > > > > Cătălin George Feștilă > > Hi, > > You might find Python course offered by edureka to be useful > http://www.edureka.co/python > > Check it out! > > Regards > Neha Hi, Please check out the below link for a demo on Edureka's Python course http://unbouncepages.com/python-training-3/ Regards Neha -- https://mail.python.org/mailman/listinfo/python-list
Re: pip3 : command not found
On 2016-11-05, Steve D'Aprano wrote: > Your implied question here: > >> Maybe he meant what you are saying, I don't know, but >> it isn't what he wrote. He clearly implied that you can run Python >> in the context of a virtualenv by just invoking that virtualenv's >> local Python without running 'activate' first. I'm curious as to >> whether this is true or not (how virtualenvs work seems to be very >> opaque). > > is a separate issue from the symbolic link question. Possibly you do have to > run `activate` first, I don't know, but either way this was not part of > your earlier question. You said nothing about `activate` in your earlier > post, and this is the first time you have mentioned it. I'm afraid I can only suggest that you try re-reading the subthread again until you manage to understand it. It wasn't really that complicated but you seem to have confused yourself greatly. -- https://mail.python.org/mailman/listinfo/python-list
Re: pip3 : command not found
On Sun, 6 Nov 2016 02:55 am, Jon Ribbens wrote: > I'm afraid I can only suggest that you try re-reading the subthread > again until you manage to understand it. It wasn't really that > complicated but you seem to have confused yourself greatly. Are you serious? Okay. Here's the start of the thread: https://mail.python.org/pipermail/python-list/2016-October/715993.html where the OP asks for help because running "pip3" causes a "command not found" error. Just as the subject line says. Here's Ben's reply: https://mail.python.org/pipermail/python-list/2016-October/715994.html where he clearly says not to rely on the "pip3" command, as it may not exist, or may not be in the environment you expect, but to specify precisely which Python interpreter you want to run: [quote] Instead, you should invoke the exact Python interpreter you want And here's your response: https://mail.python.org/pipermail/python-list/2016-October/716005.html where you quote those exact same words from Ben and ask what difference it makes as it is a symbolic link. You'll note that there is not one word about "activate" in your post, just as I said. Instead you question why Ben thinks there's a difference between running the sym link and running the direct link to the executable, even though that's not what Ben said. I can only repeat yet again: you have not understood Ben. He's not stating that there is a difference between: /direct/path/to/python3.x and /symbolic/link/to/direct/path/to/python3.x You are right to state that they will be exactly the same[1], but wrong to imagine that Ben said that they were different. That's not what Ben said. He stated that there is a difference between: /direct/path/to/python3.x -m pip ... which specifies the precise Python environment you want to run, and: pip3 ... which may not exist at all, or if it exists it may not be on the PATH, or if it is on the PATH it may not install into the Python environment that you expect. Oh, and ironically, you'll see that Ben anticipated and answered your question about activate. Here's the link again, to save you scrolling up: https://mail.python.org/pipermail/python-list/2016-October/715994.html where he says: If you already have a specific environment active and know that ‘python3’ is the correct Python interpreter from that environment, you can omit the explicit path. The implication is that the answer to your question is Yes, you can run Python in the context of a virtualenv by just invoking that virtualenv's local Python without running 'activate' first. Is Ben correct? The virtualenv documentation confirms that Ben is right: If you directly run a script or the python interpreter from the virtualenv’s bin/ directory (e.g. path/to/ENV/bin/pip or /path/to/ENV/bin/python-script.py) there’s no need for activation. https://virtualenv.pypa.io/en/stable/userguide/#activate-script [1] Technically, the application being run may invoke different behaviour depending on the name it was invoked by. Some editors do that, e.g. the "joe" editor. But I don't believe Python does anything like this. http://joe-editor.sourceforge.net/ -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
[Theory] How to speed up python code execution / pypy vs GPU
Hi, Some skeptics asked my why there is a reason to use Python against of any other "not interpreted" languages, like objective-C. As my explanation, I have answered that there is a a lot of useful APIs, language is modern, has advanced objective architecture, and what is the most important - it is dynamic and support is simply great. However the same skeptics told my that, ok we believe that it is true, however the code execution is much slower than any other compiled language. I must tell you that is the reason I started to dig into internet and searching some methods to speed up python's code. 1. What I have found is modified python interpreter - pypy - http://pypy.org that does not require any different approach to develop your code. 2. And: Gpu based computing powered by Nvidia (NumbaPro compiler): https://developer.nvidia.com/how-to-cuda-python Do you have any experience in that areas, and maybe some benchmarks showing advantage of using these technologies? Cheers, Marcin -- https://mail.python.org/mailman/listinfo/python-list
First security bug related to f-strings
Well, that didn't take very long at all.
Here's the first security bug which is related to the new (and badly
misnamed) f-string feature:
http://bugs.python.org/issue28563
Note what I'm not saying: I'm not saying that the bug is *caused* by
f-strings. It is not. The bug is actually caused by the inappropriate use
of eval. But the worrying thing here is:
Bonus: With the new string interpolation in Python 3.7, exploiting
gettext.c2py becomes trivial:
gettext.c2py('f"{os.system(\'sh\')}"')(0)
The tokenizer will recognize the entire format-string as just a string,
thus bypassing the security checks.
Yay for hiding arbitrary code evaluation inside something that pretends to
be a string!
My guess is that there is probably more code out there in the wild which is
vulnerable to code injection attacks thanks to the use of eval or exec, but
its been too hard to exploit up until now. But with f-strings, chances are
that they too will be trivially exploitable.
--
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.
--
https://mail.python.org/mailman/listinfo/python-list
SCons 2.5.1 Released
Available at: https://sourceforge.net/projects/scons/files/latest/download?source=files Changelog: SCons - a software construction tool Change Log RELEASE 2.5.1 - Mon, 03 Nov 2016 13:37:42 -0400 From William Deegan: - Add scons-configure-cache.py to packaging. It was omitted From Alexey Klimkin: - Use memoization to optimize PATH evaluation across all dependencies per node. (PR #345) RELEASE 2.5.0 - Mon, 09 Apr 2016 11:27:42 -0700 From Dirk Baechle: - Removed a lot of compatibility methods and workarounds for Python versions < 2.7, in order to prepare the work towards a combined 2.7/3.x version. (PR #284) Also fixed the default arguments for the print_tree and render_tree methods. (PR #284, too) From William Blevins: - Added support for cross-language dependency scanning; SCons now respects scanner keys for implicit dependencies. - Notes for SCons users with heterogeneous systems. - May find new (previously missed) dependencies. - May cause rebuild after upgrade due to dependency changes. - May find new dependency errors (EG. cycles). - Discovered in some of the SCons QT tests. - Resolved missing cross-language dependencies for SWIG bindings (fixes #2264). - Corrected typo in User Guide for Scanner keyword. (PR #2959) - Install builder interacts with scanner found in SCANNERS differently. - Previous: Install builder recursively scanned implicit dependencies for scanners from SCANNER, but not for built-in (default) scanners. - Current: Install builder will not scan for implicit dependencies via either scanner source. This optimizes some Install builder behavior and brings orthogonality to Install builder scanning behavior. From William Deegan: - Add better messaging when two environments have different actions for the same target (Bug #2024) - Fix issue only with MSVC and Always build where targets marked AlwaysBuild wouldn't make it into CHANGED_SOURCES and thus yield an empty compile command line. (Bug #2622) - Fix posix platform escaping logic to properly handle paths with parens in them "()". (Bug #2225) From Jakub Pola: - Intel Compiler 2016 (Linux/Mac) update for tool directories. From Adarsh Sanjeev: - Fix for issue #2494: Added string support for Chmod function. From Tom Tanner: - change cache to use 2 character subdirectories, rather than one character, so as not to give huge directories for large caches, a situation which causes issues for NFS. For existing caches, you will need to run the scons-configure-cache.py script to update them to the new format. You will get a warning every time you build until you co this. - Fix a bunch of unit tests on windows -- https://mail.python.org/mailman/listinfo/python-list
Re: First security bug related to f-strings
On 5-11-2016 18:12, Steve D'Aprano wrote: > Well, that didn't take very long at all. > > Here's the first security bug which is related to the new (and badly > misnamed) f-string feature: > > http://bugs.python.org/issue28563 I think perhaps we should have a command line option / environment variable to be able to disable 'eval' altogether Irmen -- https://mail.python.org/mailman/listinfo/python-list
Re: First security bug related to f-strings
On Sat, Nov 5, 2016 at 5:33 PM, Irmen de Jong wrote: > I think perhaps we should have a command line option / environment variable > to be able > to disable 'eval' altogether I don't think that's practical. exec and eval are commonly used by shells and IDEs such as IDLE and IPython. In the standard library, importlib and namedtuple are two important users of exec. Just try `import builtins; del builtins.exec, builtins.eval`. -- https://mail.python.org/mailman/listinfo/python-list
Re: First security bug related to f-strings
On 5-11-2016 19:08, eryk sun wrote: > On Sat, Nov 5, 2016 at 5:33 PM, Irmen de Jong wrote: >> I think perhaps we should have a command line option / environment variable >> to be able >> to disable 'eval' altogether > > I don't think that's practical. exec and eval are commonly used by > shells and IDEs such as IDLE and IPython. In the standard library, > importlib and namedtuple are two important users of exec. Just try > `import builtins; del builtins.exec, builtins.eval`. > Perhaps. But in those cases you could just leave things on the default. If you choose to run the interpreter with eval (and exec) disabled, you should be aware that you'll break tools like that. But for other situations (web server etc) it could still be useful? I do agree that not being able to use namedtuple (and perhaps other things from the stdlib) is a problem then. It was just a thought Irmen -- https://mail.python.org/mailman/listinfo/python-list
Re: pip3 : command not found
On Sun, Nov 6, 2016 at 3:53 AM, Steve D'Aprano
wrote:
> [1] Technically, the application being run may invoke different behaviour
> depending on the name it was invoked by. Some editors do that, e.g.
> the "joe" editor. But I don't believe Python does anything like this.
>
> http://joe-editor.sourceforge.net/
And quite a few other examples, too - some versions of bash will act
in a more sh-compatible way if invoked as /bin/sh, thus allowing one
to be a link to the other.
Here's how I'm probing Python for this behaviour:
1) Brand new venv
rosuav@sikorsky:~/tmp$ python3 -m venv env
2) With venv active:
(env) rosuav@sikorsky:~/tmp$ python3
Python 3.7.0a0 (default:72e64fc8746b+, Oct 28 2016, 12:35:28)
[GCC 6.2.0 20161010] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; sys.executable
'/home/rosuav/tmp/env/bin/python3'
>>> sys.path
['', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7',
'/usr/local/lib/python3.7/lib-dynload',
'/home/rosuav/tmp/env/lib/python3.7/site-packages']
3) With venv not active:
rosuav@sikorsky:~$ python3
Python 3.7.0a0 (default:72e64fc8746b+, Oct 28 2016, 12:35:28)
[GCC 6.2.0 20161010] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; sys.executable
'/usr/local/bin/python3'
>>> sys.path
['', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7',
'/usr/local/lib/python3.7/lib-dynload',
'/home/rosuav/.local/lib/python3.7/site-packages',
'/usr/local/lib/python3.7/site-packages']
4) Proof that it really is a symlink:
(env) rosuav@sikorsky:~/tmp$ ll env/bin
total 32
-rw-r--r-- 1 rosuav rosuav 2134 Nov 6 05:57 activate
-rw-r--r-- 1 rosuav rosuav 1250 Nov 6 05:57 activate.csh
-rw-r--r-- 1 rosuav rosuav 2414 Nov 6 05:57 activate.fish
-rwxr-xr-x 1 rosuav rosuav 249 Nov 6 05:57 easy_install
-rwxr-xr-x 1 rosuav rosuav 249 Nov 6 05:57 easy_install-3.7
-rwxr-xr-x 1 rosuav rosuav 221 Nov 6 05:57 pip
-rwxr-xr-x 1 rosuav rosuav 221 Nov 6 05:57 pip3
-rwxr-xr-x 1 rosuav rosuav 221 Nov 6 05:57 pip3.7
lrwxrwxrwx 1 rosuav rosuav7 Nov 6 05:57 python -> python3
lrwxrwxrwx 1 rosuav rosuav 22 Nov 6 05:57 python3 -> /usr/local/bin/python3
5) Moment of truth:
rosuav@sikorsky:~$ tmp/env/bin/python3
Python 3.7.0a0 (default:72e64fc8746b+, Oct 28 2016, 12:35:28)
[GCC 6.2.0 20161010] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys; sys.executable
'/home/rosuav/tmp/env/bin/python3'
>>> sys.path
['', '/usr/local/lib/python37.zip', '/usr/local/lib/python3.7',
'/usr/local/lib/python3.7/lib-dynload',
'/home/rosuav/tmp/env/lib/python3.7/site-packages']
So it looks like Python actually *does* take notice of the executable
path. This is presumably to allow shebangs to be created the easy way
("#!" + sys.executable) and to always represent the correct Python,
even if you had a venv active.
Now, that said, everything about the previous thread *here* was still
true. People weren't talking about symlinks. But this does mean that
(at least on recent Pythons) running Python from within a venv will
implicitly activate it. (And since the pip* files in that directory
use exactly that shebang, this is also true of running
"tmp/env/bin/pip install spam".)
I'll leave it to someone else to test the older 'virtualenv' module,
as I don't have it installed here.
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: [Theory] How to speed up python code execution / pypy vs GPU
On Sun, 6 Nov 2016 04:10 am, Mr. Wrobel wrote:
> Hi,
>
> Some skeptics asked my why there is a reason to use Python against of
> any other "not interpreted" languages, like objective-C.
Here's the "Hello World" program in Python:
--- cut ---
print("Hello World")
--- cut ---
Here's the same program in Objective C:
--- cut ---
#import
int main (int argc, const char * argv[])
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSLog (@"Hello, World!");
[pool drain];
return 0;
}
--- cut ---
Which would you rather write?
--
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.
--
https://mail.python.org/mailman/listinfo/python-list
Re: First security bug related to f-strings
On Sat, Nov 5, 2016 at 6:50 PM, Irmen de Jong wrote: > Perhaps. But in those cases you could just leave things on the default. > If you choose to run the interpreter with eval (and exec) disabled, you > should be aware > that you'll break tools like that. But for other situations (web server etc) > it could > still be useful? I do agree that not being able to use namedtuple (and > perhaps other > things from the stdlib) is a problem then. Breaking importlib at startup is not an option. An application would need to import everything before disabling exec. -- https://mail.python.org/mailman/listinfo/python-list
Re: pip3 : command not found
On 2016-11-05, Steve D'Aprano wrote: > On Sun, 6 Nov 2016 02:55 am, Jon Ribbens wrote: >> I'm afraid I can only suggest that you try re-reading the subthread >> again until you manage to understand it. It wasn't really that >> complicated but you seem to have confused yourself greatly. > > Are you serious? Yes. You haven't understood the thread, and I don't know how to explain it any simpler, so I'm afraid all I can suggest is that you re-read what was already written. > I can only repeat yet again: you have not understood Ben. You're still wrong no matter how many times you repeat yourself. I've understood him, you have understood neither him nor me. You've got hung up on the pip3 issue which is blinding you to the other implications of Ben's answer. Threads can and usually do diverge to a greater or lesser degree. Just because a post is a follow-up in a thread doesn't mean that post is entirely and solely about the original topic of that thread. Try dropping your preconceptions and reading the sub-thread again from the post before my first post without making false assumptions and while bearing in mind that I am asking a question that is tangential to the original question rather than directly descended from it. > The implication is that the answer to your question is Yes, you can run > Python in the context of a virtualenv by just invoking that virtualenv's > local Python without running 'activate' first. So you were wrong earlier when you said you couldn't do that? You're spending an awful lot of effort being pointlessly argumentative about who has or hasn't understood what while avoiding addressing the actual perfectly simple question - if running Python directly from a venv path is equivalent to running 'activate' then Python, how does it know that it's in a venv? (Yes, I know it can sort-of work out where it's run from using argv[0] but how does it know it's *in a venv*?) -- https://mail.python.org/mailman/listinfo/python-list
Re: [Theory] How to speed up python code execution / pypy vs GPU
Steve D'Aprano writes:
> On Sun, 6 Nov 2016 04:10 am, Mr. Wrobel wrote:
>
>> Hi,
>>
>> Some skeptics asked my why there is a reason to use Python against of
>> any other "not interpreted" languages, like objective-C.
>
> Here's the "Hello World" program in Python:
>
> --- cut ---
>
> print("Hello World")
>
> --- cut ---
>
>
>
> Here's the same program in Objective C:
>
> --- cut ---
>
> #import
>
> int main (int argc, const char * argv[])
> {
> NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
> NSLog (@"Hello, World!");
> [pool drain];
> return 0;
> }
>
> --- cut ---
>
> Which would you rather write?
That's a rather odd comparison. Why not
#import
int main()
{
printf("Hello world\n");
return 0;
}
? It's decades since I wrote any Objective-C (and then not much) but I
think this is the closest comparison.
--
Ben.
--
https://mail.python.org/mailman/listinfo/python-list
Re: [Theory] How to speed up python code execution / pypy vs GPU
W dniu 05.11.2016 o 22:17, Ben Bacarisse pisze:
Steve D'Aprano writes:
On Sun, 6 Nov 2016 04:10 am, Mr. Wrobel wrote:
Hi,
Some skeptics asked my why there is a reason to use Python against of
any other "not interpreted" languages, like objective-C.
Here's the "Hello World" program in Python:
--- cut ---
print("Hello World")
--- cut ---
Here's the same program in Objective C:
--- cut ---
#import
int main (int argc, const char * argv[])
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSLog (@"Hello, World!");
[pool drain];
return 0;
}
--- cut ---
Which would you rather write?
That's a rather odd comparison. Why not
#import
int main()
{
printf("Hello world\n");
return 0;
}
? It's decades since I wrote any Objective-C (and then not much) but I
think this is the closest comparison.
Wel, indeed. However the most important is second part of my question.
What do you think about using GPU processing or pypy?
--
https://mail.python.org/mailman/listinfo/python-list
Re: [Theory] How to speed up python code execution / pypy vs GPU
"Mr. Wrobel" writes: > ... However the most important is second part of my question. > > What do you think about using GPU processing or pypy? Sorry, I don't have enough experience of them to offer any useful advice. -- Ben. -- https://mail.python.org/mailman/listinfo/python-list
Re: pip3 : command not found
Steve D'Aprano writes: > Oh, and ironically, you'll see that Ben anticipated and answered your > question about activate. Here's the link again, to save you scrolling up: > > https://mail.python.org/pipermail/python-list/2016-October/715994.html > > where he says: > > If you already have a specific environment active and know that > ‘python3’ is the correct Python interpreter from that environment, > you can omit the explicit path. That's what I said, and Steven is reading its meaning correctly. -- \ “You can be a theist, and you can be a skeptic. But if you're | `\ both, you're not very good at one of them.” —Dave Silverman, | _o__) 2011-11-19 | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: [Theory] How to speed up python code execution / pypy vs GPU
On Sun, 6 Nov 2016 08:17 am, Ben Bacarisse wrote:
> Steve D'Aprano writes:
>> Here's the same program in Objective C:
>>
>> --- cut ---
>>
>> #import
>>
>> int main (int argc, const char * argv[])
>> {
>> NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
>> NSLog (@"Hello, World!");
>> [pool drain];
>> return 0;
>> }
>>
>> --- cut ---
>>
>> Which would you rather write?
>
> That's a rather odd comparison. Why not
>
> #import
>
> int main()
> {
> printf("Hello world\n");
> return 0;
> }
Because that's not Objective-C? (This is not a rhetorical question.)
I'm not an Objective-C expert, but to my eye, that doesn't look like
Objective-C. It looks like plain old regular C.
Here's where I stole the code from:
https://www.binpress.com/tutorial/objectivec-lesson-1-hello-world/41
and its not too dissimilar from the versions here:
http://rosettacode.org/wiki/Hello_world/Text#Objective-C
> ? It's decades since I wrote any Objective-C (and then not much) but I
> think this is the closest comparison.
--
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.
--
https://mail.python.org/mailman/listinfo/python-list
Re: pip3 : command not found
On Sun, 6 Nov 2016 07:55 am, Jon Ribbens wrote: >> The implication is that the answer to your question is Yes, you can run >> Python in the context of a virtualenv by just invoking that virtualenv's >> local Python without running 'activate' first. > > So you were wrong earlier when you said you couldn't do that? I'm happy to admit it when I have made a mistake, but this isn't one of those times. I didn't say it couldn't be done, I initially said: "Possibly you do have to run `activate` first, I don't know" but you already know that, since you can read just as well as I can. > You're still wrong no matter how many times you repeat yourself. > I've understood him, you have understood neither him nor me. [...] > You're spending an awful lot of effort being pointlessly argumentative > about who has or hasn't understood what while avoiding addressing the > actual perfectly simple question - if running Python directly from a > venv path is equivalent to running 'activate' then Python, how does it > know that it's in a venv? And now you change your story for the second time. As I said, I'm happy to admit when I am wrong. I was wrong to think you were discussing this in good faith. You're clearly not: you keep changing your story, keep insisting that the question you have asked is different from what you previously wrong. First you asked about symlinks, misunderstanding what Ben wrote; then you denied you asked that and insisted you really asked about activate (despite not having even mentioned activate); and now you're inventing a new question, "how does it know it is in a venv?". Life is too short to waste time in a pointless discussion with trolls who argue in bad faith, as you are doing. Welcome to my kill-file. *plonk* -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: [Theory] How to speed up python code execution / pypy vs GPU
On Sun, 6 Nov 2016 09:17 am, Mr. Wrobel wrote: > However the most important is second part of my question. > > What do you think about using GPU processing or pypy? I don't have any experience with GPU processing. I expect that it will be useful for somethings, but for number-crushing and numeric work, I am concerned that GPUs rarely provide correctly rounded IEEE-754 maths. That means that they are accurate enough for games where a few visual glitches don't matter, but they risk being inaccurate for serious work. I fear that doing numeric work in GPUs will be returning to the 1970s, when every computer was incompatible with every other computer, and it was almost impossible to write cross-platform, correct, accurate numeric code. As far as PyPy goes, it is a proven optimizing Python JIT compiler that can speed-up long-running code very well. It is not suitable for short scripts or programmers that run quickly. It takes time for the JIT to warm up and start showing optimizations. -- Steve “Cheer up,” they said, “things could be worse.” So I cheered up, and sure enough, things got worse. -- https://mail.python.org/mailman/listinfo/python-list
Re: [Theory] How to speed up python code execution / pypy vs GPU
Steve D'Aprano writes:
> On Sun, 6 Nov 2016 08:17 am, Ben Bacarisse wrote:
>
>> Steve D'Aprano writes:
>
>>> Here's the same program in Objective C:
>>>
>>> --- cut ---
>>>
>>> #import
>>>
>>> int main (int argc, const char * argv[])
>>> {
>>> NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
>>> NSLog (@"Hello, World!");
>>> [pool drain];
>>> return 0;
>>> }
>>>
>>> --- cut ---
>>>
>>> Which would you rather write?
>>
>> That's a rather odd comparison. Why not
>>
>> #import
>>
>> int main()
>> {
>> printf("Hello world\n");
>> return 0;
>> }
>
> Because that's not Objective-C? (This is not a rhetorical question.)
Yes, I think so.
> I'm not an Objective-C expert, but to my eye, that doesn't look like
> Objective-C. It looks like plain old regular C.
C has no #import directive.
--
Ben.
--
https://mail.python.org/mailman/listinfo/python-list
Re: distributed development methodology
On Saturday, October 29, 2016 at 1:20:25 PM UTC+5:30, Paul Rubin wrote: > Adam Jensen writes: > > So what are some of the more successful distributed. multi-platform, > > development models? > > Use an orchestration program to keep the systems in sync: I use ansible > (ansible.com) which is written in Python and fairly simple once you get > used to it, but there are lots of other programs in that space and > everyone has their own preferences. > > If stuff doesn't change too often, you could also use Docker or whatever > the current hotness is to make a container image and deploy it to your > fleet. Been seeing things like this of late: https://thehftguy.wordpress.com/2016/11/01/docker-in-production-an-history-of-failure/ [Disclaimer: No direct experience myself one way or other] LXC and others compared: https://www.flockport.com/alternatives-to-docker-and-lxc/ -- https://mail.python.org/mailman/listinfo/python-list
Re: pip3 : command not found
On 2016-11-06, Steve D'Aprano wrote: > *plonk* Thank feck for that, I was beginning to think he'd never shut up. I don't suppose anyone else more constructive and informed actually knows the answer to my rather simple question of how Python knows it's in a venv? ;-) -- https://mail.python.org/mailman/listinfo/python-list
Re: pip3 : command not found
On Sun, Nov 6, 2016 at 3:03 PM, Jon Ribbens wrote: > I don't suppose anyone else more constructive and informed actually > knows the answer to my rather simple question of how Python knows > it's in a venv? ;-) Two ways. 1) Normally, you 'activate' the venv by sourcing a script into your shell. This modifies $PATH, $PYTHONHOME, and I think a couple of other environment variables. 2) If Python notices that its executable comes from a venv, it uses it. (I wasn't aware of #2 until this thread.) Generally, you type "python" or "python3" and the shell searches $PATH. If you have a venv active, it'll find the one there before it finds the system one. ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: pip3 : command not found
Jon Ribbens writes: > On 2016-11-06, Steve D'Aprano wrote: > > *plonk* > > Thank feck for that, I was beginning to think he'd never shut up. Really? I didn't see a single message from Steven that wasn't solicited by an assertion from you that needed response. If your concept of “won't shut up” includes “responds to me when I demand he acknowledge my position is correct”, then it seems clear the problem has a different solution. -- \ “Prayer must never be answered: if it is, it ceases to be | `\ prayer and becomes correspondence.” —Oscar Wilde, _The Epigrams | _o__)of Oscar Wilde_, 1952 | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Delete h2 until you reach the next h2 in beautifulsoup
Considering the following html: cool stuff hizz and the following list: ignore_list = ['example','lalala'] My goal is, while going through the HTML using Beautifulsoup, I find a h2 that has an ID that is in my list (ignore_list) I should delete all the ul and lis under it until I find another h2. I would then check if the next h2 was in my ignore list, if it is, delete all the ul and lis until I reach the next h2 (or if there are no h2s left, delete the ul and lis under the current one and stop). How I see the process going: you read all the h2s from up to down in the DOM. If the id for any of those is in the ignore_list, then delete all the ul and li under the h2 until you reach the NEXT h2. If there is no h2, then delete the ul and LI then stop. Here is the full HMTL I am trying to work with: http://pastebin.com/Z3ev9c8N I am trying to delete all the UL and lis after "See_also"How would I accomplish this in Python? -- https://mail.python.org/mailman/listinfo/python-list
Re: pip3 : command not found
On 2016-11-06, Chris Angelico wrote: > On Sun, Nov 6, 2016 at 3:03 PM, Jon Ribbens wrote: >> I don't suppose anyone else more constructive and informed actually >> knows the answer to my rather simple question of how Python knows >> it's in a venv? ;-) > > Two ways. > > 1) Normally, you 'activate' the venv by sourcing a script into your > shell. This modifies $PATH, $PYTHONHOME, and I think a couple of other > environment variables. It sets VIRTUAL_ENV, adds the virtualenv's bin directory to PATH, and *unsets* PYTHONHOME if set. That's it (modulo storing old values of variables and updating PS1 which is presumably just cosmetic). > 2) If Python notices that its executable comes from a venv, it uses it. Yes. My question is *how does it notice*? -- https://mail.python.org/mailman/listinfo/python-list
Re: pip3 : command not found
On 2016-11-06, Ben Finney wrote: > Jon Ribbens writes: >> On 2016-11-06, Steve D'Aprano wrote: >> > *plonk* >> >> Thank feck for that, I was beginning to think he'd never shut up. > > Really? I didn't see a single message from Steven that wasn't solicited > by an assertion from you that needed response. > > If your concept of “won't shut up” includes “responds to me when I > demand he acknowledge my position is correct”, then it seems clear the > problem has a different solution. He consistently misunderstood almost everything I said (which is a pattern of behaviour he seems to have, i.e. heroically misunderstanding people so he can argue with them), lied about what I had said, lied about what he had said, lied about me "not arguing in good faith", was rude and insulting, and studiously avoided the actual interesting and useful topic in favour of a stupid and pointless flamewar about his misunderstanding of my original question. He clearly didn't know the answer to the question nor did he have any interest in dicussing it in a civilised manner with a view to discovering the answer, he just wants an argument. So yes, I view him bowing out of the conversation as an unmitigated blessing. -- https://mail.python.org/mailman/listinfo/python-list
Re: pip3 : command not found
Jon Ribbens writes: > He […] lied about me "not arguing in good faith" I find you to be not arguing in good faith; if you consistently engage someone in a manner that requires response, that's not consistent with also “wondering when they'd shut up”. > So yes, I view him bowing out of the conversation as an unmitigated > blessing. You could have disengaged at any time, so I don't find your statement believable. -- \ “Firmness in decision is often merely a form of stupidity. It | `\indicates an inability to think the same thing out twice.” | _o__)—Henry L. Mencken | Ben Finney -- https://mail.python.org/mailman/listinfo/python-list
Re: pip3 : command not found
On 2016-11-06, Ben Finney wrote: > Jon Ribbens writes: > >> He […] lied about me "not arguing in good faith" > > I find you to be not arguing in good faith; I find that to be particularly pompous of you. I am arguing in good faith regardless of your misguided opinion. I wasn't actually even trying to argue at all, I simply had a question, which Steven successfully derailed into a stupid argument as to whether he knew better than me what question I was asking. >> So yes, I view him bowing out of the conversation as an unmitigated >> blessing. > > You could have disengaged at any time, so I don't find your statement > believable. I plead guilty to an excess of optimism that I could persuade him to start discussing the actual topic. I had actually reached the end of my optimism in that regard and the last post I had made to him was the last one in which I had intended to bother trying to get through to him. -- https://mail.python.org/mailman/listinfo/python-list
