Re: Find the path of a shell command
On 10/12/22 14:51, Paulo da Silva wrote:
Às 19:14 de 12/10/22, Jon Ribbens escreveu:
On 2022-10-12, Paulo da Silva
wrote:
Às 05:00 de 12/10/22, Paulo da Silva escreveu:
Hi!
The simple question: How do I find the full path of a shell command
(linux), i.e. how do I obtain the corresponding of, for example,
"type rm" in command line?
The reason:
I have python program that launches a detached rm. It works pretty well
until it is invoked by cron! I suspect that for cron we need to specify
the full path.
Of course I can hardcode /usr/bin/rm. But, is rm always in /usr/bin?
What about other commands?
Thank you all who have responded so far.
I think that the the suggestion of searching the PATH env seems the
best.
Another thing that I thought of is that of the 'which', but, to avoid
the mentioned recurrent problem of not knowing where 'which' is I would
use 'type' instead. 'type' is a bash (sh?) command.
If you're using subprocess.run / subprocess.Popen then the computer is
*already* searching PATH for you.
Yes, and it works out of cron.
Your problem must be that your cron
job is being run without PATH being set, perhaps you just need to edit
your crontab to set PATH to something sensible.
I could do that, but I am using /etc/cron.* for convenience.
Or just hard-code your
program to run '/bin/rm' explicitly, which should always work (unless
you're on Windows, of course!)
It can also be in /bin, at least.
A short idea is to just check /bin/rm and /usr/bin/rm, but I prefer
searching thru PATH env. It only needs to do that once.
I've read quite a bit of this thread without completely understanding
the actual problem.
Crontabs have always been good at exposing the problem of "is my program
expecting something to be set up which isn't guaranteed?" - usually
environment variables, of which PATH is one. "But it worked from my
login shell".
Python's standard library has a tool for you:
shutil.which = which(cmd, mode=1, path=None)
Given a command, mode, and a PATH string, return the path which
conforms to the given mode on the PATH, or None if there is no such
file.
`mode` defaults to os.F_OK | os.X_OK. `path` defaults to the result
of os.environ.get("PATH"), or can be overridden with a custom search
path.
Since cron usually runs in a minimal environment, you should be able to
use this to supply a decent value for PATH, look up the command you
need, and then use that path to launch the command - or fail in some
sort of graceful way if not found, so you can hear about it and make
corrections.
--
https://mail.python.org/mailman/listinfo/python-list
Fwd: Can you help me with this Python question?
For a python class I am taking.. In this challenge, you'll be working with a DataFrame that contains data about artworks, and it contains many missing values. Your task is to create a variable called na_sum that contains the total number of missing values in the DataFrame. When that's completed, print out your answer! Hint: The code given below will give you the number of missing (NaN) values for the *Name* column in the DataFrame. How would you edit the code to get the missing values for every column in the DataFrame? Extra hint: You'll be returning a single number which is the final sum() of everything. df['Name'].isnull().sum() -- Thanks! *Sarah Wallace* sarah.wallac...@gmail.com -- Thanks! *Sarah Wallace* sarah.wallac...@gmail.com 214.300.1064 -- https://mail.python.org/mailman/listinfo/python-list
Re: Find the path of a shell command
In comp.lang.python, jkn wrote: > On Wednesday, October 12, 2022 at 6:12:23 AM UTC+1, jak wrote: >> I'm afraid you will have to look for the command in every path listed in >> the PATH environment variable. > erm, or try 'which rm' ? It is so hilarious seeing the responses to this thread. Hint: what do you think the `which` program does? $ touch /var/tmp/rm $ chmod 755 /var/tmp/rm $ env -i PATH=/etc:/usr:/lib:/var/tmp:/tmp /usr/bin/which rm /var/tmp/rm $ mv /var/tmp/rm /tmp/ $ env -i PATH=/etc:/usr:/lib:/var/tmp:/tmp /usr/bin/which rm /tmp/rm $ Elijah -- /usr/bin/busybox rm /tmp/rm -- https://mail.python.org/mailman/listinfo/python-list
Re: What might suddenly provoke this poplib error?
A further little bit of information, I tried running getCatchall.py from the command prompt and there was a long wait before it output the same error message. I.e. it looks rather as if the server is not responding to requests. (A 'long wait' is a minute or two) -- Chris Green · -- https://mail.python.org/mailman/listinfo/python-list
Re: Find the path of a shell command
On 2022-10-12, Paulo da Silva wrote:
> Às 22:38 de 12/10/22, Jon Ribbens escreveu:
>> On 2022-10-12, Jon Ribbens wrote:
>>> On 2022-10-12, Paulo da Silva wrote:
Às 19:14 de 12/10/22, Jon Ribbens escreveu:
> On 2022-10-12, Paulo da Silva
> wrote:
>> Às 05:00 de 12/10/22, Paulo da Silva escreveu:
>>> Hi!
>>>
>>> The simple question: How do I find the full path of a shell command
>>> (linux), i.e. how do I obtain the corresponding of, for example,
>>> "type rm" in command line?
>>>
>>> The reason:
>>> I have python program that launches a detached rm. It works pretty well
>>> until it is invoked by cron! I suspect that for cron we need to specify
>>> the full path.
>>> Of course I can hardcode /usr/bin/rm. But, is rm always in /usr/bin?
>>> What about other commands?
>>>
>> Thank you all who have responded so far.
>> I think that the the suggestion of searching the PATH env seems the best.
>> Another thing that I thought of is that of the 'which', but, to avoid
>> the mentioned recurrent problem of not knowing where 'which' is I would
>> use 'type' instead. 'type' is a bash (sh?) command.
>
> If you're using subprocess.run / subprocess.Popen then the computer is
> *already* searching PATH for you.
Yes, and it works out of cron.
> Your problem must be that your cron
> job is being run without PATH being set, perhaps you just need to edit
> your crontab to set PATH to something sensible.
I could do that, but I am using /etc/cron.* for convenience.
> Or just hard-code your
> program to run '/bin/rm' explicitly, which should always work (unless
> you're on Windows, of course!)
It can also be in /bin, at least.
>>>
>>> I assume you mean /usr/bin. But it doesn't matter. As already
>>> discussed, even if 'rm' is in /usr/bin, it will be in /bin as well
>>> (or /usr/bin and /bin will be symlinks to the same place).
>>>
A short idea is to just check /bin/rm and /usr/bin/rm, but I prefer
searching thru PATH env. It only needs to do that once.
>>>
>>> I cannot think of any situation in which that will help you. But if for
>>> some reason you really want to do that, you can use the shutil.which()
>>> function from the standard library:
>>>
>>> >>> import shutil
>>> >>> shutil.which('rm')
>>> '/usr/bin/rm'
>>
>> Actually if I'm mentioning shutil I should probably mention
>> shutil.rmtree() as well, which does the same as 'rm -r', without
>> needing to find or run any other executables.
> Except that you can't have parallel tasks, at least in an easy way.
> Using Popen I just launch rm's and end the script.
[threading.Thread(target=shutil.rmtree, args=(item,)).start()
for item in items_to_delete]
--
https://mail.python.org/mailman/listinfo/python-list
What might suddenly provoke this poplib error?
I have a short python3 program that collects E-Mails from a 'catchall'
mailbox, sends the few that might be interesting to me and dumps the
rest.
It has suddenly (after working for some years) started throwing the
following:-
Traceback (most recent call last):
File "/home/chris/.mutt/bin/getCatchall.py", line 83, in
pop3.pass_('brzmilla')
File "/usr/lib/python3.10/poplib.py", line 218, in pass_
return self._shortcmd('PASS %s' % pswd)
File "/usr/lib/python3.10/poplib.py", line 181, in _shortcmd
return self._getresp()
File "/usr/lib/python3.10/poplib.py", line 157, in _getresp
raise error_proto(resp)
poplib.error_proto: b'-ERR internal server error'
The section of code throwing the error is as follows:-
#
#
# Connect to the POP3 server, get message count, exit if no messages
#
for t in range(10): # retry 10 times
try:
pop3 = poplib.POP3_SSL('mail3.gridhost.co.uk',timeout=300)
break
except TimeoutError:
if t == 9:
log.err("Timed out 10 times, giving up")
exit(1)
else:
log.warn("Timed out, try " + str(t))
pop3.user('catch...@isbd.net')
pop3.pass_('brzmilla')
numMessages = len(pop3.list()[1])
if (numMessages == 0):
break
It seems to be saying that the POP3 server has a problem, if so there's not
much I can do about it as it's my hosting provider's mail server. Is it
really saying the server has a problem?
--
Chris Green
·
--
https://mail.python.org/mailman/listinfo/python-list
Re: What might suddenly provoke this poplib error?
On 2022-10-13 13:47, Chris Green wrote:
I have a short python3 program that collects E-Mails from a 'catchall'
mailbox, sends the few that might be interesting to me and dumps the
rest.
It has suddenly (after working for some years) started throwing the
following:-
Traceback (most recent call last):
File "/home/chris/.mutt/bin/getCatchall.py", line 83, in
pop3.pass_('brzmilla')
File "/usr/lib/python3.10/poplib.py", line 218, in pass_
return self._shortcmd('PASS %s' % pswd)
File "/usr/lib/python3.10/poplib.py", line 181, in _shortcmd
return self._getresp()
File "/usr/lib/python3.10/poplib.py", line 157, in _getresp
raise error_proto(resp)
poplib.error_proto: b'-ERR internal server error'
The section of code throwing the error is as follows:-
#
#
# Connect to the POP3 server, get message count, exit if no messages
#
for t in range(10): # retry 10 times
try:
pop3 = poplib.POP3_SSL('mail3.gridhost.co.uk',timeout=300)
break
except TimeoutError:
if t == 9:
log.err("Timed out 10 times, giving up")
exit(1)
else:
log.warn("Timed out, try " + str(t))
pop3.user('catch...@isbd.net')
pop3.pass_('brzmilla')
numMessages = len(pop3.list()[1])
if (numMessages == 0):
break
It seems to be saying that the POP3 server has a problem, if so there's not
much I can do about it as it's my hosting provider's mail server. Is it
really saying the server has a problem?
As you've already ascertained that it's a server error, I'd just like to
suggest that you add a sleep before retrying. If it has timed out after
5 minutes, I doubt there's much point in retrying immediately.
--
https://mail.python.org/mailman/listinfo/python-list
Re: What might suddenly provoke this poplib error?
On 2022-10-13 13:47:07 +0100, Chris Green wrote: > I have a short python3 program that collects E-Mails from a 'catchall' > mailbox, sends the few that might be interesting to me and dumps the > rest. > > It has suddenly (after working for some years) started throwing the > following:- [...] > poplib.error_proto: b'-ERR internal server error' [...] > It seems to be saying that the POP3 server has a problem, Yes. "-ERR" is the normal start of a POP error reply, so the message "-ERR internal server error" was sent by the POP server. > if so there's not much I can do about it as it's my hosting provider's > mail server. You can call you hosting provider and ask them to fix the problem. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Re: Fwd: Can you help me with this Python question?
Well, although I never used pandas and never will, if that's about artworks, that's mine. Obviously, you need to iterate columns and sum values returned by the snippet you provided. A quick search tells us to use colums property. So, it might look like this: na_sum = sum(df[name].isnull().sum() for name in df.columns) Axy On 13/10/2022 13:44, Sarah Wallace wrote: For a python class I am taking.. In this challenge, you'll be working with a DataFrame that contains data about artworks, and it contains many missing values. Your task is to create a variable called na_sum that contains the total number of missing values in the DataFrame. When that's completed, print out your answer! Hint: The code given below will give you the number of missing (NaN) values for the *Name* column in the DataFrame. How would you edit the code to get the missing values for every column in the DataFrame? Extra hint: You'll be returning a single number which is the final sum() of everything. df['Name'].isnull().sum() -- https://mail.python.org/mailman/listinfo/python-list
Re: What might suddenly provoke this poplib error?
On 14/10/2022 01.47, Chris Green wrote:
I have a short python3 program that collects E-Mails from a 'catchall'
mailbox, sends the few that might be interesting to me and dumps the
rest.
It has suddenly (after working for some years) started throwing the
following:-
Traceback (most recent call last):
File "/home/chris/.mutt/bin/getCatchall.py", line 83, in
pop3.pass_('brzmilla')
File "/usr/lib/python3.10/poplib.py", line 218, in pass_
return self._shortcmd('PASS %s' % pswd)
File "/usr/lib/python3.10/poplib.py", line 181, in _shortcmd
return self._getresp()
File "/usr/lib/python3.10/poplib.py", line 157, in _getresp
raise error_proto(resp)
poplib.error_proto: b'-ERR internal server error'
The section of code throwing the error is as follows:-
#
#
# Connect to the POP3 server, get message count, exit if no messages
#
for t in range(10): # retry 10 times
try:
pop3 = poplib.POP3_SSL('mail3.gridhost.co.uk',timeout=300)
break
except TimeoutError:
if t == 9:
log.err("Timed out 10 times, giving up")
exit(1)
else:
log.warn("Timed out, try " + str(t))
pop3.user('catch...@isbd.net')
pop3.pass_('brzmilla')
numMessages = len(pop3.list()[1])
if (numMessages == 0):
break
It seems to be saying that the POP3 server has a problem, if so there's not
much I can do about it as it's my hosting provider's mail server. Is it
really saying the server has a problem?
There's a range of possibilities here. The first question to ask (as you
probably have) is "what has changed?" - has the Python/PSL on this
computer been updated*, has the computer's SSL/TLS library been updated,
etc. Then there are possible changes at 'the other end' - per previous
responses.
* IIRC the last changes to email lib were circa 3.5, but poplib's doc
page shows more recent updates - including to timeouts in v3.9 (YMMV).
There are changes afoot in the TLS arena. Older algorithms being dumped,
and newer, stronger, harder, faster, better mechanisms (with racing
stripes) being added. Not using POP3, I can't say how much impact such
may have...
Try using an independent checker to investigate continuity of access.
From your own machine (I use "s_client -connect..." but for IMAP), thus
should be able to replicate the 'error' conversation at the level of the
POP3 'conversation'. Alternately, use an on-line service such as:
https://www.wormly.com/test-pop3-mail-server
Disclaimer: again, not using POP3, I have never used this service, it
'popped-up' whilst searching on DuckDuckGo.
Many general-purpose service/server-providers seem rather heavy-handed
when it comes to email security, often as a result of some knee-jerk
response to a 'live-issue' - ultimately caused by them not having
competent email-specialists on-staff. Accordingly, they may have
'changed policy' and 'forgotten' to let mere-clients know. (although, if
your standard email access continues unabated, this seems less-likely)
That said, it never hurts to ask/be in friendly-contact...
PS is that really your password? If so, ...
--
Regards,
=dn
--
https://mail.python.org/mailman/listinfo/python-list
Re: What might suddenly provoke this poplib error?
MRAB wrote: [snip boring code] > > > > It seems to be saying that the POP3 server has a problem, if so there's not > > much I can do about it as it's my hosting provider's mail server. Is it > > really saying the server has a problem? > > > As you've already ascertained that it's a server error, I'd just like to > suggest that you add a sleep before retrying. If it has timed out after > 5 minutes, I doubt there's much point in retrying immediately. I guess I could do that but this has only happened once in quite a few years so I'm not sure if I;ll bother! :-) The program is only run half-hourly by cron. -- Chris Green · -- https://mail.python.org/mailman/listinfo/python-list
Re: What might suddenly provoke this poplib error?
Peter J. Holzer wrote: > [-- text/plain, encoding quoted-printable, charset: us-ascii, 28 lines --] > > On 2022-10-13 13:47:07 +0100, Chris Green wrote: > > I have a short python3 program that collects E-Mails from a 'catchall' > > mailbox, sends the few that might be interesting to me and dumps the > > rest. > > > > It has suddenly (after working for some years) started throwing the > > following:- > [...] > > poplib.error_proto: b'-ERR internal server error' > [...] > > It seems to be saying that the POP3 server has a problem, > > Yes. "-ERR" is the normal start of a POP error reply, so the message > "-ERR internal server error" was sent by the POP server. > > > if so there's not much I can do about it as it's my hosting provider's > > mail server. > > You can call you hosting provider and ask them to fix the problem. > After a couple of hours someone obviously fixed something and the error has stopped now. Thanks everyone. -- Chris Green · -- https://mail.python.org/mailman/listinfo/python-list
Re: Find the path of a shell command
On 10/11/22 22:00, Paulo da Silva wrote: > Hi! > > The simple question: How do I find the full path of a shell command > (linux), i.e. how do I obtain the corresponding of, for example, > "type rm" in command line? > > The reason: > I have python program that launches a detached rm. It works pretty well > until it is invoked by cron! I suspect that for cron we need to specify > the full path. > Of course I can hardcode /usr/bin/rm. But, is rm always in /usr/bin? > What about other commands? There are certain standards that suggest where to look. For example, there's the Linux Filesystem Hiearchy Standard 3.0: https://refspecs.linuxfoundation.org/FHS_3.0/fhs/ch03s04.html In short, you want to hard code /bin for a command like rm. And yes it will always be in /bin on any standard Linux OS. Despite modern distros making /bin and /usr/bin the same directory, if the target OS is anywhere close to the standard, you can always find the basic commands in /bin. I would not hard code any script to use /usr/bin for any basic commands and I would not use anything other than /bin/sh or /bin/bash as the shell script shebang if you want any sort of portability. -- https://mail.python.org/mailman/listinfo/python-list
