Shipping python modules - best practices?
Hello, I developed some moderate-sized python scripts that I would like to distribute as python modules. I have never shipped modules before and I read http://docs.python.org/distutils/index.html. I was able to generate a source distribution, but I still have some questions. 1) My module contains some unit tests in separate scripts. I want to execute them when I generate the source distribution. How can I do it? 2) When I install 3rd party modules, I see some of them executing their unit tests during the installation. How can I do it? How can I provide an option to the user to ignore possible test failures? 3) I generate some html documentation with pydoc. How shall I distribute it with the module? Thank you very much in advance. Regards rambius -- http://mail.python.org/mailman/listinfo/python-list
Parametrized Unit Tests
Hello,
I am running one and the same unit tests that test some web application. I
would like to execute them against different servers that may host different
instances of the application.
So far I have something like
#!/usr/bin/env python
import unittest
server = ""
user = ""
password = ""
class MyTest(unittest.TestCase):
def setUp(self):
global server
global user
global password
self.conn = MyConnection(server, user, password)
def test_1(self):
result = run_smth_1(self.conn)
verify_smth_1(result)
def test_2(self):
result = run_smth_2(self.conn)
verify_smth_2(result)
def tearDown(self):
self.conn.close()
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('-u', '--user',
default=getpass.getuser())
parser.add_argument('-p', '--password')
parser.add_argument('-s', '--server')
args = parser.parse_args()
server = args.server
user = args.user
password = args.password
unittest.main()
Is there a better a way to pass the server, the user and the password to the
test without resolving to global variables?
Although I developed these tests as unit tests they are more of integration
tests. Is there an integration testing framework that supports a more
convenient passing of test parameters / data?
Thank you in advance for your responses.
Regards
Rambius
--
https://mail.python.org/mailman/listinfo/python-list
Re: Parametrized Unit Tests
Hello, петък, 21 август 2015 г., 21:43:19 UTC-4, Ben Finney написа: > > Is there a better a way to pass the server, the user and the password > > to the test without resolving to global variables? > > The ‘testscenarios’ library is one way to have a set of scenarios > applied at run-time to produce tests across all combinations > https://pypi.python.org/pypi/testscenarios/>. testscenarios worked for me. Thank you for your recommendation. Regards Ivan -- https://mail.python.org/mailman/listinfo/python-list
What's wrong with my logging configuration
Hello,
I am using python logging and as of today it stopped working. Here is
my program:
#!/usr/bin/env python
import base64
import getpass
import httplib
import logging
import logging.config
import sys
import urlparse
logging.config.fileConfig('logging.config')
logger = logging.getLogger(sys.argv[0])
def main():
from optparse import OptionParser
usage = '%prog -u [options] '
parser = OptionParser(usage=usage)
parser.add_option('-u', '--user',
help='The user that will be authenticated')
(options, args) = parser.parse_args()
if len(args) != 1:
logger.error('Please provide a url')
sys.exit(1)
url = args[0]
if not options.user:
logger.error('Please provide a user')
sys.exit(2)
password = getpass.getpass('Password: ')
if __name__ == '__main__':
main()
Here is my logging.config:
[loggers]
keys=root
[handlers]
keys=console,file
[formatters]
keys=consolefmt,filefmt
[formatter_consolefmt]
format=%(name)-12s: %(levelname)-8s %(message)s
class=logging.Formatter
[formatter_filefmt]
format=%(asctime)s %(name)-12s %(levelname)-8s %(message)s
datefmt=%d/%m/%Y %H:%M:%S
[handler_console]
class=StreamHandler
level=INFO
formatter=consolefmt
args=(sys.stdout,)
[handler_file]
class=FileHandler
level=DEBUG
formatter=filefmt
args=('dav.log', 'a')
[logger_root]
level=NOTSET
handlers=console,file
Yesterday, it was working and I have not touched it - logging has been
working for months. This morning it stopped working and I was
debugging it for half a day and I am clueless. It does not the errors
neither on the console nor in the log file.
I will appreciate any help! Thank you in advance!
Regards
Rambius
--
http://mail.python.org/mailman/listinfo/python-list
Re: What's wrong with my logging configuration
On Nov 27, 4:07 pm, rambius wrote: > Hello, > > I am using python logging and as of today it stopped working. Here is > my program: > I forgot to give my environment: $ uname -a Darwin arielmac.lan 10.5.0 Darwin Kernel Version 10.5.0: Fri Nov 5 23:20:39 PDT 2010; root:xnu-1504.9.17~1/RELEASE_I386 i386 $ python -V Python 2.6.6 Regards Rambius -- http://mail.python.org/mailman/listinfo/python-list
Re: What's wrong with my logging configuration
Hello,
I was able to reproduce the problem with even a smaller program:
#!/usr/bin/env python
import logging
import logging.config
logging.config.fileConfig('logging.config')
logger = logging.getLogger('test')
def main():
logger.fatal('test1')
import optparse
logger.fatal('test2')
if __name__ == '__main__':
main()
When I run this program the logger statement after 'import optparse'
does not appear. The first logger statement before optparse does
appear. Has anyone experienced similar behaviour?
Regards
Rambius
--
http://mail.python.org/mailman/listinfo/python-list
Re: What's wrong with my logging configuration
Hello Ben, On what environment did you try it? On Nov 27, 5:09 pm, Ben Finney wrote: > rambius writes: > > When I run this program the logger statement after 'import optparse' > > does not appear. The first logger statement before optparse does > > appear. Has anyone experienced similar behaviour? > > I use the same config file (from your first message) and your shorter > program, and I see this output: > > = > test : CRITICAL test1 > test : CRITICAL test2 > = > This is so strange. If I import optparse outside of main() it works. If I import it between the two logging statement, the second one does not appear. Regards Rambius -- http://mail.python.org/mailman/listinfo/python-list
Re: What's wrong with my logging configuration
Hello, I found the culprit. Yesterday I added a file called copy.py. It clashed with the python built-in module copy and caused that nasty error. Regards Rambius -- http://mail.python.org/mailman/listinfo/python-list
Resolving Weekday Schedules to Dates
Hello, Do you know of a library that resolves schedules like every Wednesday at 3:00pm to absolute time, that is return the datetime of the next occurrence? Regards rambius P.S. -- Tangra Mega Rock: http://www.radiotangra.com -- https://mail.python.org/mailman/listinfo/python-list
Custom help format for a choice argparse argument
Hello,
I am developing a script that accepts a time zone as an option. The
time zone can be any from pytz.all_timezones. I have
def main():
parser = argparse.ArgumentParser()
parser.add_argument("-z", "--zone", choices=pytz.all_timezones)
args = parser.parse_args()
print(args)
print(f"Specified timezone: {args.zone}")
It works, but when I run it with the -h option it dumps all entries in
pytz.all_timezones. I would like to modify the help format for just
-z|--zone option. I read the docs about HelpFormatter and argparse.py
and I ended up with
class CustomHelpFormatter(argparse.HelpFormatter):
def _metavar_formatter(self, action, default_metavar):
if action.dest == 'zone':
result = 'zone from pytz.all_timezones'
def format(tuple_size):
if isinstance(result, tuple):
return result
else:
return (result, ) * tuple_size
return format
else:
return super(CustomHelpFormatter,
self)._metavar_formatter(action, default_metavar)
def main():
parser = argparse.ArgumentParser(formatter_class=CustomHelpFormatter)
parser.add_argument("-z", "--zone", choices=pytz.all_timezones)
args = parser.parse_args()
print(args)
print(f"Specified timezone: {args.zone}")
This works, but is there a more elegant way to achieve it?
Regards
rambius
--
Tangra Mega Rock: http://www.radiotangra.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: Custom help format for a choice argparse argument
Hello,
On Fri, Jan 27, 2023 at 4:22 PM Weatherby,Gerard wrote:
>
> Why not something like:
>
>
> parser.add_argument("-z", "--zone")
>
>args = parser.parse_args()
>if args.zone and args.zone not in ptyz.all_timezones:
>
> print(“Invalid timezone”,file=sys.stderr)
>
This is what I use now. I still wonder if I can mold HelpFormatter to
do what I want it to do.
> …
>
>
>
>
> From: Python-list on
> behalf of Ivan "Rambius" Ivanov
> Date: Friday, January 27, 2023 at 3:33 PM
> To: Python
> Subject: Custom help format for a choice argparse argument
>
> *** Attention: This is an external email. Use caution responding, opening
> attachments or clicking on links. ***
>
> Hello,
>
> I am developing a script that accepts a time zone as an option. The
> time zone can be any from pytz.all_timezones. I have
>
> def main():
> parser = argparse.ArgumentParser()
> parser.add_argument("-z", "--zone", choices=pytz.all_timezones)
> args = parser.parse_args()
> print(args)
> print(f"Specified timezone: {args.zone}")
>
> It works, but when I run it with the -h option it dumps all entries in
> pytz.all_timezones. I would like to modify the help format for just
> -z|--zone option. I read the docs about HelpFormatter and argparse.py
> and I ended up with
>
> class CustomHelpFormatter(argparse.HelpFormatter):
> def _metavar_formatter(self, action, default_metavar):
> if action.dest == 'zone':
> result = 'zone from pytz.all_timezones'
> def format(tuple_size):
> if isinstance(result, tuple):
> return result
> else:
> return (result, ) * tuple_size
> return format
> else:
> return super(CustomHelpFormatter,
> self)._metavar_formatter(action, default_metavar)
>
>
> def main():
> parser = argparse.ArgumentParser(formatter_class=CustomHelpFormatter)
> parser.add_argument("-z", "--zone", choices=pytz.all_timezones)
> args = parser.parse_args()
> print(args)
> print(f"Specified timezone: {args.zone}")
>
> This works, but is there a more elegant way to achieve it?
>
> Regards
> rambius
>
> --
> Tangra Mega Rock:
> https://urldefense.com/v3/__http://www.radiotangra.com__;!!Cn_UX_p3!kiJusdm5pCptP3sOBX85KXqUJkqr2jSa4C_-WAqND7WkL-aw3BYbW50td_AcuzJ1XUPYIVO3JiLMc4gRWS885vTKFsFvaQ$
> --
> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!kiJusdm5pCptP3sOBX85KXqUJkqr2jSa4C_-WAqND7WkL-aw3BYbW50td_AcuzJ1XUPYIVO3JiLMc4gRWS885vRXq-JKLg$
--
Tangra Mega Rock: http://www.radiotangra.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: Custom help format for a choice argparse argument
Hello Cameron,
On Fri, Jan 27, 2023 at 4:45 PM Cameron Simpson wrote:
>
> On 27Jan2023 15:31, Ivan "Rambius" Ivanov
> wrote:
> >I am developing a script that accepts a time zone as an option. The
> >time zone can be any from pytz.all_timezones. I have
> >
> >def main():
> >parser = argparse.ArgumentParser()
> >parser.add_argument("-z", "--zone", choices=pytz.all_timezones)
> [...]
> >
> >It works, but when I run it with the -h option it dumps all entries in
> >pytz.all_timezones.
>
> What happens if you just presupply a `help=` parameter in
> `add_argument`?
I tried with def main():
parser = argparse.ArgumentParser()
parser.add_argument("-z", "--zone", choices=pytz.all_timezones,
help="a timezone from pytz.all_timezones")
args = parser.parse_args()
print(args)
-h still shows all the contents of pytz.all_timezones.
Regards
rambius
>
> Cheers,
> Cameron Simpson
> --
> https://mail.python.org/mailman/listinfo/python-list
--
Tangra Mega Rock: http://www.radiotangra.com
--
https://mail.python.org/mailman/listinfo/python-list
Question about logging.config.dictConfig
Hello,
I am trying to configure my loggers using dictConfig, but they do not
print anything. Here are more details.
I have several python scripts that use a similar logging setup. I put
the common configuration in a separate module myloggingconf.py:
# myloggingconf.py
import logging
def configure_logging():
default = logging.Formatter('%(asctime)s %(levelname)-7s %(name)s
%(funcName)s %(message)s')
consoleHandler = logging.StreamHandler()
consoleHandler.setFormatter(default)
rootLogger = logging.getLogger()
rootLogger.addHandler(consoleHandler)
rootLogger.setLevel(logging.INFO)
Then I use it in the other scripts as:
# uselog.py
import logging
import os
from myloggingconf import configure_logging
logger = logging.getLogger(os.path.basename(__file__))
def dosmth():
logger.debug("debug")
logger.info("info")
logger.warning("warning")
logger.error("error")
def main():
configure_logging()
dosmth()
if __name__ == '__main__':
main()
This works correctly and the log messages about INFO print out. Now I
want to try dictConfig. I changed myloggingconf.configure_logging to:
# myloggingconf.py
import logging.config
def configure_logging():
config = {
'version': 1,
'disable_existing_loggers': True,
'formatters': {
'default': {
'fmt': '%(asctime)s %(levelname)-7s %(name)s
%(funcName)s %(message)s'
}
},
'handlers': {
'stdout': {
'level': 'DEBUG',
'class': 'logging.StreamHandler',
'formatter': 'default',
}
},
'loggers': {
'': {
'handlers': ['stdout'],
'level': 'DEBUG'
}
}
}
logging.config.dictConfig(config)
When I run uselog.py it prints nothing. I am wondering what is wrong
with the second configuration. I will appreciate any help.
Regards
rambius
--
Tangra Mega Rock: http://www.radiotangra.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: Question about logging.config.dictConfig
On Tue, Feb 7, 2023 at 7:35 PM Peter J. Holzer wrote:
>
> On 2023-02-07 17:58:26 -0500, Ivan "Rambius" Ivanov wrote:
> > I am trying to configure my loggers using dictConfig, but they do not
> > print anything. Here are more details.
> [...]
> > from myloggingconf import configure_logging
> >
> > logger = logging.getLogger(os.path.basename(__file__))
> [...]
> >
> > def main():
> > configure_logging()
> > dosmth()
> [...]
> > def configure_logging():
> > config = {
> > 'version': 1,
> > 'disable_existing_loggers': True,
>
> 'disable_existing_loggers': False,
Thank you! That helped a lot!
Regards
rambius
>
> I think I just found out why I've been cargo-culting this line since my
> early Django days ;-)-
>
> If you call getLogger very early (as you do), you don't want it disabled
> by a later call to dictConfig().
>
>
> > 'formatters': {
> > 'default': {
> > 'fmt': '%(asctime)s %(levelname)-7s %(name)s %(funcName)s
> > %(message)s'
> 'format'
> > }
> > },
> [...]
> > }
> > logging.config.dictConfig(config)
> >
> > When I run uselog.py it prints nothing. I am wondering what is wrong
> > with the second configuration.
>
> See above.
>
> hp
>
> --
>_ | Peter J. Holzer| Story must make more sense than reality.
> |_|_) ||
> | | | h...@hjp.at |-- Charles Stross, "Creative writing
> __/ | http://www.hjp.at/ | challenge!"
> --
> https://mail.python.org/mailman/listinfo/python-list
--
Tangra Mega Rock: http://www.radiotangra.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: System Beep?
Hello, On Fri, Mar 8, 2019, 3:19 PM Steve wrote: > = RESTART: C:\Gork\Med Insulin codes\MedReminder 127.py > = > Traceback (most recent call last): > File "C:\Gork\Med Insulin codes\MedReminder 127.py", line 13, in > > windsound.Beep #(frequency, duration) > NameError: name 'windsound' is not defined > > Something missing in my code? > You have a typo. It is winsound, not winDsound. Regards rambius > > 98% of lawyers give the other 2% a bad name. > > > -Original Message- > From: Python-list On > Behalf Of David Raymond > Sent: Friday, March 8, 2019 3:02 PM > To: python-list@python.org > Subject: RE: System Beep? > > Windows has the built in winsound module that lets you set frequency and > duration, not sure about other OS's. > > https://docs.python.org/3.7/library/winsound.html > > > -Original Message- > From: Python-list > [mailto:python-list-bounces+david.raymond=tomtom@python.org] On Behalf > Of Steve > Sent: Friday, March 08, 2019 1:14 PM > To: python-list@python.org > Subject: System Beep? > > How can I cause a system beep using code? > This is using the computer's internal speaker, not an audio external > speaker. > Do I have control of duration or volume? > > Steve > > > > > > Footnote: > Every minute, every single day, the equivalent of a truckload of plastic > enters our oceans. > > > -- > https://mail.python.org/mailman/listinfo/python-list > -- > https://mail.python.org/mailman/listinfo/python-list > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: answer not correct
Hello, First of all, remove the asterisks around the snippet, it makes it so hard to copy and paste your code. My answer is inlined. On Sun, Nov 8, 2020 at 2:28 PM Quentin Bock wrote: > > *def add(numbers):* > * total = 1* If this is your sum, you need to initialize it to zero: total = 0 > * for x in numbers:* > * total += 1* Do you mean total += x? Regards rambius > * return total* > *print(add[1999, -672, 64]))* > > *the answer I get is 4 but it should be 1,411* > -- > https://mail.python.org/mailman/listinfo/python-list -- Tangra Mega Rock: http://www.radiotangra.com -- https://mail.python.org/mailman/listinfo/python-list
Re: EnvironmentError
Hello, On Thu, Nov 19, 2020 at 1:56 PM Usman Musa wrote: > > When I try to install a package or upgrade pip, using pip install I got > this error massage. > WARNING: Retrying (Retry(total=4, connect=None, read=None, > redirect=None, status=None)) after connection broken by > 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] > certificate verify failed: certificate is not yet valid (_ssl.c:1122)'))': > /packages/97/f3/c064343ac58d1a54c393a3f66483a29500f644a5918deeb935d28673edd9/virtualenv-20.1.0-py2.py3-none-any.whl > WARNING: Retrying (Retry(total=3, connect=None, read=None, redirect=None, > status=None)) after connection broken by > 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] > certificate verify failed: certificate is not yet valid (_ssl.c:1122)'))': > /packages/97/f3/c064343ac58d1a54c393a3f66483a29500f644a5918deeb935d28673edd9/virtualenv-20.1.0-py2.py3-none-any.whl > WARNING: Retrying (Retry(total=2, connect=None, read=None, redirect=None, > status=None)) after connection broken by > 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] > certificate verify failed: certificate is not yet valid (_ssl.c:1122)'))': > /packages/97/f3/c064343ac58d1a54c393a3f66483a29500f644a5918deeb935d28673edd9/virtualenv-20.1.0-py2.py3-none-any.whl > WARNING: Retrying (Retry(total=1, connect=None, read=None, redirect=None, > status=None)) after connection broken by > 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] > certificate verify failed: certificate is not yet valid (_ssl.c:1122)'))': > /packages/97/f3/c064343ac58d1a54c393a3f66483a29500f644a5918deeb935d28673edd9/virtualenv-20.1.0-py2.py3-none-any.whl > WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, > status=None)) after connection broken by > 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] > certificate verify failed: certificate is not yet valid (_ssl.c:1122)'))': > /packages/97/f3/c064343ac58d1a54c393a3f66483a29500f644a5918deeb935d28673edd9/virtualenv-20.1.0-py2.py3-none-any.whl > ERROR: Could not install packages due to an EnvironmentError: > HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries > exceeded with url: > /packages/97/f3/c064343ac58d1a54c393a3f66483a29500f644a5918deeb935d28673edd9/virtualenv-20.1.0-py2.py3-none-any.whl > (Caused by SSLError(SSLCertVerificationError(1, '[SSL: > CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate is not > yet valid (_ssl.c:1122)'))) > Please help me out > -- > https://mail.python.org/mailman/listinfo/python-list The error says that the certificate of files.pythonhosted.org is not valid, but this is definitely not the case. I think you sit behind some ssl proxy and it behaves as a man-in-the-middle and replaces the certificate with its own (I was experiencing that in a previous job). You have several choices: 1) pass --trusted-host files.pythonhosted.org to disable the certificate check 2) import the ssl certificate of the ssl proxy (if you have such) into python's certificate store 3) speak with the local system administrators about that. They may be of help. Regards rambius -- Tangra Mega Rock: http://www.radiotangra.com -- https://mail.python.org/mailman/listinfo/python-list
Concatenating a Hash to a String
Hello,
I want to store the hashes of strings in a database and I have
problems generating the sql statements. I generate the hashes using
hashlib and then convert it to base64 and I put the base64
representation in the sql. Here is the code:
#!/usr/bin/env python3.8
import base64
import hashlib
def gen_sql(s):
sha = hashlib.sha256()
# need to encode s as binary, otherwise the next line throws
# TypeError: Unicode-objects must be encoded before hashing
sha.update(b"{s}")
ehash = base64.b64encode(sha.digest())
sql = "insert into HASHES value ('" + ehash + "')"
s = "test"
gen_sql(s)
This code throws
$ ./hashinstr.py
Traceback (most recent call last):
File "./hashinstr.py", line 16, in
gen_sql(s)
File "./hashinstr.py", line 13, in gen_sql
sql = "insert into HASHES value ('" + ehash + "')"
TypeError: can only concatenate str (not "bytes") to str
Any help on how to concatenate ehash to the sql?
Regards
rambius
--
Tangra Mega Rock: http://www.radiotangra.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: Concatenating a Hash to a String
On Tue, Dec 1, 2020 at 12:39 AM Chris Angelico wrote: > Don't do this! DO NOT do this! Even if it might happen to work with a > base 64 encoded value, this is a terrible terrible bug just waiting to > happen. Instead, use *parameterized queries* and keep your SQL safe. OK. What are parameterized queries? Can you give an example? -- Tangra Mega Rock: http://www.radiotangra.com -- https://mail.python.org/mailman/listinfo/python-list
Re: Concatenating a Hash to a String
Hello,
Thank you all for your help.
On Tue, Dec 1, 2020 at 1:38 PM MRAB wrote:
> The bytes are all in the ASCII range, so you can convert it into a
> string using .decode('ascii').
I utilized encode and decode string methods to convert from bytes to strings
> And, of course, use parametrised queries, as ChrisA said.
I am using sqlite3 and it supports parameterized queries. Overall they
are better than constructing the sql statements manually.
Regards and thanks
rambius
--
Tangra Mega Rock: http://www.radiotangra.com
--
https://mail.python.org/mailman/listinfo/python-list
Getting stdout and stderr from subprocess in correct order
Dear colleagues,
I using subprocess module and I am wondering how I can get the output
of the spawned process's stdout and stderr in the right order. Here
are my sample programs:
$ cat subprc.py
import subprocess
import sys
f = 'hw.py'
p = subprocess.run([sys.executable, f], stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
print(p.stdout)
print(p.stderr)
$ cat hw.py
import sys
print("a", file=sys.stdout)
print("b", file=sys.stderr)
print("c", file=sys.stdout)
print("d", file=sys.stderr)
When i run hw.py I get
$ ./hw.py
a
b
c
d
When I run it through subprc.py, I do get standard out and standard
errors streams, but they are separated and not in the order above:
$ ./subprc.py
b'a\nc\n'
b'b\nd\n'
How should I use subprocess in order to get the outputs in the correct
order? Thank you for your help in advance.
Regards
Rambius
--
Tangra Mega Rock: http://www.radiotangra.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: Getting stdout and stderr from subprocess in correct order
Hello,
Thank you all for your suggestions. I will see what will apply to my use case.
Regards
Rambius
On Sat, Mar 4, 2017 at 5:37 PM, Piet van Oostrum
wrote:
> "Ivan \"Rambius\" Ivanov" writes:
>
>> Dear colleagues,
>>
>> I using subprocess module and I am wondering how I can get the output
>> of the spawned process's stdout and stderr in the right order. Here
>> are my sample programs:
>>
>> $ cat subprc.py
>> import subprocess
>> import sys
>>
>> f = 'hw.py'
>> p = subprocess.run([sys.executable, f], stdout=subprocess.PIPE,
>> stderr=subprocess.PIPE)
>> print(p.stdout)
>> print(p.stderr)
>>
>> $ cat hw.py
>> import sys
>>
>> print("a", file=sys.stdout)
>> print("b", file=sys.stderr)
>> print("c", file=sys.stdout)
>> print("d", file=sys.stderr)
>>
>> When i run hw.py I get
>>
>> $ ./hw.py
>> a
>> b
>> c
>> d
>>
>> When I run it through subprc.py, I do get standard out and standard
>> errors streams, but they are separated and not in the order above:
>>
>> $ ./subprc.py
>> b'a\nc\n'
>> b'b\nd\n'
>>
>> How should I use subprocess in order to get the outputs in the correct
>> order? Thank you for your help in advance.
>>
> If you want them in the order they are produced then you have to put
> them through a single channel with stderr=subprocess.STDOUT.
>
> p = subprocess.run([sys.executable, f], stdout=subprocess.PIPE,
> stderr=subprocess.STDOUT, encoding='ascii')
>
> I added an encoding so that it will be read as a text file rather than
> binary. And of course you should read only stdout (stderr will give
> None)
>
> Moreover you must make sure that each write is flushed. This can be done
> in three ways:
>
> 1. Add the flush=True argument to the print statements.
> print("a", file=sys.stdout, flush=True)
> print("b", file=sys.stderr, flush=True)
> print("c", file=sys.stdout, flush=True)
> print("d", file=sys.stderr, flush=True)
>
> 2. Change the files to a line buffered version.
> import os
> sys.stdout = os.fdopen(sys.stdout.fileno(), 'w', 1)
> sys.stderr = os.fdopen(sys.stderr.fileno(), 'w', 1)
>
> 3. Add a flush() call after each print statement, like
> sys.stdout.flush()
>
> Then you get them in order.
> $ python3 subprc.py
> a
> b
> c
> d
>
> Of course you won't be able to tell from which stream each line comes.
> The streams are byte streams, not message streams. You would have to put
> extra information, e.g. a prefix, in your print statements.
> --
> Piet van Oostrum
> WWW: http://pietvanoostrum.com/
> PGP key: [8DAE142BE17999C4]
> --
> https://mail.python.org/mailman/listinfo/python-list
--
Tangra Mega Rock: http://www.radiotangra.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: what type of application implemented with python?
Hello, Python's own website contains a non-exhaustive list of applications implemeted in Python: https://www.python.org/about/apps/ Regards Rambius On Sat, Apr 15, 2017 at 4:13 PM, kondaiah sinha wrote: > what type of application implemented with python?...like by using java we can > implement web,Standard alone app's,mobile based. > Can anyone tell me is it possible to implement Standard alone applications > with, if possible give any example > -- > https://mail.python.org/mailman/listinfo/python-list -- Tangra Mega Rock: http://www.radiotangra.com -- https://mail.python.org/mailman/listinfo/python-list
A Single Instance of an Object?
Hello,
I am refactoring some code and I would like to get rid of a global
variable. Here is the outline:
import subprocess
CACHE = {}
def lookup(key):
""""Runs the command cmd, parses its output, extract's the key's value,
caches it and returns it. If the key has already been in the cache,
returns its cached value. If the command cmd returns an error, the
value is set to None and cached as None."""
if key in CACHE:
return CACHE[key]
value = None
cmd = f"mycmd {key}"
proc = subprocess(cmd, capture_output=True, text=True, check=False)
if proc.returncode == 0:
value = proc.stdout.strip()
else:
logger.error("cmd returned error")
CACHE[key] = value
return value
...
def main():
while True:
keys = load_keys()
for key in keys:
call_function_that_call_function_that_calls_lookup(key)
The global cache variable made unit testing of the lookup(key) method
clumsy, because I have to clean it after each unit test. I refactored
it as:
class Lookup:
def __init__(self):
self.cache = {}
def lookup(key):
if key in self.cache:
return self.cache[key]
value = None
cmd = f"mycmd {key}"
proc = subprocess(cmd, capture_output=True, text=True, check=False)
if proc.returncode == 0:
value = proc.stdout.strip()
else:
logger.error("cmd returned error")
self.cache[key] = value
return value
Now it is easier to unit test, and the cache is not global. However, I
cannot instantiate Lookup inside the while- or for- loops in main(),
because the cache should be only one. I need to ensure there is only
one instance of Lookup - this is why I made it a global variable, so
that it is accessible to all functions in that script and the one that
actually needs it is 4 levels down in the call stack.
I have never done that in Python because I deliberately avoided such
complicated situations up to now. I know about the Singleton pattern,
but I have never implemented it in Python and I don't know if it is
Pythonish.
I am looking for the same behaviour as logging.getLogger(name).
logging.getLogger("myname") will always return the same object no
matter where it is called as long as the name argument is the same.
How would you advise me to implement that?
Regards
rambius
--
Tangra Mega Rock: http://www.radiotangra.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: A Single Instance of an Object?
On Mon, Mar 11, 2024 at 5:01 PM Chris Angelico via Python-list wrote: > > On Tue, 12 Mar 2024 at 07:54, Ivan "Rambius" Ivanov via Python-list > wrote: > > I am refactoring some code and I would like to get rid of a global > > variable. Here is the outline: > > > > ... > > > > I have never done that in Python because I deliberately avoided such > > complicated situations up to now. I know about the Singleton pattern, > > but I have never implemented it in Python and I don't know if it is > > Pythonish. > > > > A Singleton is just a global variable. Why do this? Did someone tell > you "global variables are bad, don't use them"? I have bad experience with global variables because it is hard to track what and when modifies them. I don't consider them bad, but if I can I avoid them. Regards rambius -- Tangra Mega Rock: http://www.radiotangra.com -- https://mail.python.org/mailman/listinfo/python-list
Re: A Single Instance of an Object?
On Mon, Mar 11, 2024 at 5:06 PM dn via Python-list
wrote:
>
> Good question Rambius!
>
> On 12/03/24 09:53, Ivan "Rambius" Ivanov via Python-list wrote:
> > Hello,
> >
> > I am refactoring some code and I would like to get rid of a global
> > variable. Here is the outline:
> >
> > import subprocess
> >
> > CACHE = {}
>
> First thought: don't reinvent-the-wheel, use lru_cache
> (https://docs.python.org/3/library/functools.html)
>
>
> > The global cache variable made unit testing of the lookup(key) method
> > clumsy, because I have to clean it after each unit test. I refactored
> > it as:
> >
> > class Lookup:
> > def __init__(self):
> > self.cache = {}
> >
>
> Change "cache" to be a class-attribute (it is currently an instance.
>
> Then, code AFTER the definition of Lookup can refer to Lookup.cache,
> regardless of instantiation, and code within Lookup can refer to
> self.cache as-is...
>
Thank you for your suggestions. I will research them!
Regards
rambius
--
Tangra Mega Rock: http://www.radiotangra.com
--
https://mail.python.org/mailman/listinfo/python-list
Timezone in HH:MM Format
Hello,
How can I convert a date, usually datetime.now(), into a format where
the timezone is in hours:minutes format. I was able to get that format
in shell:
$ date +%Y-%m-%dT%H:%M:%S%:z
2024-06-18T19:24:09-04:00
The closest I got in python is
from datetime import datetime
from zoneinfo import ZoneInfo
s = datetime.strftime(datetime.now(ZoneInfo("America/New_York")),
"%Y-%m-%dT%H:%M:%S%z")
print(s)
This prints the same as the shell command above except the last column:
2024-06-18T19:28:56-0400
Any help will be appreciated.
Regards
Ivan
--
Tangra Mega Rock: http://www.radiotangra.com
--
https://mail.python.org/mailman/listinfo/python-list
Re: Timezone in HH:MM Format
Thank you all for your responses!
On Tue, Jun 18, 2024 at 9:54 PM Jon Ribbens via Python-list
wrote:
>
> datetime.now(ZoneInfo("America/New_York")).isoformat()
Both .isoformat() and "%:z" work.
--
Tangra Mega Rock: http://www.radiotangra.com
--
https://mail.python.org/mailman/listinfo/python-list
