Re: Dispatch table of methods with various return value types
dn writes:
> On 19/11/2020 02:13, Loris Bennett wrote:
>> dn writes:
>>
>> Firsty, thanks for taking the time to write such a detailed reply.
>
> Bitte!
>
>
>> I have a method for manipulating the membership of groups such as:
>>
>>def execute(self, operation, users, group):
>>"""
>>Perform the given operation on the users with respect to the
>>group
>>"""
>>
>>action = {
>>'get': self.get,
>>'add': self.add,
>>'delete': self.delete,
>>}
>>
>>return action.get(operation)(users, group)
>>
>> The 'get' action would return, say, a dict of users attribute, whereas
>> the 'add/delete' actions would return, say, nothing, and all actions
>> could raise an exception if something goes wrong.
>>
>> The method which calls 'execute' has to print something to the terminal,
>> such as the attributes in the case of 'get' and 'OK' in the cases of
>> 'add/delete' (assuming no exception occurred).
>>
>> Is there a canonical way of dealing with a method which returns different
>> types of data, or should I just make all actions return the same data
>> structure so that I can generate a generic response?
>
>
> Is the problem caused by coding the first step before thinking of the
> overall
> task? Try diagramming or pseudo-coding the complete solution (with
> multiple
> approaches), ie the operations AND the printing and exception-handling.
You could have a point, although I do have a reasonable idea of what the
task is and coming from a Perl background, Python always feels a bit
like pseudocode anyway (which is one of the things I like about Python).
>>>
>>> +1 the ease of Python, but can this be seductive?
>>>
>>> Per the comment about Perl/Python experience, the operative part is the
>>> "thinking", not the tool - as revealed in responses below...
>>>
>>> Sometimes we design one 'solution' to a problem, and forget (or 'brainwash'
>>> ourselves into thinking) that there might be 'another way'.
>>>
>>> It may/not apply in this case, but adjusting from a diagram-first
>>> methodology,
>>> to the habit of 'jumping straight into code' exhibited by many colleagues,
>>> before readjusting back to (hopefully) a better balance; I felt that
>>> coding-first often caused me to 'paint myself into a corner' with some
>>> 'solutions, by being too-close to the code and not 'stepping back' to take a
>>> wider view of the design - but enough about me...
>>>
>>>
> Might it be more appropriate to complete not only the get but also its
> reporting, as a unit. Similarly the add and whatever happens after that;
> and the
> delete, likewise.
Currently I am already obtaining the result and doing the reporting in
one method, but that makes it difficult to write tests, since it
violates the idea that one method should, in general, just do one thing.
That separation would seem appropriate here, since testing whether a
data set is correctly retrieved from a database seems to be
significantly different to testing whether the
reporting of an action is correctly laid out and free of typos.
>>>
>>> SRP = design thinking! +1
>>
>> I knew the idea, but I didn't now the TLA for it ;-)
>
> Yes, there are plenty of those!
>
> You may be interested in reading about "Clean Code", instigated (IIRC) by
> "Uncle
> Bob" (Robert Martin). NB Python may/not be used for book-examples. Just the
> other day I came across "Clean Code in Python", Mariano Anaya, PacktPub,
> 2018. I
> have yet to read it, but the contents page seemed to 'tick all the boxes'. The
> book is two years old, and IIRC he presented at EuroPython a few years before
> that (YouTube videos on-line - in case you prefer that medium, or want to
> gain a
> flavor before spending money...). All of these TLAs, and others comprising the
> "SOLID Principles" appear in the ToC, along with plenty of others, eg YAGNI
> and
> EAFP; plus some specific to Python, eg MRO.
I had a look at the Europython 2016 video and found it very instructive.
I'm a not very familiar with using exceptions, but when I have tried to
use them, its seems to have generated a lot of code clutter. The
approach shown in the video seems to be an elegant solution for certain
category of exception handling code repetition.
>>> TDD = early testing! +1
>>>
>>> Agreed: The tasks are definitely separate. The first is data-related. The
>>> second
>>> is about presentation.
>>>
>>> In keeping with the SRP philosophy, keep the split of execution-flow into
>>> the
>>> three (or more) functional-tasks by data-process, but turn each of those
>>> tasks
>>> into two steps/routines. (once the reporting routine following "add" has
>>> been
>>> coded, and it comes time to implement "delete", it
Regarding Regex timeout behavior to minimize CPU consumption
Hi Team, I have noticed if our regex sometimes does not give a result and on that time regex took more time in returning response (empty response). My Question is can we set a timeout parameter (in seconds/millisecond) with re.find or anywhere in code to avoid CPU consumption if regex takes more time in execution. Below is the example, which take more time in execution: (in this case can we set timeout to kill the execution to avoid CPU consumption) regex = r'data-stid="section-room-list"[\s\S]*?>\s*([\s\S]*?)\s*' \ r'(?:class\s*=\s*"\s*sticky-book-now\s*"|\s*|id\s*=\s*"Location")' rooms_blocks_to_be_replace = re.findall(regex, html_template) Please help me, I will be very thankful for this. Thanks, -- https://mail.python.org/mailman/listinfo/python-list
Cannot marshal objects
I am getting this error. I found this recipe to fix it:
from xmlrpclib import Marshaller
from decimal import Decimal
def dump_decimal(self, value, write):
write("")
write(str(value))
write("\n")
Marshaller.dispatch[Decimal] = dump_decimal
That seems to be for Python 2. I am running Python 3.5 (I know but I am
stuck there for the moment). I tried changing the import to "from
xmlrpcl.client import Marshaller" which didn't raise an error but the
problem still persists. Do I need to do something different for Python 3?
TIA.
--
D'Arcy J.M. Cain
Vybe Networks Inc.
A unit of Excelsior Solutions Corporation - Propelling Business Forward
http://www.VybeNetworks.com/
IM:da...@vybenetworks.com VoIP: sip:da...@vybenetworks.com
OpenPGP_signature
Description: OpenPGP digital signature
--
https://mail.python.org/mailman/listinfo/python-list
Why I always have invalid error when using "pip install matplotlib" in Python 3.9
Sent from [1]Mail for Windows 10 References Visible links 1. https://go.microsoft.com/fwlink/?LinkId=550986 -- https://mail.python.org/mailman/listinfo/python-list
Re: Cannot marshal objects
> I am getting this error.
I assume you mean the email subject. It doesn't work in 3.8 either:
>>> import decimal
>>> d = decimal.Decimal(3.5)
>>> d
Decimal('3.5')
>>> import marshal
>>> marshal.dumps(d)
Traceback (most recent call last):
File "", line 1, in
ValueError: unmarshallable object
but that's not surprising to me. The marshal module is more-or-less
meant to serialize Python byte code. Pickle is more generally used for
object serialization. Why not use it?
>>> import pickle, decimal
>>> d = decimal.Decimal(3.5)
>>> pickle.dumps(d)
b'\x80\x04\x95!\x00\x00\x00\x00\x00\x00\x00\x8c\x07decimal\x94\x8c\x07Decimal\x94\x93\x94\x8c\x033.5\x94\x85\x94R\x94.'
That's meant more for serialization of data objects.
Skip
--
https://mail.python.org/mailman/listinfo/python-list
Re: Environment vars
On 26/11/2020 05:46, Bob van der Poel wrote: I've got a program which accepts an optional env variable listing a single or multiple directory for the app to use. I've done a bit of a search and see both a comma and semicolon being used/suggested as a path separator. Any consensus on which is better? Further to comments about some users expecting one option, whereas others may be thinking of another... Herewith an irreverent article from "ElReg". NB Do not read "The Register" unless you can cope with sardonic, British, humor; and especially if you do not believe that "confession is good for the soul"! It's always DNS, especially when a sysadmin makes a hash of their semicolons Remember the days when 'we made it up as we went along'? Richard Speed Mon 23 Nov 2020 // 08:15 UTC https://www.theregister.com/2020/11/23/who_me/ -- Regards =dn -- https://mail.python.org/mailman/listinfo/python-list
Re: Environment vars
On Fri, Nov 27, 2020 at 1:41 PM dn via Python-list wrote: > On 26/11/2020 05:46, Bob van der Poel wrote: > > I've got a program which accepts an optional env variable listing a > single > > or multiple directory for the app to use. I've done a bit of a search and > > see both a comma and semicolon being used/suggested as a path separator. > > Any consensus on which is better? > > > Further to comments about some users expecting one option, whereas > others may be thinking of another... > > > Herewith an irreverent article from "ElReg". > > NB Do not read "The Register" unless you can cope with sardonic, > British, humor; and especially if you do not believe that "confession is > good for the soul"! > > It's always DNS, especially when a sysadmin makes a hash of their > semicolons > Remember the days when 'we made it up as we went along'? > Richard Speed Mon 23 Nov 2020 // 08:15 UTC > https://www.theregister.com/2020/11/23/who_me/ > -- > Regards =dn > -- > https://mail.python.org/mailman/listinfo/python-list > This article clearly represents things which could go wrong in the good old days. Those days are gone forever and things like that could never go wrong today :) -- https://mail.python.org/mailman/listinfo/python-list
Re: Cannot marshal objects
On 11/27/20 4:05 PM, Skip Montanaro wrote: I am getting this error. I assume you mean the email subject. It doesn't work in 3.8 either: Yes I do and that's too bad. but that's not surprising to me. The marshal module is more-or-less meant to serialize Python byte code. Pickle is more generally used for object serialization. Why not use it? Because the marshaling is happening in the guts of xmlrpc. I may have solved it anyway. The code I was running was spitting out that error but I think it was actually happening in the server that it was connecting to. I just added that same code to the server. Have to wait until Monday to test. I will update the list if it fixes it. -- D'Arcy J.M. Cain Vybe Networks Inc. A unit of Excelsior Solutions Corporation - Propelling Business Forward http://www.VybeNetworks.com/ IM:da...@vybenetworks.com VoIP: sip:da...@vybenetworks.com OpenPGP_signature Description: OpenPGP digital signature -- https://mail.python.org/mailman/listinfo/python-list
try/except in loop
I'm using the Box API (https://developer.box.com/guides/tooling/sdks/python/).
I can get an access token, though it expires after a certain amount of
time. My plan is to store the access token on the filesystem and use it
until it expires, then fetch a new one. In the example below assume I have
an expired access token.
# This next line does not throw an error:
client.folder('0').get_items()
# But iteration does (maybe this is a lazy fetch?)
for _ in client.folder('0').get_items():
logger.debug("Using existing access token.")
return access_token
# So I use try/except
try:
for _ in client.folder('0').get_items():
logger.debug("Using existing access token.")
return access_token
except boxsdk.exception.BoxAPIException:
pass # access token invalid, let's get one
else:
pass # access token invalid, let's get one
# When running the debugger the except clause seems to catch the first
throw, but the loop I think continues, throws the error again, and that
second throw is not caught.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Cannot marshal objects
> Because the marshaling is happening in the guts of xmlrpc.
Okay, I misunderstood your original message. You used the word
"marshal" (and the xmlrpc error did as well). I thought you were
referring to the marshal module. I didn't understand (and ignored) the
references to the xmlrpc module. In Python 3, the Marshaller class has
migrated into the xmlrpc.client module.
Your code worked for me in 3.8. Note, however, that this is not going
to round trip properly. You will stuff in a Decimal object at one end
and get out a double at the other end. Recent versions support
unmarshalling to Decimal objects in the Unmarshaller class. I don't
know why the Marshaller class doesn't accept Decimal objects for
serialization. You could try this:
def dump_decimal(self, value, write):
write("")
write(str(value))
write("\n")
xmlrpc.client.Marshaller.dispatch[decimal.Decimal] = dump_decimal
I used the "ex:" prefix based on this document:
http://ws.apache.org/xmlrpc/types.html
YMMV. The tag name understood by the Unmarshaller class doesn't
include that prefix.
Skip
--
https://mail.python.org/mailman/listinfo/python-list
Re: try/except in loop
On Fri, Nov 27, 2020, 6:06 PM Jason Friedman wrote:
> I'm using the Box API (
> https://developer.box.com/guides/tooling/sdks/python/).
> I can get an access token, though it expires after a certain amount of
> time. My plan is to store the access token on the filesystem and use it
> until it expires, then fetch a new one. In the example below assume I have
> an expired access token.
>
>
> # This next line does not throw an error:
>
> client.folder('0').get_items()
>
>
> # But iteration does (maybe this is a lazy fetch?)
>
> for _ in client.folder('0').get_items():
>
> logger.debug("Using existing access token.")
>
> return access_token
>
>
> # So I use try/except
>
> try:
>
> for _ in client.folder('0').get_items():
>
> logger.debug("Using existing access token.")
>
> return access_token
>
> except boxsdk.exception.BoxAPIException:
>
> pass # access token invalid, let's get one
>
> else:
>
> pass # access token invalid, let's get one
>
>
> # When running the debugger the except clause seems to catch the first
> throw, but the loop I think continues, throws the error again, and that
> second throw is not caught.
>
It would appear that get items is a generator which uses the token exactly
once when it is first started; subsequent calls to the generator all use
the same token. You need to test the token; if it fails,
obtain a new one, then start the loop.
Bob Gailer
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: try/except in loop
>
>
>> I'm using the Box API (
>> https://developer.box.com/guides/tooling/sdks/python/).
>> I can get an access token, though it expires after a certain amount of
>> time. My plan is to store the access token on the filesystem and use it
>> until it expires, then fetch a new one. In the example below assume I have
>> an expired access token.
>>
>> # This next line does not throw an error:
>> client.folder('0').get_items()
>>
>> # But iteration does (maybe this is a lazy fetch?)
>> for _ in client.folder('0').get_items():
>> logger.debug("Using existing access token.")
>> return access_token
>>
>> # So I use try/except
>> try:
>> for _ in client.folder('0').get_items():
>> logger.debug("Using existing access token.")
>> return access_token
>> except boxsdk.exception.BoxAPIException:
>> pass # access token invalid, let's get one
>> else:
>> pass # access token invalid, let's get one
>>
>> # When running the debugger the except clause seems to catch the first
>> throw, but the loop I think continues, throws the error again, and that
>> second throw is not caught.
>>
>
> It would appear that get items is a generator which uses the token exactly
> once when it is first started; subsequent calls to the generator all use
> the same token. You need to test the token; if it fails,
> obtain a new one, then start the loop.
>
# Here's the solution I came up with:
try:
list(client.folder('0').get_items())
logger.debug("Using existing access token.")
return access_token
except boxsdk.exception.BoxAPIException:
pass # access token invalid, let's get one
# And for those who happen to be using the Box API, this command avoids all
that and is less expensive:
try:
client.user().get()
logger.debug("Using existing access token.")
return access_token
except boxsdk.exception.BoxAPIException:
pass # access token invalid, let's get one
--
https://mail.python.org/mailman/listinfo/python-list
Re: try/except in loop
I had to do this with a different API the toehr year, and made this
method:
def auth_token(self):
''' Obtain a current auth token [...]
Refreshes the cached token if stale.
'''
while True:
with self._auth_token_lock:
token = self._auth_token
if (token and token.timestamp + FM_TOKEN_LIFESPAN -
FM_TOKEN_LIFESPAN_SKEW > time.time()):
break
try:
token = self._get_new_auth_token()
except Exception as e:
error("error fetching token: %s", e)
else:
if token:
self._auth_token = token
break
error("token not refreshed, delay then retry")
time.sleep(self.DEFAULT_RETRY_DELAY)
return token
The "while True" bit was because the server was flakey and sometime you
just had to wait for it to come back.
Then I just called this to obtain a current token whenever I needed a
token. in the API, for example:
headers = {'Authorization': 'Bearer ' + self.auth_token().token}
Means you don't have to embed verbose checks all through your code -
just grab "the token" and proceeed.
Cheers,
Cameron Simpson
--
https://mail.python.org/mailman/listinfo/python-list
filtering out warnings
The Box API is noisy ... very helpful for diagnosing, and yet for
production code I'd like less noise.
I tried this:
warnings.filterwarnings(
action='ignore',
# category=Warning,
# module=r'boxsdk.*'
)
but I still see this:
WARNING:boxsdk.network.default_network:"POST
https://api.box.com/oauth2/token"; 400 83
{'Date': 'Sat, 28 Nov 2020 04:30:03 GMT', 'Content-Type':
'application/json', 'Transfer-Encoding': 'chunked', 'Connection':
'keep-alive', 'Strict-Transport-Security': 'max-age=31536000',
'Set-Cookie': 'box_visitor_id=5fc1d24b134ce6.76522820; expires=Sun,
28-Nov-2021 04:30:03 GMT; Max-Age=31536000; path=/; domain=.box.com;
secure, bv=OPS-44131; expires=Sat, 05-Dec-2020 04:30:03 GMT;
Max-Age=604800; path=/; domain=.app.box.com; secure, cn=87; expires=Sun,
28-Nov-2021 04:30:03 GMT; Max-Age=31536000; path=/; domain=.app.box.com;
secure, site_preference=desktop; path=/; domain=.box.com; secure',
'Cache-Control': 'no-store'}
{'error': 'invalid_client',
'error_description': 'The client credentials are invalid'}
My code as written I think should filter ALL warnings. I'll eventually want
to restrict the filter to only warnings generated by the Box SDK.
--
https://mail.python.org/mailman/listinfo/python-list
