Re: on the python paradox

2022-12-11 Thread Martin Di Paola
On Mon, Dec 05, 2022 at 10:37:39PM -0300, Sabrina Almodóvar wrote: The Python Paradox Paul Graham August 2004 [SNIP] Hence what, for lack of a better name, I'll call the Python paradox: if a company chooses to write

Re: Which architectures to support in a CI like Travis?

2022-09-19 Thread Martin Di Paola
I would depend on the project. In the crytoanalysis tool that I developing, "cryptonita", I just manipule bytes. Nothing that could depend on the distro so my CI picks one OS and run the tests there. Project: https://github.com/cryptonitas/cryptonita CI: https://github.com/cryptonitas/cryptonit

Re: Simple TCP proxy

2022-07-27 Thread Martin Di Paola
On Wed, Jul 27, 2022 at 08:32:31PM +0200, Morten W. Petersen wrote: You're thinking of the backlog argument of listen? From my understanding, yes, when you set up the "accepter" socket (the one that you use to listen and accept new connections), you can define the length of the queue for inco

Re: exec() an locals() puzzle

2022-07-20 Thread Martin Di Paola
I did a few tests # test 1 def f(): i = 1 print(locals()) exec('y = i; print(y); print(locals())') print(locals()) a = eval('y') print(locals()) u = a print(u) f() {'i': 1} 1 {'i': 1, 'y': 1} {'i': 1, 'y': 1} {'i': 1, 'y': 1, 'a': 1} 1 # test 2 def f(): i = 1

Re: list indices must be integers or slices, not str

2022-07-20 Thread Martin Di Paola
offtopic If you want a pure-python but definitely a more hacky implementation, you can play around with inspect.stack() and get the variables from the outer frames. # code: x = 32 y = 42 printf("Hello x={x}, y={y}", x=27) # output: Hello x=27, y=42 The implementation of printf() was never re

Re: Simple message passing system and thread safe message queue

2022-07-18 Thread Martin Di Paola
Hi, I couldn't read your posts, every time I try to open one I'm redirected to an index page. I took a look at the smps project and I as far I understand it is a SSL client that sends messages to a server that implements a store of messages. I would suggest to remove the sleep() calls and as a c

Re: TENGO PROBLEMAS AL INSTALAR PYTHON

2022-07-08 Thread Martin Di Paola
On Fri, Jul 08, 2022 at 04:15:35PM -0600, Mats Wichmann wrote: In addition... there is no "Python 10.0" ... Mmm, perhaps that's the problem :D @Angie Odette Lima Banguera, vamos a necesitar algun traceback o algo para guiarte. Podes tambien buscar en internet (youtube) q hay varios tutori

Re: Filtering XArray Datasets?

2022-06-07 Thread Martin Di Paola
Hi, I'm not an expert on this so this is an educated guess: You are calling drop=True and I presume that you want to delete the rows of your dataset that satisfy a condition. That's a problem. If the underlying original data is stored in a dense contiguous array, deleting chunks of it will leav

Re: Request for assistance (hopefully not OT)

2022-05-17 Thread Martin Di Paola
Try to reinstall python and only python and if you succeeds, then try to reinstall the other tools. For this, use "apt-get" instead of "apt" $ sudo apt-get reinstall python3 When a system is heavily broken, be extra careful and read the output of the programs. If "apt-get" says than in order to

Re: Changing calling sequence

2022-05-13 Thread Martin Di Paola
You probably want something like overload/multiple dispatch. I quick search on PyPI yields a 'multipledispatch' package. I never used, however. On Wed, May 11, 2022 at 08:36:26AM -0700, Tobiah wrote: On 5/11/22 06:33, Michael F. Stemper wrote: I have a function that I use to retrieve daily dat

Re: Accuracy of multiprocessing.Queue.qsize before any Queue.get invocations?

2022-05-13 Thread Martin Di Paola
If the queue was not shared to any other process, I would guess that its size is reliable. However, a plain counter could be much simpler/safer. The developer of multiprocessing.Queue, implemented size() thinking in how to share the size and maintain a reasonable consistency between process. He/

Re: Could frozendict or frozenmap be of some use for PEP 683 (Immortal objects)?

2022-03-09 Thread Martin Di Paola
I perhaps didn't understand the PEP completely but I think that the goal of marking some objects as immortal is to remove the refcount from they. For immutable objects that would make them truly immutable. However I don't think that the immortality could be applied to any immutable object by def

Re: Execute in a multiprocessing child dynamic code loaded by the parent process

2022-03-08 Thread Martin Di Paola
Then, you must put the initialization (dynamically loading the modules) into the function executed in the foreign process. You could wrap the payload function into a class instances to achieve this. In the foreign process, you call the instance which first performs the initialization and then exe

Re: Execute in a multiprocessing child dynamic code loaded by the parent process

2022-03-07 Thread Martin Di Paola
7; in sys.modules True So the last check proves that pickle.loads imports any necessary module. Martin. On Mon, Mar 07, 2022 at 08:28:15AM +, Barry wrote: On 7 Mar 2022, at 02:33, Martin Di Paola wrote: Yes but I think that unpickle (pickle.loads()) does that plus importing any mod

Re: Execute in a multiprocessing child dynamic code loaded by the parent process

2022-03-06 Thread Martin Di Paola
Yeup, that would be my first choice but the catch is that "sayhi" may not be a function of the given module. It could be a static method of some class or any other callable. Ah, fair. Are you able to define it by a "path", where each step in the path is a getattr() call? Yes but I think th

Re: Execute in a multiprocessing child dynamic code loaded by the parent process

2022-03-06 Thread Martin Di Paola
I'm not so sure about that. The author of the plugin knows they're writing code that will be dynamically loaded, and can therefore expect the kind of problem they're having. It could be argued that it's their responsibility to ensure that all the needed code is loaded into the subprocess. Ye

Re: Execute in a multiprocessing child dynamic code loaded by the parent process

2022-03-06 Thread Martin Di Paola
Try to use `fork` as "start method" (instead of "spawn"). Yes but no. Indeed with `fork` there is no need to pickle anything. In particular the child process will be a copy of the parent so it will have all the modules loaded, including the dynamic ones. Perfect. The problem is that `fork` is t

Re: Execute in a multiprocessing child dynamic code loaded by the parent process

2022-03-06 Thread Martin Di Paola
The way you've described it, it's a hack. Allow me to slightly redescribe it. modules = loader() objs = init(modules) def invoke(mod, func): # I'm assuming that the loader is smart enough to not load # a module that's already loaded. Alternatively, load just the # module you need,

Execute in a multiprocessing child dynamic code loaded by the parent process

2022-03-06 Thread Martin Di Paola
Hi everyone. I implemented time ago a small plugin engine to load code dynamically. So far it worked well but a few days ago an user told me that he wasn't able to run in parallel a piece of code in MacOS. He was using multiprocessing.Process to run the code and in MacOS, the default start metho

Re: library not initialized (pygame)

2022-02-19 Thread Martin Di Paola
Could you share the traceback / error that you are seeing? On Sun, May 02, 2021 at 03:23:21PM -0400, Quentin Bock wrote: Code: #imports and variables for game import pygame from pygame import mixer running = True #initializes pygame pygame.init() #creates the pygame window screen = pygame.dis

Re: venv and executing other python programs

2022-02-17 Thread Martin Di Paola
That's correct. I tried to be systematic in the analysis so I tested all the possibilities. Your test results were unexpected for `python3 -m venv xxx`. By default, virtual environments exclude the system and user site packages. Including them should require the command-line argument `--system-

Re: venv and executing other python programs

2022-02-15 Thread Martin Di Paola
If you have activated the venv then any script that uses /usr/bin/env will use executables from the venv bin folder. That's correct. I tried to be systematic in the analysis so I tested all the possibilities. I avoid all these issues by not activating the venv. Python has code to know how

Re: venv and executing other python programs

2022-02-15 Thread Martin Di Paola
I did a few experiments in my machine. I created the following foo.py import pandas print("foo") Now "pandas" is installed under Python 3 outside the venv. I can run it successfully calling "python3 foo.py". If I add the shebang "#!/usr/bin/env python3" (notice the 3), I can also run it

Re: How do you log in your projects?

2022-02-10 Thread Martin Di Paola
? Logs are not intended to be read by end users. Logs are primarily used to understand what the code is doing in a production environment. They could also be used to gather metrics data. Why should you log to give a message instead of simply using a print? You are assuming that logs and prints

Re: How do you log in your projects?

2022-02-09 Thread Martin Di Paola
- On a line per line basis? on a function/method basis? In general I prefer logging line by line instead per function. It is easy to add a bunch of decorators to the functions and get the logs of all the program but I most of the time I end up with very confusing logs. There are exceptions,

Re: Waht do you think about my repeated_timer class

2022-02-04 Thread Martin Di Paola
In _run I first set the new timer and then I execute the function. So that will go mostly OK. Yes, that's correct however you are not taking into consideration the imprecision of the timers. Timer will call the next _run() after self._interval *plus* some unknown arbitrary time (and extra

Re: sharing data across Examples docstrings

2022-01-14 Thread Martin Di Paola
Hello, I understand that you want to share data across examples (docstrings) because you are running doctest to validate them (and test). The doctest implementation evaluates each docstring separately without sharing the context so the short answer is "no". This is a limitation of doctest b

Re: Call julia from Python: which package?

2021-12-18 Thread Martin Di Paola
I played with Julia a few months ago. I was doing some data-science stuff with Pandas and the performance was terrible so I decided to give Julia a try. My plan was to do the slow part in Julia and call it from Python. I tried juliacall (if I don't remember wrong) but I couldn't set up it.

Re: threading and multiprocessing deadlock

2021-12-06 Thread Martin Di Paola
Hi!, in short your code should work. I think that the join-joined problem is just an interpretation problem. In pseudo code the background_thread function does: def background_thread() # bla print("join?") # bla print("joined") When running this function in parallel using threads, you

Re: The task is to invent names for things

2021-10-28 Thread Martin Di Paola
IMHO, I prefer really weird names. For example if I'm not sure how to name a class that I'm coding, I name it like XXXYYY (literally). Really ugly. This is a way to avoid the so called "naming paralysis". Once I finish coding the class I look back and it should be easy to see "what it does" and

Re: Select columns based on dates - Pandas

2021-09-03 Thread Martin Di Paola
You may want to reshape the dataset to a tidy format: Pandas works better with that format. Let's assume the following dataset (this is what I understood from your message): In [34]: df = pd.DataFrame({ ...: 'Country': ['us', 'uk', 'it'], ...: '01/01/2019': [10, 20, 30], ...: '02/

Re: basic auth request

2021-08-21 Thread Martin Di Paola
While it is correct to say that Basic Auth without HTTPS is absolutely insecure, using Basic Auth *and* HTTPS is not secure either. Well, the definition of "secure" depends of your threat model. HTTPS ensures encryption so the content, including the Basic Auth username and password, is secret

Re: on perhaps unloading modules?

2021-08-17 Thread Martin Di Paola
This may not answer your question but it may provide an alternative solution. I had the same challenge that you an year ago so may be my solution will work for you too. Imagine that you have a Markdown file that *documents* the expected results. --8<---cut here---st

Re: Empty list as a default param - the problem, and my suggested solution

2021-08-14 Thread Martin Di Paola
I don't know if it is useful but it is an interesting metaprogramming/reflection challenge. You used `inspect` but you didn't take its full potential. Try to see if you can simplify your code and see if you can come with a decorator that does not require special parameters. from new import N

Re: [ANN] Austin -- CPython frame stack sampler v3.0.0 is now available

2021-07-02 Thread Martin Di Paola
Very nice. I used rbspy for Ruby programs https://rbspy.github.io/ and it can give you some insights about the running code that other profiling techniques may not give you. I'll use it in my next performance-bottleneck challenge. On Fri, Jul 02, 2021 at 04:04:24PM -0700, Gabriele Tornetta wro

Re: optimization of rule-based model on discrete variables

2021-06-14 Thread Martin Di Paola
From what I'm understanding it is an "optimization problem" like the ones that you find in "linear programming". But in your case the variables are not Real (they are Integers) and the function to minimize g() is not linear. You could try/explore CVXPY (https://www.cvxpy.org/) which it's a so

Re: Recommendation for drawing graphs and creating tables, saving as PDF

2021-06-11 Thread Martin Di Paola
You could try https://plantuml.com and http://ditaa.sourceforge.net/. Plantuml may not sound as the right tool but it is quite flexible and after a few tweak you can create a block diagram as you shown. And the good thing is that you *write* which elements and relations are in your diagram an

Re: Data structure for plotting monotonically expanding data set

2021-06-05 Thread Martin Di Paola
One way to go is using Pandas as it was mentioned before and Seaborn for plotting (built on top of matplotlib) I would approach this prototyping first with a single file and not with the 1000 files that you have. Using the code that you have for parsing, add the values to a Pandas DataFrame

Re: Use Chrome's / Firefox's dev-tools in python

2021-05-23 Thread Martin Di Paola
"unselectable text" not necessary means that it is an image. There is a CSS property that you can change to make a text selectable/unselectable. And if it is an image, it very likely that it comes from the server as such, so "intercepting" the packet coming from there will be for nothing: you

Re: Use Chrome's / Firefox's dev-tools in python

2021-05-22 Thread Martin Di Paola
Hello, I'm not 100% sure but I think that I understand what you are trying to do. I faced the same problem a few months ago. I wanted to know when a particular blog posted a new article. My plan was to query the blog every day running a python script, get the list of articles it has and compari

byexample: free, open source tool to find snippets of code in your docs and execute them as regression tests

2021-05-03 Thread Martin Di Paola
Hi everyone, I would like to share a free, open source tool with you that I've been developing in the last few years. You'll be probably familiar with things like this in the Python documentation: ``` >>> 1 + 3 4 ``` byexample will find those snippets, it will execute "1 + 3" and the output