Python-Verilog (Pyverilog)

[Bobby]

I am working on my own domain-specific language (DSL). The DSL is based on 
functional requirements of a device under test (DUT). The grammar has rules 
like 'if' and 'else'. I have made a grammar of my DSL without any programming 
code in my grammar. I am using ANTLR to generate Parser, Lexer, Visitor and 
Listeners in Python. I have made the visitor pattern in python to parse the 
input I give to my grammar. The input I am parsing is in natural words as my 
grammar itself consists of natural words rules. Therefore the input is not a 
programming language. Now I have a working visitor in python which can 
successfully interpret my input.
 
I would like to generate a Verilog test bench structure from this stage on 
wards. Literally blocked at this point. I have come across this toolkit 
Pyverilog. It has a Verilog code generator as a tool. In this Verilog code 
generator, a Verilog HDL code is represented by using the AST classes defined 
in its parser called'vparser.ast'.

Can I use this code generator as a standalone for my Python visitor pattern 
which I have made from my ANTLR ? Any workaround that I can use this kit 
considering the above mentioned scenario ?

I would appreciate any suggestion. 

-- 
https://mail.python.org/mailman/listinfo/python-list

Python-Verilog (PyHDI-Pyverilog)

[Bobby]

I am working on my own domain-specific language (DSL). The DSL is based on 
functional requirements of a device under test (DUT). The grammar has rules 
like 'if' and 'else'. I have made a grammar of my DSL without any programming 
code in my grammar. I am using ANTLR to generate Parser, Lexer, Visitor and 
Listeners in Python. I have made the visitor pattern in python to parse the 
input I give to my grammar. The input I am parsing is in natural words as my 
grammar itself consists of natural words rules. Therefore the input is not a 
programming language. Now I have a working visitor in python which can 
successfully interpret my input. 
  
I would like to generate a Verilog test bench structure from this stage on 
wards. Literally blocked at this point. I have come across this toolkit 
Pyverilog (https://github.com/PyHDI/Pyverilog). It has a Verilog code generator 
as a tool. In this Verilog code generator, a Verilog HDL code is represented by 
using the AST classes defined in its parser called'vparser.ast'. 

Can I use this code generator as a standalone for my Python visitor pattern 
which I have made from my ANTLR ? Any workaround that I can use this kit 
considering the above mentioned scenario ? 

I would appreciate any suggestion.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Marko Rauhamaa]

Chris Angelico :

> On Mon, Oct 15, 2018 at 3:51 PM Marko Rauhamaa  wrote:
> I don't understand your point here. It prints a letter, then some
> spaces, then a tab, then another letter. On my terminal, that displays
> the tab by advancing to the next tab position. If I highlight to
> select, it's obvious that the spaces have not been collapsed or
> converted in any way; it is indeed printing that many spaces, then a
> tab. Universal default? Not very.

The point is that your tab stops are at every 8th column even if some
other tab stops were used in your editor.

Unless you configured your terminal (emulator) with the same tab stops
as your editor.

Then, you'll need to configure your printer, browser and other visual
tools that have the every-8th-column-tab-stop default.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Rhodri James]

On 14/10/18 09:06, Peter J. Holzer wrote:

On 2018-10-14 00:45:49 +, Grant Edwards wrote:

On 2018-10-13, Peter J. Holzer  wrote:



For "just use tabs" to work, all of those tools would have to
magically recognize that they're looking at Python source and adjust
the tab size accordingly.  That isn't going to happen.


Well, no. The idea of "just use tabs" isn't have a different tab width
per language, but a different tab width per user.


You work in a different environment than I do.  for me, tab width
varies from one project to another (sometimes even in the same
language).  I don't get to pick just one tab width.


You probably don't because you use spaces, not tabs.

If everybody used tabs, why would anyone care about your tab width?
Unless they look over your shoulder while you are typing, they won't
even know how wide your tabs are.


If you used two-space tab stops and I used (the normal :-) eight, 
comfortable indentations for you would rapidly become uncomfortably 
large indentations for me.  I'd care about that.  I'd also care if you 
used spaces to make sub-tab stop adjustments to line up appropriate text 
(inside brackets, for example).


--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

Re: ESR "Waning of Python" post

[Marko Rauhamaa]

dieter :

> Marko Rauhamaa  writes:
>> Keeping the number of long-term objects low is key.
>
> Right, if I need near realtime behaviour and must live
> with [C]Python's garbage collector.

Or any other GC ever invented.

> But, a web application does usually not need near realtime behaviour.
> An occasional (maybe once in a few days) garbage collection and
> associated reduced response time is acceptable.
> A problem only arises if a badly designed component produces
> quite frequently hundreds of thousands of temporary objects
> likely triggering (frequent) garbage collections.

But I think you are barking up the wrong tree. You could rightly blame
GC itself as an unworkable paradigm and switch to, say, C++ or Rust.

Or you could blame the parts of the software that create too many
long-term objects.

You shouldn't blame the parts of the software that churn out zillions of
short-term objects.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Rhodri James]

On 15/10/18 05:45, Marko Rauhamaa wrote:

Chris Angelico :

I'm saying I have never seen is this:

On Mon, Oct 15, 2018 at 8:56 AM Marko Rauhamaa  wrote:

However, it is trumped by an older
convention whereby the indentation levels go as follows:

0:
1: SPC SPC
2: SPC SPC SPC SPC
3: SPC SPC SPC SPC SPC SPC
4: TAB
5: TAB SPC SPC
6: TAB SPC SPC SPC SPC
7: TAB SPC SPC SPC SPC SPC SPC
8: TAB TAB


Specifically that two-space indents and tab-collapsing are a
*convention*. I have never seen this used anywhere, much less seen it
commonly enough to call it a convention.


The two-space indentation is the out-of-the-box default for emacs.


Ahem.  It's the default for certain C styles.  It's not even the default 
for C-mode itself, which is 4.  Those of us who believe that tabs are 
evil set indent-tabs-mode nil anyway to stop the annoying behaviour.


--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Marko Rauhamaa]

Rhodri James :

> On 15/10/18 05:45, Marko Rauhamaa wrote:
>> The two-space indentation is the out-of-the-box default for emacs.
>
> Ahem.  It's the default for certain C styles.  It's not even the default
> for C-mode itself, which is 4.

You must be running a different version of emacs than all the versions
I've every run.

Try running

emacs -q abc.c

and observe the indentation depth.

> Those of us who believe that tabs are evil set indent-tabs-mode nil
> anyway to stop the annoying behaviour.

Yes.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Antoon Pardon]

On 13-10-18 09:37, Peter J. Holzer wrote:
> On 2018-10-09 09:55:34 +0200, Antoon Pardon wrote:
>> On 08-10-18 19:43, Peter J. Holzer wrote:
>>> On 2018-10-08 10:36:21 +1100, Chris Angelico wrote:
 How wide my indents are on my screen shouldn't influence your screen
 or your choices.
>>> Theoretically I would agree with you: Just use a single tab per
>>> indentation level and let the user decide whether that's displayed as 2,
>>> 3, 4, or 8 spaces or 57 pixels or whatever. 
>>>
>>> In practice it doesn't work in my experience. There is always someone in
>>> a team who was "just testing that new editor" and replaced all tabs
>>> with spaces (or vice versa) or - worse - just some of them.
>> Isn't that caugth in the process of commiting to version control?
> Tabs are easy to catch. If a file contains a tab, reject it.
>
> Spaces aren't, because spaces are everywhere.

Spaces that replaced a tab by accident, are easy to catch too. They are all
those lines that show up when you do a diff with the previous version that
shouldn't show up.

-- 
Antoon.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Rhodri James]

On 15/10/18 12:28, Marko Rauhamaa wrote:

Rhodri James :


On 15/10/18 05:45, Marko Rauhamaa wrote:

The two-space indentation is the out-of-the-box default for emacs.


Ahem.  It's the default for certain C styles.  It's not even the default
for C-mode itself, which is 4.


You must be running a different version of emacs than all the versions
I've every run.

Try running

 emacs -q abc.c

and observe the indentation depth.


https://www.gnu.org/software/emacs/manual/html_node/ccmode/Customizing-Indentation.html#Customizing-Indentation

"""User Option: c-basic-offset

This style variable holds the basic offset between indentation 
levels. It's factory default is 4, but all the built-in styles set it 
themselves, to some value between 2 (for gnu style) and 8 (for bsd, 
linux, and python styles)."""


--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Marko Rauhamaa]

Rhodri James :

> On 15/10/18 12:28, Marko Rauhamaa wrote:
>> Try running
>>
>>  emacs -q abc.c
>>
>> and observe the indentation depth.
>
> """User Option: c-basic-offset
>
> This style variable holds the basic offset between indentation
> levels. It's factory default is 4, but all the built-in styles set it
> themselves, to some value between 2 (for gnu style) and 8 (for bsd,
> linux, and python styles)."""

To realize why 2 is the factory default despite the above statement, we
observe that the factory setting of c-default-style specifies "gnu" for
C-like files other than Java and awk.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: UnicodeDecodeError: 'charmap' codec can't decode byte 0x9d in position 10442: character maps to

[pjmclenon]

On Saturday, October 13, 2018 at 7:24:14 PM UTC-4, MRAB wrote:
> On 2018-10-14 00:13, pjmcle...@gmail.com wrote:
> > On Wednesday, June 13, 2018 at 7:14:06 AM UTC-4, INADA Naoki wrote:
> >> ​> 1st is this script is from a library module online open source
> >> 
> >> If it's open source, why didn't you show the link to the soruce?
> >> I assume your code is this:
> >> 
> >> https://github.com/siddharth2010/String-Search/blob/6770c7a1e811a5d812e7f9f7c5c83a12e5b28877/createIndex.py
> >> 
> >> And self.collFile is opened here:
> >> 
> >> https://github.com/siddharth2010/String-Search/blob/6770c7a1e811a5d812e7f9f7c5c83a12e5b28877/createIndex.py#L91
> >> 
> >> You need to add `encoding='utf-8'` argument.
> > 
> > 
> > 
> > hello i used this recommandtion in one of my projects in python and it 
> > worked fine im wrting today cuz i have this same unicode error in a slighty 
> > differn file code line and i added encoding utf 8 but i still get the same 
> > error
> > 
> > here is my line of code
> > 
> > with open(join("docs", path)) as f:
> > 
> > where can i add the encoding="utf8" line??
> > does anyone on this forum happen to know??
> > 
> > ok thank you jessica
> > 
> with open(join("docs", path), encoding="utf-8") as f:

THANK you very much
that works
i had forgot a comma , thats what gave me the error

thaxz again
Jesiica
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Rhodri James]

On 15/10/18 16:41, Marko Rauhamaa wrote:

Rhodri James :


On 15/10/18 12:28, Marko Rauhamaa wrote:

Try running

  emacs -q abc.c

and observe the indentation depth.


"""User Option: c-basic-offset

 This style variable holds the basic offset between indentation
levels. It's factory default is 4, but all the built-in styles set it
themselves, to some value between 2 (for gnu style) and 8 (for bsd,
linux, and python styles)."""


To realize why 2 is the factory default despite the above statement, we
observe that the factory setting of c-default-style specifies "gnu" for
C-like files other than Java and awk.


In other words, only for some  C styles.  As I said.

--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

RE: Single DB connection during class's lifetime. Metaclass,singletonand __new__() examples and references.

[Ryan Johnson]

Thank you. That clears everything up.

Sent from Mail for Windows 10

From: Cameron Simpson
Sent: Saturday, October 13, 2018 6:06 PM
To: Ryan Johnson
Cc: python-list@python.org
Subject: Re: Single DB connection during class's lifetime. 
Metaclass,singletonand __new__() examples and references.

On 12Oct2018 13:28, Ryan Johnson  wrote:
>Thanks for the clarification.
>
>If I am creating a class variable, are you suggesting I perform the “if it 
>exists, great, otherwise make it” logic in the __init__ block or in the class 
>definition block? Will that even run in a class definition?

The class definition code runs when the class is defined (as Python 
reads it in your code).

The __init__ block runs once each time a new instance of the class is 
initialised.

When do you _want_ this logic to run? That dictates where the logic 
goes.

If you run this in the class definition code it pretty much will 
unconditionally make a db connection. But in reality (a) you usually 
want to defer making the connection until you need it to reduce resource 
usage and (b) you often don't know enough to make the connection at 
class definition time i.e. you don't know the database host, the 
credentials, etc - they are often supplied to the initialiser (directly 
or via some config file).

>I never see examples do anything besides assignment operations and flow 
>control, although it would follow that if the block allows creation of 
>strings (an object), it would allow creation of connection objects. On 
>the other hand, the __init__ block seems like a natural place to put 
>the cursor instantiation.

You can do anything that is sensible in the __init__ block - it is just 
code. Your goal is to decide what is sensible.

Normally the initialiser mosts sets up various arrtributes to sane 
initial values. It can do complex things, but is usually relatively 
basic.

I'd note, as Thomas did, that the cursor is a control object associated 
with a query. You can have multiple cursors on a connection, and you 
often make one from a query, process the query results, then discard the 
cursor. SO you routinely use several during the lifetime of a 
connection.

Therefore you don't make cursors when you set up the connection; you 
make them in association with queries.

>From this answer ( 
>https://stackoverflow.com/questions/25577578/python-access-class-variable-from-instance/25577642#25577642
> ) the pythonic way to access the class variable is using type(self), but 
>doesn’t this access the local class’s variable, if someone subclasses my 
>class? Can I specify my class variable via the class name itself? Example: 
>instancevar = ClassName.classvar .
>I am hoping that because of the way python points labels to objects, this 
>should give my instance an instance var that refers to the class var. Am I 
>right?

_Usually_ I access class attributes (which you're calling variables, I 
believe - they're not) via the instance:

  def foo(self, blah=None):
if blah is None:
  blah = foo.DEFAULT_BLAH_VALUE
... work with blah ...

As you suggest, this will find DEFAULT_BLAH_VALUE from the subclass 
before it finds it from the superclass. Usually that is what I want - 
the purpose of subclassing is to (possibly) override the aspects of the 
superclass.

However, you _can_ always reach directly to a specific class to get a 
value:

  blah = MySuperCLass.DEFAULT_BLAH_VALUE

if that is sensible. All you're doing is changing the way in which the 
name "DEFAULT_BLAH_VALUE" is found: do I use the instance's name lookup 
or go somewhere direct?

In your case with a persistent database connection associated with a 
class it would be best to make the "get a connection" logic a class 
method because the "is there a connection" attribute is associated with 
the class, not the instance.

Methods are, by default, "instance" methods: they are defined like this:

  def method(self, ...):

and you largely work through "self", being the current instance. That is 
its "context".

Class method are defined like this:

  @classmethod
  def method(cls, ...)

and instead of having an instance as context (with the conventional name 
'self"), you have the class (with the conventional name "cls"). These 
are for methods which _do_ _not_ care about the instance, just the 
class. So in the case of your database connection, made on demand once 
per class, you might go:

  @classmethod
  def conn(cls):
c = cls.connection
if c is None:
  c = connect_to_the_db(.)
  cls.connection = c
return c

See that there's no "self" in here?

Then your instance methods can look like this:

  def lookup(self, ):
conn = self.conn()
cursor = conn.select()
... use the cursor to process the result ...

The instance finds the "conn" class method in the usual way: look in the 
instance, then look in the class hierarchy.

This in itself is an argument against making the connection in the 
__init__ block.

Does this help?

Cheers,
Cameron Simpso

Re: Python indentation (3 spaces)

[Peter J. Holzer]

On 2018-10-15 09:49:12 +1100, Cameron Simpson wrote:
> On 15Oct2018 00:33, Peter J. Holzer  wrote:
> > On 2018-10-15 09:06:11 +1100, Chris Angelico wrote:
> > > On Mon, Oct 15, 2018 at 8:56 AM Marko Rauhamaa  wrote:
> > > > Chris Angelico :
> > > > > Tabs for indentation have semantic meaning. Top-level has zero tabs.
> > > > > One indentation level is represented by one tab. Two indentation
> > > > > levels? Two tabs. It's about as perfect a representation as you could
> > > > > hope for.
> > > >
> > > > That *could* be the situation. However, it is trumped by an older
> > > > convention whereby the indentation levels go as follows:
> > > >
> > > >0:
> > > >1: SPC SPC
> > > >2: SPC SPC SPC SPC
> > > >3: SPC SPC SPC SPC SPC SPC
> > > >4: TAB
> > > >5: TAB SPC SPC
> > > >6: TAB SPC SPC SPC SPC
> > > >7: TAB SPC SPC SPC SPC SPC SPC
> > > >8: TAB TAB
> > 
> > That's not using tabs for indentation, that's using tabs for compressing
> > spaces (somebody already mentioned that in this thread).
> > 
> > 
> > > I've literally NEVER come across this as a convention. Not a single
> > > file that I have ever worked with has used it. Where is this
> > > convention from?
> > 
> > It's something vi does by default, and apparently emacs as well.
> > In the 1970's saving space by replacing sequences of 8 spaces
> > with tabs seemed lika a good idea.
> > 
> > There are workarounds in vi(m), but I'm not sure if you can get rid of
> > that behaviour completely. I'm sure it is possible in emacs.
> 
> I'm a "just use spaces" guy.

So am I. As I wrote somewhere near the start of this thread, I find the
"1 tab per indentation level" theoretically appealing but unworkable in
practice. 


> I use the tab _key_ as a shortcut to do a bunch
> of spaces.
> 
> My vim setup has this:
> 
>  set expandtab
> 
> which turns them into spaces.

Yep, mine, too. But that doesn't help you if you want to use tabs
(the character ASCII 0x09, not the tab key) to signify an indentation
level in your files.

So it's sort of off-topic in this sub-thread.

My point was that with expandtab off, vim sometimes "optimizes"
sequences of spaces into tabs in places where it shouldn't, and that
it's hard (maybe impossible) to get vim leave tabs where they belong and
not introduce them where they don't belong. So it isn't the ideal editor
for the "1 tab per indentation level" style.

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Peter J. Holzer]

On 2018-10-15 12:06:06 +0100, Rhodri James wrote:
> On 14/10/18 09:06, Peter J. Holzer wrote:
> > If everybody used tabs, why would anyone care about your tab width?
> > Unless they look over your shoulder while you are typing, they won't
> > even know how wide your tabs are.
> 
> If you used two-space tab stops and I used (the normal :-) eight,
> comfortable indentations for you would rapidly become uncomfortably large
> indentations for me.  I'd care about that.

Buy a bigger monitor :-).

On a more serious note, I think the number of indentation levels should
be fairly low regardless of their width. If someone uses 2-space
indentation so they can fit 20 nested ifs, the other programmers should
complain during code review: Not because the lines are too long but
because the nesting is too deep.

> I'd also care if you used spaces to make sub-tab stop adjustments to
> line up appropriate text (inside brackets, for example).

I'm not sure what a "sub-tab stop adjustment" is. I think that for the
"1 tab per indentation level" style, tabs would be used only for
indentation, nothing else, and preferably only for block-level
indentation.

So a python function might look like this with 8 spaces per tab:

def·foo(a_parameter,·another_parameter,
an_optional_parameter=DEFAULT_VALUE,
another_optional_parameter=None
):
   →x·=·a_parameter
   →a·=·[
   →{·'x':·1,···'y':·2,'z':·3·},
   →{·'x':·0.4,·'y':·3.14,·'z':·1.618·},
   →]
   →if·another_optional_parameter:
   →   →r·=·(
   →   
→(another_optional_parameter·*·a[0]['x']·+·another_parameter·*·b[0]['y'])
   →   →/
   →   →(a[1]['x']·+·a[1]['y']·+·a[1]['z'])
   →   →)
   →else:
   →   →r·=·another_parameter·*·0
   →return·r

(I'using → to represent a tab here and · to represent a space.)

And like this with 2 spaces per tab:

def·foo(a_parameter,·another_parameter,
an_optional_parameter=DEFAULT_VALUE,
another_optional_parameter=None
):
 →x·=·a_parameter
 →a·=·[
 →{·'x':·1,···'y':·2,'z':·3·},
 →{·'x':·0.4,·'y':·3.14,·'z':·1.618·},
 →]
 →if·another_optional_parameter:
 → →r·=·(
 → →(another_optional_parameter·*·a[0]['x']·+·another_parameter·*·b[0]['y'])
 → →/
 → →(a[1]['x']·+·a[1]['y']·+·a[1]['z'])
 → →)
 →else:
 → →r·=·another_parameter·*·0
 →return·r

Note that everything still lines up nicely.

hp (a four spaces per indentation level guy)

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Peter J. Holzer]

On 2018-10-15 14:12:54 +0200, Antoon Pardon wrote:
> On 13-10-18 09:37, Peter J. Holzer wrote:
> > On 2018-10-09 09:55:34 +0200, Antoon Pardon wrote:
> >> On 08-10-18 19:43, Peter J. Holzer wrote:
> >>> In practice it doesn't work in my experience. There is always someone in
> >>> a team who was "just testing that new editor" and replaced all tabs
> >>> with spaces (or vice versa) or - worse - just some of them.
> >> Isn't that caugth in the process of commiting to version control?
> > Tabs are easy to catch. If a file contains a tab, reject it.
> >
> > Spaces aren't, because spaces are everywhere.
> 
> Spaces that replaced a tab by accident, are easy to catch too. They are all
> those lines that show up when you do a diff with the previous version that
> shouldn't show up.

And where is the AI that decides which lines in a diff are should show
up?

Whether a line in a diff should or should not show up seems to me to be
even harder to determine than whether a tab fits the syntax. 

hp

-- 
   _  | Peter J. Holzer| we build much bigger, better disasters now
|_|_) || because we have much more sophisticated
| |   | h...@hjp.at | management tools.
__/   | http://www.hjp.at/ | -- Ross Anderson 


signature.asc
Description: PGP signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Chris Angelico]

On Tue, Oct 16, 2018 at 6:34 AM Peter J. Holzer  wrote:
>
> On 2018-10-15 14:12:54 +0200, Antoon Pardon wrote:
> > On 13-10-18 09:37, Peter J. Holzer wrote:
> > > On 2018-10-09 09:55:34 +0200, Antoon Pardon wrote:
> > >> On 08-10-18 19:43, Peter J. Holzer wrote:
> > >>> In practice it doesn't work in my experience. There is always someone in
> > >>> a team who was "just testing that new editor" and replaced all tabs
> > >>> with spaces (or vice versa) or - worse - just some of them.
> > >> Isn't that caugth in the process of commiting to version control?
> > > Tabs are easy to catch. If a file contains a tab, reject it.
> > >
> > > Spaces aren't, because spaces are everywhere.
> >
> > Spaces that replaced a tab by accident, are easy to catch too. They are all
> > those lines that show up when you do a diff with the previous version that
> > shouldn't show up.
>
> And where is the AI that decides which lines in a diff are should show
> up?
>
> Whether a line in a diff should or should not show up seems to me to be
> even harder to determine than whether a tab fits the syntax.
>

If there's a change, it shows up. If there's no change, it doesn't show up.

Ergo, if you accidentally replace a tab with spaces, it's a change,
and it shows up.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Cameron Simpson]

[ Marko and Rhdri discussing emacs indentation modes ... ]


I can't express how pleasing it is to see the traditional vi-vs-emacs 
wars supplanted by emacs-vs-emacs :-)


Cheers,
Cameron Simpson 
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Chris Angelico]

On Tue, Oct 16, 2018 at 8:03 AM Cameron Simpson  wrote:
>
> > [ Marko and Rhdri discussing emacs indentation modes ... ]
>
> I can't express how pleasing it is to see the traditional vi-vs-emacs
> wars supplanted by emacs-vs-emacs :-)
>

Which editor should I use - vi in emacs mode, or emacs in vi mode?

*diving for cover*

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

pipenv and wheel/pre-compiled packages

[Florian Pelgrim]

Hi there,

I'm currently using a buildhost for third party packages running

```
$ pip3 wheel --wheel-dir=/root/wheelhouse -r /requirements.txt
```

After successful build I copy the directory `/root/wheelhouse` onto a 
new machine and install the compiled packages by running


```
$ pip3 install -r /requirements.txt --no-index --find-links=/root/wheelhouse
```

Is there something similar in `pipenv`?
Everything I found in combination with `wheel` are bug reports on GitHub.

Note that copying the venv directory is not an option for me. I'm using 
a docker container and want to install the packages system wide.


Cheers
Flo
--
https://mail.python.org/mailman/listinfo/python-list

Email parsing and unicode/utf8

[Thomas Jollans]

Hi,

I just stumbled over some curious behaviour of the stdlib email parsing
APIs which accept strings rather than bytes. It appears that you can't
parse an 8-bit UTF-8 message you have as a str without first encoding it.

The docs
 do
mention some problems (which I saw after the fact):

> class email.parser.FeedParser(_factory=None, *, policy=policy.compat32)
> 
> Works like BytesFeedParser except that the input to the feed() method 
> must be a string. This is of limited utility, since the only way for such a 
> message to be valid is for it to contain only ASCII text or, if utf8 is True, 
> no binary attachments.
> 
> Changed in version 3.3: Added the policy keyword.

Okay, cool - let's try parsing a message with text only (no attachments,
no BINARYMIME), with a UTF-8 Content-Type, and a policy with utf8=True.

Python 3.7.1rc2 (default, Oct 14 2018, 15:27:05)
[GCC 8.2.1 20180831 [gcc-8-branch revision 264010]] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import email.parser, email.policy
>>> pol = email.policy.SMTPUTF8
>>> pol.utf8
True
>>> pol.cte_type
'8bit'
>>> msg = '''MIME-Version: 1.0
... Content-Type: text/plain; charset="utf-8"
... Content-Transfer-Encoding: 8bit
... Subject: ¿Will it parse? Нет.
...
... ¡This message contains two (٢) non-ASCII characters!
... '''
>>> fp = email.parser.FeedParser(policy=pol)
>>> fp.feed(msg)
>>> msg_obj = fp.close()
>>> msg_obj

>>> print(msg_obj.get_content())
�This message contains two (\u0662) non-ASCII characters!

>>> print(msg_obj['Subject'])
¿Will it parse? Нет.

I don't know WHAT it's doing with the body there... It doesn't look like
utf8 mode actually did anything. Interesting that the subject header
survived! Maybe this is what the utf8=True does?

>>> email.policy.default.utf8
False
>>> fp2 = email.parser.FeedParser(policy=email.policy.default)
>>> fp2.feed(msg)
>>> msg_obj2 = fp2.close()
>>> print(msg_obj2['Subject'])
¿Will it parse? Нет.

Nope. Apparently, contrary to what my reading of the docs suggests, the
utf8 flag does nothing at all when parsing.

Just to check that this was in fact a perfectly valid email:

>>> bfp = email.parser.BytesFeedParser(policy=pol)
>>> bfp.feed(msg.encode('utf-8'))
>>> msg_objb = bfp.close()
>>> print(msg_objb.get_content())
¡This message contains two (٢) non-ASCII characters!

>>> print(msg_objb['Subject'])
¿Will it parse? Нет.

BytesFeedParser is happy.

Question: Is this a bug? Am I missing something? Does the clause in the
docs about utf8 mean anything?

Cheers
Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Gregory Ewing]

Cameron Simpson wrote:
I can't express how pleasing it is to see the traditional vi-vs-emacs 
wars supplanted by emacs-vs-emacs :-)


We're the People's Front of Emacs, not the Emacs People's Front!

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[D'Arcy Cain]

On 10/15/18 5:54 PM, Gregory Ewing wrote:
> Cameron Simpson wrote:
>> I can't express how pleasing it is to see the traditional vi-vs-emacs
>> wars supplanted by emacs-vs-emacs :-)
> 
> We're the People's Front of Emacs, not the Emacs People's Front!

I thought we were the People's Emacs Front.

-- 
D'Arcy J.M. Cain
System Administrator, Vex.Net
http://www.Vex.Net/ IM:da...@vex.net
VoIP: sip:da...@vex.net
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Chris Angelico]

On Tue, Oct 16, 2018 at 9:16 AM D'Arcy Cain  wrote:
>
> On 10/15/18 5:54 PM, Gregory Ewing wrote:
> > Cameron Simpson wrote:
> >> I can't express how pleasing it is to see the traditional vi-vs-emacs
> >> wars supplanted by emacs-vs-emacs :-)
> >
> > We're the People's Front of Emacs, not the Emacs People's Front!
>
> I thought we were the People's Emacs Front.
>

No, that's just a front for a tab-laundering enterprise...

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python indentation (3 spaces)

[Ethan Furman]

On 10/15/2018 12:37 PM, Chris Angelico wrote:

On Tue, Oct 16, 2018 at 6:34 AM Peter J. Holzer  wrote:

On 2018-10-15 14:12:54 +0200, Antoon Pardon wrote:



Spaces that replaced a tab by accident, are easy to catch too. They are all
those lines that show up when you do a diff with the previous version that
shouldn't show up.


And where is the AI that decides which lines in a diff are should show
up?

Whether a line in a diff should or should not show up seems to me to be
even harder to determine than whether a tab fits the syntax.



If there's a change, it shows up. If there's no change, it doesn't show up.

Ergo, if you accidentally replace a tab with spaces, it's a change,
and it shows up.


Unless you have your diff tool set to ignore whitespace-only changes.  :(

--
~Ethan~
--
https://mail.python.org/mailman/listinfo/python-list

Is it dangeous when using custom metaclass?

[jfong]

class StructureMeta(type):
def __init__(self, clsname, bases, clsdict):
offset = 0
...
...
setattr(self, 'struct_size', offset)

class Structure(metaclass=StructureMeta):
...
...

class PolyHeader(Structure):
...
...

As my understanding, the metaclass's __init__ was called when a class was 
created. In the above example, both the Structure and PolyHeader called it. My 
question is: because the PolyHeader inherited Structure, is it reasonable for 
PolyHeader to call this __init__ again? Will it cause any possible trouble?

--Jach
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ESR "Waning of Python" post

[dieter]

Marko Rauhamaa  writes:
> dieter :
>> Marko Rauhamaa  writes:
>>> Keeping the number of long-term objects low is key.
>>
>> Right, if I need near realtime behaviour and must live
>> with [C]Python's garbage collector.
>
> Or any other GC ever invented.

There are "realtime garbage collection" algorithms. For them,
the creation of new objects must cooperate with the garbage
collection - likely with the need to acquire a lock for a short
period. But there is no (principle) need to block all
"normal" activity during the complete garbage collection (as [C]Python's
garbage collector has done this at the time of my problem).

>> But, a web application does usually not need near realtime behaviour.
>> An occasional (maybe once in a few days) garbage collection and
>> associated reduced response time is acceptable.
>> A problem only arises if a badly designed component produces
>> quite frequently hundreds of thousands of temporary objects
>> likely triggering (frequent) garbage collections.
>
> But I think you are barking up the wrong tree. You could rightly blame
> GC itself as an unworkable paradigm and switch to, say, C++ or Rust.

I am happy that [C]Python uses mainly reference counting for
its memory management and that GC is used quite sparingly.

> Or you could blame the parts of the software that create too many
> long-term objects.

I do not do that because I understand why in my application
there are many long living objects.

> You shouldn't blame the parts of the software that churn out zillions of
> short-term objects.

I do precisely that: the blamed component produced a very large
number of short living objects -- without need and while it should
have been aware that it operates on mass data - among others from the
fact that its complete environment took special care to work with
this mass data efficiently.

I solved my problem by replacing this single component by
one knowing what it does. No need to rewrite the complete
application, get rid of Python object caches or even
switch to a different language.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Email parsing and unicode/utf8

[dieter]

Thomas Jollans  writes:
> I just stumbled over some curious behaviour of the stdlib email parsing
> APIs which accept strings rather than bytes. It appears that you can't
> parse an 8-bit UTF-8 message you have as a str without first encoding it.

The primary purpose of an email parser is likely the parsing
of RFC 822/2045 messages which are a sequence of bytes,
encoded as dictated by RFC 822.
Therefore, I would expect some peculiarities when you feed such
a parser with general text.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: ESR "Waning of Python" post

[Chris Angelico]

On Tue, Oct 16, 2018 at 4:18 PM dieter  wrote:
>
> Marko Rauhamaa  writes:
> > Or you could blame the parts of the software that create too many
> > long-term objects.
>
> I do not do that because I understand why in my application
> there are many long living objects.
>
> > You shouldn't blame the parts of the software that churn out zillions of
> > short-term objects.
>
> I do precisely that: the blamed component produced a very large
> number of short living objects -- without need and while it should
> have been aware that it operates on mass data - among others from the
> fact that its complete environment took special care to work with
> this mass data efficiently.

Exactly. Long-term objects are NOT a problem. Tell me, how many
objects get created as a web app boots up and then are never destroyed
for the lifetime of that process? To find out, I added this line just
before a very VERY small Flask app of mine goes into its main loop:

import gc; print(len(gc.get_objects()), "objects currently tracked")

There are over 40,000 of them. Now, I can't say for sure that every
one of those objects will stick around till the process shuts down,
but I'd say a lot of them will. They're modules, functions, types,
class dictionaries... oh, and the GC doesn't track strings or numbers,
so that's another whole huge slab of objects that you'd have to count.
If you replace the refcounting GC with a pure mark-and-sweep, you have
to check every single one of them every time you do a GC pass.

(For reference, running the same GC check in an empty Python
interpreter gives around five thousand tracked objects, still mostly
functions.)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Is it dangeous when using custom metaclass?

[Gregory Ewing]

jf...@ms4.hinet.net wrote:

class Structure(metaclass=StructureMeta): ...

class PolyHeader(Structure): ...

As my understanding, the metaclass's __init__ was called when a class was
created. In the above example, both the Structure and PolyHeader called it.
My question is: because the PolyHeader inherited Structure, is it reasonable
for PolyHeader to call this __init__ again? Will it cause any possible
trouble?


It's reasonable for both to call it, because they're distinct
instances of StructureMeta, each of which need to be initialised.

Whether it will cause a problem depends on what StructureMeta's
__init__ is supposed to do. Presumably you want a given structure
class to start allocating its offsets where its base class left
off, in which case you may need to do something like this:

class StructureMeta(type):
def __init__(self, clsname, bases, clsdict):
if bases:
   offset = bases[0].offset # assuming there isn't more than one base
else:
   offset = 0
...

(BTW, why do you use setattr() to set the offset attribute
instead of just doing self.offset = offset?)

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list