Re: file.read Method Documentation (Python 2.7.10)
Chris - In the Python 2.7.10 documentation, I am referring to section 5. Built-in Types, subsection 5.9 File Objects. In that subsection, I have the following paragraph: file.read([*size*]) Read at most *size* bytes from the file (less if the read hits EOF before obtaining *size* bytes). If the *size* argument is negative or omitted, read all data until EOF is reached. The bytes are returned as a string object. An empty string is returned when EOF is encountered immediately. (For certain files, like ttys, it makes sense to continue reading after an EOF is hit.) Note that this method may call the underlying C function fread() more than once in an effort to acquire as close to *size* bytes as possible. Also note that when in non-blocking mode, less data than was requested may be returned, even if no *size* parameter was given. Note This function is simply a wrapper for the underlying fread() C function, and will behave the same in corner cases, such as whether the EOF value is cached. Stephen. On Mon, Jan 9, 2023 at 6:25 PM Chris Angelico wrote: > On Tue, 10 Jan 2023 at 01:36, Stephen Tucker > wrote: > > > > Dear Python-list, > > > > Yes, I know that Python 2.x is no longer supported. > > > > I have found that the documentation for this method is misleading when > the > > file being read is UTF-8-encoded: > > > >Instead of reading *size* bytes, the method reads *size *UTF-8 byte > > *sequences*. > > > > Has this error been corrected in the Python 3.x documentation? > > > > What documentation is this? The builtin 'file' type doesn't know > anything about encodings, and only ever returns bytes. > > ChrisA > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: file.read Method Documentation (Python 2.7.10)
On Wed, 11 Jan 2023 at 21:31, Stephen Tucker wrote: > > Chris - > > In the Python 2.7.10 documentation, I am referring to section 5. Built-in > Types, subsection 5.9 File Objects. > > In that subsection, I have the following paragraph: > > file.read([size]) > > Read at most size bytes from the file (less if the read hits EOF before > obtaining size bytes). If the size argument is negative or omitted, read all > data until EOF is reached. The bytes are returned as a string object. An > empty string is returned when EOF is encountered immediately. (For certain > files, like ttys, it makes sense to continue reading after an EOF is hit.) > Note that this method may call the underlying C function fread() more than > once in an effort to acquire as close to size bytes as possible. Also note > that when in non-blocking mode, less data than was requested may be returned, > even if no size parameter was given. > Yes, so it should be that number of bytes, which is what it does, isn't it? ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
Yes, I did understand that. In your example, "a" and "b" are the same pointer, so an operation on one is an operation on the other (because they’re the same memory block). My issue in Python came up because Python can dynamically change one or the other to a different object (memory block) so I have to be aware of that when handing this kind of situation. Jan 10, 2023, 17:31 by greg.ew...@canterbury.ac.nz: > On 11/01/23 11:21 am, Jen Kris wrote: > >> where one object derives from another object (a = b[0], for example), any >> operation that would alter one will alter the other. >> > > I think you're still confused. In C terms, after a = b[0], a and b[0] > are pointers to the same block of memory. If you change that block of > memory, then of course you will see the change through either pointer. > > Here's a rough C translation of some of your Python code: > > /* mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] */ > int **mx1 = (int **)malloc(3 * sizeof(int *)); > mx1[0] = (int *)malloc(3 * sizeof(int)); > mx1[0][0] = 1; > mx1[0][1] = 2; > mx1[0][2] = 3; > mx1[1] = (int *)malloc(3 * sizeof(int)); > mx1[1][0] = 4; > mx1[1][1] = 5; > mx1[1][2] = 6; > mx1[2] = (int *)malloc(3 * sizeof(int)); > mx1[2][0] = 7; > mx1[2][1] = 8; > mx1[2][2] = 9; > > /* arr1 = mx1[2] */ > int *arr1 = mx[2]; > > /* arr1 = [ 10, 11, 12 ] */ > arr1 = (int *)malloc(3 * sizeof(int)); > arr1[0] = 10; > arr1[1] = 11; > arr1[2] = 12; > > Does that help your understanding? > > -- > Greg > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
Op 11/01/2023 om 16:33 schreef Jen Kris via Python-list: Yes, I did understand that. In your example, "a" and "b" are the same pointer, so an operation on one is an operation on the other (because they’re the same memory block). Sorry if you feel I'm being overly pedantic, but your explanation "an operation on one is an operation on the other (because they’re the same memory block)" still feels a bit misguided. "One" and "other" still make it sound like there are two objects, and "an operation on one" and "an operation on the other" make it sound like there are two operations. Sometimes it doesn't matter if we're a bit sloppy for sake of simplicity or convenience, sometimes we really need to be precise. I think this is a case where we need to be precise. So, to be precise: there is only one object, with possible multiple names to it. We can change the object, using one of the names. That is one and only one operation on one and only one object. Since the different names refer to the same object, that change will of course be visible through all of them. Note that 'name' in that sentence doesn't just refer to variables (mx1, arr1, ...) but also things like indexed lists (mx1[0], mx1[[0][0], ...), loop variables, function arguments. The correct mental model is important here, and I do think you're on track or very close to it, but the way you phrase things does give me that nagging feeling that you still might be just a bit off. -- "Peace cannot be kept by force. It can only be achieved through understanding." -- Albert Einstein -- https://mail.python.org/mailman/listinfo/python-list
Re: file.read Method Documentation (Python 2.7.10)
Chris,
Thanks for your reply.
I hope the evidence below (taken from IDLE) clarifies my issue:
Stephen.
==
1. Create BOM.txt
-
>>> myfil = open ("BOM.txt", "wb")
>>> myfil.write ("\xef" + "\xbb" + "\xbf")
>>> myfil.close()
2. Input three bytes at once from BOM.txt and print them
>>> myfil = open ("BOM.txt", "rb")
>>> myBOM = myfil.read (3)
>>> myBOM
'\xef\xbb\xbf'
>>> myfil.close()
3. Input three bytes one at a time from BOM.txt and print them
--
>>> myfil = open ("BOM.txt", "rb")
>>> myBOM_1 = myfil.read (1)
>>> myBOM_2 = myfil.read (1)
>>> myBOM_3 = myfil.read (1)
>>> myBOM_1
'\xef'
>>> myBOM_2
'\xbb'
>>> myBOM_3
'\xbf'
>>> myfil.close()
4. Input three bytes at once from BOM.txt and print them
>>> import codecs
>>> myfil = codecs.open ("BOM.txt", mode="rb", encoding="UTF-8")
>>> myBOM = unicode (myfil.read (3))
>>> myBOM
u'\ufeff'
>>> myfil.close ()
5. Attempt to input three bytes one at a time from BOM.txt and print them
-
>>> myfil = codecs.open ("BOM.txt", mode="rb", encoding="UTF-8")
>>> myBOM_4 = myfil.read (1)
>>> myBOM_5 = myfil.read (1)
>>> myBOM_6 = myfil.read (1)
>>> myBOM_4
u'\ufeff'
>>> myBOM_5
u''
>>> myBOM_6
u''
>>> myfil.close()
Notes
A. The attempt at Part 5 actually inputs all three bytes when we ask it to
input just the first one!
B. The outcome from Part 5 shows that, actually, the request to input text
in Part 4 brought about a response from the program something like this:
Input the UTF-8-encoded character as the first "byte";
As expected, after reaching the end of the file, continue supplying an
empty string for each of the requested extra bytes.
==
On Wed, Jan 11, 2023 at 11:00 AM Chris Angelico wrote:
> On Wed, 11 Jan 2023 at 21:31, Stephen Tucker
> wrote:
> >
> > Chris -
> >
> > In the Python 2.7.10 documentation, I am referring to section 5.
> Built-in Types, subsection 5.9 File Objects.
> >
> > In that subsection, I have the following paragraph:
> >
> > file.read([size])
> >
> > Read at most size bytes from the file (less if the read hits EOF before
> obtaining size bytes). If the size argument is negative or omitted, read
> all data until EOF is reached. The bytes are returned as a string object.
> An empty string is returned when EOF is encountered immediately. (For
> certain files, like ttys, it makes sense to continue reading after an EOF
> is hit.) Note that this method may call the underlying C function fread()
> more than once in an effort to acquire as close to size bytes as possible.
> Also note that when in non-blocking mode, less data than was requested may
> be returned, even if no size parameter was given.
> >
>
> Yes, so it should be that number of bytes, which is what it does, isn't it?
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list
>
--
https://mail.python.org/mailman/listinfo/python-list
Re: file.read Method Documentation (Python 2.7.10)
On Thu, 12 Jan 2023 at 04:31, Stephen Tucker wrote:
> 1. Create BOM.txt
> 2. Input three bytes at once from BOM.txt and print them
> 3. Input three bytes one at a time from BOM.txt and print them
All of these correctly show that a file, in binary mode, reads and writes bytes.
> 4. Input three bytes at once from BOM.txt and print them
> >>> import codecs
> >>> myfil = codecs.open ("BOM.txt", mode="rb", encoding="UTF-8")
This is now a codecs file, NOT a vanilla file object. See its docs here:
https://docs.python.org/2.7/library/codecs.html#codecs.open
The output is "codec-dependent" but I would assume that UTF-8 will
yield Unicode text strings.
> 5. Attempt to input three bytes one at a time from BOM.txt and print them
> -
>
> >>> myfil = codecs.open ("BOM.txt", mode="rb", encoding="UTF-8")
> >>> myBOM_4 = myfil.read (1)
> >>> myBOM_4
> u'\ufeff'
> A. The attempt at Part 5 actually inputs all three bytes when we ask it to
> input just the first one!
On the contrary; you asked it for one *character* and it read one character.
Where were you seeing documentation that disagreed with this?
ChrisA
--
https://mail.python.org/mailman/listinfo/python-list
Re: file.read Method Documentation (Python 2.7.10)
Chris Angelico schreef op 11/01/2023 om 18:36:
On Thu, 12 Jan 2023 at 04:31, Stephen Tucker wrote:
> 1. Create BOM.txt
> 2. Input three bytes at once from BOM.txt and print them
> 3. Input three bytes one at a time from BOM.txt and print them
All of these correctly show that a file, in binary mode, reads and writes bytes.
> 4. Input three bytes at once from BOM.txt and print them
> >>> import codecs
> >>> myfil = codecs.open ("BOM.txt", mode="rb", encoding="UTF-8")
This is now a codecs file, NOT a vanilla file object. See its docs here:
https://docs.python.org/2.7/library/codecs.html#codecs.open
The output is "codec-dependent" but I would assume that UTF-8 will
yield Unicode text strings.
> 5. Attempt to input three bytes one at a time from BOM.txt and print them
> -
>
> >>> myfil = codecs.open ("BOM.txt", mode="rb", encoding="UTF-8")
> >>> myBOM_4 = myfil.read (1)
> >>> myBOM_4
> u'\ufeff'
> A. The attempt at Part 5 actually inputs all three bytes when we ask it to
input just the first one!
On the contrary; you asked it for one *character* and it read one character.
Not exactly. You're right of course that things opened with
codecs.open() behave differently from vanilla file objects.
codecs.open() returns a StreamReaderWriter instance, which combines
StreamReader and StreamWriter. For read(), StreamReader is what matters
(documented at
https://docs.python.org/3.11/library/codecs.html#codecs.StreamReader).
It's read() method is:
read(size=- 1, chars=- 1, firstline=False)
_size_ indicates the approximate maximum number of encoded bytes or code
points to read for decoding. The decoder can modify this setting as
appropriate.
_chars_ indicates the number of decoded code points or bytes to return.
The read() method will never return more data than requested, but it
might return less, if there is not enough available.
When only one parameter is provided, without name, it's _size_. So
myfil.read(1) asks to read enough bytes to decode 1 code point
(approximately). That's totally consistent with the observer behavior.
--
"Peace cannot be kept by force. It can only be achieved through understanding."
-- Albert Einstein
--
https://mail.python.org/mailman/listinfo/python-list
Re: Mailing-Lists (pointer)
Cameron Simpson wrote: > On 10Jan2023 08:45, Chris Green wrote: > >dn wrote: > >> See also the wisdom of enabling comp.lang.python and python-list as > >> 'mirrors', enabling those who prefer one mechanism/client to another, > >> yet maintaining a single 'community'. > >> > >Yes, this is important I think. Plus, if possible, if it's decided to > >move to a forum format make that accessible by E-Mail. > > There's a Discourse forum over at discuss.python.org. I use it in > "mailing list mode" and do almost all my interactions via email, exactly > as I do for python-list. Posts come to me and land in the same local > mail folder I use for python-list. My replies land on the forum as > expected (and of course also go by email to those members who have > turned that mode on). > > So I'm using both the new forum and the currently mailing list still, > and broadly in exactly the same way. > Yes, Discourse is one of the few web forums that also provides full E-Mail access. -- Chris Green · -- https://mail.python.org/mailman/listinfo/python-list
Re: Mailing-Lists (pointer)
Cameron Simpson wrote at 2023-1-11 08:37 +1100: > ... >There's a Discourse forum over at discuss.python.org. I use it in >"mailing list mode" and do almost all my interactions via email, exactly >as I do for python-list. Posts come to me and land in the same local >mail folder I use for python-list. My replies land on the forum as >expected (and of course also go by email to those members who have >turned that mode on). I am also using the Plone `Discourse` forum in "mailing list mode". It now works quite well but it took some years before reaching this state. For a very long time, my mail replies did not reach the forum reliably. My latest complaint (more than half a year ago): when I had visited the forum via `http` (I did this occasionally to verify my reply has reached the forum), it sometimes thought, I had seen a new message and did not inform me about it via mail. Meanwhile, all replies seem to arrive reliably and I no longer use `http` for access. Therefore, I do not know whether the behavior described above still persists. -- https://mail.python.org/mailman/listinfo/python-list
Re: Python - working with xml/lxml/objectify/schemas, datatypes, and assignments
aapost wrote at 2023-1-10 22:15 -0500: >On 1/4/23 12:13, aapost wrote: >> On 1/4/23 09:42, Dieter Maurer wrote: >> ... >>> You might have a look at `PyXB`, too. >>> It tries hard to enforce schema restrictions in Python code. >> ... >Unfortunately picking it apart for a while and diving deeper in to a >rabbit hole, PyXB looks to be a no-go. > >PyXB while interesting, and I respect it's complexity and depth, is >lacking in design consistency in how it operates if you are trying to >modify and work with the resulting structure intuitively. > ... problem with simple types ... I use `PyXB` in `dm.saml2` and `dm.zope.saml2`, i.e. with the SAML2 schema definitions (which include those of XML signature and XML encryption). I had no problems with simple types. I just assign them to attributes of the Python objects representing the XML elements. `PyXB` does the right thing when it serializes those objects into XML. -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
Thanks for your comments. After all, I asked for clarity so it’s not pedantic to be precise, and you’re helping to clarify. Going back to my original post, mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] arr1 = mx1[2] Now if I write "arr1[1] += 5" then both arr1 and mx1[2][1] will be changed because while they are different names, they are the assigned same memory location (pointer). Similarly, if I write "mx1[2][1] += 5" then again both names will be updated. That’s what I meant by "an operation on one is an operation on the other." To be more precise, an operation on one name will be reflected in the other name. The difference is in the names, not the pointers. Each name has the same pointer in my example, but operations can be done in Python using either name. Jan 11, 2023, 09:13 by r...@roelschroeven.net: > Op 11/01/2023 om 16:33 schreef Jen Kris via Python-list: > >> Yes, I did understand that. In your example, "a" and "b" are the same >> pointer, so an operation on one is an operation on the other (because >> they’re the same memory block). >> > > Sorry if you feel I'm being overly pedantic, but your explanation "an > operation on one is an operation on the other (because they’re the same > memory block)" still feels a bit misguided. "One" and "other" still make it > sound like there are two objects, and "an operation on one" and "an operation > on the other" make it sound like there are two operations. > Sometimes it doesn't matter if we're a bit sloppy for sake of simplicity or > convenience, sometimes we really need to be precise. I think this is a case > where we need to be precise. > > So, to be precise: there is only one object, with possible multiple names to > it. We can change the object, using one of the names. That is one and only > one operation on one and only one object. Since the different names refer to > the same object, that change will of course be visible through all of them. > Note that 'name' in that sentence doesn't just refer to variables (mx1, arr1, > ...) but also things like indexed lists (mx1[0], mx1[[0][0], ...), loop > variables, function arguments. > > The correct mental model is important here, and I do think you're on track or > very close to it, but the way you phrase things does give me that nagging > feeling that you still might be just a bit off. > > -- > "Peace cannot be kept by force. It can only be achieved through > understanding." > -- Albert Einstein > > -- > https://mail.python.org/mailman/listinfo/python-list > -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
I think the key point is that "the operation" doesn't act on "names" but on "objects" (which are different sort of things), and thus there isn't an "the other" when talking about the object being operated on. Thinking of an operation being on a "name" is the mental model error. The only operations that operate on "names" are assignment operations. Augmented assignment operations are more complicatd, as they can either work on the object the name points on, if that object is mutable, or rebinds the name to a new object if it isn't. Thus a += b is NOT neccessarilily the same as a = a + b, as a += b might just mutate the object that a is bound to or might rebind in the manner of a = a + b; Thus: a = [ 1, 2, 3] b = a a += [4] will change the single list that a and b are bound to into [1, 2, 3, 4], while a = "foo" b = a a += "bar" will change a to be bound to the string object "foobar" but not b, since the string object "foo" wasn't mutable. Brings up the point, that you need to be careful with augmented assignment operators. On 1/11/23 1:28 PM, Jen Kris via Python-list wrote: Thanks for your comments. After all, I asked for clarity so it’s not pedantic to be precise, and you’re helping to clarify. Going back to my original post, mx1 = [ [ 1, 2, 3 ], [ 4, 5, 6 ], [ 7, 8, 9 ] ] arr1 = mx1[2] Now if I write "arr1[1] += 5" then both arr1 and mx1[2][1] will be changed because while they are different names, they are the assigned same memory location (pointer). Similarly, if I write "mx1[2][1] += 5" then again both names will be updated. That’s what I meant by "an operation on one is an operation on the other." To be more precise, an operation on one name will be reflected in the other name. The difference is in the names, not the pointers. Each name has the same pointer in my example, but operations can be done in Python using either name. Jan 11, 2023, 09:13 by r...@roelschroeven.net: Op 11/01/2023 om 16:33 schreef Jen Kris via Python-list: Yes, I did understand that. In your example, "a" and "b" are the same pointer, so an operation on one is an operation on the other (because they’re the same memory block). Sorry if you feel I'm being overly pedantic, but your explanation "an operation on one is an operation on the other (because they’re the same memory block)" still feels a bit misguided. "One" and "other" still make it sound like there are two objects, and "an operation on one" and "an operation on the other" make it sound like there are two operations. Sometimes it doesn't matter if we're a bit sloppy for sake of simplicity or convenience, sometimes we really need to be precise. I think this is a case where we need to be precise. So, to be precise: there is only one object, with possible multiple names to it. We can change the object, using one of the names. That is one and only one operation on one and only one object. Since the different names refer to the same object, that change will of course be visible through all of them. Note that 'name' in that sentence doesn't just refer to variables (mx1, arr1, ...) but also things like indexed lists (mx1[0], mx1[[0][0], ...), loop variables, function arguments. The correct mental model is important here, and I do think you're on track or very close to it, but the way you phrase things does give me that nagging feeling that you still might be just a bit off. -- "Peace cannot be kept by force. It can only be achieved through understanding." -- Albert Einstein -- https://mail.python.org/mailman/listinfo/python-list -- Richard Damon -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
On Tue, 10 Jan 2023 16:59:59 -0500, Thomas Passin declaimed the following: >Just to add a possibly picky detail to what others have said, Python >does not have an "array" type. It has a "list" type, as well as some >other, not necessarily mutable, sequence types. > However, it has long had https://docs.python.org/3/library/array.html -- Wulfraed Dennis Lee Bieber AF6VN wlfr...@ix.netcom.comhttp://wlfraed.microdiversity.freeddns.org/ -- https://mail.python.org/mailman/listinfo/python-list
Re: To clarify how Python handles two equal objects
On 2023-01-11 18:49:14 +, Stefan Ram wrote: > Jen Kris writes: > >Each name has the same pointer > > ... from the C programmer's point of view. > > From the Python programmer's point of view, there are no "pointers". That's just window dressing. Pointers are evil, so we can't have pointers. So we'll just call them by a different name. > Instead, names or other targets are /assigned/ or /bound/ to objects. This is the wrong way around. The name isn't assigned to the object. The object is assigned to the name. As you quoted: > |... the sequence is asked to assign the assigned object to > |its item with that index ... Also, while the word "bind" is used in this manner in the official docs > |... the name is bound to the object ... and some members of this list, I think it is really misleading. It sounds like the name is now an attribute of the object. But it isn't. There is no way to the name (or the names) from the object. But there is a way to get from the name the object. I can't think of a snappy, unambiguous single verb to describe what's happening here, but do we need one? We already have words like "assign", "refer", "point" with fairly standardized meaning in IT. We can write that "a reference to an object is assigned to a variable", and that after the assignment "the variable refers to the object". Or we can even use the P-word. And since we know that variables in Python can only contain references and never values, we can abbreviate "a reference to an object" as "an object" in most contexts. hp -- _ | Peter J. Holzer| Story must make more sense than reality. |_|_) || | | | h...@hjp.at |-- Charles Stross, "Creative writing __/ | http://www.hjp.at/ | challenge!" signature.asc Description: PGP signature -- https://mail.python.org/mailman/listinfo/python-list
Re: Mailing-Lists (pointer)
On 11Jan2023 19:10, Dieter Maurer wrote: Cameron Simpson wrote at 2023-1-11 08:37 +1100: ... There's a Discourse forum over at discuss.python.org. I use it in "mailing list mode" and do almost all my interactions via email, exactly as I do for python-list. [...] I am also using the Plone `Discourse` forum in "mailing list mode". It now works quite well but it took some years before reaching this state. Some of this kind of thing will be because the Doscourse devs, as you might imagine, are forum/web-first people while you and I are email-first people. So they won't notice email shortcomings as readily. That said, they do seem very engaged and willing to chase and fix bugs if they can be identified. For a very long time, my mail replies did not reach the forum reliably. My latest complaint (more than half a year ago): when I had visited the forum via `http` (I did this occasionally to verify my reply has reached the forum), it sometimes thought, I had seen a new message and did not inform me about it via mail. There's certainly still an issue where some messages are not reliably sent via email when the inbound needs-spam-review filter flags a message/post, particularly the first post; they're fixing that right now :-) Meanwhile, all replies seem to arrive reliably and I no longer use `http` for access. Therefore, I do not know whether the behavior described above still persists. One day everything will be perfect! Cheers, Cameron Simpson -- https://mail.python.org/mailman/listinfo/python-list
