On 3/10/24 11:48 am, Left Right wrote:
So, streaming parsers (eg. SAX) are written for a regular language
that approximates XML.
SAX doesn't parse a whole XML document, it parses small pieces of it
independently and passes them on. It's more like a lexical analyser than
a parser in that respect
This thread is derailing.
Please consider it closed.
--
~Ethan~
Moderator
--
https://mail.python.org/mailman/listinfo/python-list
> You can't validate an IP packet without having all of it. Your notion
> of "streaming" is nonsensical.
Whoa, whoa, hold your horses! "nonsensical" needs a little bit of
justification :)
It seems you don't understand the difference between words and
languages! In my examples, IP _protocol_ is th
> One single IP packet is all you can parse.
I worked for an undisclosed company which manufactures h/w for ISPs
(4- and 8-unit boxes you mount on a rack in a datacenter).
Essentially, big-big routers. So, I had the pleasure of writing
software that parses IP _protocol_, and let me tell you: you
On Thu, 3 Oct 2024 at 08:48, Left Right wrote:
>
> > You can't validate an IP packet without having all of it. Your notion
> > of "streaming" is nonsensical.
>
> Whoa, whoa, hold your horses! "nonsensical" needs a little bit of
> justification :)
>
> It seems you don't understand the difference be
On Wed, 2 Oct 2024 at 23:53, Left Right via Python-list
wrote:
> In the same email you replied to, I gave examples of languages for
> which parsers can be streaming (in general): SCSI or IP.
You can't validate an IP packet without having all of it. Your notion
of "streaming" is nonsensical.
Chri
parser that is only useful _sometimes_.
And, in practice, languages like XML or JSON do well with streaming,
even though in general it's impossible.
I'm sorry if this comes as a surprise. On one hand I don't want to
sound condescending, on the other hand, this is something that you&
On 2/10/24 12:26 pm, avi.e.gr...@gmail.com wrote:
The real problem is how the JSON is set up. If you take umpteen data
structures and wrap them all in something like a list, then it may be a tad
hard to stream as you may not necessarily be examining the contents till the
list finishes gigabytes
relevant; the
point is that it is possible to parse integers from most significant
digit to least significant digit under a streaming model (and if you're
sufficiently clever, you can even write partial results to external
storage and/or another transmission protocol, thus allowing for numbers
generators to give you as many primes as you want, and
no more.
So, if you can store arbitrary python code as part of your JSON, you can
send quite a bit of somewhat compressed data.
The real problem is how the JSON is set up. If you take umpteen data
structures and wrap them all in something like
On 2/10/24 10:03 am, Left Right wrote:
Consider also an interesting
consequence of SCSI not being able to have infinite words: this means,
besides other things that fsync() is nonsense! :) If you aren't
familiar with the concept: UNIX filesystem API suggests that it's
possible to destage arbitrar
On 1/10/24 8:34 am, Left Right wrote:
You probably forgot that it has to be _streaming_. Suppose you parse
the first digit: can you hand this information over to an external
function to process the parsed data? -- No! because you don't know the
magnitude yet.
By that definition of "streaming",
to sync _everything_ (and it hurts!)
On Tue, Oct 1, 2024 at 5:49 PM Dan Sommers via Python-list
wrote:
>
> On 2024-09-30 at 21:34:07 +0200,
> Regarding "Re: Help with Streaming and Chunk Processing for Large JSON Data
> (60 GB) from Kenna API,"
> Left Right via Python-lis
On 2024-09-30 at 21:34:07 +0200,
Regarding "Re: Help with Streaming and Chunk Processing for Large JSON Data (60
GB) from Kenna API,"
Left Right via Python-list wrote:
> > What am I missing? Handwavingly, start with the first digit, and as
> > long as the next character
On 2024-09-30 at 18:48:02 -0700,
Keith Thompson via Python-list wrote:
> 2qdxy4rzwzuui...@potatochowder.com writes:
> [...]
> > In Common Lisp, you can write integers as #nnR[digits], where nn is the
> > decimal representation of the base (possibly without a leading zero),
> > the # and the R are
> What am I missing? Handwavingly, start with the first digit, and as
> long as the next character is a digit, multipliy the accumulated result
> by 10 (or the appropriate base) and add the next value. Oh, and handle
> scientific notation as a special case, and perhaps fail spectacularly
> instea
2qdxy4rzwzuui...@potatochowder.com writes:
[...]
> In Common Lisp, you can write integers as #nnR[digits], where nn is the
> decimal representation of the base (possibly without a leading zero),
> the # and the R are literal characters, and the digits are written in
> the intended base. So the inp
On 2024-10-01 at 09:09:07 +1000,
Chris Angelico via Python-list wrote:
> On Tue, 1 Oct 2024 at 08:56, Grant Edwards via Python-list
> wrote:
> >
> > On 2024-09-30, Dan Sommers via Python-list wrote:
> >
> > > In Common Lisp, integers can be written in any integer base from two
> > > to thirty s
On Tue, 1 Oct 2024 at 08:56, Grant Edwards via Python-list
wrote:
>
> On 2024-09-30, Dan Sommers via Python-list wrote:
>
> > In Common Lisp, integers can be written in any integer base from two
> > to thirty six, inclusive. So knowing the last digit doesn't tell
> > you whether an integer is ev
On 2024-09-30, Dan Sommers via Python-list wrote:
> In Common Lisp, integers can be written in any integer base from two
> to thirty six, inclusive. So knowing the last digit doesn't tell
> you whether an integer is even or odd until you know the base
> anyway.
I had to think about that for an
ny integer base from two to
thirty six, inclusive. So knowing the last digit doesn't tell you
whether an integer is even or odd until you know the base anyway.
Curiously, we agree: if you move the goal posts arbitrarily, then
some algorithms that parse JSON numbers will fail.
--
https://mail.python.org/mailman/listinfo/python-list
streaming parser is required.
There is also the json-stream library, on PyPi at
https://pypi.org/project/json-stream/
--
https://mail.python.org/mailman/listinfo/python-list
> Streaming won't work because the file is gzipped. You have to receive
> the whole thing before you can unzip it. Once unzipped it will be even
> larger, and all in memory.
GZip is specifically designed to be streamed. So, that's not a
problem (in principle), but you would need to have a stream
On 9/30/2024 1:00 PM, Chris Angelico via Python-list wrote:
On Tue, 1 Oct 2024 at 02:20, Thomas Passin via Python-list
wrote:
On 9/30/2024 11:30 AM, Barry via Python-list wrote:
On 30 Sep 2024, at 06:52, Abdur-Rahmaan Janhangeer via Python-list
wrote:
import polars as pl
pl.read_json("
On Tue, 1 Oct 2024 at 04:30, Dan Sommers via Python-list
wrote:
>
> But why do I need to start with the least
> significant digit?
If you start from the most significant, you don't know anything about
the number until you finish parsing it. There's almost nothing you can
say about a number given
On 2024-09-30, Dan Sommers via Python-list wrote:
> On 2024-09-30 at 11:44:50 -0400,
> Grant Edwards via Python-list wrote:
>
>> On 2024-09-30, Left Right via Python-list wrote:
>> > [...]
>> > Imagine a pathological case of this shape: 1... <60GB of digits&g
On 2024-09-30 at 11:44:50 -0400,
Grant Edwards via Python-list wrote:
> On 2024-09-30, Left Right via Python-list wrote:
> > Whether and to what degree you can stream JSON depends on JSON
> > structure. In general, however, JSON cannot be streamed (but commonly
> > it can b
On Tue, 1 Oct 2024 at 02:20, Thomas Passin via Python-list
wrote:
>
> On 9/30/2024 11:30 AM, Barry via Python-list wrote:
> >
> >
> >> On 30 Sep 2024, at 06:52, Abdur-Rahmaan Janhangeer via Python-list
> >> wrote:
> >>
> >>
> >> import polars as pl
> >> pl.read_json("file.json")
> >>
> >>
> >
>
On 9/30/2024 11:30 AM, Barry via Python-list wrote:
On 30 Sep 2024, at 06:52, Abdur-Rahmaan Janhangeer via Python-list
wrote:
import polars as pl
pl.read_json("file.json")
This is not going to work unless the computer has a lot more the 60GiB of RAM.
As later suggested a streaming par
On 2024-09-30, Left Right via Python-list wrote:
> Whether and to what degree you can stream JSON depends on JSON
> structure. In general, however, JSON cannot be streamed (but commonly
> it can be).
>
> Imagine a pathological case of this shape: 1... <60GB of digits>. Th
> On 30 Sep 2024, at 06:52, Abdur-Rahmaan Janhangeer via Python-list
> wrote:
>
>
> import polars as pl
> pl.read_json("file.json")
>
>
This is not going to work unless the computer has a lot more the 60GiB of RAM.
As later suggested a streaming parser is required.
Barry
--
https://m
Whether and to what degree you can stream JSON depends on JSON
structure. In general, however, JSON cannot be streamed (but commonly
it can be).
Imagine a pathological case of this shape: 1... <60GB of digits>. This
is still a valid JSON (it doesn't have any limits on how many digits a
Thanks Abdur Rahmaan.
I will give it a try !
Thanks
Asif
On Mon, Sep 30, 2024 at 11:19 AM Abdur-Rahmaan Janhangeer <
arj.pyt...@gmail.com> wrote:
> Idk if you tried Polars, but it seems to work well with JSON data
>
> import polars as pl
> pl.read_json("file.json")
Idk if you tried Polars, but it seems to work well with JSON data
import polars as pl
pl.read_json("file.json")
Kind Regards,
Abdur-Rahmaan Janhangeer
about <https://compileralchemy.github.io/> | blog
<https://www.pythonkitchen.com>
github <https://github.com/Abdur-Rah
Dear Python Experts,
I am working with the Kenna Application's API to retrieve vulnerability
data. The API endpoint provides a single, massive JSON file in gzip format,
approximately 60 GB in size. Handling such a large dataset in one go is
proving to be quite challenging, especially in ter
Hi,
I'm looking for a convenience function to convert a Marshmallow schema
into a valid Python class definition. That is, I want to generate python
code (class MySchema.. etc) that I could write to a .py file. Does this
exist? I tried the code below, but that is not the intended use
acters each per event. The traffic on the DB is going to be
>>very low, creating maybe a dozen events a day, mainly triggered via a
>>command-line interface, although I will probably set up one or two cron
>>jobs, each of which might generate another 0 to maybe 5 records a day.
&
t; very low, creating maybe a dozen events a day, mainly triggered via a
>> command-line interface, although I will probably set up one or two cron
>> jobs, each of which might generate another 0 to maybe 5 records a day.
>>
>> I could go for JSON (or rather LONGSTRING, as J
day, mainly triggered via a
> command-line interface, although I will probably set up one or two cron
> jobs, each of which might generate another 0 to maybe 5 records a day.
>
> I could go for JSON (or rather LONGSTRING, as JSON is just an alias for
> LONGSTRING, but JSON is not
hich I want to add in the field 'info'. This data never needs to be
>>modified, once the event has been inserted into the DB.
>>
>>What type should the info field have? JSON, PickleType, String, or
>>something else?
>
> I would use JSON, it expresses dicts well
ullable=False)
> uid = Column('gid', String(64), ForeignKey('users.uid'), nullable=False)
> info = ??
>
>The event may have arbitrary, but dict-like data associated with it,
>which I want to add in the field 'info'. This data never needs to be
>mod
ned to use JSON if the data is something that can
be easily represented that way.
--
Greg
--
https://mail.python.org/mailman/listinfo/python-list
Albert-Jan Roskam wrote:
> The event may have arbitrary, but dict-like data associated with it,
> which I want to add in the field 'info'. This data never needs to be
> modified, once the event has been inserted into the DB.
>
> What type should t
needs to be
modified, once the event has been inserted into the DB.
What type should the info field have? JSON, PickleType, String, or
something else?
I couldn't find any really reliable sounding information about the
relative
pros and cons, apart from a Red
ForeignKey('users.uid'), nullable=False)
info = ??
The event may have arbitrary, but dict-like data associated with it,
which I want to add in the field 'info'. This data never needs to be
modified, once the event has been inserted into the DB.
What type should the info field hav
On Mon, Dec 14, 2020 at 4:57 AM Ethan Furman wrote:
>
> From: Caleb Gattegno
> Date: Fri, Dec 11, 2020 at 5:02 PM
> Subject: How to specify JSON parameters in CallBack?
>
>
> Please can you suggest where should I look for advice on converting a old
> style web app which v
From: Caleb Gattegno
Date: Fri, Dec 11, 2020 at 5:02 PM
Subject: How to specify JSON parameters in CallBack?
Please can you suggest where should I look for advice on converting a old style web app which vends whole pages of html
with a cgi-bin/python script invoked bypython3 server.py, into
On 2020-12-13, Bischoop wrote:
> Here https://bpa.st/YBVA
Don't do that. Include in your post a short example that illustrates
your questions.
> I've working code with dictionary only if used dict from the code
> [...]
--
https://mail.python.org/mailman/listinfo/python-list
On 2020-12-13 00:01, Bischoop wrote:
Here https://bpa.st/YBVA I've working code with dictionary only if used dict
from the code
(it's commented now) but when I load it I can print it from load
function (line 14) but at all have not a
clue how to pass the data so could use them.
I've learnt a
On Sat, Dec 12, 2020 at 4:05 PM Bischoop wrote:
>
> Here https://bpa.st/YBVA I've working code with dictionary only if used
> dict from the code
> (it's commented now) but when I load it I can print it from load
> function (line 14) but at all have not a
> clue how to pass the data so could use
Here https://bpa.st/YBVA I've working code with dictionary only if used dict
from the code
(it's commented now) but when I load it I can print it from load
function (line 14) but at all have not a
clue how to pass the data so could use them.
I've learnt a lot when making it but here I'm comple
There is some comment for
https://docs.python.org/3.10/library/json.html.
The latest RFC number is 8259.
Thanks,
Sincerely, JSON fan Byung-Hee
--
^고맙습니다 _白衣從軍_ 감사합니다_^))//
--
https://mail.python.org/mailman/listinfo/python-list
On 2019-08-12 11:38 p.m., Peng Yu wrote:
```
import requests
s = requests.Session()
import json
s.cookies.set_cookie(requests.utils.cookiejar_from_dict(json.load(sys.stdin)))
```
I used the above command to load cookies from a json file. But I got
the following error. Does anybody know how to
```
import requests
s = requests.Session()
import json
s.cookies.set_cookie(requests.utils.cookiejar_from_dict(json.load(sys.stdin)))
```
I used the above command to load cookies from a json file. But I got
the following error. Does anybody know how to fix the error? Thanks.
```
Traceback (most
On 12/20/2018 10:42 AM, Paul Moore wrote:
I'm looking for a way to incrementally decode a JSON file. I know this
has come up before, and in general the problem is not soluble (because
in theory the JSON file could be a single object).
AFAIK, a JSON file always represents a single JSON ite
On Fri, Dec 21, 2018 at 4:30 AM Paul Moore wrote:
> On Thu, 20 Dec 2018 at 17:22, Chris Angelico wrote:
> > Proper error handling is left as an exercise for the reader, both in
> > terms of JSON errors and file errors. Also, the code is completely
> > untested. Have fun
(Sorry, hit "Send" too soon on the last try!)
On Thu, 20 Dec 2018 at 17:22, Chris Angelico wrote:
>
> On Fri, Dec 21, 2018 at 2:44 AM Paul Moore wrote:
> >
> > I'm looking for a way to incrementally decode a JSON file. I know this
> > has come up be
On Thu, 20 Dec 2018 at 17:22, Chris Angelico wrote:
>
> On Fri, Dec 21, 2018 at 2:44 AM Paul Moore wrote:
> >
> > I'm looking for a way to incrementally decode a JSON file. I know this
> > has come up before, and in general the problem is not soluble (because
> &g
On Fri, Dec 21, 2018 at 2:44 AM Paul Moore wrote:
>
> I'm looking for a way to incrementally decode a JSON file. I know this
> has come up before, and in general the problem is not soluble (because
> in theory the JSON file could be a single object). In my particular
> situati
I'm looking for a way to incrementally decode a JSON file. I know this
has come up before, and in general the problem is not soluble (because
in theory the JSON file could be a single object). In my particular
situation, though, I have a 9GB file containing a top-level array
object, with
On 01/12/2018 10:10, Marko Rauhamaa wrote:
In my case, I need an explicit terminator marker to know when a JSON
value is complete. For example, if I should read from a socket:
123
[snip]
You're having to invent a transport protocol to encapsulate your data
protocol because you
of libraries to deal with them.
Then the data files wouldn't be human readable, making debugging
a lot more hassle.
Cameron Simpson writes:
> There's a common format called Newline Delimited JSON (NDJSON) for just this
> need.
>
> Just format the outbound records as JSON
On 2018-11-30, Marko Rauhamaa wrote:
> Paul Rubin :
>> Maybe someone can convince me I'm misusing JSON but I often want to
>> write out a file containing multiple records, and it's convenient to
>> use JSON to represent the record data.
>>
>> The obvious
Marko Rauhamaa writes:
> Chris Angelico :
>
>> On Sat, Dec 1, 2018 at 10:16 PM Marko Rauhamaa wrote:
>>> and the framing format is HTTP. I will need to type something like this:
>>>
>>>POST / HTTP/1.1^M
>>>Host: localhost^M
>>>
Chris Angelico :
> On Sat, Dec 1, 2018 at 10:16 PM Marko Rauhamaa wrote:
>> and the framing format is HTTP. I will need to type something like this:
>>
>>POST / HTTP/1.1^M
>>Host: localhost^M
>> Content-type: application/json^M
&g
gt; >
> > I dunno, I type HTTP manually often enough that it can't be all *that*
> > horrible.
>
> Say I want to send this piece of JSON:
>
>{
>"msgtype": "echo-req",
>"opid": 3487547843
>}
>
> an
x. HTTP would be
>> horrible to have to type even without the content-length problem, but
>> BEEP (RFC 3080) would suffer from the content-length (and CRLF!)
>> issue as well.
>
> I dunno, I type HTTP manually often enough that it can't be all *that*
> horrible.
Say I
On Sat, Dec 1, 2018 at 9:16 PM Marko Rauhamaa wrote:
>
> Paul Rubin :
>
> > Marko Rauhamaa writes:
> >> Having rejected different options ( >> https://en.wikipedia.org/wiki/JSON_streaming>), I settled with
> >> terminating each JSON value with an ASC
Paul Rubin :
> Marko Rauhamaa writes:
>> Having rejected different options (> https://en.wikipedia.org/wiki/JSON_streaming>), I settled with
>> terminating each JSON value with an ASCII NUL character, which is
>> illegal in JSON proper.
>
> Thanks, that Wikipedi
On 30/11/2018 23:40, Marko Rauhamaa wrote:
> Paul Rubin :
>> Maybe someone can convince me I'm misusing JSON but I often want to
>> write out a file containing multiple records, and it's convenient to
>> use JSON to represent the record data.
>>
>> The ob
On Sat, Dec 1, 2018 at 9:46 AM Marko Rauhamaa wrote:
>
> Paul Rubin :
> > Maybe someone can convince me I'm misusing JSON but I often want to
> > write out a file containing multiple records, and it's convenient to
> > use JSON to represent the record data.
>
On 01Dec2018 00:40, Marko Rauhamaa wrote:
Paul Rubin :
Maybe someone can convince me I'm misusing JSON but I often want to
write out a file containing multiple records, and it's convenient to
use JSON to represent the record data.
The obvious way to read a JSON doc from a fi
Paul Rubin :
> Maybe someone can convince me I'm misusing JSON but I often want to
> write out a file containing multiple records, and it's convenient to
> use JSON to represent the record data.
>
> The obvious way to read a JSON doc from a file is with "json.load(
On Wed, 25 Apr 2018 14:32:01 -0700, Sayth Renshaw wrote:
> Hi
>
> Hoping for guidance trying to find some advanced articles or guides or
> topics for json parsing.
>
> I can parse and extract json just dandy.
>
> What I am trying to figure out is how I give myself
On Thursday, 26 April 2018 07:57:28 UTC+10, Paul Rubin wrote:
> Sayth Renshaw writes:
> > What I am trying to figure out is how I give myself surety that the
> > data I parse out is correct or will fail in an expected way.
>
> JSON is messier than people think. Here&#
On 2018-03-25 06:30:54 +1100, Chris Angelico wrote:
> On Sun, Mar 25, 2018 at 3:35 AM, Peter J. Holzer wrote:
> > On 2018-03-24 11:21:09 +1100, Chris Angelico wrote:
> >> If the database has been configured to use UTF-8 (as mentioned, that's
> >> "utf8mb4" in MySQL), you won't get that byte sequen
On Sun, Mar 25, 2018 at 3:35 AM, Peter J. Holzer wrote:
> On 2018-03-24 11:21:09 +1100, Chris Angelico wrote:
>> If the database has been configured to use UTF-8 (as mentioned, that's
>> "utf8mb4" in MySQL), you won't get that byte sequence back. You'll get
>> back valid UTF-8.
>
> Actually (with
On 2018-03-24 11:21:09 +1100, Chris Angelico wrote:
> On Sat, Mar 24, 2018 at 11:11 AM, Steven D'Aprano
> wrote:
> > On Fri, 23 Mar 2018 07:46:16 -0700, Tobiah wrote:
> >> If I changed my database tables to all be UTF-8 would this work cleanly
> >> without any decoding?
> >
> > Not reliably or saf
On Sat, 24 Mar 2018 11:21:09 +1100, Chris Angelico wrote:
>>> If I changed my database tables to all be UTF-8 would this work
>>> cleanly without any decoding?
>>
>> Not reliably or safely. It will appear to work so long as you have only
>> pure ASCII strings from the database, and then crash when
(most recent call last):
> File "", line 1, in
> File "/usr/local/lib/python2.7/json/__init__.py", line 231, in dumps
> return _default_encoder.encode(obj)
> File "/usr/local/lib/python2.7/json/encoder.py", line 195, in encode
> return e
#x27;t:
py> text_from_database = u"hello wörld".encode('latin1')
py> print text_from_database
hello w�rld
py> json.dumps(text_from_database)
Traceback (most recent call last):
File "", line 1, in
File "/usr/local/lib/python2.7/json/__init__.py"
On Sat, Mar 24, 2018 at 1:46 AM, Tobiah wrote:
> On 03/22/2018 12:46 PM, Tobiah wrote:
>>
>> I have some mailing information in a Mysql database that has
>> characters from various other countries. The table says that
>> it's using latin-1 encoding. I want
On 2018-03-23, Richard Damon wrote:
> One comment on this whole argument, the original poster asked how to get
> data from a database that WAS using Latin-1 encoding into JSON (which
> wants UTF-8 encoding) and was asking if something needed to be done
> beyond using .decode(&#x
On 2018-03-23, Chris Angelico wrote:
> On Fri, Mar 23, 2018 at 10:47 AM, Steven D'Aprano
> wrote:
>> On Fri, 23 Mar 2018 07:09:50 +1100, Chris Angelico wrote:
>>
>>>> I was reading though, that JSON files must be encoded with UTF-8. So
>>>> should
On 03/22/2018 12:46 PM, Tobiah wrote:
I have some mailing information in a Mysql database that has
characters from various other countries. The table says that
it's using latin-1 encoding. I want to send this data out
as JSON.
So I'm just taking each datum and doing 'name&#
with a
maximum of three bytes per character", in contrast to "utf8mb4" which
is, well, UTF-8.
In any case, abusing "Latin-1" to store binary data is still wrong.
That's what BLOB is for.
ChrisA
One comment on this whole argument, the original poster asked how to get
On Fri, Mar 23, 2018 at 9:29 PM, Steven D'Aprano
wrote:
> On Fri, 23 Mar 2018 18:35:20 +1100, Chris Angelico wrote:
>
>> That doesn't seem to be a strictly-correct Latin-1 decoder, then. There
>> are a number of unassigned byte values in ISO-8859-1.
>
> That's incorrect, but I don't blame you for
On Fri, 23 Mar 2018 18:35:20 +1100, Chris Angelico wrote:
> That doesn't seem to be a strictly-correct Latin-1 decoder, then. There
> are a number of unassigned byte values in ISO-8859-1.
That's incorrect, but I don't blame you for getting it wrong. Who thought
that it was a good idea to disting
On 23 March 2018 at 00:27, Thomas Jollans wrote:
> On 22/03/18 20:46, Tobiah wrote:
>> I was reading though, that JSON files must be encoded with UTF-8. So
>> should I be doing string.decode('latin-1').encode('utf-8')? Or does
>> the json module do th
On Fri, Mar 23, 2018 at 4:35 PM, Steven D'Aprano
wrote:
> On Fri, 23 Mar 2018 12:05:34 +1100, Chris Angelico wrote:
>
>> Latin-1 is not "arbitrary bytes". It is a very specific encoding that
>> cannot decode every possible byte value.
>
> Yes it can.
>
> py> blob = bytes(range(256))
> py> len(blob
On Fri, 23 Mar 2018 12:05:34 +1100, Chris Angelico wrote:
> Latin-1 is not "arbitrary bytes". It is a very specific encoding that
> cannot decode every possible byte value.
Yes it can.
py> blob = bytes(range(256))
py> len(blob)
256
py> blob[45:55]
b'-./0123456'
py> s = blob.decode('latin1')
py>
ut Python 2 style byte-
> strings, and Latin-1 is just a convenient, easy way to get that that
> ensures ASCII bytes look like ASCII characters.
The OP is talking about JSON. Reason makes no sense in that context.
And if it really is a byte string, why store it as a Latin-1 string?
Store it as the
On Fri, 23 Mar 2018 11:08:56 +1100, Chris Angelico wrote:
> On Fri, Mar 23, 2018 at 10:47 AM, Steven D'Aprano
> wrote:
>> On Fri, 23 Mar 2018 07:09:50 +1100, Chris Angelico wrote:
>>
>>>> I was reading though, that JSON files must be encoded with UTF-8. So
&
On Fri, Mar 23, 2018 at 11:39 AM, Ben Finney wrote:
> Chris Angelico writes:
>
>> There is NOT always a good reason for a suboptimal configuration.
>
> True. Did anyone claim otherwise?
>
> What I saw Steven responding to was your claim that there is *never* a
> good reason to do it.
>
> To refut
On 22/03/18 20:46, Tobiah wrote:
> I was reading though, that JSON files must be encoded with UTF-8. So
> should I be doing string.decode('latin-1').encode('utf-8')? Or does
> the json module do that for me when I give it a unicode object?
Definitely not.
Chris Angelico writes:
> There is NOT always a good reason for a suboptimal configuration.
True. Did anyone claim otherwise?
What I saw Steven responding to was your claim that there is *never* a
good reason to do it.
To refute that, it's sufficient to show that good reason can exist in
some c
On Fri, Mar 23, 2018 at 11:25 AM, Ben Finney wrote:
> Chris Angelico writes:
>
>> On Fri, Mar 23, 2018 at 10:47 AM, Steven D'Aprano
>> wrote:
>> > On Fri, 23 Mar 2018 07:09:50 +1100, Chris Angelico wrote:
>> >> Reconfigure your MySQL database to use UTF-8. There is no reason to
>> >> use Latin-1
Chris Angelico writes:
> On Fri, Mar 23, 2018 at 10:47 AM, Steven D'Aprano
> wrote:
> > On Fri, 23 Mar 2018 07:09:50 +1100, Chris Angelico wrote:
> >> Reconfigure your MySQL database to use UTF-8. There is no reason to
> >> use Latin-1 in the database.
> >
> > You don't know that. You don't know
On Fri, Mar 23, 2018 at 10:47 AM, Steven D'Aprano
wrote:
> On Fri, 23 Mar 2018 07:09:50 +1100, Chris Angelico wrote:
>
>>> I was reading though, that JSON files must be encoded with UTF-8. So
>>> should I be doing string.decode('latin-1').encode('utf-8
On Fri, 23 Mar 2018 07:09:50 +1100, Chris Angelico wrote:
>> I was reading though, that JSON files must be encoded with UTF-8. So
>> should I be doing string.decode('latin-1').encode('utf-8')? Or does
>> the json module do that for me when I give it a unico
1 - 100 of 614 matches
Mail list logo
