[Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread Ram Rachum 
Hi everyone,

Take a look at this question:

http://stackoverflow.com/questions/16122435/python-3-how-do-i-use-bytes-to-bytes-and-string-to-string-encodings/16122472?noredirect=1#comment23034787_16122472

Is there really no way to use base64 that's as short as:

b'whatever'.encode('base64')

Because doing this:

import codecs
codecs.decode(b"whatever", "base64_codec")

Or this:

import base64
encoded = base64.b64encode(b'whatever')

Is cumbersome!

Why can't I do something like b'whatever'.encode('base64')? Or maybe using
a different method than `encode`?


Thanks,
Ram.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread Victor Stinner 
Hi,

Your question is discussed since 4 years in the following issue:
http://bugs.python.org/issue7475

The last proposition is to add transform() and untransform() methods
to bytes and str types. But nobody implemented the idea. If I remember
correctly, the missing point is how to define which types are
supported by a codec (ex: only bytes for bz2 codec, bytes and str for
rot13).

Victor

2013/4/22 Ram Rachum :
> Hi everyone,
>
> Take a look at this question:
>
> http://stackoverflow.com/questions/16122435/python-3-how-do-i-use-bytes-to-bytes-and-string-to-string-encodings/16122472?noredirect=1#comment23034787_16122472
>
> Is there really no way to use base64 that's as short as:
>
> b'whatever'.encode('base64')
>
> Because doing this:
>
> import codecs
> codecs.decode(b"whatever", "base64_codec")
>
> Or this:
>
> import base64
> encoded = base64.b64encode(b'whatever')
>
> Is cumbersome!
>
> Why can't I do something like b'whatever'.encode('base64')? Or maybe using a
> different method than `encode`?
>
>
> Thanks,
> Ram.
>
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread Calvin Spealman 
if two lines is cumbersome, you're in for a cumbersome life a programmer.
On Apr 22, 2013 7:31 AM, "Ram Rachum"  wrote:

> Hi everyone,
>
> Take a look at this question:
>
>
> http://stackoverflow.com/questions/16122435/python-3-how-do-i-use-bytes-to-bytes-and-string-to-string-encodings/16122472?noredirect=1#comment23034787_16122472
>
> Is there really no way to use base64 that's as short as:
>
> b'whatever'.encode('base64')
>
> Because doing this:
>
> import codecs
> codecs.decode(b"whatever", "base64_codec")
>
> Or this:
>
> import base64
> encoded = base64.b64encode(b'whatever')
>
> Is cumbersome!
>
> Why can't I do something like b'whatever'.encode('base64')? Or maybe using
> a different method than `encode`?
>
>
> Thanks,
> Ram.
>
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/ironfroggy%40gmail.com
>
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread Paul Moore 
On 22 April 2013 12:39, Calvin Spealman  wrote:

> if two lines is cumbersome, you're in for a cumbersome life a programmer.
>

One of which is essentially Python's equivalent of a declaration...
Paul
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread Devin Jeanpierre 
On Mon, Apr 22, 2013 at 7:39 AM, Calvin Spealman  wrote:
> if two lines is cumbersome, you're in for a cumbersome life a programmer.

Other encodings are either missing completely from the stdlib, or have
corrupted behavior. For example, string_escape is gone, and
unicode_escape doesn't make any sense anymore -- python code is text,
not bytes, so why does 'abc'.encode('unicode_escape') return bytes? I
don't think this change was thought through completely before it was
implemented.

I agree base64 is a bad place to pick at the encode/decode changes, though. :(

-- Devin
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread Oleg Broytman 
On Mon, Apr 22, 2013 at 09:50:14AM -0400, Devin Jeanpierre 
 wrote:
> unicode_escape doesn't make any sense anymore -- python code is text,
> not bytes, so why does 'abc'.encode('unicode_escape') return bytes?

   AFAIU the situation is simple: unicode.encode(encoding) returns
bytes, bytes.decode(encoding) returns unicode, and neither
unicode.decode() nor bytes.encode() exist.
   Transformations like base64 and bz2 are nor encoding/decoding -- they
are bytes/bytes or unicode/unicode transformations.

Oleg.
-- 
 Oleg Broytmanhttp://phdru.name/[email protected]
   Programmers don't die, they just GOSUB without RETURN.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread Stephen J. Turnbull 
Devin Jeanpierre writes:

 > why does 'abc'.encode('unicode_escape') return bytes?

Duck-typing: encode always turns unicode into bytes.

___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread R. David Murray 
On Mon, 22 Apr 2013 09:50:14 -0400, Devin Jeanpierre  
wrote:
> On Mon, Apr 22, 2013 at 7:39 AM, Calvin Spealman  wrote:
> > if two lines is cumbersome, you're in for a cumbersome life a programmer.
> 
> Other encodings are either missing completely from the stdlib, or have
> corrupted behavior. For example, string_escape is gone, and
> unicode_escape doesn't make any sense anymore -- python code is text,
> not bytes, so why does 'abc'.encode('unicode_escape') return bytes? I
> don't think this change was thought through completely before it was
> implemented.

We use unicode_escape (actually raw_unicode_escape) in the email package,
and there we are converting between string and bytes.  It is used as an
encoder when we are supposed to have ASCII input but have other stuff,
and need ASCII output and don't want to lose information.  So yes,
that encoder does still make sense.  It would also be useful as a
transform function, but as someone has pointed out there's an issue
for that.

--David
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Simplify the code of get_attrib_from_keywords somewhat.

2013-04-22 Thread Serhiy Storchaka 

On 22.04.13 15:52, eli.bendersky wrote:

http://hg.python.org/cpython/rev/c9674421d78e
changeset:   83494:c9674421d78e
user:Eli Bendersky 
date:Mon Apr 22 05:52:16 2013 -0700
summary:
   Simplify the code of get_attrib_from_keywords somewhat.



-PyDict_DelItem(kwds, attrib_str);
+PyDict_DelItemString(kwds, ATTRIB_KEY);


PyDict_GetItemString() and PyDict_DelItemString() internally create a 
Python string. I.e. new code creates one additional string if attrib was 
found in kwds.



-if (attrib)
-PyDict_Update(attrib, kwds);
+assert(attrib);


attrib can be NULL in case of memory allocation error.


___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] cpython: Simplify the code of get_attrib_from_keywords somewhat.

2013-04-22 Thread Eli Bendersky 
On Mon, Apr 22, 2013 at 10:13 AM, Serhiy Storchaka wrote:

> On 22.04.13 15:52, eli.bendersky wrote:
>
>> http://hg.python.org/cpython/**rev/c9674421d78e
>> changeset:   83494:c9674421d78e
>> user:Eli Bendersky 
>> date:Mon Apr 22 05:52:16 2013 -0700
>> summary:
>>Simplify the code of get_attrib_from_keywords somewhat.
>>
>
>  -PyDict_DelItem(kwds, attrib_str);
>> +PyDict_DelItemString(kwds, ATTRIB_KEY);
>>
>
> PyDict_GetItemString() and PyDict_DelItemString() internally create a
> Python string. I.e. new code creates one additional string if attrib was
> found in kwds.
>
>
>  -if (attrib)
>> -PyDict_Update(attrib, kwds);
>> +assert(attrib);
>>
>
> attrib can be NULL in case of memory allocation error.
>

Thanks, I'll review this

Eli
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread Greg Ewing 

Victor Stinner wrote:

The last proposition is to add transform() and untransform() methods
to bytes and str types. ... If I remember
correctly, the missing point is how to define which types are
supported by a codec


Also, for any given codec, which direction is "transform"
and which is "untransform"?

Also also, what's so special about base64 et al that they
deserve an ultra-special way of invoking them, instead of
having to import a class or function like you do for
*every* *other* piece of library functionality?

--
Greg
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread R. David Murray 
On Tue, 23 Apr 2013 11:16:20 +1200, Greg Ewing  
wrote:
> Victor Stinner wrote:
> > The last proposition is to add transform() and untransform() methods
> > to bytes and str types. ... If I remember
> > correctly, the missing point is how to define which types are
> > supported by a codec
> 
> Also, for any given codec, which direction is "transform"
> and which is "untransform"?

You transform *into* the encoding, and untransform *out* of the encoding.
Do you have an example where that would be ambiguous?

> Also also, what's so special about base64 et al that they
> deserve an ultra-special way of invoking them, instead of
> having to import a class or function like you do for
> *every* *other* piece of library functionality?

You can ask the same question about all the other codecs.  (And that
question has indeed been asked in the past.)

(One answer is that they used to work in Python2...but the longer we go
without restoring the functionality to Python3, the weaker that particular
argument becomes.)

--David
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread Guido van Rossum 
--Guido van Rossum (sent from Android phone)
On Apr 22, 2013 6:09 PM, "R. David Murray"  wrote:
>
> On Tue, 23 Apr 2013 11:16:20 +1200, Greg Ewing <
[email protected]> wrote:
> > Victor Stinner wrote:
> > > The last proposition is to add transform() and untransform() methods
> > > to bytes and str types. ... If I remember
> > > correctly, the missing point is how to define which types are
> > > supported by a codec
> >
> > Also, for any given codec, which direction is "transform"
> > and which is "untransform"?
>
> You transform *into* the encoding, and untransform *out* of the encoding.
> Do you have an example where that would be ambiguous?
>
> > Also also, what's so special about base64 et al that they
> > deserve an ultra-special way of invoking them, instead of
> > having to import a class or function like you do for
> > *every* *other* piece of library functionality?
>
> You can ask the same question about all the other codecs.  (And that
> question has indeed been asked in the past.)

Except for rot13. :-)

> (One answer is that they used to work in Python2...but the longer we go
> without restoring the functionality to Python3, the weaker that particular
> argument becomes.)
>
> --David
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
http://mail.python.org/mailman/options/python-dev/guido%40python.org
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread Steven D'Aprano 

On 23/04/13 09:16, Greg Ewing wrote:

Victor Stinner wrote:

The last proposition is to add transform() and untransform() methods
to bytes and str types. ... If I remember
correctly, the missing point is how to define which types are
supported by a codec


Also, for any given codec, which direction is "transform"
and which is "untransform"?

Also also, what's so special about base64 et al that they
deserve an ultra-special way of invoking them, instead of
having to import a class or function like you do for
*every* *other* piece of library functionality?



As others have pointed out in the past, repeatedly, the codec system is completely general and can 
transform bytes->bytes and text->text just as easily as bytes<->text. Or indeed any 
bijection, as the docs for 2.7 point out. The question isn't "What's so special about 
base64?" The questions should be:

- What's so special about exotic legacy transformations like ISO-8859-10 and 
MacRoman that they deserve a string method for invoking them?

- Why have common transformations like base64, which worked in 2.x, been 
relegated to second-class status in 3.x?

- If it is no burden to have to import a module and call an external function 
for some transformations, why have encode and decode methods at all?


If you haven't read this, you should:

http://lucumr.pocoo.org/2012/8/11/codec-confusion/




--
Steven
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread Donald Stufft 

On Apr 22, 2013, at 10:04 PM, Steven D'Aprano  wrote:

> On 23/04/13 09:16, Greg Ewing wrote:
>> Victor Stinner wrote:
>>> The last proposition is to add transform() and untransform() methods
>>> to bytes and str types. ... If I remember
>>> correctly, the missing point is how to define which types are
>>> supported by a codec
>> 
>> Also, for any given codec, which direction is "transform"
>> and which is "untransform"?
>> 
>> Also also, what's so special about base64 et al that they
>> deserve an ultra-special way of invoking them, instead of
>> having to import a class or function like you do for
>> *every* *other* piece of library functionality?
> 
> 
> As others have pointed out in the past, repeatedly, the codec system is 
> completely general and can transform bytes->bytes and text->text just as 
> easily as bytes<->text. Or indeed any bijection, as the docs for 2.7 point 
> out. The question isn't "What's so special about base64?" The questions 
> should be:
> 
> - What's so special about exotic legacy transformations like ISO-8859-10 and 
> MacRoman that they deserve a string method for invoking them?
> 
> - Why have common transformations like base64, which worked in 2.x, been 
> relegated to second-class status in 3.x?
> 
> - If it is no burden to have to import a module and call an external function 
> for some transformations, why have encode and decode methods at all?
> 
> 
> If you haven't read this, you should:
> 
> http://lucumr.pocoo.org/2012/8/11/codec-confusion/

I may be dull, but it wasn't until I started using Python 3 that it really 
clicked in my head what encode/decode did exactly. In Python2 I just sort of 
sprinkled one or the other when there was errors until the pain stopped. I 
mostly attribute this to str.decode and bytes.encode not existing.

> 
> 
> 
> 
> -- 
> Steven
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/donald%40stufft.io


-
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA



signature.asc
Description: Message signed with OpenPGP using GPGMail
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread Guido van Rossum 
On Mon, Apr 22, 2013 at 7:04 PM, Steven D'Aprano  wrote:
> As others have pointed out in the past, repeatedly, the codec system is
> completely general and can transform bytes->bytes and text->text just as
> easily as bytes<->text. Or indeed any bijection, as the docs for 2.7 point
> out. The question isn't "What's so special about base64?" The questions
> should be:
>
> - What's so special about exotic legacy transformations like ISO-8859-10 and
> MacRoman that they deserve a string method for invoking them?
>
> - Why have common transformations like base64, which worked in 2.x, been
> relegated to second-class status in 3.x?
>
> - If it is no burden to have to import a module and call an external
> function for some transformations, why have encode and decode methods at
> all?

There are good answers to all of these, and your rhetoric is not
appreciated. The special status is for the translation between bytes
and Unicode characters (code points). There are many contexts where a
byte stream is labeled (either separately or in-line) as being encoded
using some specific encoding.

--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread Lennart Regebro 
On Tue, Apr 23, 2013 at 4:04 AM, Steven D'Aprano  wrote:
> As others have pointed out in the past, repeatedly, the codec system is
> completely general and can transform bytes->bytes and text->text just as
> easily as bytes<->text.

Yes, but the encode()/decode() methods are not, and the fact that you
now know what goes in and what comes out means that people get much
fewer Decode/EncodeErrors. Which is a good thing.

//Lennart
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread Fábio Santos 
Using decode() and encode() would break that predictability. But someone
suggested the use of transform() and untransform() instead. That would
clarify that the transformation is bytes > bytes and Unicode string >
Unicode string.
On 23 Apr 2013 05:50, "Lennart Regebro"  wrote:

> On Tue, Apr 23, 2013 at 4:04 AM, Steven D'Aprano 
> wrote:
> > As others have pointed out in the past, repeatedly, the codec system is
> > completely general and can transform bytes->bytes and text->text just as
> > easily as bytes<->text.
>
> Yes, but the encode()/decode() methods are not, and the fact that you
> now know what goes in and what comes out means that people get much
> fewer Decode/EncodeErrors. Which is a good thing.
>
> //Lennart
> ___
> Python-Dev mailing list
> [email protected]
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/fabiosantosart%40gmail.com
>
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Why can't I encode/decode base64 without importing a module?

2013-04-22 Thread Greg Ewing 

Steven D'Aprano wrote:
- If it is no burden to have to import a module and call an external 
function for some transformations, why have encode and decode methods at 
all?


Now that all text strings are unicode, the unicode codecs
are in a sense special, in that you can't do any string
I/O at all without using them at some stage. So arguably
it makes sense to have a very easy way of invoking them.

I suspect that without this, the idea of all strings
being unicode would have been even harder to sell than
it was.

--
Greg
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com