Re: [Python-Dev] [Python-ideas] Proposed addtion to urllib.parse in 3.1 (and urlparse in 2.7)

[Mart Sõmermaa]

The general consensus in python-ideas is that the following is needed, so I
bring it to python-dev to final discussions before I file a feature request
in bugs.python.org.

Proposal: add add_query_params() for appending query parameters to an URL to
urllib.parse and urlparse.

Implementation:
http://github.com/mrts/qparams/blob/83d1ec287ec10934b5e637455819cf796b1b421c/qparams.py(feel
free to fork and comment).

Behaviour (longish, guided by "simple things are simiple, complex things
possible"):

In the simplest form, parameters can be passed via keyword arguments:

>>> add_query_params('foo', bar='baz')
'foo?bar=baz'

>>> add_query_params('http://example.com/a/b/c?a=b', b='d')
'http://example.com/a/b/c?a=b&b=d'

Note that '/', if given in arguments, is encoded:

>>> add_query_params('http://example.com/a/b/c?a=b', b='d', foo='/bar')
'http://example.com/a/b/c?a=b&b=d&foo=%2Fbar'

Duplicates are discarded:

>>> add_query_params('http://example.com/a/b/c?a=b', a='b')
'http://example.com/a/b/c?a=b'

>>> add_query_params('http://example.com/a/b/c?a=b&c=q', a='b', b='d',
...  c='q')
'http://example.com/a/b/c?a=b&c=q&b=d'

But different values for the same key are supported:

>>> add_query_params('http://example.com/a/b/c?a=b', a='c', b='d')
'http://example.com/a/b/c?a=b&a=c&b=d'

Pass different values for a single key in a list (again, duplicates are
removed):

>>> add_query_params('http://example.com/a/b/c?a=b', a=('q', 'b', 'c'),
... b='d')
'http://example.com/a/b/c?a=b&a=q&a=c&b=d'

Keys with no value are respected, pass ``None`` to create one:

>>> add_query_params('http://example.com/a/b/c?a', b=None)
'http://example.com/a/b/c?a&b'

But if a value is given, the empty key is considered a duplicate (i.e. the
case of a&a=b is considered nonsensical):

>>> add_query_params('http://example.com/a/b/c?a', a='b', c=None)
'http://example.com/a/b/c?a=b&c'

If you need to pass in key names that are not allowed in keyword arguments,
pass them via a dictionary in second argument:

>>> add_query_params('foo', {"+'|äüö": 'bar'})
'foo?%2B%27%7C%C3%A4%C3%BC%C3%B6=bar'

Order of original parameters is retained, although similar keys are grouped
together. Order of keyword arguments is not (and can not be) retained:

>>> add_query_params('foo?a=b&b=c&a=b&a=d', a='b')
'foo?a=b&a=d&b=c'

>>> add_query_params('http://example.com/a/b/c?a=b&q=c&e=d',
... x='y', e=1, o=2)
'http://example.com/a/b/c?a=b&q=c&e=d&e=1&x=y&o=2'

If you need to retain the order of the added parameters, use an
:class:`OrderedDict` as the second argument (*params_dict*):

>>> from collections import OrderedDict
>>> od = OrderedDict()
>>> od['xavier'] = 1
>>> od['abacus'] = 2
>>> od['janus'] = 3
>>> add_query_params('http://example.com/a/b/c?a=b', od)
'http://example.com/a/b/c?a=b&xavier=1&abacus=2&janus=3'

If both *params_dict* and keyword arguments are provided, values from the
former are used before the latter:

>>> add_query_params('http://example.com/a/b/c?a=b', od, xavier=1.1,
... zorg='a', alpha='b', watt='c', borg='d')
'
http://example.com/a/b/c?a=b&xavier=1&xavier=1.1&abacus=2&janus=3&zorg=a&borg=d&watt=c&alpha=b
'

Do nothing with a single argument:

>>> add_query_params('a')
'a'

>>> add_query_params('arbitrary strange stuff?öäüõ*()+-=42')
'arbitrary strange stuff?\xc3\xb6\xc3\xa4\xc3\xbc\xc3\xb5*()+-=42'
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Possible py3k io wierdness

[Brian Quinlan]

I've added a new proposed patch to:
http://bugs.python.org/issue5700

The idea is:
- only IOBase implements close() (though a subclass can override close
  without causing problems so long as it calls super().close() or
  calls .flush() and ._close() directly)
- change IOBase.close to call .flush() and then ._close()
- .flush() invokes super().flush() in every class except IOBase
- ._close() invokes super()._close() in every class except IOBase
- FileIO is implemented in Python in _pyio.py so that it can have the
  same base class as the other Python-implemented files classes
- tests verify that .flush() is not called after the file is closed
- tests verify that ._close()/.flush() calls are propagated correctly

On nice side effect is that inheritance is a lot easier and MI works as 
expected i.e.


class DebugClass(IOBase):
  def flush(self):
print()
super().flush()
  def _close(self):
print(
super()._close()

class MyClass(FileIO, DebugClass): # whatever order makes sense
  ...

m = MyClass(...)
m.close()
# Will call:
#   IOBase.close()
#   DebugClass.flush()  # FileIO has no .flush method
#   IOBase.flush()
#   FileIO._close()
#   DebugClass._close()
#   IOBase._close()


Cheers,
Brian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-ideas] Proposed addtion to urllib.parse in 3.1 (and urlparse in 2.7)

[Mart Sõmermaa]

On Sun, Apr 12, 2009 at 3:23 PM, Jacob Holm  wrote:

> Hi Mart
>
>>>> add_query_params('http://example.com/a/b/c?a=b', b='d', foo='/bar')
>>'http://example.com/a/b/c?a=b&b=d&foo=%2Fbar <
>> http://example.com/a/b/c?a=b&b=d&foo=%2Fbar>'
>>
>> Duplicates are discarded:
>>
>
> Why discard duplicates?  They are valid and have a well-defined meaning.



The bad thing about reasoning about query strings is that there is no
comprehensive documentation about their meaning. Both RFC 1738 and RFC 3986
are rather vague in that matter. But I agree that duplicates actually have a
meaning (an ordered list of identical values), so I'll remove the bits that
prune them unless anyone opposes (which I doubt).


>> But if a value is given, the empty key is considered a duplicate (i.e. the
>> case of a&a=b is considered nonsensical):
>>
>
> Again, it is a valid url and this will change its meaning.  Why?


I'm uncertain whether a&a=b has a meaning, but don't see any harm in
supporting it, so I'll add the feature.


>>>>> add_query_params('http://example.com/a/b/c?a', a='b', c=None)
>>'http://example.com/a/b/c?a=b&c '
>>
>> If you need to pass in key names that are not allowed in keyword
>> arguments,
>> pass them via a dictionary in second argument:
>>
>>>>> add_query_params('foo', {"+'|äüö": 'bar'})
>>'foo?%2B%27%7C%C3%A4%C3%BC%C3%B6=bar'
>>
>> Order of original parameters is retained, although similar keys are
>> grouped
>> together.
>>
>
> Why the grouping?  Is it a side effect of your desire to discard
> duplicates?   Changing the order like that changes the meaning of the url.
>  A concrete case where the order of field names matters is the ":records"
> converter in http://pypi.python.org/pypi/zope.httpform/1.0.1 (a small
> independent package extracted from the form handling code in zope).


 It's also related to duplicate handling, but it mostly relates to the data
structure used in the initial implementation (an OrderedDict). Re-grouping
is removed now and not having to deal with duplicates simplified the code
considerably (using a simple list of key-value tuples now).

If you change it to keep duplicates and not unnecessarily mangle the field
> order I am +1, else I am -0.


Thanks for your input! Changes pushed to github (see the updated behaviour
there as well):

http://github.com/mrts/qparams/blob/4f32670b55082f8d0ef01c33524145c3264c161a/qparams.py

MS
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Needing help to change the grammar

[Harry (Thiago Leucz Astrizi)]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1


Written by "Martin v. Löwis" :
> Notice that Python source is represented in UTF-8 in the parser.  It
> might be that the C source code has a different encoding, which
> would cause the strcmp to fail.

No, all the files in the surce code were already in UTF-8. My system
is configured to treat UTF-8 as the default encoding. This is not an
encoding problem.

Written by "Jack diederich" :
> I love the idea (and most recently edited PEP 306) so here are a few
> suggestions;
>
> Brazil has many python programmers so you might be able to make
> quick progress by asking them for volunteer time.

Yes, I have plans to ask for help in the brazilian Python mailing list
when I finish to prepare the C source code for this project. Then I
expect to receive help to translate the python modules for this new
language. There's a lot of work to do.


> To bug-hunt your technical problem: try switching the "not is"
> operator to include an underscore "not_is."  The python LL(1)
> grammar checker works for python but isn't robust, and does miss
> some grammar ambiguities.  Making the operator a single word might
> reveal a bug in the parser.

Thanks for the advice, you almost guessed what went wrong. I made some
tests and already discovered what's the problem. When I change
Grammar/Grammar, Python/ast.c and Modules/parsermodule.c to transform
"is not" in "not is", everything works fine and I create a new Python
verson where "a is not None" is wrong and "a not is None" is
right. But when I translate this to "não é", always happens a
SyntaxError. So the probles is really in the grammar checker that
can't handle some letters with accent.

Well, knowing where the problem is, I think that I can try to solve it
by myself. Thanks again.


> Please consider switching your students to 'real' python part way
> through the course.  If they want to use the vast amount of python
> code on the internet as examples they will need to know the few
> English keywords.
>
> Also - most python core developers are not native English speakers
> and do OK :) PyCon speakers are about 25% non-native English
> speakers and EuroPython speakers are about the reverse (my rough
> estimate - I'd love to see some hard numbers).

Yes, I know. To a more "serious" programmer, it's essential to have a
basic understanding in english and would be better for him to start
with the real Python. But my intent is not to substitute Python in
Brazil, but to create a new language that could be learned easily by
younger people for educational purposes. My intent is to show them how
a computer software works. But surely I will warn my students that to
take programming more seriously, it's important to learn how to
program in some other language, like the original Python. But thanks
for the advice.

-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFJ4jrjmNGEzq1zP84RAvikAJ4k25vufyWWiDvj3HFZ7Q4M38zCjgCglBGC
dPQTd7mBuswKbNstpJqRuFE=
=xApj
-END PGP SIGNATURE-


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Needing help to change the grammar

[Terry Reedy]

Harry (Thiago Leucz Astrizi) wrote:


Yes, I have plans to ask for help in the brazilian Python mailing list
when I finish to prepare the C source code for this project. Then I
expect to receive help to translate the python modules for this new
language. There's a lot of work to do.


There are only a few modules that you really need to do this for for 
beginners.  Trying to convert the entire stdlib, let alone other stuff 
on pypi, strikes me as foolish.


...


Yes, I know. To a more "serious" programmer, it's essential to have a
basic understanding in english and would be better for him to start
with the real Python. But my intent is not to substitute Python in
Brazil, but to create a new language that could be learned easily by
younger people for educational purposes. My intent is to show them how
a computer software works. But surely I will warn my students that to
take programming more seriously, it's important to learn how to
program in some other language, like the original Python. But thanks
for the advice.


If possible, and I presume it is, make your interpreter dual language. 
Source code in .py files is parsed as now (and module compiles to .pyc). 
 Source in .pyb (python-brazil) is parsed with with your new parser, 
and get a brazilian equivalent of builtins, but use the same AST and 
bytecode.  Bytecode is neither English nor Brazilian ;-).  This would 
give your students access to the whole world of Python modules and allow 
 those who want to move to normal English-based international Python to 
do so without obsoleting their existing work.


Terry Jan Reedy

PS. Since this thread is not about developing Python itself, it would be 
more appropriate on the python-ideas list if continued much further.


PPS Once unicode identifiers were allowed, I considered it inevitable 
that people would also want native-language keywords, especially for 
younger students.  So I expected a project like yours, though I expected 
the first to be in Asia.  I think dual language versions, if possible, 
would be the way to do this without ghettoizing the national versions. 
But as I said, a general discussion of this belongs on python-ideas.


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-ideas] Proposed addtion to urllib.parse in 3.1 (and urlparse in 2.7)

[Lino Mastrodomenico]

2009/4/12 Mart Sõmermaa :
> The bad thing about reasoning about query strings is that there is no
> comprehensive documentation about their meaning. Both RFC 1738 and RFC 3986
> are rather vague in that matter.

FYI the HTML5 spec (http://whatwg.org/html5 ) may have a better
contact with reality than the RFCs.

>From a quick scan, two sections that may be relevant are "4.10.16.3
Form submission algorithm":



and "4.10.16.4 URL-encoded form data":



-- 
Lino Mastrodomenico
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Proposed addtion to urllib.parse in 3.1 (and urlparse in 2.7)

[Cameron Simpson]

On 12Apr2009 16:15, Mart S?mermaa  wrote:
| On Sun, Apr 12, 2009 at 3:23 PM, Jacob Holm  wrote:
| > Hi Mart
| >>>> add_query_params('http://example.com/a/b/c?a=b', b='d', foo='/bar')
| >>'http://example.com/a/b/c?a=b&b=d&foo=%2Fbar <
| >> http://example.com/a/b/c?a=b&b=d&foo=%2Fbar>'
| >>
| >> Duplicates are discarded:
| >
| > Why discard duplicates?  They are valid and have a well-defined meaning.
| 
| The bad thing about reasoning about query strings is that there is no
| comprehensive documentation about their meaning. Both RFC 1738 and RFC 3986
| are rather vague in that matter. But I agree that duplicates actually have a
| meaning (an ordered list of identical values), so I'll remove the bits that
| prune them unless anyone opposes (which I doubt).

+1 from me, with the following suggestion: it's probably worth adding the
to doco that people working with dict-style query_string params should
probably go make a dict or OrderedDict and use:

  add_query_params(..., **the_dict)

just to make the other use case obvious.

An alternative would be to have add_ and append_ methods with set and
list behaviour. Feels a little like API bloat, though the convenience
function can be nice.

Cheers,
-- 
Cameron Simpson  DoD#743
http://www.cskk.ezoshosting.com/cs/

The wonderous pulp and fibre of the brain had been substituted by brass and
iron; he had taught wheelwork to think. - Harry Wilmot Buxton 1832,
referring to Charles Babbage and his difference engine.
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Needing help to change the grammar

[Tony Nelson]

At 16:30 -0400 04/12/2009, Terry Reedy wrote:
 ...
>  Source in .pyb (python-brazil) is parsed with with your new parser,
 ...

In case anyone ever does this again, I suggest that the extension be the
language and optionally country code:

.py_pt  or  .py_pt_BR
-- 

TonyN.:'   
  '  
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [Python-ideas] Proposed addtion to url lib.parse in 3.1 (and urlparse in 2.7)

[Antoine Pitrou]

Mart Sõmermaa  gmail.com> writes:
> 
> Proposal: add add_query_params() for appending query parameters to an URL to
urllib.parse and urlparse.

Is there anything to /remove/ a query parameter?


___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Needing help to change the grammar

[Glenn Linderman]

On approximately 4/12/2009 2:41 PM, came the following characters from 
the keyboard of Tony Nelson:

At 16:30 -0400 04/12/2009, Terry Reedy wrote:
 ...

 Source in .pyb (python-brazil) is parsed with with your new parser,

 ...

In case anyone ever does this again, I suggest that the extension be the
language and optionally country code:

.py_pt  or  .py_pt_BR



Wouldn't that be a good idea for this implementation too?  It sounds 
like it is not-yet-released, as it is also not-yet-bug-free.


And actually, wouldn't it be nice if international keywords could be 
accepted as alternates if one just said


import pt_BR

An implementation along that line, except for things like reversing the 
order of "not" and "is", would allow the next national language 
customization to be done by just recoding the pt_BR module, renaming to 
pt_it or pt_fr or pt_no and translating a bunch of strings, no?


Probably it would be sufficient to allow for one language at a time, per 
module.


--
Glenn -- http://nevcal.com/
===
A protocol is complete when there is nothing left to remove.
-- Stuart Cheshire, Apple Computer, regarding Zero Configuration Networking
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Evaluated cmake as an autoconf replacement

[Giovanni Bajo]

On Mon, 30 Mar 2009 20:34:21 +0200, Christian Heimes wrote:

> Hallo Alexander!
> 
> Alexander Neundorf wrote:
>> This of course depends on the definition of "as good as" ;-) Well, I
>> have met Windows-only developers which use CMake because it is able to
>> generate project files for different versions of Visual Studio, and
>> praise it for that.
> 
> So far I haven't heard any complains about or feature requests for the
> project files. ;)

In fact, I have had one.

I asked to put all those big CJK codecs outside of python2x.dll because 
they were too big and create far larger self-contained distributions 
(aka: py2exe/pyinstaller) as would normally be required.

I was replied that it would be unconvienent to do so because of the fact 
that the build system is made by hand and it's hard to generate project 
files for each third party module.

Were those project files generated automatically, changing between 
external modules within or outside python2x dll would be a one-line 
switch in CMakeLists.txt (or similar).
-- 
Giovanni Bajo
Develer S.r.l.
http://www.develer.com

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Evaluated cmake as an autoconf replacement

[Giovanni Bajo]

On Fri, 10 Apr 2009 11:49:04 +1000, Neil Hodgson wrote:

>This means that generated Visual Studio project files will not work
> for other people unless a particular absolute build location is
> specified for everyone which will not suit most. Each person that wants
> to build Python will have to run cmake before starting Visual Studio
> thus increasing the prerequisites.

Given that we're now stuck with using whatever Visual Studio version the 
Python maintainers decided to use, I don't see this as a problem. As in: 
there is already a far larger and invasive dependency. 

CMake is readily available on all platforms, and it can be installed in a 
couple of seconds.
-- 
Giovanni Bajo
Develer S.r.l.
http://www.develer.com

___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com