Re: Search a sequence for its minimum and stop as soon as the lowest possible value is found
Jussi Piitulainen writes: > That would return 0 even when there is no 0 in xs at all. Doesn't look that way to me: >>> minabs([5,3,1,2,4]) 1 -- https://mail.python.org/mailman/listinfo/python-list
RE: Using namedtuples field names for column indices in a list of lists
Steven D'Aprano wrote, on January 07, 2017 10:43 PM
>
> On Sunday 08 January 2017 16:39, Deborah Swanson wrote:
>
> > What I've done so far:
> >
> > with open('E:\\Coding projects\\Pycharm\\Moving\\Moving
> 2017 in.csv',
> > 'r') as infile:
> > ls = list(csv.reader(infile))
> > lst = namedtuple('lst', ls[0])
> >
> > where 'ls[0]' is the header row of the csv, and it works perfectly
> > well. 'lst' is a namedtuple instance with each of the
> column titles as
> > field names.
>
> Are you sure? namedtuple() returns a class, not a list:
Yes. 'ls' is defined as 'list(csv.reader(infile))', so ls[0] is the
first row from the csv, the header row. 'lst' is the namedtuple.
Perhaps what's puzzling you is that the way I've written it, the list of
data and the namedtuple are disjoint, and that's the problem.
> py> from collections import namedtuple
> py> names = ['A', 'B', 'C']
> py> namedtuple('lst', names)
>
>
> The way namedtuple() is intended to be used is like this:
>
>
> py> from collections import namedtuple
> py> names = ['A', 'B', 'C']
> py> Record = namedtuple('Record', names)
> py> instance = Record(10, 20, 30)
> py> print(instance)
> Record(A=10, B=20, C=30)
>
>
> There is no need to call fget directly to access the
> individual fields:
>
> py> instance.A
> 10
> py> instance.B
> 20
> py> instance[1] # indexing works too
> 20
>
>
> which is *much* simpler than:
>
> py> Record.A.fget(instance)
> 10
I don't disagree with anything you've said and shown here. But I want to
use the 'instance.A' as a subscript for the list 'ls', and the only way
to do that is with .fget(). Believe me, I tried every possible way to
use instance.A or instance[1] and no way could I get ls[instance.A].
The problem I'm having here is one of linkage between the named tuple
for the column titles and the list that holds the data in the columns.
> I think you should be doing something like this:
>
> pathname = 'E:\\Coding projects\\Pycharm\\Moving\\Moving 2017
> in.csv' with open(pathname, 'r') as infile:
> rows = list(csv.reader(infile))
> Record = namedtuple("Record", rows[0])
> for row in rows[1:]: # skip the first row, the header
> row = Record(row)
> # process this row...
> if row.location == 0:
> ...
Now here you have something I didn't think of: 'row = Record(row)' in a
loop through the rows.
> [...]
> > But I haven't found a way to assign new values to a list element.
> > using namedtuple.fieldname. I think a basic problem is that
> > namedtuples have the properties of tuples, and you can't assign to
an
> > existing tuple because they're immutable.
>
> Indeed. Being tuples, you have to create a new one. You can
> do it with slicing,
> like ordinary tuples, but that's rather clunky:
>
> py> print(instance)
> Record(A=10, B=20, C=30)
> py> Record(999, *instance[1:])
> Record(A=999, B=20, C=30)
Very clunky. I don't like modifying standard tuples with slicing, and
this is even worse.
> The recommended way is with the _replace method:
>
> py> instance._replace(A=999)
> Record(A=999, B=20, C=30)
> py> instance._replace(A=999, C=888)
> Record(A=999, B=20, C=888)
>
>
> Note that despite the leading underscore, _replace is *not* a
> private method of
> the class. It is intentionally documented as public. The
> leading underscore is
> so that it won't clash with any field names.
>
>
>
>
> --
> Steven
> "Ever since I learned about confirmation bias, I've been seeing
> it everywhere." - Jon Ronson
I will have to work with this. It's entirely possible it will do what I
want it to do. The key problem I was having was getting a linkage
between the namedtuple and the list of data from the csv.
I want to implement a suggestion I got to use a namedtuple made from the
header row as subscripts for elements in the list of data, and the
example given in the docs:
EmployeeRecord = namedtuple('EmployeeRecord', 'name, age, title,
department, paygrade')
import csv
for emp in map(EmployeeRecord._make, csv.reader(open("employees.csv",
"rb"))):
print(emp.name, emp.title)
assumes the field names will be hardcoded. Reading the csv into a list
and then trying to use the namedtuple made from the header row as
subscripts is how I ended up resorting to 'Record.A.fget(instance)' to
read values, and wasn't able to assign them.
But assigning the rows of data into namedtuple instances with:
Record = namedtuple("Record", rows[0])
for row in rows[1:]:
row = Record(row)
does look like the linkage I need and wasn't finding the way I was doing
it. If 'Record(row)' is the list data and the columns are the same as
defined in 'namedtuple("Record", rows[0])', it really should work. And I
didn't get it that _replace could be used to assign new values to
namedtuples (duh. Pretty clear now that I reread it, and all the row
data is in namedtuple instances.)
The big question is whether the namedtuple instances can be used as
something recognizable as
Re: Search a sequence for its minimum and stop as soon as the lowest possible value is found
Paul Rubin writes: > seems to work, but is ugly. Maybe there's something better. def minabs2(xs): def z(): for x in xs: yield abs(x), x if x==0: break return min(z())[1] is the same thing but a little bit nicer. -- https://mail.python.org/mailman/listinfo/python-list
Re: Search a sequence for its minimum and stop as soon as the lowest possible value is found
Paul Rubin writes: > Doesn't look that way to me: > >>> minabs([5,3,1,2,4]) > 1 There's a different problem though: >>> minabs([1,2,3,0]) 1 I think Python's version of iterators is actually buggy and at least the first element of the rest of the sequence should be preserved. There are ways to fake it but they're too messy for something like this. It should be the default and might have been a good change for Python 3. def minabs2(xs): def z(): for x in xs: yield x if x==0: break return min((abs(x),x) for x in z())[1] seems to work, but is ugly. Maybe there's something better. -- https://mail.python.org/mailman/listinfo/python-list
RE: Using namedtuples field names for column indices in a list of lists
Deborah Swanson wrote:
> to do that is with .fget(). Believe me, I tried every possible way to
> use instance.A or instance[1] and no way could I get ls[instance.A].
Sorry, no.
To get a list of namedtuple instances use:
rows = csv.reader(infile)
Record = namedtuple("Record", next(rows))
records = [Record._make(row) for row in rows]
If you want a column from a list of records you need to extract it manually:
columnA = [record.A for record in records]
--
https://mail.python.org/mailman/listinfo/python-list
Re: Search a sequence for its minimum and stop as soon as the lowest possible value is found
Paul Rubin wrote: > Paul Rubin writes: >> seems to work, but is ugly. Maybe there's something better. > > def minabs2(xs): > def z(): > for x in xs: > yield abs(x), x > if x==0: break > return min(z())[1] > > is the same thing but a little bit nicer. Yes, that's another variant of the decorate/undecorate approach combined with Wolfgang's local take_until(). The generalized form that cannot rely on the sequence items being orderable would be firstitem = operator.itemgetter(0) def stopmin_paul(items, *, key, stop): def take_until(): for item in items: k = key(item) yield k, item if k <= stop: break return min(take_until(), key=firstitem)[1] -- https://mail.python.org/mailman/listinfo/python-list
RE: Using namedtuples field names for column indices in a list of lists
Peter Otten wrote, on January 08, 2017 3:01 AM
>
> Deborah Swanson wrote:
>
> > to do that is with .fget(). Believe me, I tried every > possible way
to
> > use instance.A or instance[1] and no way could I get ls[instance.A].
>
> Sorry, no.
I quite agree, I was describing the dead end I was in from peeling the
list of data and the namedtuple from the header row off the csv
separately. That was quite obviously the wrong path to take, but I
didn't know what a good way would be.
> To get a list of namedtuple instances use:
>
> rows = csv.reader(infile)
> Record = namedtuple("Record", next(rows))
> records = [Record._make(row) for row in rows]
This is slightly different from Steven's suggestion, and it makes a
block of records that I think would be iterable. At any rate all the
data from the csv would belong to a single data structure, and that
seems inherently a good thing.
a = records[i].A , for example
And I think that this would produce recognizable field names in my code
(which was the original goal) if the following works:
records[0] is the header row == ('Description', 'Location', etc.)
If I can use records[i].Location for the Location column data in row
'i', then I've got my recognizable-field-name variables.
> If you want a column from a list of records you need to
> extract it manually:
>
> columnA = [record.A for record in records]
This is very neat. Something like a list comprehension for named tuples?
Thanks Peter, I'll try it all tomorrow and see how it goes.
PS. I haven't forgotten your defaultdict suggestion, I'm just taking the
suggestions I got in the "Cleaning up Conditionals" thread one at a
time, and I will get to defaultdict. Then I'll look at all of them and
see what final version of the code will work best with all the factors
to consider.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Search a sequence for its minimum and stop as soon as the lowest possible value is found
Paul Rubin writes: > Jussi Piitulainen writes: >> That would return 0 even when there is no 0 in xs at all. > > Doesn't look that way to me: > > >>> minabs([5,3,1,2,4]) > 1 Sorry about that. I honestly meant to say it would return 1 even when there was a single 0 at the very end. Somehow I got seriously confused. You noticed the actual problem yourself. -- https://mail.python.org/mailman/listinfo/python-list
Re: Search a sequence for its minimum and stop as soon as the lowest possible value is found
Paul Rubin writes: > I think Python's version of iterators is actually buggy and at least > the first element of the rest of the sequence should be preserved. > There are ways to fake it but they're too messy for something like > this. It should be the default and might have been a good change for > Python 3. It could still be added as an option, to both takewhile and iter(_, _). -- https://mail.python.org/mailman/listinfo/python-list
RE: Using namedtuples field names for column indices in a list of lists
Deborah Swanson wrote:
> Peter Otten wrote, on January 08, 2017 3:01 AM
>>
>> Deborah Swanson wrote:
>>
>> > to do that is with .fget(). Believe me, I tried every > possible way
> to
>> > use instance.A or instance[1] and no way could I get ls[instance.A].
>>
>> Sorry, no.
>
> I quite agree, I was describing the dead end I was in from peeling the
> list of data and the namedtuple from the header row off the csv
> separately. That was quite obviously the wrong path to take, but I
> didn't know what a good way would be.
>
>> To get a list of namedtuple instances use:
>>
>> rows = csv.reader(infile)
>> Record = namedtuple("Record", next(rows))
>> records = [Record._make(row) for row in rows]
>
> This is slightly different from Steven's suggestion, and it makes a
> block of records that I think would be iterable. At any rate all the
> data from the csv would belong to a single data structure, and that
> seems inherently a good thing.
>
> a = records[i].A , for example
>
> And I think that this would produce recognizable field names in my code
> (which was the original goal) if the following works:
>
> records[0] is the header row == ('Description', 'Location', etc.)
Personally I would recommend against mixing data (an actual location) and
metadata (the column name,"Location"), but if you wish my code can be
adapted as follows:
infile = open("dictreader_demo.csv")
rows = csv.reader(infile)
fieldnames = next(rows)
Record = namedtuple("Record", fieldnames)
records = [Record._make(fieldnames)]
records.extend(Record._make(row) for row in rows)
If you want a lot of flexibility without doing the legwork yourself you
might also have a look at pandas. Example session:
$ cat places.csv
Location,Description,Size
here,something,17
there,something else,10
$ python3
Python 3.4.3 (default, Nov 17 2016, 01:08:31)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pandas
>>> places = pandas.read_csv("places.csv")
>>> places
Location Description Size
0 here something17
1there something else10
[2 rows x 3 columns]
>>> places.Location
0 here
1there
Name: Location, dtype: object
>>> places.sort(columns="Size")
Location Description Size
1there something else10
0 here something17
[2 rows x 3 columns]
>>> places.Size.mean()
13.5
Be aware that there is a learning curve...
> If I can use records[i].Location for the Location column data in row
> 'i', then I've got my recognizable-field-name variables.
>
>> If you want a column from a list of records you need to
>> extract it manually:
>>
>> columnA = [record.A for record in records]
>
> This is very neat. Something like a list comprehension for named tuples?
>
> Thanks Peter, I'll try it all tomorrow and see how it goes.
>
> PS. I haven't forgotten your defaultdict suggestion, I'm just taking the
> suggestions I got in the "Cleaning up Conditionals" thread one at a
> time, and I will get to defaultdict. Then I'll look at all of them and
> see what final version of the code will work best with all the factors
> to consider.
--
https://mail.python.org/mailman/listinfo/python-list
RE: Using namedtuples field names for column indices in a list of lists
Peter Otten wrote, on January 08, 2017 5:21 AM
>
> Deborah Swanson wrote:
>
> > Peter Otten wrote, on January 08, 2017 3:01 AM
> >>
> >> Deborah Swanson wrote:
> >>
> >> > to do that is with .fget(). Believe me, I tried every > possible
> >> > way
> > to
> >> > use instance.A or instance[1] and no way could I get
> >> > ls[instance.A].
> >>
> >> Sorry, no.
> >
> > I quite agree, I was describing the dead end I was in from
> peeling the
> > list of data and the namedtuple from the header row off the csv
> > separately. That was quite obviously the wrong path to take, but I
> > didn't know what a good way would be.
> >
> >> To get a list of namedtuple instances use:
> >>
> >> rows = csv.reader(infile)
> >> Record = namedtuple("Record", next(rows))
> >> records = [Record._make(row) for row in rows]
> >
> > This is slightly different from Steven's suggestion, and it makes a
> > block of records that I think would be iterable. At any
> rate all the
> > data from the csv would belong to a single data structure, and that
> > seems inherently a good thing.
> >
> > a = records[i].A , for example
> >
> > And I think that this would produce recognizable field names in my
> > code (which was the original goal) if the following works:
> >
> > records[0] is the header row == ('Description', 'Location', etc.)
>
> Personally I would recommend against mixing data (an actual
> location) and
> metadata (the column name,"Location"), but if you wish my code can be
> adapted as follows:
>
> infile = open("dictreader_demo.csv")
> rows = csv.reader(infile)
> fieldnames = next(rows)
> Record = namedtuple("Record", fieldnames)
> records = [Record._make(fieldnames)]
> records.extend(Record._make(row) for row in rows)
Peter, this looks really good, and yes, I didn't feel so good about
records[i].Location either, but it was the only way I could see to get
the recognizable variable names I want. By extending records from a
namedtuple of field names, I think it can be done cleanly. I'll try it
and see.
> If you want a lot of flexibility without doing the legwork
> yourself you
> might also have a look at pandas. Example session:
>
> $ cat places.csv
> Location,Description,Size
> here,something,17
> there,something else,10
> $ python3
> Python 3.4.3 (default, Nov 17 2016, 01:08:31)
> [GCC 4.8.4] on linux
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import pandas
> >>> places = pandas.read_csv("places.csv")
> >>> places
> Location Description Size
> 0 here something17
> 1there something else10
>
> [2 rows x 3 columns]
> >>> places.Location
> 0 here
> 1there
> Name: Location, dtype: object
> >>> places.sort(columns="Size")
> Location Description Size
> 1there something else10
> 0 here something17
>
> [2 rows x 3 columns]
> >>> places.Size.mean()
> 13.5
>
> Be aware that there is a learning curve...
Yes, and I'm sure the learning curve is steep. I watched a webinar on
pandas about a year ago, not to actually learn it, but just to take in
the big picture and see something people were really accomplishing with
python. I won't take this on any time right away, but I'll definitely
keep it and work with it sometime. Maybe as just an intro to pandas,
using my data from the real estate project.
> > If I can use records[i].Location for the Location column
> data in row
> > 'i', then I've got my recognizable-field-name variables.
> >
> >> If you want a column from a list of records you need to extract it
> >> manually:
> >>
> >> columnA = [record.A for record in records]
> >
> > This is very neat. Something like a list comprehension for named
> > tuples?
> >
> > Thanks Peter, I'll try it all tomorrow and see how it goes.
> >
> > PS. I haven't forgotten your defaultdict suggestion, I'm
> just taking
> > the suggestions I got in the "Cleaning up Conditionals"
> thread one at
> > a time, and I will get to defaultdict. Then I'll look at
> all of them
> > and see what final version of the code will work best with all the
> > factors to consider.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Using namedtuples field names for column indices in a list of lists
"Deborah Swanson" writes: > Peter Otten wrote, on January 08, 2017 3:01 AM >> >> columnA = [record.A for record in records] > > This is very neat. Something like a list comprehension for named tuples? Not something like - this *is* a list comprehension - it creates a list of named tuples. The thing you iterate over within the comprehension can be any iterator. (Of course you're going to run into problems if you try to construct a list from an infinite iterator.) -- https://mail.python.org/mailman/listinfo/python-list
Re: Search a sequence for its minimum and stop as soon as the lowest possible value is found
Peter Otten <__pete...@web.de> writes: > return min(take_until(), key=firstitem)[1] Actually, key=abs should work. I realized that after posting. -- https://mail.python.org/mailman/listinfo/python-list
Re: Search a sequence for its minimum and stop as soon as the lowest possible value is found
Jussi Piitulainen writes: > It could still be added as an option, to both takewhile and iter(_, _). That's too messy, it really should be pervasive in iterators. -- https://mail.python.org/mailman/listinfo/python-list
RE: Using namedtuples field names for column indices in a list of lists
Paul Rudin wrote, on January 08, 2017 6:49 AM > > "Deborah Swanson" writes: > > > Peter Otten wrote, on January 08, 2017 3:01 AM > >> > >> columnA = [record.A for record in records] > > > > This is very neat. Something like a list comprehension for named > > tuples? > > Not something like - this *is* a list comprehension - it > creates a list of named tuples. > > The thing you iterate over within the comprehension can be > any iterator. (Of course you're going to run into problems if > you try to construct a list from an infinite iterator.) Thanks Paul. I've been meaning to spend some time getting to thoroughly know list comprehensions for awhile now, but I keep running into so many new things I just haven't gotten to it. I thought it looked like one, but I hedged my wording because I wasn't sure. Infinite iterators definitely sound like something to remember! -- https://mail.python.org/mailman/listinfo/python-list
Re: Using sudo with pip3?
On Samstag, 7. Januar 2017 19:07:55 Clint Moyer wrote: > I would lightly advise against, assuming both Pip and your package > manager are trying to accomplish nearly the same thing. Stick with > updating through the repo. > > If you find that the version your OS provides is out-of-date compared > to what's on PyPi or Github, then you might want to remove from your > OS and re-install through Pip, for those discrete cases. That's the > platform agnostic route. Or take the ninja way, I do: Build all packages, you want (updated) for your distribution: https://build.opensuse.org/project/monitor/home:frispete:python https://build.opensuse.org/project/monitor/home:frispete:python3 The red labels are reminders... SCR, Pete -- https://mail.python.org/mailman/listinfo/python-list
RE: Grumpy: Python to Go compiler
Steven D'Aprano wrote, on January 07, 2017 11:37 PM > > Grumpy, an experimental project from Google, transpiles > Python code into Go, allowing Python programs to be compiled > and run as static binaries using the Go toolchain. > > > http://www.infoworld.com/article/3154624/application-development/google- boosts-python-by-turning-it-into-go.html > > >-- >Steven >"Ever since I learned about confirmation bias, I've been seeing >it everywhere." - Jon Ronson Interesting online resource, if you can stomach contact with Google. I'll spare you the diatribe of all my beefs with Google (haha, unless you ask). But this is a potentially useful tool to know about. -- https://mail.python.org/mailman/listinfo/python-list
Re: Grumpy: Python to Go compiler
On 01/08/2017 06:18 PM, Deborah Swanson wrote: > (haha, unless > you ask) C'mon, go for it ... there hasn't been a good rant here in 4 or 5 minutes ... -- https://mail.python.org/mailman/listinfo/python-list
Re: Search a sequence for its minimum and stop as soon as the lowest possible value is found
On Friday, January 6, 2017 at 6:04:33 AM UTC-8, Peter Otten wrote: > Example: you are looking for the minimum absolute value in a series of > integers. As soon as you encounter the first 0 it's unnecessary extra work > to check the remaining values, but the builtin min() will continue. > > The solution is a minimum function that allows the user to specify a stop > value: > > >>> from itertools import count, chain > >>> stopmin(chain(reversed(range(10)), count()), key=abs, stop=0) > 0 > > How would you implement stopmin()? > > Currently I raise an exception in the key function: > > class Stop(Exception): > pass > > def stopmin(items, key, stop): > """ > >>> def g(): > ... for i in reversed(range(10)): > ... print(10*i) > ... yield str(i) > >>> stopmin(g(), key=int, stop=5) > 90 > 80 > 70 > 60 > 50 > '5' > """ > def key2(value): > result = key(value) > if result <= stop: > raise Stop(value) > return result > try: > return min(items, key=key2) > except Stop as stop: > return stop.args[0] This is the simplest version I could come up with. I also like the classic 100% imperative, but it seems that is not trendy between the solutions given :D. you can test it here https://repl.it/FD5A/0 source code: from itertools import accumulate # stopmin calculates the greatest lower bound (infimum). # https://upload.wikimedia.org/wikipedia/commons/0/0a/Infimum_illustration.svg def takeuntil(pred, seq): for item in seq: yield item if not pred(item): break def stopmin(seq, stop=0): drop_ltstop = (item for item in seq if item >= stop) min_gen = (min_ for min_ in accumulate(drop_ltstop, func=min)) return list(takeuntil(lambda x: x!= stop, min_gen))[-1] seq = [1, 4, 7, -8, 0, 7, -8, 9] # 0 just until zero is generated seq = [1, 4, 7, -8, 7, -8, 9] # 1 the entire sequence is generated print(stopmin(seq, stop=0)) -- https://mail.python.org/mailman/listinfo/python-list
RE: Grumpy: Python to Go compiler
Tim Daneliuk wrote, on January 08, 2017 4:49 PM > > On 01/08/2017 06:18 PM, Deborah Swanson wrote: > > (haha, unless > > you ask) > > C'mon, go for it ... there hasn't been a good rant here in > 4 or 5 minutes ... Oh hell. (How do I tell him I was up til 8am this morning, only got a few hours sleep, and I'm just too pookie now to deliver a decent rant on all the ways Google is truly evil. Don't have to, just did.) Besides, I'm rewriting code this afternoon and really want to see how much better it is when I'm done. -- https://mail.python.org/mailman/listinfo/python-list
RE: Using namedtuples field names for column indices in a list of lists
On Sunday 08 January 2017 20:53, Deborah Swanson wrote:
> Steven D'Aprano wrote, on January 07, 2017 10:43 PM
>>
>> On Sunday 08 January 2017 16:39, Deborah Swanson wrote:
>>
>> > What I've done so far:
>> >
>> > with open('E:\\Coding projects\\Pycharm\\Moving\\Moving
>> 2017 in.csv',
>> > 'r') as infile:
>> > ls = list(csv.reader(infile))
>> > lst = namedtuple('lst', ls[0])
>> >
>> > where 'ls[0]' is the header row of the csv, and it works perfectly
>> > well. 'lst' is a namedtuple instance with each of the
>> column titles as
>> > field names.
>>
>> Are you sure? namedtuple() returns a class, not a list:
>
> Yes. 'ls' is defined as 'list(csv.reader(infile))', so ls[0] is the
> first row from the csv, the header row. 'lst' is the namedtuple.
>
> Perhaps what's puzzling you is that the way I've written it, the list of
> data and the namedtuple are disjoint, and that's the problem.
No, I'm pretty sure that's not the case. I don't have access to your CSV file,
but I can simulate it:
ls = [['Location', 'Date', 'Price'],
['here', '1/1/17', '1234'],
['there', '1/1/17', '5678'],
['everywhere', '1/1/17', '9821']
]
from collections import namedtuple
lst = namedtuple('lst', ls[0])
print(type(lst))
print(lst)
If you run that code, you should see:
which contradicts your statement:
'lst' is a namedtuple instance with each of the column
titles as field names.
and explains why you had to access the individual property method `fget`: you
were accessing the *class object* rather than an actual named tuple instance.
The code you gave was:
lst.Location.fget(l)
where l was not given, but I can guess it was a row of the CSV file, i.e. an
individual record. So:
- lst was the named tuple class, a subclass of tuple
- lst.Location returns a property object
- lst.Location.fget is the internal fget method of the property object.
I think Peter Otten has the right idea: create a list of records with something
like this:
Record = namedtuple('Record', ls[0])
data = [Record(*row) for row in ls[1:])
or if you prefer Peter's version:
data = [Record._make(row) for row in ls[1:])
Half the battle is coming up with the right data structures :-)
--
Steven
"Ever since I learned about confirmation bias, I've been seeing
it everywhere." - Jon Ronson
--
https://mail.python.org/mailman/listinfo/python-list
Temporary variables in list comprehensions
Suppose you have an expensive calculation that gets used two or more times in a loop. The obvious way to avoid calculating it twice in an ordinary loop is with a temporary variable: result = [] for x in data: tmp = expensive_calculation(x) result.append((tmp, tmp+1)) But what if you are using a list comprehension? Alas, list comps don't let you have temporary variables, so you have to write this: [(expensive_calculation(x), expensive_calculation(x) + 1) for x in data] Or do you? ... no, you don't! [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]] I can't decide whether that's an awesome trick or a horrible hack... -- Steven "Ever since I learned about confirmation bias, I've been seeing it everywhere." - Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
Re: Temporary variables in list comprehensions
On Mon, Jan 9, 2017 at 2:53 PM, Steven D'Aprano wrote: > [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]] > > > I can't decide whether that's an awesome trick or a horrible hack... A horrible hack on par with abusing a recursive function's arguments for private variables. Much better would be to refactor the append part: def this_and_one(value): return value, value + 1 [this_and_one(expensive_calculation(x)) for x in data] ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Temporary variables in list comprehensions
On Monday 09 January 2017 15:09, Chris Angelico wrote: > On Mon, Jan 9, 2017 at 2:53 PM, Steven D'Aprano > wrote: >> [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]] >> >> >> I can't decide whether that's an awesome trick or a horrible hack... > > A horrible hack on par with abusing a recursive function's arguments > for private variables. What's wrong with that? That's a perfectly legitimate technique. I prefer to hide it behind a private function rather than expose it in a public interface: # instead of this def recursive(arg, _internal=None): """Recursive function. Don't supply the _internal argument.""" ... return recursive(arg-1, spam) # I usually prefer this def recursive(arg): """Recursive function.""" return _recursive(arg, spam) def _recursive(arg, internal): ... but that's just polishing the code. > Much better would be to refactor the append > part: > > def this_and_one(value): > return value, value + 1 I wouldn't call that "much better". Requiring an extra function defeats the purpose of using a list comp, and it doesn't scale well for multiple list comps each of which needs a different helper function: def this_and_one(value): return value, value + 1 def this_less_one_and_this_plus_one_and_this(value): return value - 1, value + 1, value def this_and_that_or_something(value): return value, value.that() or something def extract_value(value): return spam[value] or ham[value] or eggs[value] and so on. Helper functions are good. Helper functions that are only used *once* are a code smell. *LOTS* of helper functions that are only used once are a sign that something is horrible, and it might just be your language... -- Steven "Ever since I learned about confirmation bias, I've been seeing it everywhere." - Jon Ronson -- https://mail.python.org/mailman/listinfo/python-list
Re: Temporary variables in list comprehensions
On Mon, Jan 9, 2017 at 3:49 PM, Steven D'Aprano wrote: > Helper functions are good. Helper functions that are only used > *once* are a code smell. *LOTS* of helper functions that are only used once > are > a sign that something is horrible, and it might just be your language... Agreed, but with a toy example like you posted, it's impossible to say which is happening :) ChrisA -- https://mail.python.org/mailman/listinfo/python-list
Re: Temporary variables in list comprehensions
On Monday, January 9, 2017 at 10:19:31 AM UTC+5:30, Steven D'Aprano wrote: > On Monday 09 January 2017 15:09, Chris Angelico wrote: > > > On Mon, Jan 9, 2017 at 2:53 PM, Steven D'Aprano wrote: > >> [(tmp, tmp + 1) for x in data for tmp in [expensive_calculation(x)]] > >> > >> > >> I can't decide whether that's an awesome trick or a horrible hack... > > > > A horrible hack on par with abusing a recursive function's arguments > > for private variables. > > What's wrong with that? That's a perfectly legitimate technique. I prefer to > hide it behind a private function rather than expose it in a public interface: > > # instead of this > def recursive(arg, _internal=None): > """Recursive function. Don't supply the _internal argument.""" > ... > return recursive(arg-1, spam) > > > # I usually prefer this > def recursive(arg): > """Recursive function.""" > return _recursive(arg, spam) > > def _recursive(arg, internal): > ... > > > but that's just polishing the code. > > > > > Much better would be to refactor the append > > part: > > > > def this_and_one(value): > > return value, value + 1 > > I wouldn't call that "much better". Requiring an extra function defeats the > purpose of using a list comp, and it doesn't scale well for multiple list > comps +1 Id call it (your original) neither an awesome trick nor a horrible hack — just a neat workaround for an obvious lacuna As for Chris' solution there is a simple test to see whether it's worth it: Does the function name make sense? If yes then the subexpression refactored- into-a-function is probably fine. If not the kludgyness shows [Im assuming the real usage is something else and your question is a highly sscce-ed version] -- https://mail.python.org/mailman/listinfo/python-list
Re: Python for WEB-page !?
On 1/5/2017 7:48 PM, Michael Torrie wrote: > While Python can do that, using a web framework to process HTTP requests > and generate HTML to display in the browser, I don't believe Python is > the appropriate language for the task at hand. Most web sites that do > interactive formula calculations like you describe do it all in the > browser using Javascript. No need to have a web server do all that > heavy lifting at all. A simple html file would contain everything you need. > > Even if you want to use Python to generate the web page and process > events, you'll still have to master Javascript at some point to make the > webpages more interactive. There are a few implementations of Python that generate code that can run the browser. I'm currently using brython and its significantly easier than struggling with JavaScript. It's not perfect by any means, it has its own quirks and bugs but the developer is very responsive, the product is getting better every month and I like to see the Python community give it a bit more love -- https://mail.python.org/mailman/listinfo/python-list
RE: Using namedtuples field names for column indices in a list of lists
Steven D'Aprano wrote, on January 07, 2017 10:43 PM > > On Sunday 08 January 2017 16:39, Deborah Swanson wrote: > > The recommended way is with the _replace method: > > py> instance._replace(A=999) > Record(A=999, B=20, C=30) > py> instance._replace(A=999, C=888) > Record(A=999, B=20, C=888) > > -- > Steven > "Ever since I learned about confirmation bias, I've been seeing > it everywhere." - Jon Ronson instance._replace(A=999) works perfectly, and editting my existing assignment statements was really easy. Thanks - a lot. -- https://mail.python.org/mailman/listinfo/python-list
RE: Using namedtuples field names for column indices in a list of lists
Peter Otten wrote, on January 08, 2017 5:21 AM
>
> Deborah Swanson wrote:
>
> > Peter Otten wrote, on January 08, 2017 3:01 AM
>
> Personally I would recommend against mixing data (an actual location)
and
> metadata (the column name,"Location"), but if you wish my code can be
> adapted as follows:
>
> infile = open("dictreader_demo.csv")
> rows = csv.reader(infile)
> fieldnames = next(rows)
> Record = namedtuple("Record", fieldnames)
> records = [Record._make(fieldnames)]
> records.extend(Record._make(row) for row in rows)
Works like a charm. I stumbled a bit changing all my subscripted
variables to namedtuples and rewriting the inevitable places my code
that didn't work the same. But actually it was fun, especially deleting
all the sections and variables I no longer needed. And it executes
correctly now too - with recognizable fieldnames instead of my quirky
2-letter code subscripts. All in all a huge win!
I do have two more questions.
1) I have a section that loops through the sorted data, compares two
adjacent rows at a time, and marks one of them for deletion if the rows
are identical.
I'm using
for i in range(len(records)-1):
r1 = records[i]
r2 = records[i+1]
if r1.xx = r2.xx:
.
.
and my question is whether there's a way to work with two adjacent rows
without using subscripts?
Even better, to get hold of all the records with the same Description as
the current row, compare them all, mark all but the different ones for
deletion, and then resume processing the records after the last one?
2) I'm using mergesort. (I didn't see any way to sort a namedtuple in
the docs.) In the list version of my code I copied and inserted the 2
columns I wanted to sort by into the beginning of the list, and then
deleted them after the list was sorted. But just looking at records, I'm
not so sure that can easily be done. I remember your code to work with
columns of the data:
columnA = [record.A for record in records]
and I can see how that would get me columnA and columnB, but then is
there any better way to insert and delete columns in an existing
namedtuple than slicing? And I don't think you can insert or delete a
whole column while slicing.
Or maybe my entire approach is not the best. I know it's possible to do
keyed sorts, but I haven't actually written or used any. So I just
pulled a mergesort off the shelf and got what I wanted by inserting
copies of those 2 columns at the front, and then deleting them when the
sort was complete. Not exactly elegant, but it works.
Any suggestions would be most welcome.
--
https://mail.python.org/mailman/listinfo/python-list
RE: Using namedtuples field names for column indices in a list of lists
Steven D'Aprano wrote, on January 08, 2017 7:30 PM
>
> On Sunday 08 January 2017 20:53, Deborah Swanson wrote:
>
> > Steven D'Aprano wrote, on January 07, 2017 10:43 PM
>
> No, I'm pretty sure that's not the case. I don't have access
> to your CSV file,
> but I can simulate it:
>
> ls = [['Location', 'Date', 'Price'],
> ['here', '1/1/17', '1234'],
> ['there', '1/1/17', '5678'],
> ['everywhere', '1/1/17', '9821']
> ]
>
> from collections import namedtuple
> lst = namedtuple('lst', ls[0])
>
> print(type(lst))
> print(lst)
>
>
>
> If you run that code, you should see:
>
>
>
>
>
> which contradicts your statement:
>
> 'lst' is a namedtuple instance with each of the column
> titles as field names.
Yes, yes. In a careless moment I called a class an instance.
> and explains why you had to access the individual property
> method `fget`: you
> were accessing the *class object* rather than an actual named
> tuple instance.
That code is deleted and long gone now, so I can't look at it in the
debugger, but yes, I'm pretty sure 'fget' is a class member.
> The code you gave was:
>
> lst.Location.fget(l)
>
> where l was not given, but I can guess it was a row of the
> CSV file, i.e. an
> individual record. So:
>
> - lst was the named tuple class, a subclass of tuple
>
> - lst.Location returns a property object
>
> - lst.Location.fget is the internal fget method of the
> property object.
And your point is? Perhaps I didn't express myself in a way that you
could recognize, but I understood all of that before I wrote to you, and
attempted to convey that understanding to you. Obviously I failed, if
you now think I need a lesson in what's going on here.
> I think Peter Otten has the right idea: create a list of
> records with something
> like this:
>
>
> Record = namedtuple('Record', ls[0])
> data = [Record(*row) for row in ls[1:])
>
>
> or if you prefer Peter's version:
>
> data = [Record._make(row) for row in ls[1:])
>
>
> Half the battle is coming up with the right data structures :-)
Can't and wouldn't disagree with any part of that!
> --
> Steven
> "Ever since I learned about confirmation bias, I've been seeing
> it everywhere." - Jon Ronson
--
https://mail.python.org/mailman/listinfo/python-list
