DFS wrote:
> getAddresses.py
>
> Scrapes addresses from www.usdirectory.com and stores them in a SQLite
> database, or writes them to text files for mailing labels, etc
>
> Now, just by typing 'fast food Taco Bell <city> 10 db all' you can find
> out how many Taco Bells are within 10 miles of you, and store all the
> addresses in your own address database.
>
> No more convoluted Googling, or hitting the 'Next Page' button, or
> fumbling with the Yellow Pages...
>
> Note: the db structure is flat on purpose, and the .csv files aren't
> quote delimited.
>
> Put the program in its own directory. It creates the SQLite database
> there, and writes files there, too.
>
> Reviews of code, bug reports, criticisms, suggestions for improvement,
> etc are all welcome.
- Avoid module-level code and global variables
- Use functions that do one thing and operate on explicitly passed arguments
- You have
if store == ...:
...
sprinkled across your module. You will have to change your code in many
places if you want to add another output format. With a linear structure
like
STORE_FUNCS = {
"db": store_in_db,
"txt": store_as_text,
"csv": store_as_csv,
}
def main():
args = read_arguments()
records = read_records(args)
records = unique(records)
if args.limit:
records = itertools.islice(records, args.limit)
STORE_FUNCS[args.storage_format](args, records)
if __name__ == "__main__":
main()
further enhancements will be a lot easier to implement.
The main() function avoids accidental uncontrolled globals. If you want one
you have to declare it:
def main():
global verbose
args = read_arguments()
verbose = args.verbose
...
--
https://mail.python.org/mailman/listinfo/python-list
