[BangPypers] python for parsing

[Rahul R]

Hello everyone ,
I am new to this community . recently i attended a python workshop in my
college and got a lot mesmerised by the language so finally decided to port
my mini project which i have partially written using LEX and Yacc into
python. i would be glad if you could provide me some valuable suggestions on
how to go about doing so.


Thanking You
Rahul
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] python for parsing

[Rahul R]

On Mon, May 24, 2010 at 12:00 AM, Dhananjay Nene
wrote:

> On Mon, May 24, 2010 at 12:51 AM, Rahul R  wrote:
>
> > Hello everyone ,
> > I am new to this community . recently i attended a python workshop in my
> > college and got a lot mesmerised by the language so finally decided to
> port
> > my mini project which i have partially written using LEX and Yacc into
> > python. i would be glad if you could provide me some valuable suggestions
> > on
> > how to go about doing so.
> >
> > I had once found pyparsing http://pyparsing.wikispaces.com/ quite nice
> though I used it in a casual experiment.
>
> I had documented my usage here :
>
> http://codeblog.dhananjaynene.com/2010/01/extracting-data-using-recursive-descent-parsing/which
> hopefully should indicate how its usage is in many ways far simpler
> than traditional lex/yacc one.
>
> Dhananjay
>
>
>
Thanks   Dhananjay & Noufal , now i have tons of options. But then again i
am trying to find the most suitable one for my program.
Well basically what i am trying to achieve is to create my own syntax
(reasonably simple ) which will help the user to write simple HTML ( +CSS
hopefully) pages without actually coding any of it .

i.e for example

if the user writes

this is heading
  -->  { the '=' would be a token for headers }

the rest paragraph


the equivalent html generated could be

  this is heading  
 the rest paragraph 

besides i am also trying to incorporate inline CSS and other basic
functualities .
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] python for parsing

[Rahul R]

Thanks Kunal , i am happy as well as sad  , happy because my job has become
much simpler and sad because everytime i think of a novel (honeslty ) idea
.. i see someone else has already executed it .. :(

On Mon, May 24, 2010 at 5:02 AM, kunal ghosh  wrote:

> Hi, Rahul
> It looks like you are trying to create an abstraction above the HTML+JS
> etc.
> Have you looked at pyjamas pyjs.org ?
> Though GWT uses java for the same thing.
>
> On Mon, May 24, 2010 at 2:02 AM, Rahul R  wrote:
>
> > On Mon, May 24, 2010 at 12:00 AM, Dhananjay Nene
> > wrote:
> >
> > > On Mon, May 24, 2010 at 12:51 AM, Rahul R  wrote:
> > >
> > > > Hello everyone ,
> > > > I am new to this community . recently i attended a python workshop in
> > my
> > > > college and got a lot mesmerised by the language so finally decided
> to
> > > port
> > > > my mini project which i have partially written using LEX and Yacc
> into
> > > > python. i would be glad if you could provide me some valuable
> > suggestions
> > > > on
> > > > how to go about doing so.
> > > >
> > > > I had once found pyparsing http://pyparsing.wikispaces.com/ quite
> nice
> > > though I used it in a casual experiment.
> > >
> > > I had documented my usage here :
> > >
> > >
> >
> http://codeblog.dhananjaynene.com/2010/01/extracting-data-using-recursive-descent-parsing/which
> > > hopefully should indicate how its usage is in many ways far simpler
> > > than traditional lex/yacc one.
> > >
> > > Dhananjay
> > >
> > >
> > >
> > Thanks   Dhananjay & Noufal , now i have tons of options. But then again
> i
> > am trying to find the most suitable one for my program.
> > Well basically what i am trying to achieve is to create my own syntax
> > (reasonably simple ) which will help the user to write simple HTML ( +CSS
> > hopefully) pages without actually coding any of it .
> >
> > i.e for example
> >
> > if the user writes
> >
> > this is heading
> >   -->  { the '=' would be a token for headers }
> >
> > the rest paragraph
> >
> >
> > the equivalent html generated could be
> >
> >   this is heading  
> >  the rest paragraph 
> >
> > besides i am also trying to incorporate inline CSS and other basic
> > functualities .
> > ___
> > BangPypers mailing list
> > BangPypers@python.org
> > http://mail.python.org/mailman/listinfo/bangpypers
> >
>
>
>
> --
> regards
> ---
> Kunal Ghosh
> Dept of Computer Sc. & Engineering.
> Sir MVIT
> Bangalore,India
>
> Blog:kunalghosh.wordpress.com
> Website:www.kunalghosh.net46.net
> ___
> BangPypers mailing list
> BangPypers@python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] Tcp/ip server client program

[Rahul R]

welll i did try a client server program in TCP/IP , from

http://www.jroller.com/RickHigh/entry/notes_on_creating_a_socket

this tutorial . It helped me a lot , i hope it would help u too.

On Fri, May 28, 2010 at 7:26 AM, Senthil Kumaran wrote:

> On Thu, May 27, 2010 at 11:46:36PM +0900, murugadoss wrote:
> > I am trying out client/server program in python using both udp and
> tcp/ip.
> > UDP is working,but for tcp/ip program, i get an error as
>
> Please share the snippet of client and server code.
>
> > KeyboardInterrupt
>
> means you pressed CNTL+C or something else sent a SIGINT signal.
>
> --
> Senthil
>
>  that's a Kludge(TM)
>  It Works(tm)
>  AIX works(TM)
>  no it doesn't
>  =>
> ___
> BangPypers mailing list
> BangPypers@python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] 2D plotting libraries.

[Rahul R]

On Fri, Jun 4, 2010 at 5:22 PM, Noufal Ibrahim  wrote:

> On Fri, Jun 4, 2010 at 6:49 PM, Shiv Shankar  wrote:
>
> >  The whole ode thing I've prepared for the
> >> pycon in Singapore is an attempt.
>
> http://github.com/nibrahim/Devious-machinations
>
> Let me see how well it's received. If it's good, I'll present it here
> at the local BangPypers meetings if there's sufficient interest.
> It's a simple game that ties up a physics engine (ODE) and a graphics
> library (PyGame). A "clone" of the famous TIM
> (http://en.wikipedia.org/wiki/The_Incredible_Machine)
>
> --
> ~noufal
> http://nibrahim.net.in
> _
>


I would love to  see that . besides there is one more game engine called
panda3D  built by carniege mellon univ students , i
liked it .



> __
> BangPypers mailing list
> BangPypers@python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

[BangPypers] python with c bindings

[Rahul R]

well recently i learned about handling shell scripts inside c files . well
i  was wondering since python is also scripting language can i bind the
python script inside a "C" file and then run it.well on the front it may
look like normal compilation of c program, but in the backend i guess the
python script is being executed.
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] python with c bindings

[Rahul R]

On Tue, Jun 15, 2010 at 11:24 AM, kunal ghosh  wrote:

> On Tue, Jun 15, 2010 at 12:45 PM, Rahul R  wrote:
>
> > well recently i learned about handling shell scripts inside c files .
> well
> > i  was wondering since python is also scripting language can i bind the
> > python script inside a "C" file and then run it.well on the front it may
> > look like normal compilation of c program, but in the backend i guess the
> > python script is being executed.
> >
>
> you can #include
> in your c program and then call the python modules in a python script
> as explained here http://www.linuxjournal.com/article/8497
>


i rather prefer running it as a script binding into a "C" (if thats possible
, well that certainly works for sh scripts i dont know whether that works
for python )
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] python with c bindings

[Rahul R]

On Tue, Jun 15, 2010 at 6:03 PM, Noufal Ibrahim  wrote:

> On Tue, Jun 15, 2010 at 9:11 PM, Rahul R  wrote:
> [..]
> > i rather prefer running it as a script binding into a "C" (if thats
> possible
> > , well that certainly works for sh scripts i dont know whether that works
> > for python )
>
> I'm not sure what you mean by "binding". But is possible to embed a
> Python interpreter into your compiled C application so that it can
> execute scripts. It's often used to provide "scriptability".
>
>
well thats wat exactly i wanted .. to embed the python interpreter into C
app .
Can u throw some light on it .. on how to go about doing so.

I apologise for not being articulate since , i did know the right jargon to
express it.
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] June meet up

[Rahul R]

On Thu, Jun 24, 2010 at 10:49 AM, Noufal Ibrahim  wrote:

> On Tue, Jun 22, 2010 at 11:13 AM, Abhishek Mishra 
> wrote:
> > +1 for the apac talk, I and a friend would definitely be interested.
>
> That's 3 people including me. Anyone else?
>
> What about the venue? We could ask TW but I'd prefer doing it
> somewhere else this time.
>
> Suggestions/ideas?
>
>
well i an newbie and have recently joined the community , can i come too ?
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

[BangPypers] python program to fetch input data in complete bit format

[Rahul R]

i was wondering if there is a way to fetch the raw data of any file i input
for example
if i accept a text file or a video file (particualr large video files ). i
could  fetch the the equivalent bits and store it in another file






---
Rahul R
Under Grad Student
Computer Science
Sir MVIT
Bangalore

Email : rahul8...@gmail.com
Site : www.rahulr.110mb.com
Blog : <http://www.rahulr.blog.co.uk>www.rahulr.blog.co.uk
---

Google Talk: rahul8...@gmail.com
[image: Linkedin] <http://www.linkedin.com/in/rahul8590>[image:
MySpace]<http://www.myspace.com/rahul8590>[image:
Twitter] 
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] python program to fetch input data in complete bit format

[Rahul R]

On Thu, Jul 8, 2010 at 1:24 AM, Noufal Ibrahim  wrote:

> Rahul R  writes:
>
> > i was wondering if there is a way to fetch the raw data of any file i
> > input for example if i accept a text file or a video file (particualr
> > large video files ). i could fetch the the equivalent bits and store
> > it in another file
>
> [...]
>
> I'm not sure what you want to do. What do you mean by the following
> terms
>  - fetch (as in "fetch the raw data")
>  - raw data
>  - accept (as in "accept a text file")
>  - equivalent (as in "equivalent bits")
>  - store (as in "store it in another file")
>
> IF you're looking to make copies, it's simple enough. Just read from one
> (source) and write into the other (destination).
>
>
>

yep similar to that .. but i want the complete text or video file in bit
level and if possible store it in a list.
Well i never tried storing such huge amount of data to be stored in a list
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

[BangPypers] catching exceptions from SimpleHTTPServer

[Rahul R]

how to catch exceptions from the simpleHTTPServer .  ( srry if this sounds
ridiculous )
the below is a small code snippet for a basic webserver . every time
whenever there is a HTTP GET/POST request i can see the feedback  in the
terminal . how do i catch those feedbacks ?

code snippet:
http://codepad.org/eKWUhrnO
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] catching exceptions from SimpleHTTPServer

[Rahul R]

>
> 
> --- browser.py  2010-07-20 22:51:33.0 +0530
> +++ browser-new.py  2010-07-20 22:55:20.0 +0530
> @@ -18,4 +18,10 @@
>
>  sa = httpd.socket.getsockname()
>  print "Serving HTTP on", sa[0], "port", sa[1], "..."
> -httpd.serve_forever()
> +import sys
> +out = open("browser.log", "w")
> +sys.stderr = out
> +try:
> +httpd.serve_forever()
> +except:
> +out.close()
> 
>
>
> negative aint working  , i tried a few changes myself . when i referred to
the class BaseHTTPServer s methods ( http://bit.ly/aYqnQE ) i found a
particular method log_message which is similar to sys.stderr but for some
reason it isnt appending ne data into the log file created.

the revised code snippet:
http://codepad.org/PRPXVUpY


--Rahul
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] July meetup

[Rahul R]

On Wed, Jul 21, 2010 at 1:10 PM, Noufal Ibrahim  wrote:

>
> Hello everyone,
>We'll need atleast a mini meetup in July atleast to collect
> registration money from offline registrants.
>
>How's this Sunday look?
>
> Thanks.
>

i would be delighted to come . this would be my first meet .
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

[BangPypers] diffrerence between lambda function and ordinary one

[Rahul R]

i was writing some basic code . for understanding lambda functions , but
couldnt understand the difference between a lambda function and an ordinary
function.

for example

>>>def f (x): return x**2
...
>>> print f(8)
>>> 64
>>> g = lambda x: x**2
>>>
>>> print g(8)
>>> 64

wats a need for using lambda functions ..?
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

[BangPypers] In Multiprocess module how to find which worker process is executing the job

[Rahul R]

I was wondering , if there is a way to find out which process among the
Pool has executed a particular job submitted.

For example ,


def start_exe():
 ##Does some task ###

if __name__ == '__main__':
 p = Pool(5)
 result = p.apply_async(start_exe)
 print result.get()


how do i find out , which worker process from the Pool has executed the
job.

Regards,
Rahul
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] Python script to limit access to Internet...

[Rahul R]

Use Squid Proxy sever

--Rahul


On Wed, Jun 27, 2012 at 1:34 PM, Vishal  wrote:

> Hello,
>
> I would like to limit internet access at my home, for some computers.
> Basically for a given time during the day (read schedule), I wish that the
> a given computer should not be able to access the internet.
> However, the computer is still connected to the internal LAN, so file/print
> sharing should be allowed.
> Something like what this software does:
> http://www.blumentals.net/inetprot/
>
>
> And, of course I would like to do this in Python.
>
> Any pointers ? Thanks in advance.
>
> Best regards,
> Vishal Sapre
> ___
> BangPypers mailing list
> BangPypers@python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] [ANNOUNCEMENT] Retask, a Task Queue implementation for human beings

[Rahul R]

hey ,

i was checking out ReTask , its a nice tool to have. But i am just curious
how is it different from using Redis with its python client. If i am not
wrong , redis natively supports pub-sub model.

Thanks,
Rahul



On Wed, Jul 4, 2012 at 10:15 PM, Kushal Das  wrote:

> Hi all,
>
> I am happy to announce Retask [1], a Task Queue implementation for
> human beings. It uses Redis [2] it the backend.
>
> User can enqueue and dequeue tasks in the queues they manage. Each
> task can contain any serializable python objects. We use JSON
> internally to store the tasks in the queues.
>
> [1] http://retask.readthedocs.org
> [2] http://redis.io
>
> Kushal
> --
> http://fedoraproject.org
> http://kushaldas.in
> ___
> BangPypers mailing list
> BangPypers@python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] Executing Commands using sudo

[Rahul R]

$pip install sh (http://amoffat.github.com/sh/)
pretty much does everything u asked for.

http://amoffat.github.com/sh/tutorials/2-interacting_with_processes.html


--Rahul

On Thu, Nov 15, 2012 at 10:01 PM, Vid  wrote:

> On Thu, Nov 15, 2012 at 6:49 AM, davidsnt  wrote:
> > Hello Team,
> >
> > I have a bunch of servers where I have to login as user1 and then sudo as
> > super user and again change to user2 and execute a bunch of commands, can
> > you guys help with any thought how can I get this done in python.
>
> Your mail is not clear about what you are attempting to do after you
> login. Anyway, if you are deploying software across multiple remote
> servers, "Fabric" is your friend. HTH.
>
> Regards,
> Vid  ॥ http://svaksha.com ॥
> ___
> BangPypers mailing list
> BangPypers@python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] Minutes of the usergroup meeting - 24-11-2012

[Rahul R]

I would love to take up the opportunity to manage the website.

--Rahul

On Mon, Nov 26, 2012 at 11:15 PM, Anand Chitipothu wrote:

> On Mon, Nov 26, 2012 at 7:05 PM, Noufal Ibrahim 
> wrote:
> > kracekumar ramaraju  writes:
> >
> >
> > [...]
> >
> >> Since we have are hosting wiki and planet, how about running static blog
> >> generator like pelican - https://github.com/getpelican/pelican/ , we
> >> already have ngnix running, this shouldn't be difficult to serve.
> >
> > [...]
> >
> > The idea is nascent but the plan is to give control of a top level
> > domain to each user group in the country. Then we'd have
> > bangalore.python.org.in etc. Under that, the webmaster is free to do
> > what he wants with the buy in from his community etc.
> >
> > I think we should formalise that and then kick start it before doing ad
> > hoc stuff.
>
> +1
>
> Anyone interested to manage the bangalore website?
>
> Anand
> ___
> BangPypers mailing list
> BangPypers@python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] Question on Pattern Matching and Regular Expression

[Rahul R]

Hey David,

Assuming , its not a continuous stream of data being written to file all
the time. you could do something like this  https://gist.github.com/4477268 .
you can enhance it and make it more pythonic. :)

Thanks,
Rahul





On Mon, Jan 7, 2013 at 5:02 PM, davidsnt  wrote:

> No this file changes very often, like once in 5 minutes and the values are
> updated, also there is no way I can move it to a database.
>
> Regards,
> Davidsanthosh L
>
>
> On Mon, Jan 7, 2013 at 4:55 PM, Amit Sethi  >wrote:
>
> > On Mon, Jan 7, 2013 at 3:52 PM, davidsnt  wrote:
> > > Gora,
> > >
> > > Can you help me with few links that you have handy to which I can refer
> > to
> > > build a parser instead of RE
> > Can you elaborate on the idea of "build a parser" , in any case you
> > will have to use regex.
> >
> > >  I also need this app to be
> > >able to do a search in the file based on the title.
> > What kind of file are you are working with , How often does it change?
> >  It might be good move the data to a database in case file does not
> > change very often .
> >
> > --
> > A-M-I-T S|S
> > ___
> > BangPypers mailing list
> > BangPypers@python.org
> > http://mail.python.org/mailman/listinfo/bangpypers
> >
> ___
> BangPypers mailing list
> BangPypers@python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

[BangPypers] Reorder Dictionary Size in python

[Rahul R]

Hey  Guys,

Is it possible to forcibly reorder the python dictionary after "n" number
of inserts and deletions. As far as i know, python dictionary performs lazy
deletes. Thus , even if the data is deleted, python has a dummy data their
in order to preserve consistency. The python dictionary
keeps expanding when the size of dict is increasing, but after deleting a
few parameters the size does not decrease. Is there a way , where I can
forcibly resize the dictionary ?

I was thinking of copying content from existing dictionary to new dict and
deleting the previous one.But thats a cumbersome operation.

Thanks,
./Rahul
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] Reorder Dictionary Size in python

[Rahul R]

Ahh , sorry If i wasnt clear the first time. I dint mean reorder the data
in dictionary. I meant resize the dictionary.

./Rahul

On Wed, Apr 17, 2013 at 9:05 PM, Anand Chitipothu wrote:

> dictionaries are unordered. It is not a good idea to expect any order in
> dictionaries, even if you are seeing some order by chance.
>
> If you need order, then use OrderedDict from collections module (new in
> Python 2.7).
>
> Anand
>
>
> On Wed, Apr 17, 2013 at 9:00 PM, Rahul R  wrote:
>
> > Hey  Guys,
> >
> > Is it possible to forcibly reorder the python dictionary after "n" number
> > of inserts and deletions. As far as i know, python dictionary performs
> lazy
> > deletes. Thus , even if the data is deleted, python has a dummy data
> their
> > in order to preserve consistency. The python dictionary
> > keeps expanding when the size of dict is increasing, but after deleting a
> > few parameters the size does not decrease. Is there a way , where I can
> > forcibly resize the dictionary ?
> >
> > I was thinking of copying content from existing dictionary to new dict
> and
> > deleting the previous one.But thats a cumbersome operation.
> >
> > Thanks,
> > ./Rahul
> > ___
> > BangPypers mailing list
> > BangPypers@python.org
> > http://mail.python.org/mailman/listinfo/bangpypers
> >
>
>
>
> --
> Anand
> http://anandology.com/
> ___
> BangPypers mailing list
> BangPypers@python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] Reorder Dictionary Size in python

[Rahul R]

As far as i know, python performs a lazy deletion of values , when we
delete content from a dictionary (correct me if i am wrong) .  So, when we
insert a lot of values the dictionary automatically expands. I don't see
dict shrinking when we delete values from dictionary. In such case, is
there a way to forcibly reduce the dictionary size ?

./Rahul

On Wed, Apr 17, 2013 at 9:23 PM, Ramdas S  wrote:

> On Wed, Apr 17, 2013 at 9:22 PM, Rahul R  wrote:
>
> > Ahh , sorry If i wasnt clear the first time. I dint mean reorder the data
> > in dictionary. I meant resize the dictionary.
> >
>
> What do you mean by resize?
>
>
> >
> > ./Rahul
> >
> > On Wed, Apr 17, 2013 at 9:05 PM, Anand Chitipothu  > >wrote:
> >
> > > dictionaries are unordered. It is not a good idea to expect any order
> in
> > > dictionaries, even if you are seeing some order by chance.
> > >
> > > If you need order, then use OrderedDict from collections module (new in
> > > Python 2.7).
> > >
> > > Anand
> > >
> > >
> > > On Wed, Apr 17, 2013 at 9:00 PM, Rahul R  wrote:
> > >
> > > > Hey  Guys,
> > > >
> > > > Is it possible to forcibly reorder the python dictionary after "n"
> > number
> > > > of inserts and deletions. As far as i know, python dictionary
> performs
> > > lazy
> > > > deletes. Thus , even if the data is deleted, python has a dummy data
> > > their
> > > > in order to preserve consistency. The python dictionary
> > > > keeps expanding when the size of dict is increasing, but after
> > deleting a
> > > > few parameters the size does not decrease. Is there a way , where I
> can
> > > > forcibly resize the dictionary ?
> > > >
> > > > I was thinking of copying content from existing dictionary to new
> dict
> > > and
> > > > deleting the previous one.But thats a cumbersome operation.
> > > >
> > > > Thanks,
> > > > ./Rahul
> > > > ___
> > > > BangPypers mailing list
> > > > BangPypers@python.org
> > > > http://mail.python.org/mailman/listinfo/bangpypers
> > > >
> > >
> > >
> > >
> > > --
> > > Anand
> > > http://anandology.com/
> > > ___
> > > BangPypers mailing list
> > > BangPypers@python.org
> > > http://mail.python.org/mailman/listinfo/bangpypers
> > >
> > ___
> > BangPypers mailing list
> > BangPypers@python.org
> > http://mail.python.org/mailman/listinfo/bangpypers
> >
>
>
>
> --
> Ramdas S
> +91 9342 583 065
> My Personal Blog on http://ramdaz.wordpress.com
> ___
> BangPypers mailing list
> BangPypers@python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] Reorder Dictionary Size in python

[Rahul R]

 I dont intent to pre optimize things , I am rather driven by curiosity on
how one could do that.

Thanks,
./Rahul

On Thu, Apr 18, 2013 at 5:53 AM, Anand Chitipothu wrote:

> On Wed, Apr 17, 2013 at 9:30 PM, Rahul R  wrote:
>
> > As far as i know, python performs a lazy deletion of values , when we
> > delete content from a dictionary (correct me if i am wrong) .  So, when
> we
> > insert a lot of values the dictionary automatically expands. I don't see
> > dict shrinking when we delete values from dictionary. In such case, is
> > there a way to forcibly reduce the dictionary size ?
> >
>
> Don't try to optimize something that is not required. Python core
> developers are smarted than you, trust them.
>
> Premature optimization is root cause of all evil.
>
> Anand
> ___
> BangPypers mailing list
> BangPypers@python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] Reorder Dictionary Size in python

[Rahul R]

As far as i know , gc.collect() works on objects which are not used
anymore. Not for resizing existing objects.

Thanks,
./Rahul

On Thu, Apr 18, 2013 at 9:25 AM, Abdul Muneer  wrote:

> Expecting this behaviour from built-in dict is not a good idea. However try
> if garbage collection helps.
> >>> import gc
> >>> gc.collect()
> I have not tried it out myself, though.
>
> Regards,
> Abdul Muneer
>
> --
> Follow me on Twitter: @abdulmuneer <http://twitter.com/#%21/abdulmuneer>
>
>
> On Thu, Apr 18, 2013 at 5:53 AM, Anand Chitipothu  >wrote:
>
> > On Wed, Apr 17, 2013 at 9:30 PM, Rahul R  wrote:
> >
> > > As far as i know, python performs a lazy deletion of values , when we
> > > delete content from a dictionary (correct me if i am wrong) .  So, when
> > we
> > > insert a lot of values the dictionary automatically expands. I don't
> see
> > > dict shrinking when we delete values from dictionary. In such case, is
> > > there a way to forcibly reduce the dictionary size ?
> > >
> >
> > Don't try to optimize something that is not required. Python core
> > developers are smarted than you, trust them.
> >
> > Premature optimization is root cause of all evil.
> >
> > Anand
> > ___
> > BangPypers mailing list
> > BangPypers@python.org
> > http://mail.python.org/mailman/listinfo/bangpypers
> >
> ___
> BangPypers mailing list
> BangPypers@python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] Reorder Dictionary Size in python

[Rahul R]

Harish , you nailed it!  Thanks for the video.  So, decreasing the size
will not be possible afterall.
But what confuses me is why is copying the items to new dictionary will not
consume less space ? By copying you will get rid of all the dummy variables
and lookup time will also decrease.

Thanks a lot!
./Rahul

On Thu, Apr 18, 2013 at 1:14 PM, Harish Vishwanath  wrote:

> Here is a good talk on how dictionaries are implemented in python :
>
> http://blip.tv/pycon-us-videos-2009-2010-2011/pycon-2010-the-mighty-dictionary-55-3352147
>
> Due to possibility of collisions in the hash table, dict keeps allocating
> 4x or 2x the size of existing allocation and *will* reorder the items when
> such an operation happens. They go and acquire more memory when the dict is
> 2/3rds full. But the reverse is not possible. A dummy key has to be
> inserted in a free slot, so that valid items in the dictionary can be
> found. Since all this depends on the order of insertion, you cannot
> guarantee that copying existing items into a new dictionary will consume
> less space.
>
> If look up times is not a concern, and you are trying to optimize for space
> the stackoverflow link that Anand shared earlier could help.
>
> Regards,
> Harish
>
>
> On Thu, Apr 18, 2013 at 12:06 PM, Hrishikesh Kulkarni
> wrote:
>
> > On Wed, Apr 17, 2013 at 9:30 PM, Rahul R  wrote:
> >
> > > As far as i know, python performs a lazy deletion of values , when we
> > > delete content from a dictionary (correct me if i am wrong) .  So, when
> > we
> > > insert a lot of values the dictionary automatically expands. I don't
> see
> > > dict shrinking when we delete values from dictionary. In such case, is
> > > there a way to forcibly reduce the dictionary size ?
> > >
> >
> >
> > Are you measuring this shrinking?
> > the following  in dictobject.c should guide you with the internals. The
> > deleted item is simply replaced with a refcounted dummy. The deleted item
> > is marked for cleanup by gc with decref.
> >
> > PyDict_DelItem () :
> > ..
> > old_key = ep->me_key;
> > Py_INCREF(dummy);
> > ep->me_key = dummy;
> > old_value = ep->me_value;
> > ep->me_value = NULL;
> > mp->ma_used--;
> > Py_DECREF(old_value);
> > Py_DECREF(old_key);
> > ..
> >
> >
> > dict_length():
> > ..
> > return mp->ma_used;
> > ..
> >
> > PyDict_Size():
> > ..
> > return ((PyDictObject *)mp)->ma_used;
> > ..
> >
> >
> >
> > regards,
> > Rishi
> >
> >
> >
> >
> > >
> > > ./Rahul
> > >
> > > On Wed, Apr 17, 2013 at 9:23 PM, Ramdas S  wrote:
> > >
> > > > On Wed, Apr 17, 2013 at 9:22 PM, Rahul R 
> wrote:
> > > >
> > > > > Ahh , sorry If i wasnt clear the first time. I dint mean reorder
> the
> > > data
> > > > > in dictionary. I meant resize the dictionary.
> > > > >
> > > >
> > > > What do you mean by resize?
> > > >
> > > >
> > > > >
> > > > > ./Rahul
> > > > >
> > > > > On Wed, Apr 17, 2013 at 9:05 PM, Anand Chitipothu <
> > > anandol...@gmail.com
> > > > > >wrote:
> > > > >
> > > > > > dictionaries are unordered. It is not a good idea to expect any
> > order
> > > > in
> > > > > > dictionaries, even if you are seeing some order by chance.
> > > > > >
> > > > > > If you need order, then use OrderedDict from collections module
> > (new
> > > in
> > > > > > Python 2.7).
> > > > > >
> > > > > > Anand
> > > > > >
> > > > > >
> > > > > > On Wed, Apr 17, 2013 at 9:00 PM, Rahul R 
> > > wrote:
> > > > > >
> > > > > > > Hey  Guys,
> > > > > > >
> > > > > > > Is it possible to forcibly reorder the python dictionary after
> > "n"
> > > > > number
> > > > > > > of inserts and deletions. As far as i know, python dictionary
> > > > performs
> > > > > > lazy
> > > > > > > deletes. Thus , even if the data is deleted, python has a dummy
> > > data
> > > > > > their
> > > > > > > in order to preserve consistency. The python dictionary
&g

Re: [BangPypers] Reorder Dictionary Size in python

[Rahul R]

Hey Hrishikesk,

So are you saying, if i run gc.collect() in my program I will get rid of
all the dummy variables in the dictionary ?

Thanks,
./Rahul

On Thu, Apr 18, 2013 at 12:06 PM, Hrishikesh Kulkarni
wrote:

> On Wed, Apr 17, 2013 at 9:30 PM, Rahul R  wrote:
>
> > As far as i know, python performs a lazy deletion of values , when we
> > delete content from a dictionary (correct me if i am wrong) .  So, when
> we
> > insert a lot of values the dictionary automatically expands. I don't see
> > dict shrinking when we delete values from dictionary. In such case, is
> > there a way to forcibly reduce the dictionary size ?
> >
>
>
> Are you measuring this shrinking?
> the following  in dictobject.c should guide you with the internals. The
> deleted item is simply replaced with a refcounted dummy. The deleted item
> is marked for cleanup by gc with decref.
>
> PyDict_DelItem () :
> ..
> old_key = ep->me_key;
> Py_INCREF(dummy);
> ep->me_key = dummy;
> old_value = ep->me_value;
> ep->me_value = NULL;
> mp->ma_used--;
> Py_DECREF(old_value);
> Py_DECREF(old_key);
> ..
>
>
> dict_length():
> ..
> return mp->ma_used;
> ..
>
> PyDict_Size():
> ..
> return ((PyDictObject *)mp)->ma_used;
> ..
>
>
>
> regards,
> Rishi
>
>
>
>
> >
> > ./Rahul
> >
> > On Wed, Apr 17, 2013 at 9:23 PM, Ramdas S  wrote:
> >
> > > On Wed, Apr 17, 2013 at 9:22 PM, Rahul R  wrote:
> > >
> > > > Ahh , sorry If i wasnt clear the first time. I dint mean reorder the
> > data
> > > > in dictionary. I meant resize the dictionary.
> > > >
> > >
> > > What do you mean by resize?
> > >
> > >
> > > >
> > > > ./Rahul
> > > >
> > > > On Wed, Apr 17, 2013 at 9:05 PM, Anand Chitipothu <
> > anandol...@gmail.com
> > > > >wrote:
> > > >
> > > > > dictionaries are unordered. It is not a good idea to expect any
> order
> > > in
> > > > > dictionaries, even if you are seeing some order by chance.
> > > > >
> > > > > If you need order, then use OrderedDict from collections module
> (new
> > in
> > > > > Python 2.7).
> > > > >
> > > > > Anand
> > > > >
> > > > >
> > > > > On Wed, Apr 17, 2013 at 9:00 PM, Rahul R 
> > wrote:
> > > > >
> > > > > > Hey  Guys,
> > > > > >
> > > > > > Is it possible to forcibly reorder the python dictionary after
> "n"
> > > > number
> > > > > > of inserts and deletions. As far as i know, python dictionary
> > > performs
> > > > > lazy
> > > > > > deletes. Thus , even if the data is deleted, python has a dummy
> > data
> > > > > their
> > > > > > in order to preserve consistency. The python dictionary
> > > > > > keeps expanding when the size of dict is increasing, but after
> > > > deleting a
> > > > > > few parameters the size does not decrease. Is there a way ,
> where I
> > > can
> > > > > > forcibly resize the dictionary ?
> > > > > >
> > > > > > I was thinking of copying content from existing dictionary to new
> > > dict
> > > > > and
> > > > > > deleting the previous one.But thats a cumbersome operation.
> > > > > >
> > > > > > Thanks,
> > > > > > ./Rahul
> > > > > > ___
> > > > > > BangPypers mailing list
> > > > > > BangPypers@python.org
> > > > > > http://mail.python.org/mailman/listinfo/bangpypers
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Anand
> > > > > http://anandology.com/
> > > > > ___
> > > > > BangPypers mailing list
> > > > > BangPypers@python.org
> > > > > http://mail.python.org/mailman/listinfo/bangpypers
> > > > >
> > > > ___
> > > > BangPypers mailing list
> > > > BangPypers@python.org
> > > > http://mail.python.org/mailman/listinfo/bangpypers
> > > >
> > >
> > >
> > >
> > > --
> > > Ramdas S
> > > +91 9342 583 065
> > > My Personal Blog on http://ramdaz.wordpress.com
> > > ___
> > > BangPypers mailing list
> > > BangPypers@python.org
> > > http://mail.python.org/mailman/listinfo/bangpypers
> > >
> > ___
> > BangPypers mailing list
> > BangPypers@python.org
> > http://mail.python.org/mailman/listinfo/bangpypers
> >
> ___
> BangPypers mailing list
> BangPypers@python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers

Re: [BangPypers] Reorder Dictionary Size in python

[Rahul R]

Afaik,  python gc is threshold based. So, gc should trigger after a given
threshold value (which can be changed) . but defintely the code is worth
looking. thanks for pointing it out.

./Rahul

On Thu, Apr 18, 2013 at 2:15 PM, Hrishikesh Kulkarni
wrote:

> The code snippet shows the old_value and old_keys are marked by decref not
> the dummy. I wouldnt worry about dummy keys so much. btw checkout
> dictresize
> which is called during insert/set which does resize the dict. When? that
> would be a good learning exercise.
>
>
>
> On Thu, Apr 18, 2013 at 2:09 PM, Rahul R  wrote:
>
> > Hey Hrishikesk,
> >
> > So are you saying, if i run gc.collect() in my program I will get rid of
> > all the dummy variables in the dictionary ?
> >
> > Thanks,
> > ./Rahul
> >
> > On Thu, Apr 18, 2013 at 12:06 PM, Hrishikesh Kulkarni
> > wrote:
> >
> > > On Wed, Apr 17, 2013 at 9:30 PM, Rahul R  wrote:
> > >
> > > > As far as i know, python performs a lazy deletion of values , when we
> > > > delete content from a dictionary (correct me if i am wrong) .  So,
> when
> > > we
> > > > insert a lot of values the dictionary automatically expands. I don't
> > see
> > > > dict shrinking when we delete values from dictionary. In such case,
> is
> > > > there a way to forcibly reduce the dictionary size ?
> > > >
> > >
> > >
> > > Are you measuring this shrinking?
> > > the following  in dictobject.c should guide you with the internals. The
> > > deleted item is simply replaced with a refcounted dummy. The deleted
> item
> > > is marked for cleanup by gc with decref.
> > >
> > > PyDict_DelItem () :
> > > ..
> > > old_key = ep->me_key;
> > > Py_INCREF(dummy);
> > > ep->me_key = dummy;
> > > old_value = ep->me_value;
> > > ep->me_value = NULL;
> > > mp->ma_used--;
> > > Py_DECREF(old_value);
> > > Py_DECREF(old_key);
> > > ..
> > >
> > >
> > > dict_length():
> > > ..
> > > return mp->ma_used;
> > > ..
> > >
> > > PyDict_Size():
> > > ..
> > > return ((PyDictObject *)mp)->ma_used;
> > > ..
> > >
> > >
> > >
> > > regards,
> > > Rishi
> > >
> > >
> > >
> > >
> > > >
> > > > ./Rahul
> > > >
> > > > On Wed, Apr 17, 2013 at 9:23 PM, Ramdas S  wrote:
> > > >
> > > > > On Wed, Apr 17, 2013 at 9:22 PM, Rahul R 
> > wrote:
> > > > >
> > > > > > Ahh , sorry If i wasnt clear the first time. I dint mean reorder
> > the
> > > > data
> > > > > > in dictionary. I meant resize the dictionary.
> > > > > >
> > > > >
> > > > > What do you mean by resize?
> > > > >
> > > > >
> > > > > >
> > > > > > ./Rahul
> > > > > >
> > > > > > On Wed, Apr 17, 2013 at 9:05 PM, Anand Chitipothu <
> > > > anandol...@gmail.com
> > > > > > >wrote:
> > > > > >
> > > > > > > dictionaries are unordered. It is not a good idea to expect any
> > > order
> > > > > in
> > > > > > > dictionaries, even if you are seeing some order by chance.
> > > > > > >
> > > > > > > If you need order, then use OrderedDict from collections module
> > > (new
> > > > in
> > > > > > > Python 2.7).
> > > > > > >
> > > > > > > Anand
> > > > > > >
> > > > > > >
> > > > > > > On Wed, Apr 17, 2013 at 9:00 PM, Rahul R 
> > > > wrote:
> > > > > > >
> > > > > > > > Hey  Guys,
> > > > > > > >
> > > > > > > > Is it possible to forcibly reorder the python dictionary
> after
> > > "n"
> > > > > > number
> > > > > > > > of inserts and deletions. As far as i know, python dictionary
> > > > > performs
> > > > > > > lazy
> > > > > > > > deletes. Thus , even if the data is deleted, python has a
> dummy
> > > > data
> > > > > > > their
> > > > >

Re: [BangPypers] Need help for python coding

[Rahul R]

os.path.realpath(__filename__)
that should do the trick.

./Rahul



On Thu, Apr 18, 2013 at 6:15 PM, Kamalakar gs  wrote:

> HI all,
> I have a query that i have doc which is  read through python.But instead of
> giving path and filename
> in the coding.I want to manipulate it like "python has to ask USER for a
> specific filename so that python recognizes the directory internally and
> read the file ".How do i do this.I have tried with
> filename = input ("filename") but it just file name and will not recognises
> the path.
>
> For example:
>
> file name = input ("file name: ")
>
>  file name = python
>
> it has to recognizes the path like"C:\users\desktop\python"
>
>
> Thanks
> Regards
> Kamalakar
> ___
> BangPypers mailing list
> BangPypers@python.org
> http://mail.python.org/mailman/listinfo/bangpypers
>
___
BangPypers mailing list
BangPypers@python.org
http://mail.python.org/mailman/listinfo/bangpypers