Is it possible to pass CSV Reader Object As Argument to another Python File ???
I have following two python scripts -namelookupWrapper.py -namelookup.py The namelookupWrapper.py takes input of "memberId", "memberName" from CLI and has following code snippet idf = sys.argv[1] namef = sys.argv[2] real_script = "C:\\Splunk\\etc\\apps\\search\\bin\\namelookup.py" r = csv.reader(sys.stdin) os.execv(python_executable, [ python_executable, real_script ] + sys.argv[1:] ) Wondering how would i pass csv reader object "r" as an argument using os.execv() to another python script i.e. namelookup.py -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to pass CSV Reader Object As Argument to another Python File ???
On Jan 26, 11:30 am, Emile van Sebille wrote: > On 1/26/2011 7:51 AM bansi said... > > > > > > > I have following two python scripts > > -namelookupWrapper.py > > -namelookup.py > > > The namelookupWrapper.py takes input of "memberId", "memberName" from > > CLI and has following code snippet > > > idf = sys.argv[1] > > namef = sys.argv[2] > > real_script = "C:\\Splunk\\etc\\apps\\search\\bin\\namelookup.py" > > r = csv.reader(sys.stdin) > > os.execv(python_executable, [ python_executable, real_script ] + > > sys.argv[1:] ) > > > Wondering how would i pass csv reader object "r" as an argument using > > os.execv() to another python script i.e. namelookup.py > > I suspect you're on the wrong path. You probably want to import > namelookup within namelooupWrapper to use the functions it defines. > > Consider: > > [root@fcfw2 src]# cat > test1.py > > def say(what): print what > > [root@fcfw2 src]# cat > test2.py > > #!/usr/local/bin/python > import sys > from test1 import say > say(sys.argv[1]) > > [root@fcfw2 src]# chmod a+x test2.py > > [root@fcfw2 src]# ./test2.py hello > hello > > HTH, > > Emile- Hide quoted text - > > - Show quoted text - Emile, Thanks for quick response. I am not sure if "import namelookup within namelooupWrapper" helps because they are two independent scripts which has to be executed in sequence. First namelookupWrapper.py running under Python 2.6 accept arguments from stdin and uses csv reader object to read it i.e. r=csv.reader(sys.stdin) And then it has to pass csv reader object to another python script namelookup.py running under Python 2.7 because it uses pyodbc to connect to database and iterates thru reader object Any better ideas/suggestions will be greatly appreciated -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to pass CSV Reader Object As Argument to another Python File ???
On Jan 26, 1:31 pm, Chris Rebert wrote: > On Wed, Jan 26, 2011 at 7:51 AM, bansi wrote: > > I have following two python scripts > > -namelookupWrapper.py > > -namelookup.py > > > The namelookupWrapper.py takes input of "memberId", "memberName" from > > CLI and has following code snippet > > > idf = sys.argv[1] > > namef = sys.argv[2] > > real_script = "C:\\Splunk\\etc\\apps\\search\\bin\\namelookup.py" > > r = csv.reader(sys.stdin) > > os.execv(python_executable, [ python_executable, real_script ] + > > sys.argv[1:] ) > > > Wondering how would i pass csv reader object "r" as an argument using > > os.execv() to another python script i.e. namelookup.py > > It's not possible to pass Python objects between processes in such a > manner. Given that "independent" scripts can't directly take objects > as input anyway, I doubt the two scripts are truly independent from > each other. I would therefore concur with van Sebille that you should > just rewrite them so that one script imports from the other rather > than spawning the other. It should not be too hard to port the Python > 2.6 script to Python 2.7 (or vice-versa if necessary). > > Cheers, > Chris > --http://blog.rebertia.com- Hide quoted text - > > - Show quoted text - Thanks Chris. Sorry for mis-communicating, the two python scripts are dependant in a way that namelookupWrapper.py needs to pass csv record object to another python script If thats not possible then please let me know how to do the workaround i didnt understood the import thing and not sure if it helps in my case Here are the details namelookupwrapper.py - takes input from stdin. Using csv reader object i iterate thru the input which looks like as shown below [MemberId, MemberName] [123, ] [456, ] [989, ] Now i have another script i.e. namelookup.py running under Python 2.7 using pyodbc to retrieve Member Names from database for a given Member Id in namelooupWrapper.py So please let me know how to accomplish this -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to pass CSV Reader Object As Argument to another Python File ???
On Jan 26, 4:36 pm, Ben Finney wrote: > bansi writes: > > Thanks Chris. Sorry for mis-communicating, the two python scripts are > > dependant in a way that namelookupWrapper.py needs to pass csv record > > object to another python script > > Why have you structured them that way, though? What constraint is > keeping you from doing the work in a single process, where the CSV > reader object can be shared? > > > If thats not possible then please let me know how to do the workaround > > i didnt understood the import thing and not sure if it helps in my > > case > > The problem as you've described it so far is best solved by having a > single process accessing the CSV reader object in memory. If that > doesn't suit your use case, you'll need to explain why not. > > -- > \ “To have the choice between proprietary software packages, is | > `\ being able to choose your master. Freedom means not having a | > _o__) master.” —Richard M. Stallman, 2007-05-16 | > Ben Finney Thanks Ben for quick response. The constraint is in using third party tool called Splunk which has in-built Python 2.6 which doesnt support pyodbc on Windows 64 bit OS. Hence i have to install Python 2.7 for pyodbc. That means namelookupwrapper.py acts as a wrapper running under Splunk environment taking input from Splunk via stdin and storing it in csv reader object and then needs to call actual script namelookup.py under Python 2.7 for making connection to database Hope i have clarified a bit -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to pass CSV Reader Object As Argument to another Python File ???
On Jan 26, 6:25 pm, Ethan Furman wrote: > bansi wrote: > > > First namelookupWrapper.py running under Python 2.6 accept arguments > > from stdin and uses csv reader object to read it i.e. > > r=csv.reader(sys.stdin) > > > > And then it has to pass csv reader object to another python script > > namelookup.py running under Python 2.7 because it uses pyodbc to > > connect to database and iterates thru reader object > > > > > > Ben Finney wrote: > > bansi writes: > > >> Thanks Chris. Sorry for mis-communicating, the two python scripts are > >> dependant in a way that namelookupWrapper.py needs to pass csv record > >> object to another python script > > > Why have you structured them that way, though? What constraint is > > keeping you from doing the work in a single process, where the CSV > > reader object can be shared? > > >> If thats not possible then please let me know how to do the workaround > >> i didnt understood the import thing and not sure if it helps in my > >> case > > > The problem as you've described it so far is best solved by having a > > single process accessing the CSV reader object in memory. If that > > doesn't suit your use case, you'll need to explain why not. > > In other words, why can't you use Python 2.7 to accept input and > generate a csv.reader? > > ~Ethan~- Hide quoted text - > > - Show quoted text - Ethan, The python script takes the input from Splunk (http://www.splunk.com/ base/Documentation/) which supports only Python 2.6 So the real constraint is Splunk supports only Python 2.6 . As you know Python 2.6 doesnt support or doesnt have pyodbc install for Windows 64 bit OS So i installed Python 2.7 and thereafter pyodbc install for Windows 64 bit OS for Python 2.7 -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to pass CSV Reader Object As Argument to another Python File ???
On Jan 26, 8:31 pm, MRAB wrote: > On 27/01/2011 00:57, bansi wrote: > > > > > > > On Jan 26, 6:25 pm, Ethan Furman wrote: > >> bansi wrote: > > >> > First namelookupWrapper.py running under Python 2.6 accept arguments > >> > from stdin and uses csv reader object to read it i.e. > >> > r=csv.reader(sys.stdin) > > >> > And then it has to pass csv reader object to another python script > >> > namelookup.py running under Python 2.7 because it uses pyodbc to > >> > connect to database and iterates thru reader object > > >> Ben Finney wrote: > >>> bansi writes: > > >>>> Thanks Chris. Sorry for mis-communicating, the two python scripts are > >>>> dependant in a way that namelookupWrapper.py needs to pass csv record > >>>> object to another python script > > >>> Why have you structured them that way, though? What constraint is > >>> keeping you from doing the work in a single process, where the CSV > >>> reader object can be shared? > > >>>> If thats not possible then please let me know how to do the workaround > >>>> i didnt understood the import thing and not sure if it helps in my > >>>> case > > >>> The problem as you've described it so far is best solved by having a > >>> single process accessing the CSV reader object in memory. If that > >>> doesn't suit your use case, you'll need to explain why not. > > >> In other words, why can't you use Python 2.7 to accept input and > >> generate a csv.reader? > > >> ~Ethan~- Hide quoted text - > > >> - Show quoted text - > > > Ethan, > > The python script takes the input from Splunk (http://www.splunk.com/ > > base/Documentation/) which supports only Python 2.6 > > So the real constraint is Splunk supports only Python 2.6 . > > > As you know Python 2.6 doesnt support or doesnt have pyodbc install > > for Windows 64 bit OS > > So i installed Python 2.7 and thereafter pyodbc install for Windows 64 > > bit OS for Python 2.7 > > Have you actually tried Splunk with Python 2.7? It might not work with > versions which are earlier than Python 2.6, but that doesn't > necessarily mean that it won't work with versions of Python 2 which are > later than Python 2.6 (unless the documentation says that it must be > Python 2.6).- Hide quoted text - > > - Show quoted text - Splunk's latest version 4.1.6 doesn't support Python 2.7 I tried the import trick but it didnt work because the real script which runs under Python 2.7 has import pyodbc so it results in following error c:\Splunk\etc\apps\search\bin>splunk cmd python namelookupWrapper.py memberId memberName < memberInput.csv Traceback (most recent call last): File "namelookupWrapper.py", line 3, in import namelookup File "c:\Splunk\etc\apps\search\bin\namelookup.py", line 7, in import pyodbc ImportError: DLL load failed: The specified module could not be found. Please let me know if i am missing something on import. If so please provide me with an example -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to pass CSV Reader Object As Argument to another Python File ???
On Jan 28, 9:46 am, bansi wrote: > On Jan 26, 8:31 pm, MRAB wrote: > > > > > > > On 27/01/2011 00:57, bansi wrote: > > > > On Jan 26, 6:25 pm, Ethan Furman wrote: > > >> bansi wrote: > > > >> > First namelookupWrapper.py running under Python 2.6 accept arguments > > >> > from stdin and uses csv reader object to read it i.e. > > >> > r=csv.reader(sys.stdin) > > > >> > And then it has to pass csv reader object to another python script > > >> > namelookup.py running under Python 2.7 because it uses pyodbc to > > >> > connect to database and iterates thru reader object > > > >> Ben Finney wrote: > > >>> bansi writes: > > > >>>> Thanks Chris. Sorry for mis-communicating, the two python scripts are > > >>>> dependant in a way that namelookupWrapper.py needs to pass csv record > > >>>> object to another python script > > > >>> Why have you structured them that way, though? What constraint is > > >>> keeping you from doing the work in a single process, where the CSV > > >>> reader object can be shared? > > > >>>> If thats not possible then please let me know how to do the workaround > > >>>> i didnt understood the import thing and not sure if it helps in my > > >>>> case > > > >>> The problem as you've described it so far is best solved by having a > > >>> single process accessing the CSV reader object in memory. If that > > >>> doesn't suit your use case, you'll need to explain why not. > > > >> In other words, why can't you use Python 2.7 to accept input and > > >> generate a csv.reader? > > > >> ~Ethan~- Hide quoted text - > > > >> - Show quoted text - > > > > Ethan, > > > The python script takes the input from Splunk (http://www.splunk.com/ > > > base/Documentation/) which supports only Python 2.6 > > > So the real constraint is Splunk supports only Python 2.6 . > > > > As you know Python 2.6 doesnt support or doesnt have pyodbc install > > > for Windows 64 bit OS > > > So i installed Python 2.7 and thereafter pyodbc install for Windows 64 > > > bit OS for Python 2.7 > > > Have you actually tried Splunk with Python 2.7? It might not work with > > versions which are earlier than Python 2.6, but that doesn't > > necessarily mean that it won't work with versions of Python 2 which are > > later than Python 2.6 (unless the documentation says that it must be > > Python 2.6).- Hide quoted text - > > > - Show quoted text - > > Splunk's latest version 4.1.6 doesn't support Python 2.7 > I tried the import trick but it didnt work because the real script > which runs under Python 2.7 has import pyodbc so it results in > following error > > c:\Splunk\etc\apps\search\bin>splunk cmd python namelookupWrapper.py > memberId memberName < memberInput.csv > Traceback (most recent call last): > File "namelookupWrapper.py", line 3, in > import namelookup > File "c:\Splunk\etc\apps\search\bin\namelookup.py", line 7, in > > import pyodbc > ImportError: DLL load failed: The specified module could not be found. > > Please let me know if i am missing something on import. If so please > provide me with an example- Hide quoted text - > > - Show quoted text - Here are some more details from my earlier posting. Please click the below link http://answers.splunk.com/questions/11145/its-getting-mysterious-to-make-the-lookup-script-work -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to pass CSV Reader Object As Argument to another Python File ???
On Jan 28, 1:52 pm, Benjamin Kaplan wrote: > On Fri, Jan 28, 2011 at 1:33 PM, bansi wrote: > > On Jan 28, 9:46 am, bansi wrote: > >> On Jan 26, 8:31 pm, MRAB wrote: > > >> > On 27/01/2011 00:57, bansi wrote: > > >> > > On Jan 26, 6:25 pm, Ethan Furman wrote: > >> > >> bansi wrote: > > >> > >> > First namelookupWrapper.py running under Python 2.6 accept > >> > >> arguments > >> > >> > from stdin and uses csv reader object to read it i.e. > >> > >> > r=csv.reader(sys.stdin) > > >> > >> > And then it has to pass csv reader object to another python > >> > >> script > >> > >> > namelookup.py running under Python 2.7 because it uses pyodbc to > >> > >> > connect to database and iterates thru reader object > > >> > >> Ben Finney wrote: > >> > >>> bansi writes: > > >> > >>>> Thanks Chris. Sorry for mis-communicating, the two python scripts > >> > >>>> are > >> > >>>> dependant in a way that namelookupWrapper.py needs to pass csv > >> > >>>> record > >> > >>>> object to another python script > > >> > >>> Why have you structured them that way, though? What constraint is > >> > >>> keeping you from doing the work in a single process, where the CSV > >> > >>> reader object can be shared? > > >> > >>>> If thats not possible then please let me know how to do the > >> > >>>> workaround > >> > >>>> i didnt understood the import thing and not sure if it helps in my > >> > >>>> case > > >> > >>> The problem as you've described it so far is best solved by having a > >> > >>> single process accessing the CSV reader object in memory. If that > >> > >>> doesn't suit your use case, you'll need to explain why not. > > >> > >> In other words, why can't you use Python 2.7 to accept input and > >> > >> generate a csv.reader? > > >> > >> ~Ethan~- Hide quoted text - > > >> > >> - Show quoted text - > > >> > > Ethan, > >> > > The python script takes the input from Splunk (http://www.splunk.com/ > >> > > base/Documentation/) which supports only Python 2.6 > >> > > So the real constraint is Splunk supports only Python 2.6 . > > >> > > As you know Python 2.6 doesnt support or doesnt have pyodbc install > >> > > for Windows 64 bit OS > >> > > So i installed Python 2.7 and thereafter pyodbc install for Windows 64 > >> > > bit OS for Python 2.7 > > >> > Have you actually tried Splunk with Python 2.7? It might not work with > >> > versions which are earlier than Python 2.6, but that doesn't > >> > necessarily mean that it won't work with versions of Python 2 which are > >> > later than Python 2.6 (unless the documentation says that it must be > >> > Python 2.6).- Hide quoted text - > > >> > - Show quoted text - > > >> Splunk's latest version 4.1.6 doesn't support Python 2.7 > >> I tried the import trick but it didnt work because the real script > >> which runs under Python 2.7 has import pyodbc so it results in > >> following error > > >> c:\Splunk\etc\apps\search\bin>splunk cmd python namelookupWrapper.py > >> memberId memberName < memberInput.csv > >> Traceback (most recent call last): > >> File "namelookupWrapper.py", line 3, in > >> import namelookup > >> File "c:\Splunk\etc\apps\search\bin\namelookup.py", line 7, in > >> > >> import pyodbc > >> ImportError: DLL load failed: The specified module could not be found. > > >> Please let me know if i am missing something on import. If so please > >> provide me with an example- Hide quoted text - > > >> - Show quoted text - > > > Here are some more details from my earlier posting. Please click the > > below link > > >http://answers.splunk.com/questions/11145/its-getting-mysterious-to-m... > > -- > >http://mail.python.org/mailman/listinfo/python-list > > Have you tried downloading the source for PyODBC and compiling it > yourself? All you need to do is python setup.py install. My guess > would be that it works just fine on 64-bit Python 2.6, they just never > released a re-compiled version of it for that platform.- Hide quoted text - > > - Show quoted text - Thanks Benjamin. Please point me to the website from where i can download pyodbc for Windows 64 bit OS under Python 2.6 and installation instructions -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to pass CSV Reader Object As Argument to another Python File ???
On Jan 28, 4:22 pm, Benjamin Kaplan wrote: > On Fri, Jan 28, 2011 at 3:42 PM, bansi wrote: > > On Jan 28, 1:52 pm, Benjamin Kaplan wrote: > >> On Fri, Jan 28, 2011 at 1:33 PM, bansi wrote: > >> > On Jan 28, 9:46 am, bansi wrote: > >> >> On Jan 26, 8:31 pm, MRAB wrote: > > >> >> > On 27/01/2011 00:57, bansi wrote: > > >> >> > > On Jan 26, 6:25 pm, Ethan Furman wrote: > >> >> > >> bansi wrote: > > >> >> > >> > First namelookupWrapper.py running under Python 2.6 accept > >> >> > >> arguments > >> >> > >> > from stdin and uses csv reader object to read it i.e. > >> >> > >> > r=csv.reader(sys.stdin) > > >> >> > >> > And then it has to pass csv reader object to another python > >> >> > >> script > >> >> > >> > namelookup.py running under Python 2.7 because it uses pyodbc > >> >> > >> to > >> >> > >> > connect to database and iterates thru reader object > > >> >> > >> Ben Finney wrote: > >> >> > >>> bansi writes: > > >> >> > >>>> Thanks Chris. Sorry for mis-communicating, the two python > >> >> > >>>> scripts are > >> >> > >>>> dependant in a way that namelookupWrapper.py needs to pass csv > >> >> > >>>> record > >> >> > >>>> object to another python script > > >> >> > >>> Why have you structured them that way, though? What constraint is > >> >> > >>> keeping you from doing the work in a single process, where the CSV > >> >> > >>> reader object can be shared? > > >> >> > >>>> If thats not possible then please let me know how to do the > >> >> > >>>> workaround > >> >> > >>>> i didnt understood the import thing and not sure if it helps in > >> >> > >>>> my > >> >> > >>>> case > > >> >> > >>> The problem as you've described it so far is best solved by > >> >> > >>> having a > >> >> > >>> single process accessing the CSV reader object in memory. If that > >> >> > >>> doesn't suit your use case, you'll need to explain why not. > > >> >> > >> In other words, why can't you use Python 2.7 to accept input and > >> >> > >> generate a csv.reader? > > >> >> > >> ~Ethan~- Hide quoted text - > > >> >> > >> - Show quoted text - > > >> >> > > Ethan, > >> >> > > The python script takes the input from Splunk > >> >> > > (http://www.splunk.com/ > >> >> > > base/Documentation/) which supports only Python 2.6 > >> >> > > So the real constraint is Splunk supports only Python 2.6 . > > >> >> > > As you know Python 2.6 doesnt support or doesnt have pyodbc install > >> >> > > for Windows 64 bit OS > >> >> > > So i installed Python 2.7 and thereafter pyodbc install for Windows > >> >> > > 64 > >> >> > > bit OS for Python 2.7 > > >> >> > Have you actually tried Splunk with Python 2.7? It might not work with > >> >> > versions which are earlier than Python 2.6, but that doesn't > >> >> > necessarily mean that it won't work with versions of Python 2 which > >> >> > are > >> >> > later than Python 2.6 (unless the documentation says that it must be > >> >> > Python 2.6).- Hide quoted text - > > >> >> > - Show quoted text - > > >> >> Splunk's latest version 4.1.6 doesn't support Python 2.7 > >> >> I tried the import trick but it didnt work because the real script > >> >> which runs under Python 2.7 has import pyodbc so it results in > >> >> following error > > >> >> c:\Splunk\etc\apps\search\bin>splunk cmd python namelookupWrapper.py > >> >> memberId memberName < memberInput.csv > >> >> Traceback (most recent call last): > >> >> File "namelookupWrapper.py", line 3, in > >> >&g
