Creating a session in windows to auth to remote machines
Hi, I'm trying to auth to remote machines so I can plunder WMI to get logs and settings and the like. My script works for most of my machines because they're all in the same domain and I run the script as somebody who has enough access to get at this stuff but for machines off the domain I'm stuck. Now I'm primarily a sys/network admin and not a programmer. As a sys admin if I had this problem while trying to pull eventlogs manually with eventvwr I would simply map a drive to the remote machine. That would allow me to enter a username/password and then once I was authed I'd have a session and I'd be able to piggyback on that session to pull logs. I'm looking to do exactly that from inside my script. I could probably import os and call the net use command to map a drive and get a session that way but that feels really sloppy to me. I want to be able to explicitly close this session when I'm done with it too. So I figure there's a com object I could call to open this session for me but I can't figure out what it is. Any help would be appreciated! Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating a session in windows to auth to remote machines
On Apr 4, 6:39 pm, ericwoodwo...@gmail.com wrote: > Hi, > I'm trying to auth to remote machines so I can plunder WMI to get > logs and settings and the like. My script works for most of my > machines because they're all in the same domain and I run the script > as somebody who has enough access to get at this stuff but for > machines off the domain I'm stuck. > > Now I'm primarily a sys/network admin and not a programmer. As a > sys admin if I had this problem while trying to pull eventlogs > manually with eventvwr I would simply map a drive to the remote > machine. That would allow me to enter a username/password and then > once I was authed I'd have a session and I'd be able to piggyback on > that session to pull logs. > > I'm looking to do exactly that from inside my script. I could > probably import os and call the net use command to map a drive and get > a session that way but that feels really sloppy to me. I want to be > able to explicitly close this session when I'm done with it too. > > So I figure there's a com object I could call to open this > session for me but I can't figure out what it is. Any help would be > appreciated! > > Thanks Also I am writing this in Python. So I can use win32com if that's the way to go or I can use anything python 2.6 has built in. I mentioned com objects because I'm using a lot of those currently but if there's a more pythonic way to do what I"m after then I'm all ears. Thanks -- http://mail.python.org/mailman/listinfo/python-list
Killing threads
Hi, I'm new to python and even newer to threading and it seems as though I'm missing something fundamental about threads. Basically I have a program that looks like this: class ThreadOne(threading.Thread): while 1: do stuff class ThreadTwo(threading.Thread): while 1: do other stuff first = ThreadOne() second = ThreadTwo() while 1: do stuff The issue that I'm having is...I don't know how to kill this app in window. I hit ctrl-c but that only seems to kill one of the threads. The rest of the app just lingers. There's got to be a more graceful way but so far I haven't googled anything up. I'm using queues to talk between these threads so I could certainly put some kind of message on the queue that causes the threads to commit suicide but I'm thinking there's a more built in way to do what I want. I'm just not sure what it is. I'd appreciate it if somebody could point me in the right direction. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Killing threads
On Apr 5, 12:22 am, a...@pythoncraft.com (Aahz) wrote: > In article <4b52f7d7-81d5-4141-9385-ee8cfb90a...@l1g2000yqk.googlegroups.com>, > > wrote: > > >I'm using queues to talk between these threads so I could certainly > >put some kind of message on the queue that causes the threads to > >commit suicide but I'm thinking there's a more built in way to do what > >I want. I'm just not sure what it is. > > There isn't, you have the right idea about using queues. > -- > Aahz (a...@pythoncraft.com) <*> http://www.pythoncraft.com/ > > "Debugging is twice as hard as writing the code in the first place. > Therefore, if you write the code as cleverly as possible, you are, by > definition, not smart enough to debug it." --Brian W. Kernighan Ok good to know. I was letting the search for a really cool solution stop me from rolling out what I think I already know how to do. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Creating a session in windows to auth to remote machines
On Apr 5, 2:11 am, Tim Golden wrote:
> ericwoodwo...@gmail.com wrote:
> > Hi,
> > I'm trying to auth to remote machines so I can plunder WMI to get
> > logs and settings and the like. My script works for most of my
> > machines because they're all in the same domain and I run the script
> > as somebody who has enough access to get at this stuff but for
> > machines off the domain I'm stuck.
>
> Unless I'm missing something here, you can already specify
> username etc. with WMI. Just Dispatch on "WbemScripting.SWbemLocator"
> and call .ConnectServer. Just in case you haven't already, I
> (naturally :) ) recommend my wmi module [1] which wraps a fair bit
> of the plumbing for you, including this. Eg,
>
>
> import wmi
>
> c = wmi.WMI ("some-machine", user="tim", password="password")
>
> for log in c.Win32_NTLogEvent (
> Logfile="Application",
> Type="error"
> ):
> print log.RecordNumber, log.SourceName, log.Message
>
>
>
> [1]http://timgolden.me.uk/python/wmi.html
>
> TJG
That's very cool. I was using win32com.client but I'll check this
out. Thanks!
--
http://mail.python.org/mailman/listinfo/python-list
Re: Creating a session in windows to auth to remote machines
On Apr 5, 9:09 am, ericwoodwo...@gmail.com wrote:
> On Apr 5, 2:11 am, Tim Golden wrote:
>
>
>
>
>
> > ericwoodwo...@gmail.com wrote:
> > > Hi,
> > > I'm trying to auth to remote machines so I can plunder WMI to get
> > > logs and settings and the like. My script works for most of my
> > > machines because they're all in the same domain and I run the script
> > > as somebody who has enough access to get at this stuff but for
> > > machines off the domain I'm stuck.
>
> > Unless I'm missing something here, you can already specify
> > username etc. with WMI. Just Dispatch on "WbemScripting.SWbemLocator"
> > and call .ConnectServer. Just in case you haven't already, I
> > (naturally :) ) recommend my wmi module [1] which wraps a fair bit
> > of the plumbing for you, including this. Eg,
>
> >
> > import wmi
>
> > c = wmi.WMI ("some-machine", user="tim", password="password")
>
> > for log in c.Win32_NTLogEvent (
> > Logfile="Application",
> > Type="error"
> > ):
> > print log.RecordNumber, log.SourceName, log.Message
>
> >
>
> > [1]http://timgolden.me.uk/python/wmi.html
>
> > TJG
>
> That's very cool. I was using win32com.client but I'll check this
> out. Thanks!- Hide quoted text -
>
> - Show quoted text -
Thanks Tim. I looked thru your code and you pointed me at just what I
need: SWbemLocator.ConnectServer Method which is awesome. I already
have the WMI portion of my program working so I was using SWbemLocator
already...I just didn't know about that specific method. That's a big
help. I've bookmarked that page...lot of good stuff there.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Killing threads
On Apr 5, 11:07 pm, Dennis Lee Bieber wrote: > On Sun, 5 Apr 2009 17:27:15 -0700 (PDT), imageguy > declaimed the following in > gmane.comp.python.general: > > > In threading.Event python 2.5 docs say; > > "This is one of the simplest mechanisms for communication between > > threads: one thread signals an event and other threads wait for it. " > > > Again, I have limited experience, however, in my reading of the > > threading manual and review examples, Events were specifically design > > to be a thread safe way to communicate a 'state' to running threads ? > > In the OP's example 'do stuff' was open to wide interpretation, > > however, if within the thread's main 'while' loop the tread checks to > > see if the 'keepgoing' Event.isSet(), in what scenario would this > > create deadlock ? > > If you are going to perform a CPU intensive polling loop, there is > no sense in using the Event system in the first place... Just create a > globally accessible flag and set it to true when you want to signal the > threads (or false if you don't want to use the negation "while not > flagged: do next processing step") > > Event is optimized for the case wherein threads can WAIT (block) on > the Event object. > -- > Wulfraed Dennis Lee Bieber KD6MOG > wlfr...@ix.netcom.com wulfr...@bestiaria.com > HTTP://wlfraed.home.netcom.com/ > (Bestiaria Support Staff: web-a...@bestiaria.com) > HTTP://www.bestiaria.com/ Well it turns out my problem was with queues not with threads. I had a self.die prop in my thread object that defaults to FALSE and that I set to true when i wanted the thread to die. then my loop would be while not die: It seemed pretty simple so I didn't know why it was failing. What I didn't know, because I'm quite new to python, is that queue.get was blocking. So my producer thread why dying immediately but my worker threads were all blocking on their queue.gets. So they were never falling off the loop. I changed it to queue.get_nowait() and added a queue.empty exception and everything worked as expected. So I thought I knew what was going on and that I was having a really esoteric problem when i was actually having a pretty boring problem I didn't recognize. Thanks everybody for the help! -- http://mail.python.org/mailman/listinfo/python-list
Re: Killing threads
On Apr 6, 3:45 am, bieff...@gmail.com wrote: > On 6 Apr, 05:25, ericwoodwo...@gmail.com wrote: > > > > > On Apr 5, 11:07 pm, Dennis Lee Bieber wrote: > > > > On Sun, 5 Apr 2009 17:27:15 -0700 (PDT), imageguy > > > declaimed the following in > > > gmane.comp.python.general: > > > > > In threading.Event python 2.5 docs say; > > > > "This is one of the simplest mechanisms for communication between > > > > threads: one thread signals an event and other threads wait for it. " > > > > > Again, I have limited experience, however, in my reading of the > > > > threading manual and review examples, Events were specifically design > > > > to be a thread safe way to communicate a 'state' to running threads ? > > > > In the OP's example 'do stuff' was open to wide interpretation, > > > > however, if within the thread's main 'while' loop the tread checks to > > > > see if the 'keepgoing' Event.isSet(), in what scenario would this > > > > create deadlock ? > > > > If you are going to perform a CPU intensive polling loop, there is > > > no sense in using the Event system in the first place... Just create a > > > globally accessible flag and set it to true when you want to signal the > > > threads (or false if you don't want to use the negation "while not > > > flagged: do next processing step") > > > > Event is optimized for the case wherein threads can WAIT (block) > > > on > > > the Event object. > > > -- > > > Wulfraed Dennis Lee Bieber KD6MOG > > > wlfr...@ix.netcom.com wulfr...@bestiaria.com > > > HTTP://wlfraed.home.netcom.com/ > > > (Bestiaria Support Staff: web-a...@bestiaria.com) > > > HTTP://www.bestiaria.com/ > > > Well it turns out my problem was with queues not with threads. I had > > a self.die prop in my thread object that defaults to FALSE and that I > > set to true when i wanted the thread to die. then my loop would be > > while not die: It seemed pretty simple so I didn't know why it was > > failing. What I didn't know, because I'm quite new to python, is that > > queue.get was blocking. So my producer thread why dying immediately > > but my worker threads were all blocking on their queue.gets. So they > > were never falling off the loop. I changed it to queue.get_nowait() > > and added a queue.empty exception and everything worked as expected. > > > So I thought I knew what was going on and that I was having a really > > esoteric problem when i was actually having a pretty boring problem I > > didn't recognize. > > > Thanks everybody for the help!> > > I've gone through that also, when I started with python threads :-) > Be aware that using get_nowait may lead to your thread using too much > CPU in checking a queue often empty. I tend to use Queue.get with a > timeout, smaller enough to keep the thread responsive but large enough > not > to waste CPU in too-frequent checks. > > Ciao > - > FB Ok thanks - good to now. I'm trying to throttle it with a 1/2 sec sleep statement in the loop but I might just have the "main" loop toss some stuff on that queue as another solution. I'm still kicking it around -- http://mail.python.org/mailman/listinfo/python-list
Python, MS SQL, and batch inserts
Hi, I have a python script I'm writing that grabs some data from a com object, does a little formatting, and then inserts that data into a MS SQL 2003 DB. Because I'm using COM objects I'm importing win32com.client. That also allows me to use ADODB.connection and ADODB.command objects for working with SQL. The program works fine but it's a little slow. Inserting ~5500 rows of data takes about 10 seconds using a DB that is on the same machine running the script. I've done some general searches on how to speed this up and in other languages people suggest sending batches of inserts off at a time instead of executing 1 insert at a time. For java and .net people recommend using a stringbuilder function to create strings quickly. I don't know of such a function in python s I tried grouping my inserts into a single string using string += syntax. I knew that would be terrible but I wanted to see how terrible. Final reults: It was pretty terrible. Script went from taking ~18sec to taking 240sec. The overhead for recreating the strings was monster. No real surprise there. So I then loaded up the commands into a list and at the end I used the strong join method to create the string. This was far faster than using += to create my strings but still took twice as long as just running my inserts one at a time. So I'm looking for suggestions. Basically I have 5000 SQL inserts that I want to do as quickly as possible. This is purely academic as I can live with the 18 seconds the script needs to run (9 to talk to the com object and format the data and 10 to write to SQL) but I'm still curious how to improve on what I have running. Thanks in advance for any help, Eric -- http://mail.python.org/mailman/listinfo/python-list
Re: Python, MS SQL, and batch inserts
On Apr 21, 2:15 pm, Philip Semanchuk wrote: > On Apr 21, 2009, at 2:02 PM, ericwoodwo...@gmail.com wrote: > > > > > Hi, > > I have a python script I'm writing that grabs some data from a > > com object, does a little formatting, and then inserts that data into > > a MS SQL 2003 DB. Because I'm using COM objects I'm importing > > win32com.client. That also allows me to use ADODB.connection and > > ADODB.command objects for working with SQL. > > > The program works fine but it's a little slow. Inserting ~5500 > > rows of data takes about 10 seconds using a DB that is on the same > > machine running the script. > > > I've done some general searches on how to speed this up and in > > other languages people suggest sending batches of inserts off at a > > time instead of executing 1 insert at a time. For java and .net > > people recommend using a stringbuilder function to create strings > > quickly. I don't know of such a function in python s I tried grouping > > my inserts into a single string using string += syntax. I knew that > > would be terrible but I wanted to see how terrible. Final reults: It > > was pretty terrible. Script went from taking ~18sec to taking > > 240sec. The overhead for recreating the strings was monster. No real > > surprise there. > > > So I then loaded up the commands into a list and at the end I > > used the strong join method to create the string. This was far faster > > than using += to create my strings but still took twice as long as > > just running my inserts one at a time. So I'm looking for > > suggestions. > > > Basically I have 5000 SQL inserts that I want to do as quickly as > > possible. This is purely academic as I can live with the 18 seconds > > the script needs to run (9 to talk to the com object and format the > > data and 10 to write to SQL) but I'm still curious how to improve on > > what I have running. > > Are you sure your logjam is in Python? Inserting 5500 rows can take a > few seconds if you're COMMITting after each INSERT. Wrap the whole > thing in an explicit transaction and see if that helps. > > Also, toss in a few print statements containing timestamps so you know > more about where the script is spending time. > > bye > Philip I'm not 100% sure it's python and not SQL but I do suspect there's a better way to do this than just serial inserts. I could be wrong about that which is what i'm trying to explore. I already do use the time stamps and this is what I see: at 9 secs in I've gotten my data, formatted it, and placed it on the list at 9.047 secs in the string.join() is done and I have my command string at 35 secs the program ends. So somehow my SQL is taking a lot longer when I format it as single string. How would I make the whole thing one transaction? Just insert BEGIN TRANSACTION at the start and COMMIT at the end? Is that enough to do it? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python, MS SQL, and batch inserts
On Apr 21, 3:36 pm, Scott David Daniels wrote: > Philip Semanchuk wrote: > > ... If you're doing a mass insert to populate a blank table it also often > > helps to postpone index creation until after the table is populated > > I forget the name of the SQL Server bulk loader, but for large loads, I > used to populate a fresh table with the bulk data, then do UPDATEs and > INSERTs to get the data spread out into the main tables. You (the OP) > might try a scheme like that. > > --Scott David Daniels > scott.dani...@acm.org Hmm..I think I'm going to move my question over to a SQL forum because this is starting to feel like a SQL, rather than a python issue to me. Three times now after letting the system "rest" where I go off an do other things and then run my script it completes in 10 seconds. If I drop tables and start fresh immediately after that it takes 35 seconds. If I drop the tables and wait an hour and then run the script it'll finish in 10 seconds again. That makes me think it's a SQL config or optimization issue more than a python issue. oh and the times I listed above were totals from the start of execution so the string.join() was taking 0.047 seconds to run. It was taking 9 seconds to get my data from the com object and format it but the join was quite fast. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python, MS SQL, and batch inserts
On Apr 21, 4:01 pm, ericwoodwo...@gmail.com wrote: > On Apr 21, 3:36 pm, Scott David Daniels wrote: > > > Philip Semanchuk wrote: > > > ... If you're doing a mass insert to populate a blank table it also often > > > helps to postpone index creation until after the table is populated > > > I forget the name of the SQL Server bulk loader, but for large loads, I > > used to populate a fresh table with the bulk data, then do UPDATEs and > > INSERTs to get the data spread out into the main tables. You (the OP) > > might try a scheme like that. > > > --Scott David Daniels > > scott.dani...@acm.org > > Hmm..I think I'm going to move my question over to a SQL forum because > this is starting to feel like a SQL, rather than a python issue to me. > > Three times now after letting the system "rest" where I go off an do > other things and then run my script it completes in 10 seconds. If I > drop tables and start fresh immediately after that it takes 35 > seconds. If I drop the tables and wait an hour and then run the > script it'll finish in 10 seconds again. > > That makes me think it's a SQL config or optimization issue more than > a python issue. > > oh and the times I listed above were totals from the start of > execution so the string.join() was taking 0.047 seconds to run. It > was taking 9 seconds to get my data from the com object and format it > but the join was quite fast. Also if I restart SQL it will respond very quickly as well...down to 10 secs again. Not sure why. -- http://mail.python.org/mailman/listinfo/python-list
