Re: Trying to understand html.parser.HTMLParser
On 05/16/2011 03:06 AM, David Robinow wrote:
On Sun, May 15, 2011 at 4:45 PM, Andrew Berg wrote:
I'm trying to understand why HMTLParser.feed() isn't returning the whole
page. My test script is this:
import urllib.request
import html.parser
class MyHTMLParser(html.parser.HTMLParser):
def handle_starttag(self, tag, attrs):
if tag == 'a' and attrs:
print(tag,'-',attrs)
url = 'http://x264.nl/x264/?dir=./64bit/8bit_depth'
page = urllib.request.urlopen(url).read()
parser = MyHTMLParser()
parser.feed(str(page))
I can do print(page) and get the entire HTML source, but
parser.feed(str(page)) only spits out the information for the top links
and none of the "revision" links. Ultimately, I just want to find
the name of the first "revision" link (right now it's
"revision1995", when a new build is uploaded it will be "revision2000"
or whatever). I figure this is a relatively simple page; once I
understand all of this, I can move on to more complicated pages.
You've got bad HTML. Look closely and you'll see the there's no space
between the "revision" strings and the style tag following.
The parser doesn't like this. I don't know a solution other than
fixing the html.
(I created a local copy, edited it and it worked.)
Hello,
Use regular expression for bad HTLM or beautifulSoup (google it), below
a exemple to extract all html links:
linksList = re.findall('.*?',htmlSource)
for link in linksList:
print link
Cheers
Karim
--
http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Sun, May 15, 2011 at 10:41 PM, Littlefield, Tyler wrote: > Hello: > Thanks all for your information and ideas. I like the idea of open source; I > have a fairly large (or large, by my standards anyway) project that I am > working on that is open source. > > Here's kind of what I want to prevent. I want to write a multi-player online > game; everyone will essentually end up connecting to my server to play the > game. I don't really like the idea of security through obscurity, but I > wanted to prevent a couple of problems. > 1) First I want to prevent people from hacking at the code, then using my > server as a test for their new setups. I do not want someone to gain some > extra advantage just by editing the code. > Is there some other solution to this, short of closed-source? > Thanks, I don't know that closing the source does you much more good than obfuscating it. The obvious attack surface here is pretty much totally exposed via network traffic, which any legitimate client can gain access to. A better approach would be to simply write more secure code in the first place. Geremy Condra -- http://mail.python.org/mailman/listinfo/python-list
Re: Import on case insensitive filesystem
En Fri, 13 May 2011 15:43:23 -0300, Mitchell Hashimoto escribió: I'm developing an app which runs Python on a filesystem which is not case sensitive (Mac OS X), but is mounted as an NFS drive on a remote machine. This causes errors because of the import being case sensitive but accessing an FS which is case insensitive. Short of copying the entire directory tree over to another filesystem, is there anything I can do to flag Python to act as though it were on a case sensitive FS? Try creating an environment variable PYTHONCASEOK with any value. See http://www.python.org/dev/peps/pep-0235/ for details. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Convert AWK regex to Python
Good morning all,
Wondering if you could please help me with the following query:-
I have just started learning Python last weekend after a colleague of mine
showed me how to dramatically cut the time a Bash script takes to execute by
re-writing it in Python. I was amazed at how fast it ran. I would now like to
do the same thing with another script I have.
This other script reads a log file and using AWK it filters certain fields from
the log and writes them to a new file. See below the regex the script is
executing. I would like to re-write this regex in Python as my script is
currently taking about 1 hour to execute on a log file with about 100,000
lines. I would like to cut this time down as much as possible.
cat logs/pdu_log_fe.log | awk -F\- '{print $1,$NF}' | awk -F\. '{print $1,$NF}'
| awk '{print $1,$4,$5}' | sort | uniq | while read service command status; do
echo "Service: $service, Command: $command, Status: $status, Occurrences: `grep
$service logs/pdu_log_fe.log | grep $command | grep $status | wc -l | awk '{
print $1 }'`" >> logs/pdu_log_fe_clean.log; done
This AWK command gets lines which look like this:-
2011-05-16 09:46:22,361 [Thread-4847133] PDU D
And outputs lines like this:-
CC_SMS_SERVICE_51408 submit_resp: 0
I have tried writing the Python script myself but I am getting stuck writing
the regex. So far I have the following:-
#!/usr/bin/python
# Import RegEx module
import re as regex
# Log file to work on
filetoread = open('/tmp/ pdu_log.log', "r")
# File to write output to
filetowrite = file('/tmp/ pdu_log_clean.log', "w")
# Perform filtering in the log file
linetoread = filetoread.readlines()
for line in linetoread:
filter0 = regex.sub(r"http://mail.python.org/mailman/listinfo/python-list
Re: Convert AWK regex to Python
On Mon, May 16, 2011 at 6:19 PM, J wrote:
> cat logs/pdu_log_fe.log | awk -F\- '{print $1,$NF}' | awk -F\. '{print
> $1,$NF}' | awk '{print $1,$4,$5}' | sort | uniq | while read service command
> status; do echo "Service: $service, Command: $command, Status: $status,
> Occurrences: `grep $service logs/pdu_log_fe.log | grep $command | grep
> $status | wc -l | awk '{ print $1 }'`" >> logs/pdu_log_fe_clean.log; done
Small side point: Instead of "| sort | uniq |", you could use a Python
dictionary. That'll likely speed things up somewhat!
Chris Angelico
--
http://mail.python.org/mailman/listinfo/python-list
Re: Convert AWK regex to Python
Good morning Angelico, Do I understand correctly? Do you mean incorporating a Python dict inside the AWK command? How can I do this? -- http://mail.python.org/mailman/listinfo/python-list
Re: Convert AWK regex to Python
On Mon, May 16, 2011 at 6:43 PM, J wrote: > Good morning Angelico, > Do I understand correctly? Do you mean incorporating a Python dict inside the > AWK command? How can I do this? No, inside Python. What I mean is that you can achieve the same uniqueness requirement by simply storing the intermediate data in a dictionary and then retrieving it at the end. Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Sun, 15 May 2011 23:41:23 -0600, Littlefield, Tyler wrote: > Here's kind of what I want to prevent. I want to write a multi-player > online game; everyone will essentually end up connecting to my server to > play the game. I don't really like the idea of security through > obscurity, but I wanted to prevent a couple of problems. 1) First I want > to prevent people from hacking at the code, then using my server as a > test for their new setups. I do not want someone to gain some extra > advantage just by editing the code. Is there some other solution to > this, short of closed-source? Thanks, Closed source is not a solution. Please wipe that out of your mind. People successfully hack closed source applications. The lack of source is hardly a barrier at all: it's like painting over the door to your house in camouflage colours so from a distance people won't see it. To a guy with a network sniffer and debugger, the lack of source is no barrier at all. You're trying to solve a hard problem, and by hard, I mean "impossible". It simply isn't possible to trust software on a machine you don't control, and pretty damn hard on a machine you do control. To put it in a nutshell, you can't trust *anything*. See the classic paper by Ken Thompson, "Reflections on Trusting Trust": http://cm.bell-labs.com/who/ken/trust.html Now, in a more practical sense, you might not fear that the operating system will turn on you, or the Python compiler. Some threats you don't care about. The threat model you do care about is a much more straight- forward one: how to trust the desktop client of your game? Alas, the answer is, you can't. You can't trust anything that comes from the client until you've verified it is unmodified, and you can't verify it is unmodified until you can trust the information it sends you. A vicious circle. You're fighting physics here. Don't think that obscuring the source code will help. On-line game servers are engaged in a never-ending arms race against "punks" who hack the clients. The servers find a way to detect one hack and block it, and the punks find another hack that goes unnoticed for a while. It's like anti-virus and virus, or immune systems and germs. The question you should be asking is not "how do I make this secure against cheats?", but "how much cheating can I afford to ignore?". If your answer is "No cheating is acceptable", then you have to do all the computation on the server, nothing on the client, and to hell with performance. All your client does is the user interface part. If the answer is, "Its a MUD, who's going to cheat???" then you don't have to do anything. Trust your users. If the benefit from "cheating" is small enough, and the number of cheaters low, who cares? You're not running an on-line casino for real money. See also here: http://web.archiveorange.com/archive/v/bqumydkHsi2ytdsX7ewa Another approach might be to use psychology on your users. Run one server for vanilla clients to connect to, and another server where anything goes. Let the punks get it out of their system by competing with other punks. Run competitions to see who can beat the most souped up, dirty, cheating turbo-powered clients, for honour and glory. Name and shame the punks who cheat on the vanilla server, praise the best cheaters on the anything-goes machine, and you'll (hopefully!) find that the level of cheating on the vanilla server is quite low. Who wants to be the low-life loser who wins by cheating when you can challenge your hacker peers instead? (Note: I don't know if this approach ever works, but I know it does *not* work when real money or glory is involved. Not even close.) If Blizzard can't stop private servers, rogue clients and hacked accounts, what makes you think you can? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Mon, May 16, 2011 at 6:49 PM, Steven D'Aprano wrote: > If your answer is "No cheating is acceptable", then you have to do all > the computation on the server, nothing on the client, and to hell with > performance. All your client does is the user interface part. > > If the answer is, "Its a MUD, who's going to cheat???" then you don't > have to do anything. Trust your users. If the benefit from "cheating" is > small enough, and the number of cheaters low, who cares? You're not > running an on-line casino for real money. The nearest I've seen to the latter is Dungeons and Dragons. People can cheat in a variety of ways, but since they're not playing *against* each other, cheating is rare. As to the former, though... the amount of computation that you can reliably offload to even a trusted client is low, so you don't lose much by doing it all on the server. The most computationally-intensive client-side work would be display graphics and such, and that's offloadable if and ONLY if there's no game-sensitive information hidden behind things. Otherwise someone could snoop the traffic-stream and find out what's behind that big nasty obstacle, or turn the obstacle transparent, or whatever... not safe. There's an old OS/2 game called Stellar Frontier that moves sprites around on the screen using clientside code, but if there's a bit of lag talking to the server, you see a ship suddenly yoinked to its new position when the client gets the latest location data. That's a fair compromise, I think; the client predicts where the ship "ought to be", and the server corrects it when it can. Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: turn monitor off and on
En Sat, 14 May 2011 03:08:44 -0300, Astan Chee escribió: I'm trying to turn off my monitor, pause and then turn it on again. I'm doing this in python 2.6 and windows xp. Here is my script so far (that doesn't work): def turnOnMonitor(): SC_MONITORPOWER = 0xF170 win32gui.SendMessage(win32con.HWND_BROADCAST, win32con.WM_SYSCOMMAND, SC_MONITORPOWER, -1) For some reason, the script doesn't turn the monitor back on. What am I doing wrong here or are there any other alternative? Your script worked fine for me, 2.6 and XP also. Perhaps your monitor device driver is buggy or does not implement the required functionality. Mine is from Philips. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
Littlefield, Tyler wrote: Hello: Thanks all for your information and ideas. I like the idea of open source; I have a fairly large (or large, by my standards anyway) project that I am working on that is open source. Here's kind of what I want to prevent. I want to write a multi-player online game; everyone will essentually end up connecting to my server to play the game. I don't really like the idea of security through obscurity, but I wanted to prevent a couple of problems. 1) First I want to prevent people from hacking at the code, then using my server as a test for their new setups. I do not want someone to gain some extra advantage just by editing the code. Is there some other solution to this, short of closed-source? Thanks, If your App meet some success, you'll need some help. You'll be able to get some only if the community grows and has access to your code. If you want to battle versus hackers, you have already lost (if your app hos no success, there will be no hacker anyway :o) ) Otherwise I guess that most online games execute all decisions and state machine transitions at server side, which is the only code you can trust. The client only forwards user inputs to the server, and display the resulting effect . JM -- http://mail.python.org/mailman/listinfo/python-list
Re: Convert AWK regex to Python
J wrote:
> Good morning all,
> Wondering if you could please help me with the following query:-
> I have just started learning Python last weekend after a colleague of mine
> showed me how to dramatically cut the time a Bash script takes to execute
> by re-writing it in Python. I was amazed at how fast it ran. I would now
> like to do the same thing with another script I have.
>
> This other script reads a log file and using AWK it filters certain fields
> from the log and writes them to a new file. See below the regex the
> script is executing. I would like to re-write this regex in Python as my
> script is currently taking about 1 hour to execute on a log file with
> about 100,000 lines. I would like to cut this time down as much as
> possible.
>
> cat logs/pdu_log_fe.log | awk -F\- '{print $1,$NF}' | awk -F\. '{print
> $1,$NF}' | awk '{print $1,$4,$5}' | sort | uniq | while read service
> command status; do echo "Service: $service, Command: $command, Status:
> $status, Occurrences: `grep $service logs/pdu_log_fe.log | grep $command |
> grep $status | wc -l | awk '{ print $1 }'`" >> logs/pdu_log_fe_clean.log;
> done
>
> This AWK command gets lines which look like this:-
>
> 2011-05-16 09:46:22,361 [Thread-4847133] PDU D
> CC_SMS_SERVICE_51408_656-ServerThread-
VASPSessionThread-7ee35fb0-7e87-11e0-a2da-00238bce423b-TRX
> - 2011-05-16 09:46:22 - OUT - (submit_resp: (pdu: L: 53 ID: 8004
> Status: 0 SN: 25866) 98053090-7f90-11e0-a2da-00238bce423b (opt: ) ) >
>
> And outputs lines like this:-
>
> CC_SMS_SERVICE_51408 submit_resp: 0
>
> I have tried writing the Python script myself but I am getting stuck
> writing the regex. So far I have the following:-
For the moment forget about the implementation. The first thing you should
do is to describe the problem as clearly as possible, in plain English.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Converting a set into list
Chris Torek wrote: > >>> x = [3, 1, 4, 1, 5, 9, 2, 6] > >>> x > [3, 1, 4, 1, 5, 9, 2, 6] > >>> list(set(x)) > [1, 2, 3, 4, 5, 6, 9] > >>> > > Of course, this trick only works if all the list elements are > hashable. > > This might not be the best example since the result is sorted > "by accident", while other list(set(...)) results are not. A minor change to your example makes it out of order even for integers: >>> x = [7, 8, 9, 1, 4, 1] >>> list(set(x)) [8, 9, 1, 4, 7] or for that mattter: >>> list(set([3, 32, 4, 32, 5, 9, 2, 6])) [32, 2, 3, 4, 5, 6, 9] -- Duncan Booth http://kupuguy.blogspot.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Convert AWK regex to Python
Hello Peter, Angelico, Ok lets see, My aim is to filter out several fields from a log file and write them to a new log file. The current log file, as I mentioned previously, has thousands of lines like this:- 2011-05-16 09:46:22,361 [Thread-4847133] PDU D All the lines in the log file are similar and they all have the same length (same amount of fields). Most of the fields are separated by spaces except for couple of them which I am processing with AWK (removing "http://mail.python.org/mailman/listinfo/python-list
Re: Convert AWK regex to Python
On Mon, 16 May 2011 03:57:49 -0700, J wrote:
> Most of the fields are separated by
> spaces except for couple of them which I am processing with AWK
> (removing " to do is evaluate each line in the log file and break them down into
> fields which I can call individually and write them to a new log file
> (for example selecting only fields 1, 2 and 3).
fields = line.split(' ')
output.write(fields[1] + ' ')
output.write(fields[2] + ' ')
output.write(fields[3] + '\n')
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Convert AWK regex to Python
J wrote: > Hello Peter, Angelico, > > Ok lets see, My aim is to filter out several fields from a log file and > write them to a new log file. The current log file, as I mentioned > previously, has thousands of lines like this:- 2011-05-16 09:46:22,361 > [Thread-4847133] PDU D CC_SMS_SERVICE_51408_656-ServerThread- VASPSessionThread-7ee35fb0-7e87-11e0-a2da-00238bce423b-TRX > - 2011-05-16 09:46:22 - OUT - (submit_resp: (pdu: L: 53 ID: 8004 > Status: 0 SN: 25866) 98053090-7f90-11e0-a2da-00238bce423b (opt: ) ) > > > All the lines in the log file are similar and they all have the same > length (same amount of fields). Most of the fields are separated by > spaces except for couple of them which I am processing with AWK (removing > " evaluate each line in the log file and break them down into fields which I > can call individually and write them to a new log file (for example > selecting only fields 1, 2 and 3). > > I hope this is clearer now Not much :( It doesn't really matter whether there are 100, 1000, or a million lines in the file; the important information is the structure of the file. You may be able to get away with a quick and dirty script consisting of just a few regular expressions, e. g. import re filename = ... def get_service(line): return re.compile(r"[(](\w+)").search(line).group(1) def get_command(line): return re.compile(r"http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Sun, 15 May 2011 23:41:23 -0600, Littlefield, Tyler wrote: > Here's kind of what I want to prevent. I want to write a multi-player > online game; everyone will essentually end up connecting to my server to > play the game. I don't really like the idea of security through > obscurity, but I wanted to prevent a couple of problems. > 1) First I want to prevent people from hacking at the code, then using > my server as a test for their new setups. I do not want someone to gain > some extra advantage just by editing the code. > Is there some other solution to this, short of closed-source? Closed source will not help in the slightest. What will help is to remember the fundamental rule of client-server security: Don't Trust The Client. If you don't remember this rule, you have no security whatsoever, whether the source is open or closed. Obfuscating the source won't prevent someone from running it under a modified Python interpreter, or running an unmodified Python interpreter under a debugger, or with modified DLLs (or even device drivers). To give just one example, Blizzard has a whole team of people working on anti-cheating measures, most of which involve installing various pieces of privacy-invading, security-endangering malware on their customers' systems. And it still doesn't work. -- http://mail.python.org/mailman/listinfo/python-list
Re: problem with GKT module?
On Sun, 15 May 2011 20:42:46 -0500, harrismh777 wrote:
> Alister Ware wrote:
>> I have a simple call back defined for a radio button widget when I use
>>> widget.name in linux I get a value of None, windows returns the
>>> widget name as I would expect.
>>>
>>>
> First, not familiar with your issue...
>
> ... but might be able to help you think through it...
>
> I am assuming that you are building a Python script using a glade
> interface that provides the widgets drag-an-drop style and then allowing
> you to take the default code, add to it, or otherwise modify it.
>
> I am also assuming that the /call back/ is returning a Python None, on
> the linux platform... otherwise you're getting the widget name from the
> same script on the windows platform?? right?
>
> It might be helpful to examine both scripts to see where (if any) they
> differ. More likely than not, this little snag is a difference in the
> way that the windows version of gtk+ libraries are working, than the
> original ones on the linux platform.
>
> On the other hand, the Python wrappers for the gtk+ library on the linux
> platform may be hiding the return values. Python functions return 'None'
> if the 'return' is not explicitly coded. The gtk+ libraries may be
> returning a value but the 'builder' is not generating the right Python
> wrapper. I'm making this up, but you get the idea, and you can probably
> check from here.
>
> On the other hand, folks here can enter into a discussion with you
> regarding the generated Python code (output from the builder) if you
> provide relevant code snippets.
>
>
> Kind regards,
> m harris
glade generates an xml file that is processed by the python script
it is the same ml file & the same python script running on both platforms
the basic structure of my call back for testing is:-
import gtk
class GUI:
def __init__(self):
builder=gtk.Builder()
builder.add_from_file('glade.test') # xml file from glade
builder.get_object('window1').show()
builder.connect_signals(self)
def callback(self,widget,data=None):
print widget#gives reference to radio button ok
print widget.name #widget name on windoze, None on linux
def main (self):
gtk.main()
def main():
gui=GUI()
gui.main()
if __name__ =='__main__':main()
I can provide more detailed sample code if required (including the xml
from glade)
--
Has everyone noticed that all the letters of the word "database" are
typed with the left hand? Now the layout of the QWERTYUIOP typewriter
keyboard was designed, among other things, to facilitate the even use
of both hands. It follows, therefore, that writing about databases is
not only unnatural, but a lot harder than it appears.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Parsing a graph image
Hi,
the project sounds like the exact tool that i need but regarding the
user manual one has to mark the points on the graph manually. Therefore
it's more work to get the data out than doing it without a tool. Or may
I miss something here?
Greets
Basti
Am Fri, 13 May 2011 14:38:45 -0500
schrieb Robert Kern :
> On 5/13/11 7:24 AM, Bastian Ballmann wrote:
> > Hi,
> >
> > Am Fri, 13 May 2011 14:01:48 +0200
> > schrieb Ulrich Eckhardt:
> >
> >> I'm not sure I understand 100% what you want. If you want to
> >> extract ("parse") the data that is contained in an image file, I
> >> have no clue how to do that.
> >
> > Yes, I want to extract the data that is contained in an image file.
>
> There is nothing in Python that solves this problem, per se, but
> there are free and open source tools for this out there. E.g.
>
>http://digitizer.sourceforge.net/
>
signature.asc
Description: PGP signature
--
http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On 2011-05-16, Ben Finney wrote: > "Littlefield, Tyler" writes: > >> I'm putting lots of work into this. I would rather not have some >> script kiddy dig through it, yank out chunks and do whatever he wants. >> I just want to distribute the program as-is, not distribute it and >> leave it open to being hacked. > > How do these arguments apply to your code base when they don't apply to, > say, LibreOffice or Linux or Python or Apache or Firefox? One obvious way that those arguments don't apply is that the OP didn't put lots of work into LibreOffice, Linux, Python, Apache or Firefox and therefore doesn't have any right to control their distribution. > How is your code base going to be harmed by having the source code > available to recipients, when that demonstrably doesn't harm > countless other code bases out there? The owner of something is free to determine how it is distributed -- he doesn't have any obligation to prove to you that some particular method of distribution is harmful to him or anybody else. -- Grant Edwards grant.b.edwardsYow! at BI-BI-BI-BI-BI-BI-BI-BI-BI-BI-BI-BI-BI-BI-BI-BI-BI-BI-BI-BI-BI-BI-BI-BI- gmail.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Convert AWK regex to Python
Thanks for the sugestions Peter, I will give them a try
Peter Otten wrote:
> J wrote:
>
> > Hello Peter, Angelico,
> >
> > Ok lets see, My aim is to filter out several fields from a log file and
> > write them to a new log file. The current log file, as I mentioned
> > previously, has thousands of lines like this:- 2011-05-16 09:46:22,361
> > [Thread-4847133] PDU D > CC_SMS_SERVICE_51408_656-ServerThread-
> VASPSessionThread-7ee35fb0-7e87-11e0-a2da-00238bce423b-TRX
> > - 2011-05-16 09:46:22 - OUT - (submit_resp: (pdu: L: 53 ID: 8004
> > Status: 0 SN: 25866) 98053090-7f90-11e0-a2da-00238bce423b (opt: ) ) >
> >
> > All the lines in the log file are similar and they all have the same
> > length (same amount of fields). Most of the fields are separated by
> > spaces except for couple of them which I am processing with AWK (removing
> > " > evaluate each line in the log file and break them down into fields which I
> > can call individually and write them to a new log file (for example
> > selecting only fields 1, 2 and 3).
> >
> > I hope this is clearer now
>
> Not much :(
>
> It doesn't really matter whether there are 100, 1000, or a million lines in
> the file; the important information is the structure of the file. You may be
> able to get away with a quick and dirty script consisting of just a few
> regular expressions, e. g.
>
> import re
>
> filename = ...
>
> def get_service(line):
> return re.compile(r"[(](\w+)").search(line).group(1)
>
> def get_command(line):
> return re.compile(r"
> def get_status(line):
> return re.compile(r"Status:\s+(\d+)").search(line).group(1)
>
> with open(filename) as infile:
> for line in infile:
> print get_service(line), get_command(line), get_status(line)
>
> but there is no guarantee that there isn't data in your file that breaks the
> implied assumptions. Also, from the shell hackery it looks like your
> ultimate goal seems to be a kind of frequency table which could be built
> along these lines:
>
> freq = {}
> with open(filename) as infile:
> for line in infile:
> service = get_service(line)
> command = get_command(line)
> status = get_status(line)
> key = command, service, status
> freq[key] = freq.get(key, 0) + 1
>
> for key, occurences in sorted(freq.iteritems()):
> print "Service: {}, Command: {}, Status: {}, Occurences: {}".format(*key
> + (occurences,))
--
http://mail.python.org/mailman/listinfo/python-list
Re: Convert AWK regex to Python
J writes:
> cat logs/pdu_log_fe.log | awk -F\- '{print $1,$NF}' | awk -F\. '{print
> $1,$NF}' | awk '{print $1,$4,$5}' | sort | uniq | while read service command
> status; do echo "Service: $service, Command: $command, Status: $status,
> Occurrences: `grep $service logs/pdu_log_fe.log | grep $command | grep
> $status | wc -l | awk '{ print $1 }'`" >> logs/pdu_log_fe_clean.log; done
>
> This AWK command gets lines which look like this:-
>
> 2011-05-16 09:46:22,361 [Thread-4847133] PDU D CC_SMS_SERVICE_51408_656-ServerThread-VASPSessionThread-7ee35fb0-7e87-11e0-a2da-00238bce423b-TRX
> - 2011-05-16 09:46:22 - OUT - (submit_resp: (pdu: L: 53 ID: 8004 Status:
> 0 SN: 25866) 98053090-7f90-11e0-a2da-00238bce423b (opt: ) ) >
>
> And outputs lines like this:-
>
> CC_SMS_SERVICE_51408 submit_resp: 0
>
i see some discrepancies in the description of your problem
1. if i echo a properly quoted line "like this" above in the pipeline
formed by the first three awk commands i get
$ echo $likethis | awk -F\- '{print $1,$NF}' \
| awk -F\. '{print$1,$NF}' \
| awk '{print $1,$4,$5}'
2011 ) )
$
not a triple 'service command status'
2. with regard to the final product, you script outputs lines like in
echo "Service: $service, [...]"
and you say that it produces lines like
CC_SMS_SERVICE_51408 submit_resp:
WHATEVER, the abnormous run time is due to the fact that for every
output line you rescan again and again the whole log file
IF i had understood what you want, imho you should run your data
through sort and uniq -c
$ awk -F\- '{print $1,$NF}' < $file \
| awk -F\. '{print$1,$NF}' \
| awk '{print $1,$4,$5}'| sort | uniq -c | format_program
uniq -c drops repeated lines from a sorted input AND prepends to each
line the count of equal lines in the original stream
hth
g
--
http://mail.python.org/mailman/listinfo/python-list
Re: Converting a set into list
Thomas Rachel wrote: > Which loops do you mean here? list(set) has been proved to largely win against list = [] for item in set: list.append(item) or [list.append(item) for item in set] -- goto /dev/null -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
>Funny you should mention this "now" I don't go around parading the info, until I have to. >Yes I agree Flash is not very accessible (never has been). >Web Standards web apps and such however are quite >accessible! If I was making a browser-based game, yes. As I'm not though... Anyway, thanks to everyone else who answered this thread. I've not done much like this besides muds, and all the logic is on the server there, I think I will build the client in python, open source it for people to fix/add to if they want and make sure to keep the server as secure as it can be. -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Mon, May 16, 2011 at 12:17 AM, Littlefield, Tyler wrote: >>Write your "game" for the "web". >>Write is as a SaaS (Software as a Service) - even if it's free and open >> source. > I understood you loud and clear. And that makes a lot of assumptions on my > game and the design. I don't really care to host this over the web. I want a > centralized server that would perform the logic, where I can offload the > playing of sounds (through a soundpack that's already installed) to the > client-side. > Not only that, but a lot of web technologies that would be used for this > wouldn't really work, as I am doing this for the blind; Flash as well as a > lot > of the popular setups are not very accessible. Probably the best thing you can do is just to treat your client as untrusted. That means: 1) Avoid sending it any data that you would not want the user to have. If they want it, they'll find a way to get it. 2) Don't rely on the client to restrict the user. If they're not supposed to be able to send a command 10 times a second, that should be enforced by the server. If the client has no useful data beyond what is normally presented to the user, and if every input that the server will accept can be done with the vanilla client, then there is no way to cheat, and the only reason remaining to hack up the client that could be detrimental to gameplay is for botting. For that, I'm afraid you'll just have to employ detection algorithms and enforce a strict no-botting policy. Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing a graph image
On 5/16/11 8:38 AM, Bastian Ballmann wrote: Hi, the project sounds like the exact tool that i need but regarding the user manual one has to mark the points on the graph manually. Therefore it's more work to get the data out than doing it without a tool. Or may I miss something here? You are probably looking at the tutorials for manually digitizing graphs. Check out this one: http://digitizer.sourceforge.net/usermanual/tutorautolinegraph.html -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: Convert AWK regex to Python
This doesn't directly bear upon the posted example, but I found the following tutorial extremely helpful for learning how to parse log files with idiomatic python. Maybe you'll might find it useful, too. http://www.dabeaz.com/generators/ http://www.dabeaz.com/generators/Generators.pdf -- http://mail.python.org/mailman/listinfo/python-list
KAJAL old pictures
she is good and most poplar actress she is looking very cute and milky white and big eyes and good looking she always smiley face she act wit top and young hero's Ramcharan teja, JR.Ntr, Ram,Prabas,Kalyanram,Allu Arjun, and also top tamil hero's her more pictures wallpapers photos for you watch and enjoy http://trisha4uuu.blogspot.com/2011/05/kajal-hot-old-wallpapers.html -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On 05/15/2011 10:29 PM, Ben Finney wrote: What is it you think you would gain by obfuscating the code, and why is that worthwhile? What evidence do you have that code obfuscation would achieve that? Based on past experience at several employers', the preeminent reason for obfuscating is to make it hard for people to see the absolutely atrocious coding that takes place behind closed doors. Having seen behind the curtain, every time I hear the buzzword(s) "enterprise software", I now equate that with "shovelware pushed out the door driven by a marketing agenda rather than customer needs; held together by baling wire and string; lacking test-suites, UI testing, or attention to failure conditions; random hit-or-miss deployment processes; usually a lack of decent revision control; etc". -tkc -- http://mail.python.org/mailman/listinfo/python-list
Re: Convert AWK regex to Python
On 16/05/2011 09:19, J wrote:
[snip]
#!/usr/bin/python
# Import RegEx module
import re as regex
# Log file to work on
filetoread = open('/tmp/ pdu_log.log', "r")
# File to write output to
filetowrite = file('/tmp/ pdu_log_clean.log', "w")
# Perform filtering in the log file
linetoread = filetoread.readlines()
for line in linetoread:
filter0 = regex.sub(r"
[snip]
If you don't need the power of regex, it's faster to use string methods:
filter0 = line.replace("http://mail.python.org/mailman/listinfo/python-list
regular expression i'm going crazy
pls help me fixing this: import re s = "linka la baba" re_s = re.compile(r'(link|l)a' , re.IGNORECASE) print re_s.findall(s) output: ['link', 'l'] why? i want my re_s to find linka and la, he just find link and l and forget about the ending a. can anyone help me? trying the regular expression in redemo.py (program provided with python to explore the use of regular expression) i get what i want, so i guess re_s is ok, but it still fail... why? help! Nico -- http://mail.python.org/mailman/listinfo/python-list
Re: regular expression i'm going crazy
On 5/16/11 11:25 AM, Tracubik wrote: pls help me fixing this: import re s = "linka la baba" re_s = re.compile(r'(link|l)a' , re.IGNORECASE) print re_s.findall(s) output: ['link', 'l'] why? i want my re_s to find linka and la, he just find link and l and forget about the ending a. can anyone help me? trying the regular expression in redemo.py (program provided with python to explore the use of regular expression) i get what i want, so i guess re_s is ok, but it still fail... why? The parentheses () create a capturing group, which specifies that the contents of the group should be extracted. See the "(...)" entry here: http://docs.python.org/library/re#regular-expression-syntax You can use the non-capturing version of parentheses if you want to just isolate the | from affecting the rest of the regex: """ (?:...) A non-capturing version of regular parentheses. Matches whatever regular expression is inside the parentheses, but the substring matched by the group cannot be retrieved after performing a match or referenced later in the pattern. """ [~] |1> import re [~] |2> s = "linka la baba" [~] |3> re_s = re.compile(r'(?:link|l)a' , re.IGNORECASE) [~] |4> print re_s.findall(s) ['linka', 'la'] -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -- http://mail.python.org/mailman/listinfo/python-list
Re: regular expression i'm going crazy
On 16.05.2011 18:25, Tracubik wrote: pls help me fixing this: import re s = "linka la baba" re_s = re.compile(r'(link|l)a' , re.IGNORECASE) print re_s.findall(s) output: ['link', 'l'] why? As the docs say: "If one or more groups are present in the pattern, return a list of groups;" http://docs.python.org/library/re.html?highlight=findall#re.findall i want my re_s to find linka and la, he just find link and l and forget about the ending a. Try with non-grouping parentheses: re_s = re.compile(r'(?:link|l)a' , re.IGNORECASE) -- http://mail.python.org/mailman/listinfo/python-list
Re: regular expression i'm going crazy
On 16/05/11 17:25, Tracubik wrote:
pls help me fixing this:
import re
s = "linka la baba"
re_s = re.compile(r'(link|l)a' , re.IGNORECASE)
print re_s.findall(s)
output:
['link', 'l']
why?
i want my re_s to find linka and la, he just find link and l and forget
about the ending a.
The round brackets define a 'capturing group'. I.e. when you do findall
it returns those elements in the string that match what's inside the
brackets. If you want to get linka and la, you need something like this:
>>> re_s = re.compile(r'((link|l)a)' , re.IGNORECASE)
>>> print re_s.findall(s)
[('linka', 'link'), ('la', 'l')]
Then just look at the first element in each of the tuples in the array
(which matches the outside set of brackets).
see:
http://www.regular-expressions.info/python.html
--
http://mail.python.org/mailman/listinfo/python-list
Image processing to auto adjust colors in python ?
Hi, I'm processing thumbnails with python but one thing I need to do is to auto-adjust an image for the "best" colors. I am using ffmpeg to get thumbs three seconds into a video, sometimes the image is too dark. Normally I'd go into something like the windows program Irfan View and do "Auto Adjust Colors", but I need a comand-line solution to processes hundreds of images in a loop. I'm using Ubuntu and Python. Is there an image precessing package i could use to do this? thanks, Lee G. -- http://mail.python.org/mailman/listinfo/python-list
Re: Image processing to auto adjust colors in python ?
On Mon, 16 May 2011 10:11:43 -0700, goldtech wrote: > I'm processing thumbnails with python but one thing I need to do is to > auto-adjust an image for the "best" colors. I am using ffmpeg to get > thumbs three seconds into a video, sometimes the image is too dark. > Normally I'd go into something like the windows program Irfan View and > do "Auto Adjust Colors", but I need a comand-line solution to > processes hundreds of images in a loop. I'm using Ubuntu and Python. > Is there an image precessing package i could use to do this? PIL (Python Imaging Library) is the most widely-used general-purpose image library. For more advanced tasks, the usual solution is to use PIL to load/save images and do the processing with NumPy/SciPy (array-processing library). But you might want to check whether either ImageMagick (e.g. "convert -equalize ...") or NetPBM (e.g. pnmhisteq) do what you want. If they do, it will probably be the simplest solution. -- http://mail.python.org/mailman/listinfo/python-list
Call for Papers (CFP)
= Journal of Emerging Trends in Computing and Information Sciences Call for Research Papers (Vol. 2 No. 6) June 2011 http://cisjournal.org/ = Dear Sir/ Madam, Journal of Emerging Trends in Computing and Information Sciences (E- ISSN 2218-6301/ ISSN 2079-8407) is an international refereed research publishing journal, focused on promoting and publishing original high quality research work in both theoretical and scientific aspects of all disciplines of Computing and Information Sciences. The objectives of the journal are to promote and publish original high quality research and to provide a forum to the researchers and industry practitioners for exchanging ideas, knowledge, and experience. We welcome original research and industry experience papers. Contributions should be written for one of the following categories: Original research Literature Review / Systematic Literature Review Short Articles on ongoing research Preliminary Findings Technical Reports / Notes Results previously published in conferences and/or journals may be submitted as extended versions. For more information about Journal and Publication Charges, please visit http://www.cisjournal.org/. You are requested to circulate this message among your colleagues and college/university fellows. Sincerely Yours, Editor Journal of Emerging Trends in Computing and Information Sciences URL: http://www.cisjournal.org E-mail:edi...@cisjournal.org -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
Steven D'Aprano wrote: To put it in a nutshell, you can't trust*anything*. See the classic paper by Ken Thompson, "Reflections on Trusting Trust": This is true, but there's another way to put it pro-active--- ... expect the client to be untrustworthy. In other words, write the server code with a protocol that 'expects' the client to be hacked. Yes, it takes three times the code and at least five times the work, but its worth it. What do you do with syn floods? What do you do with attempted overruns? What if someone builds a client emulator, just to hammer your protocol and slow the server down, just for fun...? You must build your server side 'assuming' that *all* of these things are going to happen (and more), and then be able to handle them when they do. That is what makes server-side coding so difficult. In other words, you build the server in such a way that you can confidently hand Mr junior cracker your client source code and be confident that your gaming server is going to be a.o.k. Many, many, coders don't want to go to all this trouble (and don't)... mainly because they're just glad if they can get simple sockets to work. So, they don't handle attempted overruns, or syn flood open attempts, or other. One thing to remember (think about this) is whether your server/client is in a push or pull mode. *Never* allow the client to be in control (pushing) while your server is passively (pulling). The server must control everything so that the untrusted client will be *controlled* regardless of client side hacks. I realize that this probably means redesign of your server. Do it. Happy gaming! m harris -- http://mail.python.org/mailman/listinfo/python-list
Re: problem with GKT module?
alister ware wrote: def callback(self,widget,data=None): print widget #gives reference to radio button ok print widget.name #widget name on windoze, None on linux Well, you're obviously using Python 2.x ... ... have you tried this in Python 3.x ? Neither here nor there... you need to know who/what is generating 'widget'. Is widget buried in gtk where you can't see how it works, or is widget a Python class (or wrapper) where you can see what its doing? The fact that widget.name returns None on the linux platform tells me that widget is a Python class (or wrapper) ... so you probably have a widget.py file somewhere... or some other module that widget is a class definition in... who knows... (at this point). It seems to me that the 'builder' is behaving differently on the two platforms (rather than Python is behaving differently on the two platforms). What happens if you change the data=None pair? What happens if you omit data=None pair? Do you see this difference with *all* widgets, or just radio buttons? kind regards, m harris -- http://mail.python.org/mailman/listinfo/python-list
Re: Get the IP address of WIFI interface
Neal Becker writes:
> Here's some useful snippits for linux:
>
> def get_default_if():
> f = open('/proc/net/route')
> for i in csv.DictReader(f, delimiter="\t"):
> if long(i['Destination'], 16) == 0:
> return i['Iface']
> return None
>
> def get_ip_address(ifname):
> s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
> return socket.inet_ntoa(fcntl.ioctl(
> s.fileno(),
> 0x8915, # SIOCGIFADDR
> struct.pack('256s', ifname[:15])
> )[20:24])
One possible solution in Linux is asking NetworkManager, if it's in
use. It knows which interfaces are active and what kind they are (LAN,
WLAN, WWAN etc.) NetworkManager communicates via dbus and even
includes python example scripts. So here's my scriptlet based on
NetworkManager example nm-state.py. This one prints out all active
devices and their type and IP address. Easily modified to print only
WLAN types.
import dbus, socket, struct
bus = dbus.SystemBus()
proxy = bus.get_object("org.freedesktop.NetworkManager",
"/org/freedesktop/NetworkManager")
manager = dbus.Interface(proxy, "org.freedesktop.NetworkManager")
# Get device-specific state
devices = manager.GetDevices()
for d in devices:
dev_proxy = bus.get_object("org.freedesktop.NetworkManager", d)
prop_iface = dbus.Interface(dev_proxy, "org.freedesktop.DBus.Properties")
# Get the device's current state and interface name
state = prop_iface.Get("org.freedesktop.NetworkManager.Device", "State")
name = prop_iface.Get("org.freedesktop.NetworkManager.Device", "Interface")
ifa = "org.freedesktop.NetworkManager.Device"
type = prop_iface.Get(ifa, "DeviceType")
addr = prop_iface.Get(ifa, "Ip4Address")
# and print them out
if state == 8: # activated
addr_dotted = socket.inet_ntoa(struct.pack('http://mail.python.org/mailman/listinfo/python-list
Re: problem with GKT module?
Alister Ware writes: > On Fri, 13 May 2011 13:13:00 +, alister ware wrote: > >> I am using gtk.builder with a glade generated GUI >> >> I have a simple call back defined for a radio button widget when I use >> widget.name in linux I get a value of None, windows returns the widget >> name as I would expect. >> >> is this a bug? >> if not how should i find the name of the widget that has triggered a >> call back? >> >> (I would like all my radio buttons to go to the same callback routine if >> possible to make code maintenance easier) >> > So nobody has any Ideas on this at all? You may want to post runnable code demonstrating your problem. -- http://mail.python.org/mailman/listinfo/python-list
Re: Memcached in python 3
Do you need something shared across multiple hosts? Across multiple CPU's of the same host? Or single process? By "Python Dictionary Manager", do you mean the manager stuff in the multiprocessing module? On Sun, May 15, 2011 at 9:52 PM, Navkirat Singh wrote: > Hi Guys, > > How can I used memcached with python 3? Are there any other good > alternatives to memcached? What about python dictionary manager, would it > compare to memcached if I were to use it for storing in-memory information? > > Any light on this matter will be appreciated. > > Regards, > Navkirat > > -- > http://mail.python.org/mailman/listinfo/python-list > > -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Mon, 16 May 2011 03:21:00 +0100, Daniel Kluev wrote: On Mon, May 16, 2011 at 1:04 PM, Littlefield, Tyler wrote: Hello all: Finally, is there a good way to accomplish this? I know that I can make .pyc files, but those can be disassembled very very easily with the disassembler and shipping these still means that the person needs the modules that are used. Is there another way to go about this? No, there is no way to prevent users from getting access to raw python sources. By its nature and design, python is not meant to be used this way, and even obfuscation would not harm readability much. However, you can write all parts you want to hide in C/C++/Cython and distribute them as .so/.dll ...which is, of course, not exactly secure either. A sufficiently determined hacker won't have much trouble disassembling a shared library even if you do strip out all the debug information. By chance I'm having to do something closely related to this at work just at the moment; it's hard, but far from impossible. -- Rhodri James *-* Wildebeest Herder to the Masses -- http://mail.python.org/mailman/listinfo/python-list
pythonwebkit-gtk, pythonwebkit-dfb
in preparation for a 0.8 release of pyjamas, a bit of work has been
done on pythonwebkit (http://www.gnu.org/software/pythonwebkit) that
makes it easier to compile and install.
pythonwebkit provides full and complete (see caveats below!) bindings
to web browser functionality... in python. what you would normally
expect to be able to do in javascript "in-browser", you can do EXACTLY
the same thing, in a "declarative" programming style, in python:
import gtk
import pywebkitgtk
url = "http://www.gnu.org/software/pythonwebkit";
wv = pywebkitgtk.WebView(1024,768, url=url)
def _doc_loaded(*args):
doc = wv.GetDomDocument()
txt = doc.createTextNode("hello")
doc.body.appendChild(txt)
wv.SetDocumentLoadedCallback(_doc_loaded)
gtk.main()
yes, that's really python, doing a createTextNode and an appendChild,
*not* javascript. not interpreted javascript, not interpreted python,
*real* python, byte-coded and everything. throw in some AJAX, some
browser event callbacks (onclick etc.) and some web browser timer
callbacks and it all starts to get a bit weird, as two or maybe three
disparate programming worlds that should never really have been
brought together suddenly.. um... well, are brought together.
the bit that's easier about installing pythonwebkit is that it is no
longer necessary to download and patch up the
http://code.google.com/p/pywebkitgtk project in order to use
pythonwebkit. you can simply do "./autogen.sh" followed by the usual
"make" and "make install". a new and absolute minimalist python
module is created and installed which will get you a blank window -
just like if you were firing up a python-GTK application or a
python-QT4 application.
anyway - just a bit of an informal not-really-announcement because,
well, it's a side-dependency to the pyjamas project, even if it is a
whopping 20mb one. those caveats btw are that a) you can't set CSS
properties as if they were python object properties: you have to use
the method "setProperty", duh, and b) there are *no* 2D or 3D SVG
Canvas objects or functions available, yet, because it would take a
good full-time 7 to 10 days to smack the codegenerator into shape and
i'm waiting for someone to step forward and fund that work. am still
servicing £20,000 in debt and still have to find a way to pay back a
complete stranger who incredibly kindly paid £4,000 in owed rent so
that we did not end up with a County Court Judgement against us.
myself, my partner and our 25 month old daughter still got evicted,
but that's another story.
against this kind of background, perhaps i might be forgiven for not
doing "freebie" free software development, i trust.
l.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Get the IP address of WIFI interface
Thanks, this code works perfectly in ubuntu 10.04.
one question though, is dbus usually implemented in other distribution of
linux?
On Mon, May 16, 2011 at 12:57 PM, Anssi Saari wrote:
> Neal Becker writes:
>
> One possible solution in Linux is asking NetworkManager, if it's in
> use. It knows which interfaces are active and what kind they are (LAN,
> WLAN, WWAN etc.) NetworkManager communicates via dbus and even
> includes python example scripts. So here's my scriptlet based on
> NetworkManager example nm-state.py. This one prints out all active
> devices and their type and IP address. Easily modified to print only
> WLAN types.
>
> import dbus, socket, struct
>
> bus = dbus.SystemBus()
>
> proxy = bus.get_object("org.freedesktop.NetworkManager",
> "/org/freedesktop/NetworkManager")
> manager = dbus.Interface(proxy, "org.freedesktop.NetworkManager")
>
> # Get device-specific state
> devices = manager.GetDevices()
> for d in devices:
>dev_proxy = bus.get_object("org.freedesktop.NetworkManager", d)
>prop_iface = dbus.Interface(dev_proxy,
> "org.freedesktop.DBus.Properties")
>
># Get the device's current state and interface name
>state = prop_iface.Get("org.freedesktop.NetworkManager.Device", "State")
>name = prop_iface.Get("org.freedesktop.NetworkManager.Device",
> "Interface")
>ifa = "org.freedesktop.NetworkManager.Device"
>type = prop_iface.Get(ifa, "DeviceType")
>addr = prop_iface.Get(ifa, "Ip4Address")
>
># and print them out
>if state == 8: # activated
>addr_dotted = socket.inet_ntoa(struct.pack('
>s = "Device %s is activated and has type %s and address %s"
>print s % (name, type, addr_dotted)
>else:
>print "Device %s is not activated" % name
>
>
>
> --
> http://mail.python.org/mailman/listinfo/python-list
>
--
http://mail.python.org/mailman/listinfo/python-list
Deleting a file?
A file can be deleted by opening it with mode os.O_TEMPORARY and then closing it. How can a file be moved to the Recycle Bin, a la Windows? -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
"Littlefield, Tyler" writes: > I wanted to make the client in python, and the server possibly, though > I'm not really sure on that. I was not worried about the code for the > server being stolen, as much as I was worried about people tinkering > with the client code for added advantages. Thank you for making your constraints explicit; that's more than most people do when asked. As Steven said, you're trying to solve a problem which is very difficult, and obfuscating the code won't be of much help. If people have the program running on their own computers, they can hack it. You can't stop that, so you have to consider other ways of making it ineffective. -- \ “The fact that a believer is happier than a skeptic is no more | `\ to the point than the fact that a drunken man is happier than a | _o__) sober one.” —George Bernard Shaw | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
Grant Edwards writes: > On 2011-05-16, Ben Finney wrote: > > "Littlefield, Tyler" writes: > > > >> I'm putting lots of work into this. I would rather not have some > >> script kiddy dig through it, yank out chunks and do whatever he > >> wants. I just want to distribute the program as-is, not distribute > >> it and leave it open to being hacked. > > > > How do these arguments apply to your code base when they don't apply > > to, say, LibreOffice or Linux or Python or Apache or Firefox? > > One obvious way that those arguments don't apply is that the OP didn't > put lots of work into LibreOffice, Linux, Python, Apache or Firefox Yet the copyright holders *did* put lots of effort into those works respectively. So the arguments would apply equally well; which is to say, they don't. > > How is your code base going to be harmed by having the source code > > available to recipients, when that demonstrably doesn't harm > > countless other code bases out there? > > The owner of something is free to determine how it is distributed -- > he doesn't have any obligation to prove to you that some particular > method of distribution is harmful to him or anybody else. Note that I didn't say anything about obligation or harm to persons. I asked only about the code base and the distribution thereof. In the meantime, Tyler has come back to us with arguments that *do* differentiate between the above cases and his own. So thanks, Tyler, for answering the questions. -- \“Of course, everybody says they're for peace. Hitler was for | `\ peace. Everybody is for peace. The question is: what kind of | _o__)peace?” —Noam Chomsky, 1984-05-14 | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
"Littlefield, Tyler" writes: > Anyway, thanks to everyone else who answered this thread. I've not > done much like this besides muds, and all the logic is on the server > there, I think I will build the client in python, open source it for > people to fix/add to if they want and make sure to keep the server as > secure as it can be. Sounds like a good approach to me that doesn't treat users as necessarily hostile. I wish you good fortune in building a strong community around the game so that it can defend itself from cheaters, and a free-software client will IMO promote exactly that. -- \ “I do not believe in immortality of the individual, and I | `\consider ethics to be an exclusively human concern with no | _o__) superhuman authority behind it.” —Albert Einstein, letter, 1953 | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Converting a set into list
TheSaint writes: > Thomas Rachel wrote: > > > Which loops do you mean here? > > list(set) has been proved to largely win against > list = [] > for item in set: > list.append(item) > or [list.append(item) for item in set] Remember that the criterion of speed is a matter of the implementation, and what's fast on one won't necessarily be fast on others. Which implementations did you try? Where I do agree is that ‘list(foo)’ wins over the other examples you show on the important criteria of concision and readability. -- \ “A thing moderately good is not so good as it ought to be. | `\Moderation in temper is always a virtue; but moderation in | _o__) principle is always a vice.” —Thomas Paine | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Trying to understand html.parser.HTMLParser
On 2011.05.16 02:26 AM, Karim wrote:
> Use regular expression for bad HTLM or beautifulSoup (google it), below
> a exemple to extract all html links:
>
> linksList = re.findall('.*?',htmlSource)
> for link in linksList:
> print link
I was afraid I might have to use regexes (mostly because I could never
understand them).
Even the BeautifulSoup website itself admits it's awful with Python 3 -
only the admittedly broken 3.1.0 will work with Python 3 at all.
ElementTree doesn't seem to have been updated in a long time, so I'll
assume it won't work with Python 3.
lxml looks promising, but it doesn't say anywhere whether it'll work on
Python 3 or not, which is puzzling since the latest release was only a
couple months ago.
Actually, if I'm going to use regex, I might as well try to implement
Versions* in Python.
Thanks for the answers!
*http://en.totalcmd.pl/download/wfx/net/Versions (original, made for
Total Commander) and
https://addons.mozilla.org/en-US/firefox/addon/versions-wfx_versions/
(clone implemented as a Firefox add-on; it's so wonderful, I even wrote
the docs for it!)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Converting a set into list
>Chris Torek wrote: >> >>> x = [3, 1, 4, 1, 5, 9, 2, 6] >> >>> list(set(x)) >> This might not be the best example since the result is sorted >> "by accident", while other list(set(...)) results are not. In article , Duncan Booth wrote: >A minor change to your example makes it out of order even for integers: > x = [7, 8, 9, 1, 4, 1] list(set(x)) >[8, 9, 1, 4, 7] > >or for that mattter: > list(set([3, 32, 4, 32, 5, 9, 2, 6])) >[32, 2, 3, 4, 5, 6, 9] Yes, but then it is no longer "as easy as pi". :-) -- In-Real-Life: Chris Torek, Wind River Systems Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603 email: gmail (figure it out) http://web.torek.net/torek/index.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Deleting a file?
"garyr" writes: > A file can be deleted by opening it with mode os.O_TEMPORARY and then > closing it. Much simpler: ‘os.remove(path)’. > How can a file be moved to the Recycle Bin, a la Windows? That's not deleting it (as you probably know), so you might better change the subject field for the thread. -- \ “Unix is an operating system, OS/2 is half an operating system, | `\Windows is a shell, and DOS is a boot partition virus.” —Peter | _o__)H. Coffin | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Deleting a file?
On Mon, May 16, 2011 at 5:23 PM, garyr wrote: > A file can be deleted by opening it with mode os.O_TEMPORARY and then > closing it. How can a file be moved to the Recycle Bin, a la Windows? I see a send2trash module (http://hg.hardcoded.net/send2trash and http://www.hardcoded.net/articles/send-files-to-trash-on-all-platforms.htm) The source code looks pretty straightforward, but I don't think there's anything in the standard library that does that. -- Jerry -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
"Littlefield, Tyler" wrote: > Anyway, thanks to everyone else who answered this thread. I've not done > much like this besides muds, and all the logic is on the server there, I > think I will build the client in python, open source it for people to > fix/add to if they want and make sure to keep the server as secure as it > can be. The browser-based game Lacuna Expanse actually open sources the Perl client for their game, it might be a good place for ideas on how to approach this: https://github.com/plainblack/Lacuna-Web-Client The MMO EVE uses Stackless Python for both the client & server. Here's a slightly older doc detailing their architecture: http://www.slideshare.net/Arbow/stackless-python-in-eve Hope this helps. -- http://mail.python.org/mailman/listinfo/python-list
Re: Memcached in python 3
On Tue, May 17, 2011 at 4:07 AM, Dan Stromberg wrote: > > Do you need something shared across multiple hosts? Across multiple CPU's > of the same host? Or single process? > > By "Python Dictionary Manager", do you mean the manager stuff in the > multiprocessing module? > > On Sun, May 15, 2011 at 9:52 PM, Navkirat Singh wrote: > >> Hi Guys, >> >> How can I used memcached with python 3? Are there any other good >> alternatives to memcached? What about python dictionary manager, would it >> compare to memcached if I were to use it for storing in-memory information? >> >> Any light on this matter will be appreciated. >> >> Regards, >> Navkirat >> >> -- >> http://mail.python.org/mailman/listinfo/python-list >> >> > Hi, I need something that can be scalable. Something that if needed, can scale to multiple hosts. Yes, I mean the manager stuff in the multiprocessing module. Regards, Nav -- http://mail.python.org/mailman/listinfo/python-list
Python 3 vs Python 2.7 dilemma
Hi Guys, I have been trying to fight this issue for sometime now. I know that a large part of the python 3rd party software base has not been ported to python 3 yet. I am trying to build a web-based enterprise solution for my client. Most of reputed frameworks like Django and Turbo gears are yet in the 2.x stage. I know that these frameworks are extremely good. But I wanted to build my base with python 3 as that is what is going to prevail in the future. I have built my own little architecture using python3. Here is what I have accomplished till now: a) A multiprocessing webserver built directly using low level sockets for maximum control, conforming to RFC 2616 (not completely right now). b) A HTTP message parser for parsing HTTP/1.1 requests and generating response messages c) A session control base using python multiprocessing dictionary manager d) A partially build MVC model, without a templating engine at the moment. I am planning to put Jinja 3 there. I have spent months of free time doing this. I have learnt a lot, but well I am not sure if the path I am on is the right one. My question to everyone is whether I should go ahead with this approach, or should I just use 2.x technology? I am not sure if I will be able to port all the code to python3 later. I will really appreciate any input. Thanks and regards, Navkirat -- http://mail.python.org/mailman/listinfo/python-list
indirect assignment question
Hi,
I have some lines of code which currently look like this:
self.window = self.wTree.get_widget("mainWindow")
self.outputToggleMenu = self.wTree.get_widget("menuitem_output_on")
self.outputToggleButton =
self.wTree.get_widget("button_toggle_output")
self.logView = self.wTree.get_widget("textview_log")
self.logScrollWindow = self.wTree.get_widget("scrolledwindow_log")
and I would like (for tidiness / compactness's sake) to replace them
with something like this:
widgetDic = {
"mainWindow": self.window,
"menuitem_output_on": self.outputToggleMenu,
"button_toggle_output": self.outputToggleButton,
"textview_log": self.logView,
"scrolledwindow_log": self.logScrollWindow
}
for key in widgetDic:
... set the variable in dic[key] to point to
self.wTree.get_widget(key) somehow
what I need is some kind of indirect assignment where I can assign to a
variable whose name is referenced in a dictionary value.
Is there a way of doing this in python?
thanks,
andy baxter
--
http://highfellow.org
--
http://mail.python.org/mailman/listinfo/python-list
Re: indirect assignment question
On Mon, May 16, 2011 at 10:13 PM, Andy Baxter wrote:
> Hi,
>
> I have some lines of code which currently look like this:
>
> self.window = self.wTree.get_widget("mainWindow")
> self.outputToggleMenu = self.wTree.get_widget("menuitem_output_on")
> self.outputToggleButton = self.wTree.get_widget("button_toggle_output")
> self.logView = self.wTree.get_widget("textview_log")
> self.logScrollWindow = self.wTree.get_widget("scrolledwindow_log")
>
> and I would like (for tidiness / compactness's sake) to replace them with
> something like this:
> widgetDic = {
> "mainWindow": self.window,
> "menuitem_output_on": self.outputToggleMenu,
> "button_toggle_output": self.outputToggleButton,
> "textview_log": self.logView,
> "scrolledwindow_log": self.logScrollWindow
> }
> for key in widgetDic:
> ... set the variable in dic[key] to point to
> self.wTree.get_widget(key) somehow
>
> what I need is some kind of indirect assignment where I can assign to a
> variable whose name is referenced in a dictionary value.
>
> Is there a way of doing this in python?
You can achieve almost the same level of brevity, with less use of
magic, by simply using a local variable to refer to
self.wTree.get_widget:
w = self.wTree.get_widget # or choose some other similarly short variable name
self.window = w("mainWindow")
self.outputToggleMenu = w("menuitem_output_on")
self.outputToggleButton = w("button_toggle_output")
self.logView = w("textview_log")
self.logScrollWindow = w("scrolledwindow_log")
Python functions/methods are first-class; exploit this feature!
Cheers,
Chris
--
http://rebertia.com
--
http://mail.python.org/mailman/listinfo/python-list
Re: indirect assignment question
Andy Baxter writes:
> with something like this:
> widgetDic = {
>"mainWindow": self.window,
>"menuitem_output_on": self.outputToggleMenu,
>"button_toggle_output": self.outputToggleButton,
>"textview_log": self.logView,
>"scrolledwindow_log": self.logScrollWindow
> }
> for key in widgetDic:
>... set the variable in dic[key] to point to
> self.wTree.get_widget(key) somehow
>
> what I need is some kind of indirect assignment where I can assign to
> a variable whose name is referenced in a dictionary value.
for (name, value) in widgetDic.iteritems():
setattr(self, name, value)
--
\ “The double standard that exempts religious activities from |
`\ almost all standards of accountability should be dismantled |
_o__) once and for all.” —Daniel Dennett, 2010-01-12 |
Ben Finney
--
http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Mon, May 16, 2011 at 07:40, Chris Angelico wrote: > And I'm sure Steven will agree with me that this is not in any way a > bad thing. I've written hundreds of such programs myself (possibly > thousands), and they have all served their purposes. On a slightly > larger scale, there are even more programs that have never left the > walls of my house, having been written for my own family - not because > I'm afraid someone else will steal them, but because they simply are > of no value to anyone else. But hey, if anyone wants a copy of my code > that's basically glue between [obscure application #1] and [obscure > application #2] that does [obscure translation] as well to save a > human from having to do it afterwards, sure! You're welcome to it! :) > > However, I do not GPL my code; I prefer some of the other licenses > (such as CC-BY-SA), unless I'm working on a huge project that's not > meant to have separate authors. For something that by and large is one > person's work, I think it's appropriate to give attribution. But > discussion of exactly _which_ open source license to use is a can of > worms that's unlikely to be worth opening at this stage. > Actually, Chris, those applications are probably no less valuable to be open source than Linux or Firefox. The reason is that when one goes to learn a new language it is valuable to look at existing real world code. However, the code available online generally falls into one of two categories: 1) Simple sample code, which demonstrates a principle or technique 2) Full-blown FOSS application with hundreds of source files and a build It sounds to me like your home-brew code might be one of the missing links between the two. It won't be so tiny as to be trivial, but it won't be so huge as to be beyond the grasp of novices. I for one would love to look over such code. I'll learn something, without a doubt. Maybe someone might even spot a bug or make a suggestion to improve it. And almost invariably, any problem that I've ever had someone has had first. So while you might have been one of the first have a need to interface FooWidget with PlasmoidBar, someone after you will in fact need just the code to do that. -- Dotan Cohen http://gibberish.co.il http://what-is-what.com -- http://mail.python.org/mailman/listinfo/python-list
Re: obviscating python code for distribution
On Tue, May 17, 2011 at 4:16 PM, Dotan Cohen wrote: > Actually, Chris, those applications are probably no less valuable to > be open source than Linux or Firefox. The reason is that when one goes > to learn a new language it is valuable to look at existing real world > code. However, the code available online generally falls into one of > two categories: > 1) Simple sample code, which demonstrates a principle or technique > 2) Full-blown FOSS application with hundreds of source files and a build > > It sounds to me like your home-brew code might be one of the missing > links between the two. It won't be so tiny as to be trivial, but it > won't be so huge as to be beyond the grasp of novices. You have a point there. Although I can't guarantee that all my code is particularly *good*, certainly not what I'd want to hold up for a novice to learn from - partly because it dates back anywhere up to two decades, and partly because quite a few of the things I was working with are completely undocumented! But if you have Pastel Accounting Version 5, running in a Windows 3.1 virtual session, and you want to export some of its data to a DB2 database, I can help you quite a bit. Assuming you have an OS/2 system to run it on, of course. (You see what I mean about obscure?) I should probably dust off some of the slightly-more-useful pieces and put them up on either The Esstu Pack (my old web site) or rosuav.com (my new web site, doesn't have any better name than that), but that kinda requires time, a resource that I don't have an awful lot of. I'm sure there'll be a few oddments in there where at least one half of the glue is more useful. Back then, though, I didn't know Python, nor Pike, nor any of quite a few other awesome languages, but REXX and C++ are at least available open source. Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
