Simple algorithm question - how to reorder a sequence economically
What is the easiest way to reorder a sequence pseudo-randomly? That is, for a sequence 1,2,3,4 to produce an arbitrary ordering (eg 2,1,4,3) that is different each time. I'm writing a simulation and would like to visit all the nodes in a different order at each iteration of the simulation to remove the risk of a fixed order introducing spurious evidence of correlation. -- http://mail.python.org/mailman/listinfo/python-list
Using ACLs in JSON
I'm designing a system that should allow different views to different
audiences. I understand that I can use application logic to control
the access security, but it seems to me that it'd make more sense to
have this documented in the data-stream so that it's data-driven.
I was wondering if there was any standard way of doing this in JSON.
Alternatively, is there a better way of organising this in Python
that's compatible with JSON?
I've put an example of the sort of thing that I mean below. The idea
is that this object is accessible for viewing or deleting by the role
'HR' and available for change only to the owner of the record itself.
In addition, the record can be viewed by those with the role
'manager'. The syntax may be wrong, but I hope that my intention is
reasonably clear.
Is there an existing practice or standard for doing this sort of
thing?
{
"title" : "Example Schema",
"type" : "object",
"version" : "1.0",
"properties": {
"firstname" : {
"type": "string"
},
"lastname" : {
"type": "string"
},
"age" : {
"description" : "Age in years",
"type": "integer",
"minimum": 0
}
},
"ACL-view": ["HR","Manager",["firstname","lastname"]],
"ACL-change": ["firstname","Lastname"],
"ACL-delete": ["HR"],
"required": ["firstname","lastname"]
}
--
http://mail.python.org/mailman/listinfo/python-list
Re: Simple algorithm question - how to reorder a sequence economically
Thank you all for those most helpful suggestions! random.shuffle does precisely the job that I need quickly. Thank you for introducing me to itertools, though, I should have remembered APL did this in a symbol or two and I'm sure that itertools will come in handy in future. Thanks for the warnings about random numbers too - I hope my lists will be short enough for the permutations of the function to be irrelevant. I don't need every single sequence to be unique, only that the same sequence only occurs occasionally. My lists might get up to the ~2k length one day, and I'll keep in mind the need, at that stage, to use a different pseudo-random generator. Actually, thinking about it, there is probably a source of non-algorithmically-derived 'random' numbers somewhere on the net that would do the job nicely. -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple algorithm question - how to reorder a sequence economically
On May 24, 5:00 pm, Carlos Nepomuceno wrote: > > > I don't know what "spurious evidence of correlation" is. Can you give a > mathematical definition? > If I run the simulation with the same sequence, then, because event E1 always comes before event E2, somebody might believe that there is a causative connection between them in the world that's being simulated, when, in fact, they only correlate in this way because the sequence is not being shuffled. That's what it means. Actually it'll be a bit more subtle than that, because each iteration of the simulation updates all nodes in one time interval, the events will not usually show the order of iteration - but, where there are any secondary effects, that are related to the order in which the nodes are updated, these will always happen the same way, which is my concern. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using ACLs in JSON
On May 24, 6:42 pm, Michael Torrie wrote: > On 05/24/2013 02:18 AM, Peter Brooks wrote: > > > I'm designing a system that should allow different views to different > > audiences. I understand that I can use application logic to control > > the access security, but it seems to me that it'd make more sense to > > have this documented in the data-stream so that it's data-driven. > > > I was wondering if there was any standard way of doing this in JSON. > > Alternatively, is there a better way of organising this in Python > > that's compatible with JSON? > > While I don't understand exactly what you're using JSON for, and nor do > I understand the purpose of the JSON structure you posted, I can say > that ACLs have nothing to do with JSON. > > JSON is simply a data markup format, like HTML, XML, plain text, or an > INI file. It's merely data. If you want to restrict who sees what when > they request a chunk of data formatted using JSON, then you have to > enforce that in the code that's processing the request for data using > another mechanism. And that mechanism depends on how your clients ask > for JSON data, and what code is serving or generating the JSON data. > Yes, you're right, I know that. However, I want the permissions embedded in the data so that it's easy to verify, from the data, who has access to which objects. The reason is to enforce transparency of the access rights and to make them easy to grant and change without needing to make any coding changes. My question was whether anybody had taken a similar approach and found a way that worked well with python or, even better, if there was a standard way of doing this. If nobody has done this, then I'm happy to invent my own method, but I don't really want to reinvent wheels that might have been nicely designed already. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using ACLs in JSON
On May 24, 6:13 pm, Carlos Nepomuceno wrote: > Not exactly what you want but you may consider Google ACL XML[1]. > > If there aren't any system integration restrictions you can do what you think > it's best... for now. > > [1]https://developers.google.com/storage/docs/accesscontrol#applyacls > Thank you for the reference. I specifically don't want to use XML itself, but I can adapt that to my purpose - great! -- http://mail.python.org/mailman/listinfo/python-list
Re: Simple algorithm question - how to reorder a sequence economically
On May 24, 11:33 pm, Carlos Nepomuceno wrote: > > > > > > > > > > > > Date: Fri, 24 May 2013 12:01:35 -0700 > > Subject: Re: Simple algorithm question - how to reorder a sequence > > economically > > From: peter.h.m.bro...@gmail.com > > To: python-l...@python.org > > > On May 24, 5:00 pm, Carlos Nepomuceno > > wrote: > > >> I don't know what "spurious evidence of correlation" is. Can you give a > >> mathematical definition? > > > If I run the simulation with the same sequence, then, because event E1 > > always comes before event E2, somebody might believe that there is a > > causative connection between them in the world that's being simulated, > > when, in fact, they only correlate in this way because the sequence is > > not being shuffled. That's what it means. > > Correlation does not imply causation. If "somebody" is an expert system and > you want to avoid it's recognition and/or triggering of some kind, and you > can't or don't want to change it's behavior, you may take the random way > because it's cheaper. > > > Actually it'll be a bit more subtle than that, because each iteration > > of the simulation updates all nodes in one time interval, the events > > will not usually show the order of iteration - but, where there are > > any secondary effects, that are related to the order in which the > > nodes are updated, these will always happen the same way, which is my > > concern. > > You should have a more precise understanding of the dependence of the > variables you taking in consideration before randomizing the series of events > your are using for tests. > If the scenario could be modelled mathematically, then there'd be no point in writing the simulation. -- http://mail.python.org/mailman/listinfo/python-list
Solving the problem of mutual recursion
I'm not sure if this'll interest anybody, but I expect that I'm going
to get some mutual recursion in my simulation, so I needed to see how
python handled it. Unfortunately, it falls over once it detects a
certain level of recursion. This is reasonable as, otherwise, the
stack eventually over-fills.
Mostly the recommendation, when this happens, is to re-write the code
as loops. Of course, that's not as satisfactory as using recursion.
The problem really is that the methods or functions called recursively
never exit because they're waiting for the return of the function
they've called. Now, if your recursion relies upon knowing what's
returned, then this solution won't help you. Often, though, you're not
interested in what's returned and would just like the routine to exit
once it's called itself, or another process, recursively.
If that's the case, this solution, using threads, allows you to have
functions call themselves, or each other, indefinitely. It works OK on
a macbook pro and the thread count varies from 2-3, so it handles the
thread queuing well. I'm not sure how well this will work on other
architecture - it'd be interesting to know if anybody tries it.
#!/usr/bin/python
# Two functions, Jim and Fred, call each other recursively and
indefinitely, while the main program continues to execute as well
import threading, time
def jim(arg,count):
print "Running jim:", arg,count," Thread Count
",threading.active_count()
thread = threading.Thread(target=fred, args=(" From Jim ",count
+1))
thread.start()
print count," Jim complete. Thread
Count",threading.active_count()
def fred(arg,counter):
print "Running fred:", arg,counter
thread = threading.Thread(target=jim, args=(" From Fred
",counter+1))
thread.start()
print counter,"Fred complete",threading.currentThread()
thread = threading.Thread(target=jim, args=(" From Main",0))
thread.start()
print "Jim run from main"
count = 0
while True:
count += 1
print 'In main program',count
--
http://mail.python.org/mailman/listinfo/python-list
Re: Solving the problem of mutual recursion
On May 26, 5:09 pm, Jussi Piitulainen wrote: > > A light-weighter way is to have each task end by assigning the next > task and returning, instead of calling the next task directly. When a > task returns, a driver loop will call the assigned task, which again > does a bounded amount of work, assigns the next task, and returns. > Tasks can even pass parameters in the same way. > Yes, that's true - there are a number of ways of making it linear. What I'm particularly pleased about with my method is the parallelism that it achieves - with so little effort! The simulation is going to be computationally intense and this is going to make sure that the CPUs are all giving it their best shot. When I run this on my macbook, the python interpreter takes over 140% of CPU - with a bit of fine- tuning, it should be possible to have more concurrent threads and to use the four cores optimally. Naturally I'll need to be careful with the concurrency, but this is so simple and clean that it should be easy to avoid the main problems with accessing the same variables. -- http://mail.python.org/mailman/listinfo/python-list
Re: Solving the problem of mutual recursion
On 26 May, 20:09, Carlos Nepomuceno wrote: > > > > > > > > > > > > Date: Sun, 26 May 2013 10:21:05 -0700 > > Subject: Re: Solving the problem of mutual recursion > > From: peter.h.m.bro...@gmail.com > > To: python-l...@python.org > > > On May 26, 5:09 pm, Jussi Piitulainen > > wrote: > > >> A light-weighter way is to have each task end by assigning the next > >> task and returning, instead of calling the next task directly. When a > >> task returns, a driver loop will call the assigned task, which again > >> does a bounded amount of work, assigns the next task, and returns. > >> Tasks can even pass parameters in the same way. > > > Yes, that's true - there are a number of ways of making it linear. > > > What I'm particularly pleased about with my method is the parallelism > > that it achieves - with so little effort! The simulation is going to > > be computationally intense and this is going to make sure that the > > CPUs are all giving it their best shot. When I run this on my macbook, > > the python interpreter takes over 140% of CPU - with a bit of fine- > > tuning, it should be possible to have more concurrent threads and to > > use the four cores optimally. > > > Naturally I'll need to be careful with the concurrency, but this is so > > simple and clean that it should be easy to avoid the main problems > > with accessing the same variables. > > -- > >http://mail.python.org/mailman/listinfo/python-list > > Python threads run in the same process and won't run concurrently: > > "CPython implementation detail: In CPython, due to the Global Interpreter > Lock, only one thread can execute Python code at once (even though certain > performance-oriented libraries might overcome this limitation). If you want > your application to make better use of the computational resources of > multi-core machines, you are advised to use multiprocessing. However, > threading is still an appropriate model if you want to run multiple I/O-bound > tasks simultaneously."[1] > > How can you get 140% of CPU? IS that a typo?? > No, on a multi-core machine it's normal. The machine shows python running multiple threads - and the number of threads change as the program runs. Perhaps the OS/X implementation of python does allow concurrency when others don't. It certainly looks like it! -- http://mail.python.org/mailman/listinfo/python-list
Re: Solving the problem of mutual recursion
On 26 May, 20:22, Carlos Nepomuceno wrote: > > > > Date: Sun, 26 May 2013 11:13:12 -0700 > > Subject: Re: Solving the problem of mutual recursion > > From: peter.h.m.bro...@gmail.com > > To: python-l...@python.org > [...] > >> How can you get 140% of CPU? IS that a typo?? > > > No, on a multi-core machine it's normal. The machine shows python > > running multiple threads - and the number of threads change as the > > program runs. Perhaps the OS/X implementation of python does allow > > concurrency when others don't. It certainly looks like it! > > I pretty sure it doesn't run on multiple cores on Linux and Windows. > > I've tested it and have been trying to find a better way achieve concurrency > in Python. One of the ways is the multiprocessing module[1]. > > Do Mac OS shows "140%" CPU load when more than a single core is been used? > lol Apple sucks!!! lol > It's not uncommon - HP-UX does exactly the same thing. -- http://mail.python.org/mailman/listinfo/python-list
Re: Solving the problem of mutual recursion
On May 27, 12:16 am, Chris Angelico wrote: > On Mon, May 27, 2013 at 5:35 AM, Ian Kelly wrote: > > I'm pretty sure that CPython uses the GIL regardless of platform. And > > yes you can have multiple OS-level threads, but because of the GIL > > only one will actually be running at a time. Other possibilities > > include: > > 6) It's spinning in a function that has released the GIL. Python > threads can certainly execute concurrently; they just can't be > manipulating Python objects. Most of the I/O functions will release > the GIL before doing a potentially-blocking operation, and some > CPU-heavy functions can do the same (I'm given to understand that > numpy does this) - just depends on having a long job that involves no > refcounted objects. > This makes complete sense - any atomic action should be atomic, so two threads can't be doing it at the same time. They can be doing anything else though. If two threads create a new object at the same time, for example, there's potentially the problem that they'll get the same space, so the object will be owned by both. To prevent this, the new object call should be run in only one thread. If you have two objects running their methods on their own local variables, then there's no potential for conflict, so there's no need for them to be blocked. This is an interesting subject.. There's nothing wrong with the tool I'm using to report threads - 'Activity Monitor' is the standard process monitor. It counts cores as 'CPUs', which seems perfectly reasonable to me. As I said, other Unixes, such as HP-UX, do the same thing. I'm not quite sure of the best way to examine the exact behaviour. Since the blocking works at the level of atomic operations, it's difficult to detect - a call to an atomic operation may block all other threads, but any snapshot of the number of active threads won't be running at this time, so will only report when there are multiple threads active. -- http://mail.python.org/mailman/listinfo/python-list
