Re: simple journal-based persistenсe for Clojure

2009-12-12 Thread Sergey Didenko
I'm going to postpone the discussed above removing of "locking" - the write bottleneck. Because: 1. In-memory writes are very quick 2. there is no demand right now 3. it makes the backup workflow more complex The next thing I'm going to add is snapshotting. However it is not a priority for me ri

Re: simple journal-based persistenсe for Clojure

2009-12-10 Thread Sergey Didenko
Yes, it uses agents for writing operations. So on the account of disk failure it's possible that in-memory state will be a few transactions ahead of what is actually persisted. Thus it's not a strict transactional solution. I consider it as a good fit for project prototyping and may be production

Re: simple journal-based persistenсe for Clojure

2009-12-10 Thread Laurent PETIT
Hello, without going too deep in the detail of your implementation, I just can't keep thinking that, as far as I know, it's currently impossible with clojure to extend the scope of the transaction outside the STM, without changes in the STM itself (STM being aware of TransactionManagers being cer

Re: simple journal-based persistenсe for Clojure

2009-12-09 Thread Sergey Didenko
I see one issue with this approach. If we get rid of "locking" then some transactions occur reordered in journal files. So simple backuping of journal files can be dangerous. Imagine the case when two transactions are reordered and got into different files: 1.journal: (tr1) ;1 (tr3) ;3 4.journal

Re: simple journal-based persistenсe for Clojure

2009-12-05 Thread Sergey Didenko
Nice idea, Luc! Useful details, Jon. So I'm implementing "(dosync (let [ tr-id (swap! tr-atom inc)] (transaction-itself) " and then loading transactions in their tr-id's order in the next release. Probably with some floating "window" of postponed transactions in order not to load all the transac

Re: simple journal-based persistenсe for Clojure

2009-12-05 Thread John Harrop
On Fri, Dec 4, 2009 at 7:40 PM, Luc Préfontaine wrote: > I was about to say that. There's no need for the id's to be "contiguous", > only to get them to grow to preserve ordering. > If you can find a way in your design to increment the atom each time a > transaction is retried then you would > p

Re: simple journal-based persistenсe for Clojure

2009-12-04 Thread Luc Préfontaine
I was about to say that. There's no need for the id's to be "contiguous", only to get them to grow to preserve ordering. If you can find a way in your design to increment the atom each time a transaction is retried then you would preserve ordering. The last value would be the one used in the "final

Re: simple journal-based persistenсe for Clojure

2009-12-04 Thread Sergey Didenko
That was for Windows and client JVM. On my Linux server JVM sometimes even 100 of "(dosync (alter myref inc))" has the same nanoTimes. So it's rather dangerous approach, unless your program follows the condition. > "(dosync (alter myref inc))" takes much longer than timer granularity on my > Wind

Re: simple journal-based persistenсe for Clojure

2009-12-04 Thread Sergey Didenko
Unfortunately: "No guarantees are made about how frequently values change." "Note that while the unit of time of the return value is a millisecond, the granularity of the value depends on the underlying operating system and may be larger." (see System nanoTime, currentTimeMillis javadoc) Howeve

Re: simple journal-based persistenсe for Clojure

2009-12-04 Thread John Harrop
On Fri, Dec 4, 2009 at 6:50 AM, Sergey Didenko wrote: > Without the global transaction counter another problem arises. > > Suppose transaction B depends on ( results of ) transaction A. And they are > executed from different threads. However they have the right order when > executing the first tim

Re: simple journal-based persistenсe for Clojure

2009-12-04 Thread Sergey Didenko
Without the global transaction counter another problem arises. Suppose transaction B depends on ( results of ) transaction A. And they are executed from different threads. However they have the right order when executing the first time. How to guarantee that the execution order is the same when t

Re: simple journal-based persistenсe for Clojure

2009-12-04 Thread Sergey Didenko
Here is the GitHub project: http://github.com/SergeyDidenko/Simple-Persistence-for-Clojure -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that posts from new members are moderated -

Re: simple journal-based persistenсe for Clojure

2009-12-04 Thread John Harrop
On Thu, Dec 3, 2009 at 6:31 PM, Sergey Didenko wrote: > Well, I'm not fluent with git yet. I'll create the github project, that can > not be hard. > > In comparison with Prevayler, the persister does not block the reads, > because it relies on Clojure STM. However it blocks the writes as > Prevay

Re: simple journal-based persistenсe for Clojure

2009-12-03 Thread Sergey Didenko
Well, I'm not fluent with git yet. I'll create the github project, that can not be hard. In comparison with Prevayler, the persister does not block the reads, because it relies on Clojure STM. However it blocks the writes as Prevayler, because currently there is no way to reliably get/ generate a

Re: simple journal-based persistenсe for Clojure

2009-12-03 Thread Pelle Braendgaard
I was just starting to research if there was a prevayler like solution for clojure. Thanks. I'll start playing with it. Do you have a github project for it yet? P On Wed, Dec 2, 2009 at 9:18 AM, Sergey Didenko wrote: > It creates journals in readable and executable form: > 1.journal > " > (tr-

Re: simple journal-based persistenсe for Clojure

2009-12-02 Thread Sergey Didenko
It creates journals in readable and executable form: 1.journal " (tr-fn 1 2) ;1 (tr-fn 10 20) ;2 (tr-fn-swap) ;3 (tr-inc) ;4 " -- You received this message because you are subscribed to the Google Groups "Clojure" group. To post to this group, send email to clojure@googlegroups.com Note that po