In some email I received from Aaron Stone <[EMAIL PROTECTED]> on Thu, 1 May 2003 13:50:06 -0700 (PDT), wrote:
> > > On Thu, 1 May 2003, lou wrote: > > > In some email I received from Aaron Stone <[EMAIL PROTECTED]> on Thu, 1 May > > 2003 > > 12:37:53 -0700 (PDT), wrote: > > > > > You wouldn't be able to separate reads and writes because the IMAP/POP3 > > > servers are so much faster than the database replication that even > > > something as simple as moving a message from one folder to another would > > > appear to have failed but then suddenly take effect a few seconds later. > > > The delay would get worse as the load went up, making a medium load > > > system appear even laggier and buggier than a heavily loaded one. > > > > right, dbmail is not optimized for parallel operations nor mysql is. > > however you answer is quite scary:) > > > > I had never thought about this one, either, but then during discussion a > few months ago Eelco brought this us and indeed it is quite scary! For > anyone else who didn't get my last sentence, it was tyoped [sic]: "The > delay gets worse as the load goes up, making a medium load split database > feel much laggier and buggier than a heavily loaded single database." Basically, that's the hard bit, some time ago i did a research on HA/parallel databases, it's a great thing to have one of these around, because it will aid our problems(at least collisions and stuff, and we dont have to worry about uniqueness). Even if we want to do so (one day) we have to move from the current design to a new one more backend/platform generic/independent, this is a long discussion and i'm not going to get into it right now, however we can start discussing the database design concenring the distributed services ( in a heterogenous env) with comming on/going off members(read servers in our case) and since mail is so atomic it wouldnt be hard to achive. Regarding you typo, Aaron dont think in mysql manner, since we know that the replication wont aid this issue, it will make it worse, all we want for the data is to be safe and accessible, that means we have to distinguish each member in a different DB space where he works, and each backend has this thing, basically all of them are using different factor for data mining in its very own place in the database. I'm really worried about one example that Jesse posted when i crafted the basic PoC patch - parallel operations. It'll be much harder to bring clustering rather than a backup/fail-over solution. Also consider the huge amount of falsepositives out there attacking a parallel service like mail. > > > Allowing a slave to step up to master status presents another problem > > > because at the time of resynchronization, you are likely to find that > > > changes were committed to the failed master before they could be > > > propagated. This leaves the sequence numbers in a perfect position to > > > collide and cause a full halt of all replication until manually restored > > > (this is further explained in the MySQL manual). > > > > that depends, depends what's the factor to generate those numbers.. > > also consider each server working i its own dimension of the database where > > the others > > doesnt have write permissions, also the access shouldnt be async but > > synch, we all > > know that sync access with transactions is more safe(not super fast, but in > > some cases > > it is) > > > > Is it possible to have a transaction that forces all replicas to be up to > date before committing to the master? I'm not sure if that's any kind of > solution, or if it just makes it worse... but interesting to know of... That depends, it wont make it worse but there might be a few issues, let say a lack of bandwidth which will slow down not only one but the whole group which will be waiting for an answer from the slow one. _BUT_ that will happen if you have a cluster around the world, if this whole thing is on one network or let say on a really reliable network that wont be an issue i'm sure. But yes you're on the right track. Right, PgSQL has it in PgSQL-r, there is one thing i can say about this guys - they are great. Basically they are using spread (spread.org ) for group communication, let consider a group of 5 multimaster PgSQL-r servers, basically the replication works ONLY and ONLY with transactions and indeces, let say you do something like: begin; insert blahblah; insert blahblah; it will do on all of them and then notify for the result, if one fail i think the transaction will fail since it's a multimaster env, the data wont be consistent if one failes. commit; or rollback;; Next thing, the group communication is very interesting, so you have those 5 servers, if the connection of 1 of them is broken to the others it goes into a read-only mode, and the other five continue their work. same if you have 2 vs 3, 2 will be read-only and 3 will be continue, basically you get the picture. so and so. Basically this thing doesnt aid much a parallel services but I'm sure they will get there somewhen. > > > I've been thinking about this problem, and it might be possible to > > > eliminate the auto_increment fields, > > I was trying to tell you this since i wrote this very basic thing, > > auto increment suck in general especially when you cant change the > > increment factor > > > > (on pgsql you can but the replication is not for productional use though) > > > > Also why dont you push for per server table for sequences, that how you can > > easily > > implemend a group communication between the servers and make it lot more > > robust.. > > anyway the sequence generation have to be coded but i dont see this as a big > > problem... > > > > just from my experience with mysql/pgsql/soso, i've never used auto > > increment with any > > database, simply because it's not portable (if you want to move it to a > > different > > database) and doesnt give you this really important management layer, like > > in this > > case. > > > > > using only the unique id of each > > > message as its handle. It's just a thought, and would require a lot of > > > work to implement the full semantics to maintain referential integrity, > > > but might be a good place to start. > > > > if you're interested i can send over the design paper? > > > > Please do. There's been a lot of work on the list lately involving the > database design, and if this is the sort of thing that isn't any much > harder to do anyways, I'd like to propose that the v2.0 schema provide its > own sequence tracking that avoids inter-server collisions like this. For > those who've been keeping track of my personal soapboxing, I still think > that the new schema is a long term project, but it *does* need a wishlist, > after all :-) Is there any place where i can, get the new schema? Cause i've working on the one available from the dbmail PDF manual and the other from the cvs source? Any major changes since then? i'm working on it (it's nearly done though), i have a friend that's helping, that should speed the whole process. In any way i dont want to rush without researching all available possibilities and covering most of the major and minor aspects of a perfect mail system:))) Which I'm sure will dbmail be. cheers
