Hi Benoit, it's important to be precise. I tried to lock down what you meant by a non-fragmented database but I'm afraid we're not there yet. In this reply you say
> A fragmented or sharded database is the clustered one which I'd read as saying that as a Q=1,N=1 database still counts as a fragmented or sharded database, but I don't think that's what you meant. I'm honestly not sure which specific databases you want to move out of shards/ and into the top-level. > Also I am wondering right now how the transition from a the current > couchdb to the new one can be done. It would be quite easier imo to > have 1 HTTP layer and flag the "special databases" to not expose them > in the HTTP API except for the admins. Now that's an interesting discussion. We've mulled it over internally at Cloudant a bit in the past but never really came to a conclusion. I don't find the :5984 / :5986 split elegant, but I'll also say that getting the API for local vs. clustered databases exactly right is not at the top of my priority list. In fact I think it may be the sort of thing that we could defer until after the first release that includes clustering capabilities. Adam On Mar 2, 2014, at 11:30 PM, Benoit Chesneau <[email protected]> wrote: > On Mon, Mar 3, 2014 at 3:26 AM, Adam Kocoloski <[email protected]> wrote: >> On Feb 28, 2014, at 4:31 AM, Benoit Chesneau <[email protected]> wrote: >> >>> Looking at the code of mem3 it seems that all databases are oblgatory >>> fragmented (sharded). I understand that you can have only 1 shard / >>> database but then it will be still stored at >>> `<database_dir>/shard/<dbname>`. >>> >>> Is there any plan of having non fragmented databases stored in their >>> own place ie `<database_dir>/<dbname>` ? Would be interesting when >>> you want to manage different policies of backup depending on the type >>> of database (fragmented or not). Which place should I look to make it >>> happen? >>> >>> - benoit >> >> I'm not entirely certain I understand what you're looking for there Benoit. >> I classify databases in two categories: "clustered" and "local". > >> >> A clustered database is available via the clustered HTTP interface and has >> its data stored in files inside <database_dir>/shards/<range>/<dbname>. >> This is true even for a Q=1,N=1 database where this is only one file for the >> database in the entire cluster. >> >> On the other hand, a local database exists on a specific node in the >> cluster, and is not accessible via the clustered HTTP interface. The data >> for this database is stored at <database_dir>/<dbname>. >> >> What exactly did you mean by a non-fragmented database? It sort of sounds >> like you're wanting to conflate a Q=1,N=1 clustered database with a local >> database, but those are very different concepts (not least because multiple >> nodes in a cluster can have a local database with the same name but >> different content). I can honestly say we've never considered storing the >> files for Q=1 or Q=1,N=1 databases somewhere different than other clustered >> databases. > > A fragmented or sharded database is the clustered one. I call them > fragmented databases since afaik a clustered database is spitted in > multiple pieces. > >> >> Can you say more about your motivations here? You talked about backup >> policies; while I could absolutely see wanting to implement a different >> backup policy for N=1 databases it's not obvious to me why you'd want to >> treat Q=1 and Q>1 databases separately. > > Having them in there own folder instead of shard them them easily more > distinct on the fs than having them in shards/ and i could easily > find db.couch by its name without using the API on the disk. > > Also I am wondering right now how the transition from a the current > couchdb to the new one can be done. It would be quite easier imo to > have 1 HTTP layer and flag the "special databases" to not expose them > in the HTTP API except for the admins. > > - benoit
