Re: [elephant-devel] Querying Advice

2007-02-28 Thread Ian Eslick
Daniel, Not sure if you're still interested, but I got motivated to start thinking about a query strategy for Elephant. I have a much more sophisticated version I'll check into contrib/eslick/queries in a day or two, but my first experimental hack can be found in the main elephant direct

Re: [elephant-devel] Querying Advice

2006-11-15 Thread Ian Eslick
It's not entirely misleading. Berkeley DB is a database in the sense that it provides efficient indexing and retrieval. However it's much lower level than SQL (and is often used to implement relational databases). Elephant is an ODB in the sense that BDB - it stores objects, values and can effic

Re: [elephant-devel] Querying Advice

2006-11-15 Thread Ian Eslick
Daniel, There are a couple of things about the Elephant model that should be made clear. Elephant is not a very high level DB system. It's focus is persistence of data. In effect, it is a collection of indexing mechanisms and a metaobject protocol that helps us to store/retrieve data. There a

Re: [elephant-devel] Querying Advice

2006-11-14 Thread Daniel Salama
Robert,Thanks for the clarification. I think I got it and agree.BTW, after running your code using persistent objects and adding 5000 users, the total on-disk storage is now 5MB (as compared to 9MB by using non-persistent objects), which concurs with what Ian mentioned yesterday. Anyway, just wante

Re: [elephant-devel] Querying Advice

2006-11-14 Thread Robert L. Read
On Tue, 2006-11-14 at 14:50 -0500, Daniel Salama wrote: Ian et al, Based on my comment to Robert and that of Pierre, could you, or anyone, please clarify this for me (and maybe others): If making a class persistent means that there is no need to add it to root or to any persistent colle

Re: [elephant-devel] Querying Advice

2006-11-14 Thread Daniel Salama
Ian et al, Based on my comment to Robert and that of Pierre, could you, or anyone, please clarify this for me (and maybe others): If making a class persistent means that there is no need to add it to root or to any persistent collection, when I look at Robert's sample code, I see (excerpt

Re: [elephant-devel] Querying Advice

2006-11-14 Thread Ian Eslick
Persistent classes in the current model aren't easy to fully delete - you can only make the unreachable by having no reference to them reachable from the controller or class roots. You can drop an object from the index and there is a way to delete the object from the main system BTree - but until

Re: [elephant-devel] Querying Advice

2006-11-14 Thread Pierre THIERRY
Scribit Daniel Salama dies 13/11/2006 hora 21:00: > So, if I make my class persistent, it's persistent... period! I don't > need to add it to a collection or to the root. Well, I did not know that before that discussion. I think it's not clearly indicated in the docs... And then I wonder: if I us

Re: [elephant-devel] Querying Advice

2006-11-13 Thread Robert L. Read
On Mon, 2006-11-13 at 21:00 -0500, Daniel Salama wrote: I think I just understood what you've all been saying. Sorry about this. So, if I make my class persistent, it's persistent... period! I don't need to add it to a collection or to the root. I guess that's also what Ian w

Re: [elephant-devel] Querying Advice

2006-11-13 Thread Daniel Salama
I think I just understood what you've all been saying. Sorry about this.So, if I make my class persistent, it's persistent... period! I don't need to add it to a collection or to the root. I guess that's also what Ian was saying about using class indices as well.Thanks,DanielOn Nov 13, 2006, at 10:

Re: [elephant-devel] Querying Advice [w/code example]

2006-11-13 Thread Venkat Manakkal
Daniel Salama wrote: > I'm not familiar with GBBopen and will read up on it. Would anyone care > to comment on it? For the purpose of trying to "develop" a querying > facility for Elephant, will it be useful/needed? Not at all, its another framework for application development I have found very us

Re: [elephant-devel] Querying Advice [w/code example]

2006-11-13 Thread Robert L. Read
I wrote that code in about an hour and a half; it is a throw-away spike solution. Furthermore I developed it using the SQL backend rather than the BDB. I would never claim it is an ideal solution.  Morever, coding style isn't really my strong suit. As Ian mentioned, our serialization could be

Re: [elephant-devel] Querying Advice [w/code example]

2006-11-13 Thread Daniel Salama
I'm running 0.6.0 downloaded from the web site. On Nov 13, 2006, at 1:48 PM, Ian Eslick wrote: Are running from HEAD or 0.6.0? I'll answer the size question tonight (on travel today). -Ian Sent via BlackBerry from T-Mobile ___ elephant-devel sit

Re: [elephant-devel] Querying Advice [w/code example]

2006-11-13 Thread Ian Eslick
Are running from HEAD or 0.6.0? I'll answer the size question tonight (on travel today). -Ian Sent via BlackBerry from T-Mobile -Original Message- From: Daniel Salama <[EMAIL PROTECTED]> Date: Mon, 13 Nov 2006 13:43:03 To:Elephant bugs and development Subject: Re: [ele

Re: [elephant-devel] Querying Advice [w/code example]

2006-11-13 Thread Daniel Salama
Ok.I got Elephant to work again with SBCL on PPC. I guess I was still using BDB 4.3 when 4.4 seems to be required. I couldn't find that anywhere in the docs.I see what you're saying and can start to envision where this can go. I will keep playing and provide more feedback later today.FYI, for curio

Re: [elephant-devel] Querying Advice

2006-11-13 Thread Robert L. Read
On Mon, 2006-11-13 at 09:47 -0500, Daniel Salama wrote: So, would you say that we based on our approach, we could just store  a bunch of random objects (whether person, state, zip, order, etc) in  the root and use class indices? Sounds interesting. I have to go to the dent

Re: [elephant-devel] Querying Advice [w/code example]

2006-11-13 Thread Robert L. Read
Some notes; forgive me if these are out of order: *) (format t "blah~A" x) is not as efficient as (concatenate 'string "blah" x), but I typically use it for debugging and its fine unless the strings are long. *) If you have many tables/persistent classes, then you are quite correct,  the code

Re: [elephant-devel] Querying Advice [w/code example]

2006-11-13 Thread Ian Eslick
Daniel, I think GBBopen would be an application-oriented approach to indexing rather than an infrastructure that one might use to build a query engine for elephant; from what I can tell you have to buy into their model of objects, spaces and dimensions, etc. You could mix it with DCM, perhaps - b

Re: [elephant-devel] Querying Advice [w/code example]

2006-11-13 Thread Daniel Salama
I'm not familiar with GBBopen and will read up on it. Would anyone care to comment on it? For the purpose of trying to "develop" a querying facility for Elephant, will it be useful/needed? Thanks, Daniel On Nov 13, 2006, at 9:16 AM, Venkat Manakkal wrote: Daniel Salama wrote: 1) The nature

Re: [elephant-devel] Querying Advice

2006-11-13 Thread Daniel Salama
On Nov 13, 2006, at 7:57 AM, Ian Eslick wrote: I think a macro system makes sense when we find we want to have a query syntax (a simpler and more elegant version of what SQL tries to do) that concisely expresses the class of operations we want to perform and behind which we need to put a quer

Re: [elephant-devel] Querying Advice

2006-11-13 Thread Daniel Salama
On Nov 13, 2006, at 7:37 AM, Ian Eslick wrote: Class indexing accomplishes the same thing - it creates a BTree for each indexed class type that contains all instances of the class. Slot indexes can be added/deleted and are secondary BTrees with the class BTree as primary. So, would you

Re: [elephant-devel] Querying Advice

2006-11-13 Thread Daniel Salama
I agree that macros are not a necessity. I was thinking in abstracting the predicate building functions so that you wouldn't have to "hard-code" some of the functionality. In essence, kind of like building a querying language. Granted, I agree that it should be flexible enough to allow the

Re: [elephant-devel] Querying Advice

2006-11-13 Thread Robert L. Read
On Sun, 2006-11-12 at 23:17 -0500, Daniel Salama wrote: I believe (and hope) that we should have no problem mapping the GUI to the requests. Just out of curiosity (and I don't mean to divert from the topic of this thread): if you're using DCM for your konsenti (BTW, nice concept) site, how

Re: [elephant-devel] Querying Advice [w/code example]

2006-11-13 Thread Venkat Manakkal
Daniel Salama wrote: > 1) The nature of dynamic queries as presented in my original email: your Just briefly coming out of lurk mode, when designing your application you might want to use something like GBBopen from gbbopen.org that implements a main memory blackboard for dynamic queries to in mem

Re: [elephant-devel] Querying Advice

2006-11-13 Thread Ian Eslick
Performance and functionality of search/sort will depend a great deal on the model we have of how data is stored, how it is cached, how it is indexed and what kind of queries we want to support. Robert's approach is more than adequate if everything is maintained in lists in memory; it is more expe

Re: [elephant-devel] Querying Advice

2006-11-13 Thread Ian Eslick
Daniel Salama wrote: >> Were you storing persistent-metaclass objects or simply normal objects? >> Normal objects are huge relative to persistent objects as currently all >> the slot names are also serialized (potentially in 32-bit unicode if on >> SBCL). I have some improvements planned to redu

Re: [elephant-devel] Querying Advice

2006-11-13 Thread Pierre THIERRY
Scribit Daniel Salama dies 12/11/2006 hora 10:28: > I guess we would have to sequentially navigate thru the results in > order to "manually" select each record based on all the other > possible search arguments. I suppose, in a way, this can be done > relatively painless by using macros I d

Re: [elephant-devel] Querying Advice

2006-11-12 Thread Daniel Salama
Were you storing persistent-metaclass objects or simply normal objects? Normal objects are huge relative to persistent objects as currently all the slot names are also serialized (potentially in 32-bit unicode if on SBCL). I have some improvements planned to reduce the storage and processin

Re: [elephant-devel] Querying Advice [w/code example]

2006-11-12 Thread Daniel Salama
Wow! I wasn't expecting to hear back from you until Monday :)I haven't been able to run your code yet. I downloaded SBCL for PPC but have not been able to successfully compile Elephant with it. I will continue trying. I used to have it working before, but stopped using SBCL a while ago for OpenMCL.

Re: [elephant-devel] Querying Advice

2006-11-12 Thread Ian Eslick
>> >> First of all, ask yourself, what is the size of your dataset? Can >> you fit it all into memory? >> If so, you have the full power of lisp at your command in dealing >> with the querying. You >> will not have to write any macros to do this. You might find the DCM >> package, in the "contr

Re: [elephant-devel] Querying Advice

2006-11-12 Thread Daniel Salama
I'd love to hear/read about your take on the subject.It would be really great to be able to have some querying or even better a reporting tool that worked with Elephant. May be something a la Crystal Reports, but that may be shooting too high :). How about something like http://www.agata.org.br/?An

Re: [elephant-devel] Querying Advice

2006-11-12 Thread Daniel Salama
On Nov 12, 2006, at 3:48 PM, Robert L. Read wrote: This requires a philosophical response.  In general, I think it will be way easier than you image, once you have been pointed in the right direction.  Take my advice with a grain of salt.I certainly hope so. As I have been learning lisp, I have fu

Re: [elephant-devel] Querying Advice [w/code example]

2006-11-12 Thread Robert L. Read
Dear Daniel and Team,     I think the code below, which I have tested on SBCL, illustrated a typical problem that Daniel Salama introduces.  To paraphrase, you have a datatype (perhaps compound) which has a lot of slots; you have a GUI, perhaps web-based, that you use to both select or filter

Re: [elephant-devel] Querying Advice

2006-11-12 Thread Ian Eslick
phant bugs and development Subject: Re: [elephant-devel] Querying Advice This requires a philosophical response.  In general, I think it will be way easier than you image, once you have been pointed in the right direction.  Take my advice with a grain of salt. First of all, ask yourself

Re: [elephant-devel] Querying Advice

2006-11-12 Thread Robert L. Read
This requires a philosophical response.  In general, I think it will be way easier than you image, once you have been pointed in the right direction.  Take my advice with a grain of salt. First of all, ask yourself, what is the size of your dataset?  Can you fit it all into memory? If so, you