Re: Validating unique objects in CoreData

2010-02-14 Thread Sean McBride
Chris Hanson ([email protected]) on 2010-02-14 15:29 said: >> I have >> found NSManagedObject's objectID method largely useless since, as the >> docs say, "if the receiver has not yet been saved, the object ID is a >> temporary value that will change when the object is saved”. > >These aren’t as useless

Re: Validating unique objects in CoreData

2010-02-14 Thread Chris Hanson
On Feb 13, 2010, at 2:32 AM, daniele malcom wrote: > Here you will found my simple storage data model: > http://img197.imageshack.us/img197/4388/screenshot20100213at111.jpg If you’re going to represent a UUID in Core Data, you are likely to be far better off storing and searching on it as two 64

Re: Validating unique objects in CoreData

2010-02-14 Thread Chris Hanson
On Feb 13, 2010, at 4:32 PM, Sean McBride wrote: > I have > found NSManagedObject's objectID method largely useless since, as the > docs say, "if the receiver has not yet been saved, the object ID is a > temporary value that will change when the object is saved”. These aren’t as useless for your

Re: Validating unique objects in CoreData

2010-02-14 Thread Roland King
Please report back, I'd be very interested to hear what you find out. Even though I said 1000 objects a second seemed ok you know .. you're right, it's not. Have you tried a few things like not doing the check to see how fast you can just blast objects into the database without caring if they ar

Re: Validating unique objects in CoreData

2010-02-14 Thread daniele malcom
Thanks Roland, saving each x records drops searching time. That's great. These are my benchmarks with 30k objects: CoreData without saving each X insertions: about 5-6 minutes CoreData with saving each 500 insertions: about 30 seconds CoreData with auxiliary indexes dictionary: about 2 seconds Ho

Re: Validating unique objects in CoreData

2010-02-13 Thread Roland King
ok I downloaded your project. I agree with Jerry there's a memory leak, actually worse than that, you aren't actually remembering the article to set its parent if you create it, so [ DDArticle newArticleWithID: messageid context:ctx ]; should be article = [ DDArticle newArtic

Re: Validating unique objects in CoreData

2010-02-13 Thread Sean McBride
Roland King ([email protected]) on 2010-02-13 22:06 said: >That's not a horrible solution, except for the feeling that core data >ought to let you do what you want without having to implement your own >UUID cache. I have found it useful, on many occasions, to add a 'uuid' attribute to many of my Core

Re: Validating unique objects in CoreData

2010-02-13 Thread Jerry Krinock
On 2010 Feb 13, at 13:51, daniele malcom wrote: > Hi Roland, in fact indices table exists (for DDArticle entity): > Enter SQL statements terminated with a ";" > sqlite> .tables > ZDDARTICLEZ_METADATAZ_PRIMARYKEY > sqlite> .indices ZDDARTICLE > ZDDARTICLE_ZMESSAGEID_INDEX > ZDDARTICLE_ZPAR

Re: Validating unique objects in CoreData

2010-02-13 Thread daniele malcom
Hi Roland, in fact indices table exists (for DDArticle entity): Enter SQL statements terminated with a ";" sqlite> .tables ZDDARTICLE    Z_METADATA    Z_PRIMARYKEY sqlite> .indices ZDDARTICLE ZDDARTICLE_ZMESSAGEID_INDEX ZDDARTICLE_ZPARENT_INDEX With my macbook pro insertion of 30k articles took ab

Re: Validating unique objects in CoreData

2010-02-13 Thread Roland King
.. oh and one other thing, there's a core data instruments tool in XCode, well there is for OSX, not for iPhoneOS which I develop for which may be why I never saw it before. You could try that. On 13-Feb-2010, at 9:36 PM, Roland King wrote: > ok, I don't see anything wrong with the predicate c

Re: Validating unique objects in CoreData

2010-02-13 Thread Roland King
ok, I don't see anything wrong with the predicate code, but I'm no core data expert. I'll make one totally challengable statement. Assuming that core data uses sqllite in a rational way to store objects (eg not storing everything as blobs of opaque data) for instance one table per entity where

Re: Validating unique objects in CoreData

2010-02-13 Thread daniele malcom
On Sat, Feb 13, 2010 at 4:06 AM, Roland King wrote: > That's not a horrible solution, except for the feeling that core data ought > to let you do what you want without having to implement your own UUID cache. > I'm still a bit surprised that a lookup for an object by one attribute is > taking so

Re: Validating unique objects in CoreData

2010-02-12 Thread Roland King
That's not a horrible solution, except for the feeling that core data ought to let you do what you want without having to implement your own UUID cache. I'm still a bit surprised that a lookup for an object by one attribute is taking so long, over just 30,000 objects. You do have the uuid attrib

Re: Validating unique objects in CoreData

2010-02-12 Thread Jerry Krinock
On 2010 Feb 12, at 10:06, malcom wrote: > A first solution is ... 60k+ queries takes ... a minute or more. > > My second solution is to create a second auxiliary NSMutableDictionary ... > the entire process takes about 5 seconds to finish So you confirmed what I what I was loathe to say, which

Re: Validating unique objects in CoreData

2010-02-12 Thread malcom
I've tried to make an alternative method. I'll try to describe it. I would to use a second an auxiliary index where to save my uuid<->objectURI; results are better, so I try to make a summary of the problem and the solution. If anyone have a better idea I'll be happy to talk about it :) I've abou

Re: Validating unique objects in CoreData

2010-02-12 Thread Sean McBride
On 2/12/10 9:05 PM, Roland King said: >2) Where does the uuid on the object you are inserting come from? If you >are setting a UUID on the objects when you are creating them then using >one of the uuid_generate functions the UUID generated can be 'reasonably >considered unique Or instead of those

Re: Validating unique objects in CoreData

2010-02-12 Thread Jerry Krinock
On 2010 Feb 12, at 03:45, malcom wrote: > Any idea to improve performance of this check? Well, 30,000 uuid is only a megabyte or so. I really hate to say this, but, just as a wild and crazy experiment, fetch the uuids from those 30,000 into a giant NSSet, and determine uniqueness by seeing if

Re: Validating unique objects in CoreData

2010-02-12 Thread malcom
On Feb 12, 2010, at 2:05 PM, Roland King wrote: > 1) does this uuid need to be something you generate externally and then set > onto the object? Each core data object already has an objectID which is > guaranteed unique in the database, can you make use of that one instead? You > are allowed t

Re: Validating unique objects in CoreData

2010-02-12 Thread Roland King
two questions, neither of which actually answers yours I think. 1) does this uuid need to be something you generate externally and then set onto the object? Each core data object already has an objectID which is guaranteed unique in the database, can you make use of that one instead? You are a

Validating unique objects in CoreData

2010-02-12 Thread malcom
Hello, I would use CoreData in order to mantain an archive of objects: each object contains an attribute called "uuid"; in the same db only one object can have the same attribute value. This mean I should validate the proposed new object to insert and check if the another object with the same uu