Hi Jeff, some of your concerns would have been valid with the pre-
release versions but not with the final release 1.0 of Twig. See
below for explanations
On 10 Mar 2010, at 16:14, Jeff Schnitzer wrote:
On Tue, Mar 9, 2010 at 8:47 PM, John Patterson
<[email protected]> wrote:
For the sake of comparison, see below for the Twig equivalent
class UserSkill {
@Parent User user;
String skill
int ability;
}
Note that no key is required and user is referenced directly so
your model
has no dependency on the low-level datastore
...and instead must make two fetches every time you load a UserSkill
object.
No that is incorrect. You actually have very fine control over what
is loaded and when using "activation" [1]
Firstly, the relationship you ask about "@Parent User user" is
absolutely not even required in this example. Although Objectify
requires the parent field, Twig does not. The returnParentsNow()
method works perfectly fine without it. I was just showing how much
cleaner the exact equivalent code was with Twig.
Because the field is not needed, obviously the developer would only
add that field as a convenience if they actually needed to access the
parent.
Secondly, even if the developer only wanted some of the parents
directly they could still add the field and not activate it:
class UserSkill {
@Activate(0) @Parent User user;
String skill
int ability;
}
Now the field contains an "unactivated" instance that does not require
a fetch at all - the key is cached with a weak reference so when you
want to activate the instance just call refresh(user). Convenient and
efficient.
Is it at least smart enough to do all the parent fetching as
a single batch get?
Yes it is at least that smart :) Parents are fetched in a single
batch using the exact same chunk-size as the child elements. See
PrefetchParentInstanceIterator for details.
Does a put() on an entity cascade to its parent?
Storing any instance cascades to every reachable instance irrespective
of the relationship. An instance can only be stored once so if the
parent is already persistent it will not be stored again.
This is one of my gripes with the Objectify API - there is no
distinction between "store" and "update" which makes it easy to
accidentally clobber existing data. The two actions are very
different in intent and should be different methods.
Not only is the parent fetch efficient but it is much more readable.
Sorry Jeff, but this is just a train wreck:
Objectify :
List<Key<User>> userKeys = new ArrayList<Key<User>>();
for (Key<UserSkill> key: ofy.query(UserSkill.class).filter("skill",
"java").filter("ability >", 5).fetchKeys())
userKeys.add(key.getParent());
Collection<User> users = ofy.get(userKeys).values();
Do you really expect your maintenance developer to understand that in
2 years time? Im sure complex queries become even more convoluted.
I notice you return an Iterable rather than an Iterator. The problem
with this is that every time the Iterable is accessed the query is
executed again. A little dangerous and non-obvious in my opinion.
Now without keys this is much more readable - almost like a sentence:
Twig:
Iterator<User> users = datastore.find()
.type(UserSkill.class)
.addFilter("skill", EQUAL, "java")
.addFilter("ability", GREATER_THAN, 5)
.returnParentsNow();
You really get the best of both worlds - precise control over fetching
and readability.
Finally on this point, even if the parent property is not defined it
is always possible to get the Key that is related to that instance
using "associatedKey(Object)" and then load the parents directly by
Key - just like Objectify - although I would not recommend this.
Claiming that "Twig is the only datastore interface to support...Plain
old Java object data models with no datastore dependencies" is really
too much.
Really? I thought it was very simple. Twig uses Pojos with no
dependencies, the others don't! Can you give an example of a Twig
data model that depends on the datastore? You are not confusing "data
model" with DAO code are you?
All that you can honestly say is that Twig lets you avoid
datastore dependencies...except when you can't.
You will have to clarify what you mean by "except when you can't".
There is no reason to have any dependency on the datastore in a Twig
data model.
The same can be said
for JDO or Objectify.
Neither JDO or Objectify support direct unowned relationships which
makes your models dependent on the datastore. Also, Objectify
requires every persistent instance to define a key - this means that
no class can be persisted unless it is specifically designed to be a
data model class. The simple examples above show this nicely.
I could go into how Twigs TypeConverter framework makes it easy to
convert any user type into datastore types without the need for any
dependencies in your data model - but its all in the docs.
Neither of these problems affect Twig so I really don't understand
what you are trying to refute here.
Can you avoid a Key reference if you don't want to perform two fetches
for every single load of a parented object?
Yes. If you don't want to load any referenced instance you can set
its activation depth to 0. The Keys *are there* - they are just
managed for you by the KeyCache.
Can you avoid a Key reference if you don't want to eager-load a
collection of entities?
Yes. Same as above. Also, on the road map is "auto activation" -
however todays activation settings work well. They are actually
modelled off Db4o but more easily configurable on a field-by-field
basis.
Can you avoid a Key reference if you have a one-to-many relationship
where the many-side might have 5001 values?
Yes. Just use a Relation Index Entity or keep the reference in the
many side. Really this is a general datastore issue no different for
Objectify or JDO.
If you really want to avoid Key objects, you'll probably get a lot
farther with DataNucleus' support for collection proxies. But I doubt
either will let you build efficient real-world applications completely
without relying on a few Key references.
That is just rubbish. You need to understand that the Keys *are*
there - they are just managed for you. In other words the Entities
actually stored in the datastore are very similar from Twig to
Objectify - but the data models are much cleaner in Twig.
Incidentally, you can generally avoid Key references using Objectify
if you so choose. Keys only become mandatory when you are dealing
with parented entities -- which should be used infrequently for
performance reasons. There's no reason you can't just use simple ids
as foreign key references.
That is a much worse solution than using Keys even! You really expect
developers to translate long ids into Keys and then look them up??? I
think this is getting a little ridiculous.
Iterator<User> users = datastore.find()
.type(UserSkill.class)
.addFilter("skill", EQUAL, "java")
.addFilter("ability", GREATER_THAN, 5)
.returnParentsNow();
This is how easy it is to use Relation Index Entities with the new
release
of Twig
While I'm solidifying the opinion that the RIE pattern is actually
pretty useless,
being able to fetch parent keys (or parents) does seem
to come up enough that we may add a shorthand for it:
Iterable<User> users = ofy.query(UserSkill.class).filter("skill",
"java").filter("ability >", 5).fetchParents();
As you have seen - it makes such common queries in Twig very
readable. One feature I have not mentioned is that you can also call
returnParentsCommandNow() which returns a command object that gives
you even finer control over the parent results including filtering out
entities at the low-level. This can save a lot of CPU.
Note that anyone can trivially fabricate their own implementation of
this boilerplate right now using Objectify's QueryWrapper class.
Behind the scenes two queries are run *in parallel* asynchronously
and then
merged together into a single Iterator. Notice also that the two
queries
both inherit the common filter and sort on "ability".
This does seem quite interesting, and I congratulate you for working
out the underlying details :-)
Cheers :) It is very fast - I recommend you use Twig on your next
project to give it a shot. Just kidding!
The question I have is for our Google overlords: Why isn't the Java
SDK opensourced like the Python implementation? Yes, the DN plugin is
opensource, but that's the least interesting part!
Agree 100%
John.
--
You received this message because you are subscribed to the Google Groups "Google
App Engine for Java" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
