Do you have any quantitative numbers? I'd really like to know how much this saves.
Jeff On Thu, Apr 1, 2010 at 9:25 AM, Eli Jones <[email protected]> wrote: > Compression will probably show the biggest benefit when you have a > significant difference between the un-compressed and compressed sizes and > the amount of data you're bringing over the wire is large enough to cause a > noticeable slowdown.. > So..if all you are doing is pulling over one entity with properties that add > up to 200 bytes total.. compression is just going to slow you down... > If you are pulling over an entity that has properties of 200 kilobytes > total, the time it takes to get the entity decompress it, change it, > compress it and put it back to the datastore will be faster than just > getting, changing and putting a non-compressed version. > Granted my assumptions in this case are based on tests I did against two > Models: a decompressed version with two large un-indexed properties and then > a compressed Model that had one property = a compressed protobuf of the > decompressed Model. (The protobuf before compression was 390KB in size.. > after compression it was 40KB). Even without using compression on the > protobuf of the big model.. just putting one large property (containing a > protobuf of the two property model) was much faster than directly putting > the two prop Model (even with indexed = false for the props). > After adding compression to the mix, the roundtrip of getting, > decompressing, changing, compressing and putting took less time and less cpu > overall. Than getting, changing, putting the un-compressed protobuf.. > never mind comparing it to putting a larger entity with multiple defined > (but un-indexed) properties. > In the end, compression may be less important than just stuffing the an > entire entity into a BlobProperty as a protobuf. (I seem to remember that > the benefit from compressing the large protobuf was nowhere near as drastic > as the benefit from turning the entire entity into a protobuf to be stuffed > into one prop.) > On Thu, Apr 1, 2010 at 2:01 AM, Jeff Schnitzer <[email protected]> wrote: >> >> On Wed, Mar 31, 2010 at 9:57 PM, Robert Kluin <[email protected]> >> wrote: >> > >> > Although I have not personally tested with _really_ large entities, >> > I see very little difference in performance based on the size of the >> > entity. We have some models with 20 or 30 string and float fields >> > that seem to perform similar to models with 5 or 6 string fields. >> > There have been a number of threads discussing this in the past. I >> > think a post had some benchmarks in December or January. >> >> This has been my experience as well. Additional indexes cost a lot, >> but additional unindexed properties seem to be almost "free" in the >> datastore. >> >> I would ask of anyone asking for select at a property level: Have you >> run any performance tests of your application with big vs small >> entities? Are you sure it matters? >> >> Jeff >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
