That is substantially faster, and more in line with my expectation of what the performance difference would be. I had thought I was using the XS module, but clearly something was amiss! This is good news.
I would think implementing within MARC::Record would be the winner between those two approaches, rather than coming up with an intermediary format. However, I had been imagining utilizing the YAML for caching rather than as persistent data, making the accommodation of internal changes to MARC::Record a matter of flushing the cache rather than regenerating the serialized records. Cheers, Clay 2010/10/26 Frederic Demians <[email protected]> > > I did some (very limited) testing on storing and retrieving MARC in YAML. >> The results were not encouraging. IIRC, I just did a direct conversion of >> the MARC::Record object into YAML and back. Perhaps there's a way to >> optimize the formatting that would improve performance, but my testing >> showed sometimes even worse performance than XML. >> > > Did you use YAML or YAML::XS? My tests with YAML::XS shows a very > significant improvement with YAML: see attached file. Of course, we should > define an serialization format independent from MARC::Record object if we > don't want to break the process when MARC::Record internal data structure > ever change. > > > MARCXML is a performance killer at this point, but there's no other >> apparent way to handle large bib records. The parsing is the issue, not the >> data transfer load. Perhaps cached BSON-formatted MARC::Record objects are a >> way out of this. >> > > Benchmark should be done with all available serialization formats. > > We also could implement serialization/deserialization logic directly into > MARC::Record library, as ISO2709 and XML format, in order gain control. > -- > Frédéric > > _______________________________________________ > Koha-devel mailing list > [email protected] > http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel >
_______________________________________________ Koha-devel mailing list [email protected] http://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
