On Fri, Oct 27, 2017 at 5:47 PM Travis Scrimshaw <tsc...@ucdavis.edu> wrote:
> > > On Friday, October 27, 2017 at 10:19:31 PM UTC+10, Erik Bray wrote: > >> On Fri, Oct 27, 2017 at 10:42 AM, Jeroen Demeyer <jdem...@cage.ugent.be> >> wrote: >> > This is a topic which comes up now and then but which hasn't been >> resolved >> > so far. >> > >> > Sage has a "pickle jar" stored in src/ext/pickle_jar/pickle_jar.tar.bz2 >> and >> > there are tests in src/sage/structure/sage_object.pyx which check that >> every >> > object in the pickle jar can be unpickled without raising an exception. >> > >> > The theory is that this ensures that people having old pickles on their >> > machines can still use them with newer versions of Sage. However, the >> pickle >> > jar does not achieve this goal because >> > >> > (1) The doctest only tests that the pickles can be unpickled. There are >> no >> > tests that the unpickled objects still function correctly. See >> > https://trac.sagemath.org/ticket/16311 >> > >> > (2) Nobody adds new pickles to the pickle jar. The last time that a >> pickle >> > was added was in 2011. So, if anything, it only tests that pickles >> which are >> > at least 6 years old still work correctly. >> > >> > Since the pickle jar doesn't do what it is meant to do, I suggest to be >> > pragmatic and remove it completely. >> > >> > Having the pickle jar is a burden for development because it prevents >> > certain refactorings or removals. For example finite_field_ext_pari.py >> was >> > deprecated in 2014 but we cannot remove it because it breaks the pickle >> jar. >> > I know that I could probably fix this with register_unpickle_override() >> but >> > that feels a waste of time because I wonder if anybody really cares. In >> the >> > mean time, people need to maintain that unused deprecated file to make >> it >> > Python 3 compatible for example. >> > >> > I'm sure that some people have suggestions for improving the pickle jar >> > procedure. But still, the fact remains that many pickles in the current >> > pickle jar are broken. So there is no point in keeping them. >> >> I am also +1 if there are no issues with the notebooks or CoCalc. > Although I do not think this affects them because inorder to use the > objects, I have to recreate them instead of having them being > (functionally) persistent across sessions. > There is definitely no issue with CoCalc. >> Plus, while pickling has many valid runtime use-cases, particularly >> for IPC, and short-term preservation of objects between interpreter >> sessions, it was *never* intended for long-term data storage, in part >> precisely because it's directly tied to the source code that was used >> to produce the pickle file. The only truly "correct" way to restore >> old pickled objects is to do so with the same version of the software >> the pickle was created with. >> > > The TestSuite does a standard pickle/unpickle test for this and IMO every > object should have at least one TestSuite(foo).run() test (typically in its > __init__ method). We have also made it easier with git tags to revert to > previous versions of Sage and then if you really need your object, you can > translate it into key data in a file that you can then read back and > reconstruct in the later version if the pickling has broken in the > meantime. While this means we are less backwards compatible, I think this > will not affect too many (any?) users, or at least make then not quit using > Sage. > If we were to get rid of the pickle jar, then we should also add documentation (e.g., for the save/load functions, which save/load objects), saying -- basically -- do not trust this at all, except for ephemeral movement of objects, unless you are sure to keep the specific version of sage you used around. We should say that we gave up and decided not to put even the slightest effort into making save/load work over time. William > > Best, > Travis > > > -- > You received this message because you are subscribed to the Google Groups > "sage-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-devel+unsubscr...@googlegroups.com. > To post to this group, send email to sage-devel@googlegroups.com. > Visit this group at https://groups.google.com/group/sage-devel. > For more options, visit https://groups.google.com/d/optout. > -- -- William Stein -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
