On Dec 4, 2009, at 3:39 AM, tmb wrote: >> You can put the .sage/sage_notebook.sagenb/home/ directory under >> revision control. You should probably only hg add the >> worksheet.html >> and worksheet_conf.pickle files, and ignore everything else. > > That doesn't really work with the current directory structure. For > example, imagine I add two different worksheets on two systems and > then try to synchronize--the directory names will conflict even though > the worksheets have different names. Conversely, the same worksheet > may be in two different directories, so it would never get merged. > Also, I would need to manually add and commit things from the command > line. So, using version control right now is at least as much work as > doing it by hand, and it makes merging and recovering from problems > even more complex than it already is. > > The problem is really with the .sage directory structure; without > changing that, I think it will be hard for Sage to get reliable > version control or error recovery. Fortunately, the changes may not > have to be big. > > Right now, .sage is essentially an opaque directory that I can't > reliably do anything with. You as a developer may know what's in > there and how it works, but I don't.
Most of what's there is caching, temporary files, user prefs ,etc. (More than just the notebook, btw.) > Look at the path: > > .sage/sage_notebook/worksheets/admin/10 > > So, here are some sample questions that come up: > > -- What worksheet is that? (I need to look inside--makes it hard to > organize manually.) > -- How do I copy a new worksheet into that directory from somewhere > else? > -- Is the title stored anywhere else? Which of the files contains the > authoritative title? > -- Is anything cached? On disk? In memory? When does the cache get > updated? > -- Can I just edit worksheet.txt? > -- What happens if I make changes while the server is running, will it > blow those away? > -- If I copy in a different worksheet.txt in there, do things just > work? Do I need to delete something else? I don't know the answer to all those questions, but a little experimentation might shed some light on it (as would reading the code as you mention.) It is accurate to say the current structure is a implementation detail, not a promised API. > I can read a bit between the lines if I go into the Python API > documentation for sage.server.notebook.worksheet.Worksheet. But the > existence of a programmatic API like that may even suggest that I > should make no assumptions about the representation of worksheets at > all. > > My suggestion would be the following: > > -- For single user mode, put all the worksheets in ~/SageMath (that > tells me that I'm supposed to look at them) It's standard to make a .foo directory to hold application defaults and data. Making a visible top-level directory is more invasive. Not treating admin specially also makes the code cleaner. > -- Put all notebook files at the toplevel with descriptive names, like > ~/SageMath/fft.sws Where would the descriptive names come from? How would one handle naming conflicts? Worksheet renaming? > -- Put all cache/computed files in ~/SageMath/.cache/... > -- Guarantee the user that he can change anything in ~/SageMath > (outside .cache) by hand and things will keep working; the server will > detect changes and rebuild cache files as necessary. > -- Adding a new worksheet is the same as copying a file into ~/ > SageMath/foo.sws > -- Put in hooks to trigger "hg add", "hg commit", etc. when necessary > (create a .hgignore containing .cache automatically) > -- Trigger commits fairly frequently (they're cheap) > > With a directory structure like that, things get a lot simpler for me > because I actually understand what I can do: > > -- Make a backup: cp ~/SageMath/*.sws ~/my-sage-backup > -- Restore backup: cp ~/my-sage-backup/* ~/SageMath This works already (with ~/.sage/sage_notebook) > -- Restore a few files: cp ~/my-sage-backup/improc*.sws ~/SageMath > -- Fix something in the fft worksheet: vi ~/SageMath/fft.sws Note a .sws file may is actually a archive of several relevant files (e.g. images that have been uploaded into DATA). > -- Duplicate a worksheet: cp ~/SageMath/fft.sws ~/SageMath/new-fft.sws > -- Synchronize repositories: cd ~/SageMath; hg pull ssh://.../ > SageMath; > hg update > -- Merging of worksheets in version control actually works as > expected. One still has the issue of naming conflicts between multiple systems. I think it makes more sense to put a repository inside each worksheet. Otherwise two servers have to be totally in sync (exactly the same set of worksheets) or not syncable at all. It would still be easy to write a script to sync every worksheet (say, for a given user) one at a time. But the idea of putting this data under revision control is a good one--no one's had time to implement it yet. Do you want to help out? > I understand that somehow I can probably do many of these things > already. But the structure of the current directory tree doesn't make > it easy or even easy to figure out, and the documentation has not been > that much help to me either. > > With a clearer directory tree layout and some small guarantees about > the cache, you wouldn't need much documentation for people to be able > to work with sage worksheets at the directory level reliably. And, in > particular, it would make it much easier for people like me to recover > from data loss due to other bugs in Sage. > > Tom > > PS: For multiuser using the current setup, you could use ~/SageMath > for admin, and ~/SageUsers/user-name/... for all other user > accounts. > > (Of course, in a university environment, we'd really like to use LDAP > for authentication and ~user/SageMath for each valid user.) > > -- > To post to this group, send email to sage-support@googlegroups.com > To unsubscribe from this group, send email to > sage-support-unsubscr...@googlegroups.com > For more options, visit this group at > http://groups.google.com/group/sage-support > URL: http://www.sagemath.org -- To post to this group, send email to sage-support@googlegroups.com To unsubscribe from this group, send email to sage-support-unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sage-support URL: http://www.sagemath.org
