OK, let's consider the following setup: we have three Cython modules, a, b, and c, and b cimports a, so we have (at least) three files
a.pyx a.pxd b.pyx c.pyx Having just built sage, we also have a.c b.c c.c with *newer* timestamps than any of the above. Now we switch branches (forward, backwards, or to a completely separate branch) that modifies a.pxd. Git sets a.pxd's timestamp to now, and when we run cythonize we correctly deduce that because a.c and b.c are older than a.pxd we must rebuild them. c.pyx is untouched, and c.c is not rebuilt. What I'm not following here is how this is not the correct behavior, nor how hashes would help other than adding complexity (e.g. one must store the hash of a.pxd inside b.c). Also, manually setting timestamps would risk incorrect builds (remember the sage -ba days). The fact that one must build large parts of sage when a large number of files (or files high in the dependency tree) change is to be expected. cycache is the right answer here (and, yes, I am becoming increasingly motivated to do something about it). On Mon, Nov 21, 2016 at 6:41 AM, Erik Bray <erik.m.b...@gmail.com> wrote: > On Fri, Nov 18, 2016 at 8:18 AM, William Stein <wst...@gmail.com> wrote: >> >> Ideas: >> >> >> What about changing Cython to optionally use sha1 or md5 hashes instead of >> timestamps? > > I've had this thought as well--it would be fairly straightforward to > output a list of hashes for all Cython sources and compare those when > deciding whether or not a file needs to be rebuilt. > >> On Thu, Nov 17, 2016 at 11:12 PM David Roe <roed.m...@gmail.com> wrote: >>> >>> If I checkout an old branch (say, from one or two versions of Sage ago), >>> it essentially forces a rebuild of all of Sage, even if I think better of it >>> and checkout develop immediately. The rebuild is a consequence of the fact >>> that Cython builds based on timestamp and all of the files have been >>> touched. >>> >>> I'd like to solicit ideas to make this less painful. Here's one, based on >>> a conversation with Julian Rüth. >>> >>> Create a new git trac subcommand to replace `git trac checkout 1234`, say >>> `git trac old 1234`. This would fetch the branch, check it out into a >>> completely separate folder within ($SAGE_ROOT/merge_tree or something), >>> merge in develop. If the merge is successful, create a new branch and pull >>> the changes in. This ends up with only a few files changing if you started >>> at develop. If the merge is not successful, report to the user and ask them >>> to fix the merge in >>> $SAGE_ROOT/merge_tree. There would then be some way to resume and pull in >>> the changes. >>> >>> There are some details to fill in, but I think that an approach like this >>> can work. It does mean having another 100MB working tree floating around >>> just for merging into, and also stepping a bit further away from normal git >>> practices. >>> >>> Any other ideas? >>> David >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "sage-devel" group. >>> To unsubscribe from this group and stop receiving emails from it, send an >>> email to sage-devel+unsubscr...@googlegroups.com. >>> To post to this group, send email to sage-devel@googlegroups.com. >>> Visit this group at https://groups.google.com/group/sage-devel. >>> For more options, visit https://groups.google.com/d/optout. >> >> -- >> Sent from my massive iPhone 6 plus. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sage-devel" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to sage-devel+unsubscr...@googlegroups.com. >> To post to this group, send email to sage-devel@googlegroups.com. >> Visit this group at https://groups.google.com/group/sage-devel. >> For more options, visit https://groups.google.com/d/optout. > > -- > You received this message because you are subscribed to the Google Groups > "sage-devel" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sage-devel+unsubscr...@googlegroups.com. > To post to this group, send email to sage-devel@googlegroups.com. > Visit this group at https://groups.google.com/group/sage-devel. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "sage-devel" group. To unsubscribe from this group and stop receiving emails from it, send an email to sage-devel+unsubscr...@googlegroups.com. To post to this group, send email to sage-devel@googlegroups.com. Visit this group at https://groups.google.com/group/sage-devel. For more options, visit https://groups.google.com/d/optout.
