On Fri, Jan 7, 2011 at 11:54 PM, William Stein <wst...@gmail.com> wrote:
> On Fri, Jan 7, 2011 at 10:57 PM, Robert Bradshaw
> <rober...@math.washington.edu> wrote:
>> On Fri, Jan 7, 2011 at 2:55 PM, William Stein <wst...@gmail.com> wrote:
>>> On Fri, Jan 7, 2011 at 1:37 PM, emil <emil.widm...@gmail.com> wrote:
>>>>
>>>>
>>>> On Jan 7, 4:18 pm, William Stein <wst...@gmail.com> wrote:
>>>>> The best solution to the problem we're discussing in the context of
>>>>> SAGE_ROOT/devel/sage would be to change the Sage library to build all
>>>>> .so's in place, so that the SAGE_ROOT/devel/sage/build would in fact
>>>>> be redundant, and there would only ever be exactly one copy of each
>>>>> file (or .so) in SAGE_ROOT/devel/sage.
>>>>
>>>> Would that mean rewriting sage build scripts?
>>>
>>> Yes. It would be a nontrivial project. I hope somebody does it sometime!
>>>
>>>>
>>>>>This is what happens with
>>>>> "python setup.py develop" or "setup.py build_ext --inplace <list of
>>>>> extension names>", when using setuptools instead of distutils. For
>>>>> example, with the Sage Notebook and PSAGE
>>>>> (http://code.google.com/p/purplesage/), which both use setuptools, we
>>>>> do all of our work this way.
>>>>
>>>> I guess you use a core of sage for your PSAGE, does it mean
>>>> work could concentrate on the packages you have excluded from psage?
>>>
>>> By "PSAGE" I meant http://code.google.com/p/purplesage/, which is a
>>> separate Python package, which itself has nothing to do with anything
>>> currently in Sage. It's just a new Python package that depends on
>>> Sage.
>>>
>>>
>>>>
>>>>> Another plus to this change is that if foo is a function and you type:
>>>>>
>>>>> sage: foo??
>>>>>
>>>>> you'll see the actual location of the file you should edit to modify
>>>>> foo, rather than some highly misleading file in SAGE_ROOT/local (which
>>>>> is a major source of confusion at first).
>>>>>
>>>>> That said, it could be significant and painfully disruptive work for
>>>>> somebody to change the core Sage library to work this way.
>>>>
>>>> Do you have a rough estimate how long it takes for someone with no
>>>> prior knowledge of setuptools?
>>>
>>> It depends entirely on how good that person is, and what they know
>>> about Sage. If they are sufficiently smart, maybe 1 week? It's
>>> impossible to know what issues might arise though until one try.
>>>
>>>
>>>>> I hope it
>>>>> happens at some point. Confusion will be reduced overall, as will
>>>>> disk space usage.
>>>> A Rough estimate of size reduction?
>>>
>>> I estimate 189MB, i.e., the size of SAGE_ROOT/devel/sage/sage/.
>>
>> This would of course mean no reasonable tracebacks, if the source
>> couldn't be found....
>> - Robert
>
> No, you're wrong. The source would be found, and the tracebacks would
> be excellent.
I was talking about a completely stripped binary (i.e no source).
> Observe the following:
>
> deep:sage wstein$ pwd
> /Users/wstein/sd27/sage-4.6.1.rc1/devel/sage
> deep:sage wstein$ ls -lh build/sage/rings/arith.py
> -rw-r--r-- 1 wstein staff 127K Jan 5 00:22 build/sage/rings/arith.py
> deep:sage wstein$ ls -lh sage/rings/arith.py
> -rw-r--r-- 1 wstein staff 127K Jan 5 00:22 sage/rings/arith.py
>
> Notice that there are two distinct but identical files. This is a
> waste of disk space. This can be fixed with absolutely no reduction in
> functionality or capabilities of Sage, e.g., by switching to using
> setuptools and building the so modules in place.
I agree 100% here.
> I think I was wrong in my space saving estimate (189MB) above. The
> SAGE_ROOT/devel/sage directory in sage-4.6.1.rc1 is 768MB:
>
> deep:sage wstein$ du -sch *
> 4.0K MANIFEST.in
> 4.0K PKG-INFO
> 4.0K README.txt
> 367M build
> 4.0K bundle
> 4.5M c_lib
> 207M doc
> 4.0K export
> 4.0K install
> 68K module_list.py
> 40K module_list.pyc
> 4.0K pull
> 189M sage
> 4.0K sage-push
> 40K setup.py
> 4.0K spkg-delauto
> 4.0K spkg-dist
> 4.0K spkg-install
> 768M total
>
>
> ------
>
> For Sage to run, I think all that's *needed* is:
>
> SAGE_ROOT/devel/sage/build/sage
>
> since that is exactly the thing that
> SAGE_ROOT/local/lib/python/site-packages/sage is linked to, so it is
> all that Python sees at runtime. This directory is 128MB:
>
> deep:sage wstein$ du -sch build/sage
> 128M build/sage
> 128M total
>
>
> -----
>
> So one could strip 640MB away from SAGE_ROOT/devel/sage/build/sage and
> still have a Sage that runs. And this would have docstring
> introspection even for Cython code, since the .pyx files are in
> SAGE_ROOT/devel/sage/build/sage.
> The autogenerated C/C++ files are not there, so development ("sage
> -br") would be painfully slow the first time, as these files would all
> get regenerated. The SAGE_ROOT/devel/sage/doc/ directory would also
> be missing, which is used by the notebook *only* when browsing the
> manuals (not for function_name? introspection).
>
> I didn't figure out the exact savings from just switching to
> setuptools, but I think it would be at least 240MB.
I don't even think we need setuptools--just build extensions inplace
(there's an option for this) and put devel/sage in the python path
(this was what I did for enabling concurrent execution of distinct
branches, though that still needs a bit of work
http://trac.sagemath.org/sage_trac/ticket/9967). Then one wouldn't
even need to do a "sage -b" to work on (non-Cython) code.
- Robert
--
To post to this group, send an email to sage-devel@googlegroups.com
To unsubscribe from this group, send an email to
sage-devel+unsubscr...@googlegroups.com
For more options, visit this group at http://groups.google.com/group/sage-devel
URL: http://www.sagemath.org
