Re: [sage-devel] Re: profiling Sage startup

2011-08-17 Thread Julian Rüth
* William Stein [2011-08-12 11:30:21 -0700]: > On Fri, Aug 12, 2011 at 10:05 AM, Julian Rüth wrote: > > I'm not sure if this discussion has been picked up in a different thread > > since February but the problem with zipimport seems to be that it can't load > > .so files. Since we have plenty of

Re: [sage-devel] Re: profiling Sage startup

2011-08-12 Thread William Stein
On Fri, Aug 12, 2011 at 10:05 AM, Julian Rüth wrote: > I'm not sure if this discussion has been picked up in a different thread > since February but the problem with zipimport seems to be that it can't load > .so files. Since we have plenty of them, just loading the .py files from a > zip file and

[sage-devel] Re: profiling Sage startup

2011-08-12 Thread Julian Rüth
I'm not sure if this discussion has been picked up in a different thread since February but the problem with zipimport seems to be that it can't load .so files. Since we have plenty of them, just loading the .py files from a zip file and the .so files from the file system might not be such a big

Re: [sage-devel] Re: profiling Sage startup

2011-03-03 Thread Harald Schilly
On Wednesday, February 23, 2011 10:56:10 PM UTC+1, robertwb wrote: > > And this is with the directory presumably in the FS cache. ... just because I read this, you can flush the disk cache via: sudo sh -c "sync && echo 3 > /proc/sys/vm/drop_caches" H -- To post to this group, send an email

Re: [sage-devel] Re: profiling Sage startup

2011-03-01 Thread Robert Bradshaw
On Tue, Mar 1, 2011 at 10:07 AM, David Kirkby wrote: > On 1 March 2011 15:38, daly wrote: >> Sage could follow the same technique used by Gimp. >> Gimp loads a huge number of files at startup and can >> take over a minute to start on small systems. They >> have a startup splash screen that shows

[sage-devel] Re: profiling Sage startup

2011-03-01 Thread luisfe
On Mar 1, 10:13 am, Robert Bradshaw wrote: > On Tue, Mar 1, 2011 at 12:48 AM, Johan S. R. Nielsen > See lazy-import. Doing this for everything may incur significant > delays the first time a function is called (rather than before the > prompt) and there are issues with Sage being fragile about t

Re: [sage-devel] Re: profiling Sage startup

2011-03-01 Thread Robert Bradshaw
On Tue, Mar 1, 2011 at 4:56 AM, luisfe wrote: > On Mar 1, 1:32 pm, "Johan S. R. Nielsen" > wrote: >> On Mar 1, 10:13 am, Robert Bradshaw >> wrote: >> >> Nice! I weren't aware of this module. When you get a good idea, >> there's a good chance that someone else thought of it before ;-) I >> like t

Re: [sage-devel] Re: profiling Sage startup

2011-03-01 Thread Robert Bradshaw
On Tue, Mar 1, 2011 at 4:32 AM, Johan S. R. Nielsen wrote: > On Mar 1, 10:13 am, Robert Bradshaw > wrote: >> On Tue, Mar 1, 2011 at 12:48 AM, Johan S. R. Nielsen >> >> >> >> wrote: >> > On Feb 23, 11:03 pm, Jason Grout wrote: >> >> On 2/23/11 3:56 PM, Robert Bradshaw wrote: >> >> >> > On Wed, F

Re: [sage-devel] Re: profiling Sage startup

2011-03-01 Thread David Kirkby
On 1 March 2011 15:38, daly wrote: > Sage could follow the same technique used by Gimp. > Gimp loads a huge number of files at startup and can > take over a minute to start on small systems. They > have a startup splash screen that shows the files > which are being loaded. That way the user sees p

Re: [sage-devel] Re: profiling Sage startup

2011-03-01 Thread daly
Sage could follow the same technique used by Gimp. Gimp loads a huge number of files at startup and can take over a minute to start on small systems. They have a startup splash screen that shows the files which are being loaded. That way the user sees progress during startup. -- To post to this g

[sage-devel] Re: profiling Sage startup

2011-03-01 Thread luisfe
On Mar 1, 3:43 pm, "Johan S. R. Nielsen" wrote: > On Mar 1, 1:56 pm, luisfe wrote: > > > > > > > No, the lazy_import object keeps wrapping the original object, but > > when accessing the lazy_import object it imports the real object in > > the namespace. So, if the lazy_import has not been refe

[sage-devel] Re: profiling Sage startup

2011-03-01 Thread Johan S. R. Nielsen
On Mar 1, 1:56 pm, luisfe wrote: > > Yes, it is not used much. Ideally, the way of improving sage > startuptime would be: > > 1.- On first execution by a user, create a pickle of lazy_import of > the global names and save it in the .sage directory. This should not > be slower than current startup.

[sage-devel] Re: profiling Sage startup

2011-03-01 Thread Johan S. R. Nielsen
On Mar 1, 1:56 pm, luisfe wrote: > > No, the lazy_import object keeps wrapping the original object, but > when accessing the lazy_import object it imports the real object in > the namespace. So, if the lazy_import has not been referenced by > another object then it will be collected as garbage. I

[sage-devel] Re: profiling Sage startup

2011-03-01 Thread luisfe
On Mar 1, 1:32 pm, "Johan S. R. Nielsen" wrote: > On Mar 1, 10:13 am, Robert Bradshaw > wrote: > > Nice! I weren't aware of this module. When you get a good idea, > there's a good chance that someone else thought of it before ;-) I > like the fact that one can dynamically hack into an object's >

[sage-devel] Re: profiling Sage startup

2011-03-01 Thread Johan S. R. Nielsen
On Mar 1, 10:13 am, Robert Bradshaw wrote: > On Tue, Mar 1, 2011 at 12:48 AM, Johan S. R. Nielsen > > > > wrote: > > On Feb 23, 11:03 pm, Jason Grout wrote: > >> On 2/23/11 3:56 PM, Robert Bradshaw wrote: > > >> > On Wed, Feb 23, 2011 at 1:47 PM, Jason Grout > >> >  wrote: > >> >> On 2/23/11 3:

Re: [sage-devel] Re: profiling Sage startup

2011-03-01 Thread Robert Bradshaw
On Tue, Mar 1, 2011 at 12:48 AM, Johan S. R. Nielsen wrote: > On Feb 23, 11:03 pm, Jason Grout wrote: >> On 2/23/11 3:56 PM, Robert Bradshaw wrote: >> >> >> >> > On Wed, Feb 23, 2011 at 1:47 PM, Jason Grout >> >  wrote: >> >> On 2/23/11 3:06 PM, Robert Bradshaw wrote: >> >> >>> On Wed, Feb 23, 2

[sage-devel] Re: profiling Sage startup

2011-03-01 Thread Johan S. R. Nielsen
On Feb 23, 11:03 pm, Jason Grout wrote: > On 2/23/11 3:56 PM, Robert Bradshaw wrote: > > > > > On Wed, Feb 23, 2011 at 1:47 PM, Jason Grout > >  wrote: > >> On 2/23/11 3:06 PM, Robert Bradshaw wrote: > > >>> On Wed, Feb 23, 2011 at 11:34 AM, William Stein     > >>> wrote: > > On Wed, Feb 23,

[sage-devel] Re: profiling Sage startup

2011-02-23 Thread koffie
On Feb 23, 12:47 pm, Niles wrote: > A couple of other threads have mentioned ways to possibly shorten the > startup time for Sage.  This made me wonder: What is a good way to > tell *what* is making Sage take so long to start up?  Are certain > sections of the library taking the majority of the

[sage-devel] Re: profiling Sage startup

2011-02-23 Thread Jason Grout
On 2/23/11 3:56 PM, Robert Bradshaw wrote: On Wed, Feb 23, 2011 at 1:47 PM, Jason Grout wrote: On 2/23/11 3:06 PM, Robert Bradshaw wrote: On Wed, Feb 23, 2011 at 11:34 AM, William Steinwrote: On Wed, Feb 23, 2011 at 10:57 AM, Jason Grout wrote: On 2/23/11 12:28 PM, William Stein

Re: [sage-devel] Re: profiling Sage startup

2011-02-23 Thread Robert Bradshaw
On Wed, Feb 23, 2011 at 1:47 PM, Jason Grout wrote: > On 2/23/11 3:06 PM, Robert Bradshaw wrote: >> >> On Wed, Feb 23, 2011 at 11:34 AM, William Stein  wrote: >>> >>> On Wed, Feb 23, 2011 at 10:57 AM, Jason Grout >>>  wrote: On 2/23/11 12:28 PM, William Stein wrote: > > At lunch

[sage-devel] Re: profiling Sage startup

2011-02-23 Thread Jason Grout
On 2/23/11 3:06 PM, Robert Bradshaw wrote: On Wed, Feb 23, 2011 at 11:34 AM, William Stein wrote: On Wed, Feb 23, 2011 at 10:57 AM, Jason Grout wrote: On 2/23/11 12:28 PM, William Stein wrote: At lunch yesterday Robert Bradshaw made the interesting suggestion to read the docs for importlib

Re: [sage-devel] Re: profiling Sage startup

2011-02-23 Thread Robert Bradshaw
On Wed, Feb 23, 2011 at 11:34 AM, William Stein wrote: > On Wed, Feb 23, 2011 at 10:57 AM, Jason Grout > wrote: >> On 2/23/11 12:28 PM, William Stein wrote: >>> >>> At lunch yesterday Robert Bradshaw made the interesting suggestion to >>> read the docs for importlib >>> (http://docs.python.org/de

Re: [sage-devel] Re: profiling Sage startup

2011-02-23 Thread William Stein
On Wed, Feb 23, 2011 at 10:57 AM, Jason Grout wrote: > On 2/23/11 12:28 PM, William Stein wrote: >> >> At lunch yesterday Robert Bradshaw made the interesting suggestion to >> read the docs for importlib >> (http://docs.python.org/dev/library/importlib.html) and write a >> customized import hook,

[sage-devel] Re: profiling Sage startup

2011-02-23 Thread Jason Grout
On 2/23/11 12:28 PM, William Stein wrote: At lunch yesterday Robert Bradshaw made the interesting suggestion to read the docs for importlib (http://docs.python.org/dev/library/importlib.html) and write a customized import hook, so that every time during Sage startup that a module is imported, the

[sage-devel] Re: profiling Sage startup

2011-02-23 Thread Jason Grout
On 2/23/11 12:28 PM, William Stein wrote: At lunch yesterday Robert Bradshaw made the interesting suggestion to read the docs for importlib (http://docs.python.org/dev/library/importlib.html) and write a customized import hook, so that every time during Sage startup that a module is imported, the

Re: [sage-devel] Re: profiling Sage startup

2011-02-23 Thread William Stein
On Wed, Feb 23, 2011 at 7:09 AM, Volker Braun wrote: > Its true that there are lots of failed attempts to get files, which is > normal for any system where you have a multiple directories which can > contain any given file: > > [vbraun@volker-desktop ~]$ echo quit | strace -f sage |& grep ENOENT |

Re: [sage-devel] Re: profiling Sage startup

2011-02-23 Thread Robert Bradshaw
On Wed, Feb 23, 2011 at 4:43 AM, Volker Braun wrote: > I take it that the slow thing is reading ~2000 sage library files from a > harddisk into the filesystem cache. I'm using SSDs and Sage starts > consistently within about 1 second. There is "sage -startuptime" to profile. > The way I see it, th

[sage-devel] Re: profiling Sage startup

2011-02-23 Thread Volker Braun
Its true that there are lots of failed attempts to get files, which is normal for any system where you have a multiple directories which can contain any given file: [vbraun@volker-desktop ~]$ echo quit | strace -f sage |& grep ENOENT | wc 24322 298143 3532797 But, really, that just means tha

[sage-devel] Re: profiling Sage startup

2011-02-23 Thread Jason Grout
On 2/23/11 6:43 AM, Volker Braun wrote: I take it that the slow thing is reading ~2000 sage library files from a harddisk into the filesystem cache. I'm using SSDs and Sage starts consistently within about 1 second. There is "sage -startuptime" to profile. The way I see it, the only way to make

[sage-devel] Re: profiling Sage startup

2011-02-23 Thread Volker Braun
I take it that the slow thing is reading ~2000 sage library files from a harddisk into the filesystem cache. I'm using SSDs and Sage starts consistently within about 1 second. There is "sage -startuptime" to profile. The way I see it, the only way to make a significant dent into the startup tim