Don’t know innards of zeppelin, but in browser land it will be able to handle several thousand item arrays from object/memory perspective, but it’s the rendering and DOM that’s a killer.
In taking a quick peak don’t think string work/concat is issue, maybe bit extra overhead for regex replace but shouldn’t be the long running script issue. Its probably just fact that DOM isn’t happy rendering many thousand rows. Other minor things that won't help performance is that DOM is being set, then a call to .perfectScrollbar(), which is probably a bit of a killer itself for list that large since it appears to be boxing the list and handling the scrolling/moving of the <div> of massive list itself which assume is setting overflow:hidden. The DOM is touched another time right after when the height is set.., any touch will cause a reflow event. Don’t have time at moment to dig into code, but couple quick recommendations would be to: -don’t support lists as-is that are greater some sane size (would start with 10K, then start increasing until things start really slowing down -for larger lists don’t depend solely on perfectScrollbar() for going through the rows, use pagination that loads the windowed pages off the in-memory list and maybe scrollbar for scrolling that smaller page/sub lsit, will then be able to support very large lists -try to remove/minimize any extra DOM touches such as the height set after the fact Nate -----Original Message----- From: moon soo Lee [mailto:[email protected]] Sent: Wednesday, July 1, 2015 4:30 PM To: [email protected] Subject: Re: Scalability of Zeppelin Hi, Thanks for sharing problem. Yes, zeppelin-web is not really optimized at the moment. For rendering table, following two functions from paragraph.js are related. loadTableData() - parse text output from backed by \n,\t and construct array for rows and columns. renderTable() - a lot of string concatenation. I didn't profiled them, but it may take some time for array construction and string concatenation. If you can help on it, that'll be really great. Thanks, moon On Wed, Jul 1, 2015 at 1:51 PM Andres Celis <[email protected]> wrote: > Hi Zeppelin team! > > I have been working on an interpreter for zeppelin that connects to MS > SQL Server. It is pretty much finished, but I had some questions as > to the scalability of Zeppelin and how to contribute. > > Recently I ran a select statement that pulls in ~45,000 rows from SQL > Server. Afterwards the application was extremely sluggish in its > performance (on chrome) and it would crash (on internet explorer). I > was hoping you could help me pinpoint where these problems occur so > that I could see if I could help make Zeppelin more scalable. > > Internet explorer would say there were "long running scripts" which > were making it sluggish, and when I was debugging I also saw that > actions like trying to "persist" or "save" the note were taking a > while. I also wonder whether the renderTable function in paragraph.js > could be the culprit that is slowing everything down since it uses a lot of > string concatenation. > These are just starting points of what might be slow, I am still new > to zeppelin so I am hoping that with your experience you will have a > better understanding of what could be going wrong. > > Looking forward to working with you, > Andres >
