one option is to keep the data in separate files. notes.json can contain the code and the data can be a pointer to /path/to/file. import/export can choose to include or exclude the data. when it is included the data files are added to a tgz file containing notes.json otherwise you just export notes.json
On Mon, Sep 12, 2016 at 10:33 AM, moon soo Lee <m...@apache.org> wrote: > Right big note.json file is a problem. > But at the same time, i think having data in the note.json helps make > import/export simple and make notebook renderable without run it. > > So far, i didn't see much discussion about this subject on mailing list or > on the issue tracker. > > If there's an good idea that can handle large data while keeping > import/export simple and ability to render without run, that would be a > great starting point of the discussions. > > Thanks, > moon > > On Wed, Sep 7, 2016 at 9:40 PM Vikash Kumar <vikash.ku...@resilinc.com> > wrote: > >> Hi moon, >> >> Yes that was the way that I was using. But is there any plan for future >> releases to removing the data from note and storing only configuration? >> >> Because storing the configuration with data when there is no max result >> limit will create a big note.json file. >> >> >> >> Thanks & Regards, >> >> *Vikash Kumar* >> >> *From:* moon soo Lee [mailto:m...@apache.org] >> *Sent:* Wednesday, September 7, 2016 8:39 PM >> *To:* users@zeppelin.apache.org >> *Subject:* Re: Hbase configuration storage without data >> >> >> >> Hi, >> >> >> >> For now, code and result data are mixed in note.json, which is >> represented by 'class Note' [1]. And every Notebook storage layer need to >> implement 'NotebookRepo.get()' [2] to read note.json from underlying >> storage and convert it into 'class Note'. >> >> >> >> As you see the related API and class definition, NotebookRepo actually >> doesn't have any restriction how 'class Note' is serialized and saved in >> the storage. >> >> >> >> So you can event new format, you can exclude result data from saving, and >> so on. >> >> >> >> Hop this helps. >> >> >> >> Thanks, >> >> moon >> >> >> >> [1] https://github.com/apache/zeppelin/blob/master/zeppelin- >> zengine/src/main/java/org/apache/zeppelin/notebook/Note.java >> >> [2] https://github.com/apache/zeppelin/blob/master/zeppelin- >> zengine/src/main/java/org/apache/zeppelin/notebook/repo/ >> NotebookRepo.java#L47 >> >> >> >> On Wed, Sep 7, 2016 at 3:47 AM Vikash Kumar <vikash.ku...@resilinc.com> >> wrote: >> >> Hi all, >> >> We are storing the note.json configuration into hbase as >> it is stored into File system. As default behavior in note.json the query >> data is stored along with configuration. But we want to store the >> configurations only and when user loading its note then query should get >> executed and data generated. This feature we are using for phoenix >> interpreter. So how can we remove the data from note.json? Is there any >> plan for that? >> >> >> >> >> >> Thanks & Regards, >> >> *Vikash Kumar* >> >>
