Hi, But storing the data in a separate file approach will need to maintain the link between both files. And also this approach is not preferable when the data is obtained on access basis. like in my case data which comes from hbase through phoenix is tenant base. So storing that data into note.json or in different file is breaking the point of multi tenancy. So as an approach can we store only configuration and retrieve the data when we are loading the note by running the all paragraph for first time load.
But at the same time, i think having data in the note.json helps make import/export simple and make notebook render able without run it. So for import/export providing the data is it good? Data is always confidential and cannot be shared with anyone in form of json. So in this approach any one can open the note.json and can access the data. Thanks & Regards, Vikash Kumar From: Felix Cheung [mailto:felixcheun...@hotmail.com] Sent: Wednesday, September 14, 2016 6:24 AM To: users@zeppelin.apache.org; users@zeppelin.apache.org Subject: Re: Hbase configuration storage without data I like that approach - though you should be able to clear result output before exporting the note, if all you want is the config? The should remove all output data, keeping it smaller? _____________________________ From: Mohit Jaggi <mohitja...@gmail.com<mailto:mohitja...@gmail.com>> Sent: Monday, September 12, 2016 10:38 AM Subject: Re: Hbase configuration storage without data To: <users@zeppelin.apache.org<mailto:users@zeppelin.apache.org>> one option is to keep the data in separate files. notes.json can contain the code and the data can be a pointer to /path/to/file. import/export can choose to include or exclude the data. when it is included the data files are added to a tgz file containing notes.json otherwise you just export notes.json On Mon, Sep 12, 2016 at 10:33 AM, moon soo Lee <m...@apache.org<mailto:m...@apache.org>> wrote: Right big note.json file is a problem. But at the same time, i think having data in the note.json helps make import/export simple and make notebook renderable without run it. So far, i didn't see much discussion about this subject on mailing list or on the issue tracker. If there's an good idea that can handle large data while keeping import/export simple and ability to render without run, that would be a great starting point of the discussions. Thanks, moon On Wed, Sep 7, 2016 at 9:40 PM Vikash Kumar <vikash.ku...@resilinc.com<mailto:vikash.ku...@resilinc.com>> wrote: Hi moon, Yes that was the way that I was using. But is there any plan for future releases to removing the data from note and storing only configuration? Because storing the configuration with data when there is no max result limit will create a big note.json file. Thanks & Regards, Vikash Kumar From: moon soo Lee [mailto:m...@apache.org<mailto:m...@apache.org>] Sent: Wednesday, September 7, 2016 8:39 PM To: users@zeppelin.apache.org<mailto:users@zeppelin.apache.org> Subject: Re: Hbase configuration storage without data Hi, For now, code and result data are mixed in note.json, which is represented by 'class Note' [1]. And every Notebook storage layer need to implement 'NotebookRepo.get()' [2] to read note.json from underlying storage and convert it into 'class Note'. As you see the related API and class definition, NotebookRepo actually doesn't have any restriction how 'class Note' is serialized and saved in the storage. So you can event new format, you can exclude result data from saving, and so on. Hop this helps. Thanks, moon [1] https://github.com/apache/zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/Note.java [2] https://github.com/apache/zeppelin/blob/master/zeppelin-zengine/src/main/java/org/apache/zeppelin/notebook/repo/NotebookRepo.java#L47 On Wed, Sep 7, 2016 at 3:47 AM Vikash Kumar <vikash.ku...@resilinc.com<mailto:vikash.ku...@resilinc.com>> wrote: Hi all, We are storing the note.json configuration into hbase as it is stored into File system. As default behavior in note.json the query data is stored along with configuration. But we want to store the configurations only and when user loading its note then query should get executed and data generated. This feature we are using for phoenix interpreter. So how can we remove the data from note.json? Is there any plan for that? Thanks & Regards, Vikash Kumar
