I’ve also been entertaining the idea of using something like pachyderm in our workflow. So filesystem context is much more deliberate and versioned. I think this might be a better approach than to try and keep the FS in sync
Get Outlook for iOS<https://aka.ms/o0ukef> ________________________________ From: [email protected] on behalf of Luciano Resende <[email protected]> Sent: Monday, February 4, 2019 9:43 am To: [email protected] Subject: Re: [jupyter] Docker jupyter kernel On Sun, Feb 3, 2019 at 2:21 PM Ray Hilton <[email protected]> wrote: > > The idea of splitting out the kernels into separate container just makes a > lot of sense to me. I’ve used enterprise gateway and it works pretty well. > Glad to hear that. > However, we had one big gotcha with this approach; the filesystem for the > kernel is not the same as in Jupiter - our users are used to being able to > reference other files they upload/see in the Jupiter UI. I tried a few > approaches to this (mounting same home directory into kernel, copying files > to kernel) but they all resulted in more complexity and there were > consistency issues. > > Given I was running everything in k8s and that the kernels shares so much > with the notebook image, it just seemed easier to keep them in one image for > now. > > I’d be very curious to see how others solved or circumvented this. Indeed, we were discussing this exact issue a couple days ago. Assuming you have the files available in both Jupyter and Kernel filesystem (e.g. mounts), we still noticed some issues that were not mirroring the work dirs between Jupyter and Kernel and we should have a proper fix for that in the next few days. In the meantime, a workaround is to add a notebook cell that adds the notebook folder to the path (e.g. sys.path.append('/my_notebooks/model')) -- Luciano Resende http://twitter.com/lresende1975 http://lresende.blogspot.com/ -- You received this message because you are subscribed to the Google Groups "Project Jupyter" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/CAGU5spcqoUbqJ0iKV_Mdj_X5Dzj8ATfmWjd8pgGr0w0qSjOb3g%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "Project Jupyter" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/SY3PR01MB2185762D3BB6362C771D7826C16C0%40SY3PR01MB2185.ausprd01.prod.outlook.com. For more options, visit https://groups.google.com/d/optout.
