*Arg! I can't seem to get formatting down on google forums. I apologize. Here is my response in full.*
Hi Roland, Thanks for your response. Your suggestion makes sense. But does that mean going with a custom server extension <http://jupyter-notebook.readthedocs.io/en/stable/extending/handlers.html> is the way to go? I spent yesterday attempting to accomplish this solely though jupyter configuration options. For a running kernel with information { "shell_port": 63391, "iopub_port": 63394, "stdin_port": 63392, "control_port": 63393, "hb_port": 63396, "ip": "127.0.0.1", "key": "fff108f8-b1bf9b17aa337ce7d3d14565", "transport": "tcp", "signature_scheme": "hmac-sha256", "kernel_name": "" } I set the following variables in jupyter_notebook_config.py. c.ConnectionFileMixin.connection_file = '/Users/ebanner/Library/Jupyter/runtime/kernel-10962.json' c.ConnectionFileMixin.control_port = 63393 c.ConnectionFileMixin.iopub_port = 63394 c.ConnectionFileMixin.stdin_port = 63392 c.ConnectionFileMixin.hb_port = 63396 c.ConnectionFileMixin.ip = '127.0.0.1' c.ConnectionFileMixin.shell_port = 63391 c.ConnectionFileMixin.transport = 'tcp' However when I created a new notebook I get errors like [I 12:53:44.981 NotebookApp] KernelRestarter: restarting kernel (1/5) [D 12:53:44.982 NotebookApp] Starting kernel: [ '/Users/ebanner/.anaconda/envs/py36/bin/python', '-m', 'ipykernel_launcher', '-f', '/Users/ebanner/Library/Jupyter/runtime/kernel-f746726f-0d27-4d63-b520-2f1c84bf5196.json' ] [D 12:53:44.986 NotebookApp] Connecting to: tcp://127.0.0.1:63393 Traceback (most recent call last): File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/runpy.py", line 193 , in _run_module_as_main "__main__", mod_spec) File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/runpy.py", line 85, in _run_code exec(code, run_globals) File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel_launcher.py" , line 16, in <module> app.launch_new_instance() File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/traitlets/config/application.py" , line 657, in launch_instance app.initialize(argv) File "<decorator-gen-123>", line 2, in initialize File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/traitlets/config/application.py" , line 87, in catch_config_error return method(app, *args, **kwargs) File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py" , line 448, in initialize self.init_sockets() File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py" , line 238, in init_sockets self.shell_port = self._bind_socket(self.shell_socket, self.shell_port) File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py" , line 180, in _bind_socket s.bind("tcp://%s:%i" % (self.ip, port)) File "zmq/backend/cython/socket.pyx", line 495, in zmq.backend.cython. socket.Socket.bind (zmq/backend/cython/socket.c:5653) File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython. checkrc._check_rc (zmq/backend/cython/socket.c:10014) zmq.error.ZMQError: Address already in use until it hits the maximum number of retries. Presumably this is because the ipykernel command attempts to create a *new* socket but since it already exists it fails to do so. Thinking that I could "trick" jupyter into not using ipykernel and just use the existing sockets I set c.KernelManager.kernel_cmd = [ '/Users/ebanner/.anaconda/envs/py36/bin/python', '/Users/ebanner/bar.py'] where bar.py contains while True: pass After creating another new notebook I get [D 12:59:37.283 NotebookApp] Opening websocket /api/kernels/9a6c1e7d-12c4- 4d98-a0c6-1f227627f784/channels [D 12:59:37.283 NotebookApp] Connecting to: tcp://127.0.0.1:63391 [D 12:59:37.284 NotebookApp] Connecting to: tcp://127.0.0.1:63394 [D 12:59:37.284 NotebookApp] Connecting to: tcp://127.0.0.1:63392 which gave me hope! But then when I tried to execute a cell it just hangs. I did notice that http://localhost:8888/api/kernels has the following information. [ { id: "9b260abf-072f-48e2-b5f0-213d934418a2", name: "python3", last_activity: "2018-02-13T21:10:41.460182Z", execution_state: "starting", connections: 0 } ] However the execution_state being stuck at "starting" and connections being at 0 make me think that bar.py isn't quite doing enough. Digging into the code a bit I found the object returned from the call to ipykernel is a subprocess.Popen() object. But I couldn't find anywhere else in the code except KernelManager.is_alive() where Popen.poll() is called on it <https://github.com/jupyter/jupyter_client/blob/38b81cc07ddf70c2f534f11c4f4ef0103bfd5fe2/jupyter_client/multikernelmanager.py#L204-L205> . I also tried creating a custom kernel manager class and set c.NotebookApp.kernel_manager_class = 'kernelmanager.ExistingMappingKernelManager' where kernelmanager.ExistingMappingKernelManager is the class from tornado import gen, web from jupyter_client import KernelManager from notebook.services.kernels.kernelmanager import MappingKernelManager class ExistingMappingKernelManager(MappingKernelManager): """A KernelManager that just connects to an existing kernel.""" @gen.coroutine def start_kernel(self, kernel_id=None, path=None, **kwargs): kernel_id = 1 km = KernelManager(kernel_name='python3') kc = km.client() kc.load_connection_file( '/Users/ebanner/Library/Jupyter/runtime/kernel-10962.json') kc.start_channels() try: kc.wait_for_ready() except RuntimeError: kc.stop_channels() raise raise gen.Return(kernel_id) but when I create a new notebook I get the error [E 13:08:17.169 NotebookApp] Unhandled error in API request Traceback (most recent call last): File "/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/base/handlers.py" , line 516, in wrapper result = yield gen.maybe_future(method(self, *args, **kwargs)) File "/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/concurrent.py" , line 238, in result raise_exc_info(self._exc_info) File "<string>", line 4, in raise_exc_info File "/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", line 1063, in run yielded = self.gen.throw(*exc_info) File "/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/handlers.py" , line 75, in post type=mtype)) File "/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", line 1055, in run value = future.result() File "/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/concurrent.py" , line 238, in result raise_exc_info(self._exc_info) File "<string>", line 4, in raise_exc_info File "/Users/ebanner/.anaconda/lib/python3.6/site-packages/tornado/gen.py", line 1069, in run yielded = self.gen.send(value) File "/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py" , line 81, in create_session self.save_session(session_id, path=path, name=name, type=type, kernel_id=kernel_id) File "/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py" , line 125, in save_session return self.get_session(session_id=session_id) File "/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py" , line 170, in get_session return self.row_to_model(row) File "/Users/ebanner/.anaconda/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py" , line 209, in row_to_model raise KeyError KeyError [E 13:08:17.171 NotebookApp] { "Host": "localhost:8888", "Connection": "keep-alive", "Content-Length": "96", "Accept": "application/json, text/javascript, */*; q=0.01", "Origin": "http://localhost:8888";, "X-Requested-With": "XMLHttpRequest", "X-Xsrftoken": "2|e5e3448b|a67102a2af982e0bece9d040d46b1c87|1516914005", "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36", "Content-Type": "application/json", "Referer": "http://localhost:8888/notebooks/Kernel%20Client.ipynb";, "Accept-Encoding": "gzip, deflate, br", "Accept-Language": "en-US,en;q=0.9", "Cookie": "_xsrf=2|e5e3448b|a67102a2af982e0bece9d040d46b1c87|1516914005; username-localhost-8889=\"2|1:0|10:1517089627|23:username-localhost-8889|44:M2NiNzEzOGRjOGVhNDhlMzk0YjRlZGI2OWQ4M2RkZjk=|fcb4100a390a3d9608150659dc573ec135609a1e778f386e293b029e1fe016e3\"; user name-localhost-8890=\"2|1:0|10:1518302103|23:username-localhost-8890|44:OTdjNGRhZGRjYTRlNGMxZjg0YzY4ZDIzZjU1NjEzYTQ=|6c9c7647d95641d6f2d7375d9d45130a96e9ed9ec690b2bca01b6652c23ccd05\"; username-localhost-8888=\"2|1:0|10:1518555822|23:username-localhost-8888|44:M2M3OTdhYTh iNjQ2NDE3ZWEyZjA3Mjk2MzdjMjZlMmI=|e2c3fe789c16638c5fa4573a9d7626b87a00cb4d33ed1488122c6543b8832398\"" } presumably because I did not call super(MappingKernelManager, self).start_kernel(**kwargs) which presumably does some important setup work. I *did* get a hacky solution "working" by the end of the day, though! I put the following code snippet in a jupyter notebook. from jupyter_client import BlockingKernelClient client = BlockingKernelClient() client.load_connection_file( '/Users/ebanner/Library/Jupyter/runtime/kernel-10962.json') client.start_channels() Then I am able to do client.execute_interactive(code) as many times as I want! Using this approach one way forward would be to create a magic command that wraps code in a call to client.execute_interactive(). However for a single notebook that would require having to put the magic inside every code cell, which I would rather avoid if possible. So my question is can I start a new jupyter notebook and attach it to a running IPython kernel with configurables alone? If so am I close? If not what would be the most promising direction to proceed? The closest example I can think to study is nb2kg <https://github.com/jupyter-incubator/nb2kg> which allows a jupyter notebook to attach to a remote kernel. However that kernel was started by a jupyter notebook so it still might not be close enough for my use case to solve my problem I truly appreciate any help I can get as I am running out of ideas. On Tuesday, February 13, 2018 at 1:28:46 PM UTC-8, Edward Banner wrote: > > Hi Roland, > > Thanks for your response. Your suggestion makes sense. But does that mean > going with a custom server extension > <http://jupyter-notebook.readthedocs.io/en/stable/extending/handlers.html> > is the way to go? > > I spent yesterday attempting to accomplish this solely though jupyter > configuration options. For a running kernel with information > > { > "shell_port": 63391, > "iopub_port": 63394, > "stdin_port": 63392, > "control_port": 63393, > "hb_port": 63396, > "ip": "127.0.0.1", > "key": "fff108f8-b1bf9b17aa337ce7d3d14565", > "transport": "tcp", > "signature_scheme": "hmac-sha256", > "kernel_name": "" > } > > I set the following variables in jupyter_notebook_config.py. > > c.ConnectionFileMixin.connection_file = > '/Users/ebanner/Library/Jupyter/runtime/kernel-10962.json' > c.ConnectionFileMixin.control_port = 63393 > c.ConnectionFileMixin.iopub_port = 63394 > c.ConnectionFileMixin.stdin_port = 63392 > c.ConnectionFileMixin.hb_port = 63396 > c.ConnectionFileMixin.ip = '127.0.0.1' > c.ConnectionFileMixin.shell_port = 63391 > c.ConnectionFileMixin.transport = 'tcp' > > However when I created a new notebook I get errors like > > [I 12:53:44.981 NotebookApp] KernelRestarter: restarting kernel (1/5) > [D 12:53:44.982 NotebookApp] Starting kernel: [ > '/Users/ebanner/.anaconda/envs/py36/bin/python', '-m', > 'ipykernel_launcher', '-f', > '/Users/ebanner/Library/Jupyter/runtime/kernel-f746726f-0d27-4d63-b520-2f1c84bf5196.json' > ] > [D 12:53:44.986 NotebookApp] Connecting to: tcp://127.0.0.1:63393 > Traceback (most recent call last): > File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/runpy.py", line > 193, in _run_module_as_main > "__main__", mod_spec) > File "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/runpy.py", line > 85, in _run_code > exec(code, run_globals) > File > "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel_launcher.py" > , line 16, in <module> > app.launch_new_instance() > File > "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/traitlets/config/application.py" > , line 657, in launch_instance > app.initialize(argv) > File "<decorator-gen-123>", line 2, in initialize > File > "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/traitlets/config/application.py" > , line 87, in catch_config_error > return method(app, *args, **kwargs) > File > "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py" > , line 448, in initialize > self.init_sockets() > File > "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py" > , line 238, in init_sockets > self.shell_port = self._bind_socket(self.shell_socket, self.shell_port > ) > File > "/Users/ebanner/.anaconda/envs/py36/lib/python3.6/site-packages/ipykernel/kernelapp.py" > , line 180, in _bind_socket > s.bind("tcp://%s:%i" % (self.ip, port)) > File "zmq/backend/cython/socket.pyx", line 495, in zmq.backend.cython. > socket.Socket.bind (zmq/backend/cython/socket.c:5653) > File "zmq/backend/cython/checkrc.pxd", line 25, in zmq.backend.cython. > checkrc._check_rc (zmq/backend/cython/socket.c:10014) > zmq.error.ZMQError: Address already in use > > until it hits the maximum number of retries. Presumably this is because > the ipykernel command attempts to create a *new* socket but since it > already exists it fails to do so. > > Thinking that I could "trick" jupyter into not using ipykernel and just > use the existing sockets I set > > c.KernelManager.kernel_cmd = [ > '/Users/ebanner/.anaconda/envs/py36/bin/python', '/Users/ebanner/bar.py'] > > where bar.py contains > > while True: > pass > > After creating another new notebook I get > > [D 12:59:37.283 NotebookApp] Opening websocket /api/kernels/9a6c1e7d-12c4- > 4d98-a0c6-1f227627f784/channels > [D 12:59:37.283 NotebookApp] Connecting to: tcp://127.0.0.1:63391 > [D 12:59:37.284 NotebookApp] Connecting to: tcp://127.0.0.1:63394 > [D 12:59:37.284 NotebookApp] Connecting to: tcp://127.0.0.1:63392 > > which gave me hope! But then when I tried to execute a cell it just hangs. > I did notice that http://localhost:8888/api/kernels has the following > information. > > <span > -- You received this message because you are subscribed to the Google Groups "Project Jupyter" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jupyter/64cc3540-1cea-4260-9d85-a5122300419d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
