Configuring a poetry environment in ein

[dcguim]

I would expect ein to be able pick up my poetry jupyter kernel:

 > poetry run jupyter kernelspec list
Available kernels:
  python3          /Users/dguim/Library/Caches/pypoetry/virtualenvs/llama-app-backend-BtvjZBul-py3.11/share/jupyter/kernels/python3

Of course I would be able to run, in the terminal from my "llama-app-backend-BtvjZBul-py3.11" virtualenv

>>> import langchain_openai
>>> 

Unfortunately when selecting python3 ipykernel from ein's notebooklist menu, and executing the import statement again:

In [2]:
import langchain_openai

Truncated Traceback (Use C-c C-$ to view full TB):
Cell In[2], line 1
----> 1 import langchain_openai

ModuleNotFoundError: No module named 'langchain_openai'

system info

;; Use EIN for Jupyter notebook integration
(use-package ein
  :ensure t
  :init
  ;; Set default notebook directory to current directory
  (setq ein:jupyter-server-use-command "/Users/dguim/Library/Caches/pypoetry/virtualenvs/llama-app-backend-BtvjZBul-py3.11/bin/jupyter")
;;  (custom-set-variables '(ein:jupyter-server-use-subcommand "run jupyter notebook"))
  (setq ein:jupyter-server-use-containers nil)
  (setq ein:jupyter-default-notebook-directory default-directory)
  :config
  ;; Enable auto-completion and undo in notebooks
  (setq ein:use-auto-complete t)
  (setq ein:worksheet-enable-undo t)
  ;; Keybindings for opening notebooks and connecting to a running Jupyter server
  :bind (("C-c C-j l" . ein:notebooklist-login)))

My poetry env:

> poetry env info

Virtualenv
Python:         3.11.10
Implementation: CPython
Path:           /Users/dguim/Library/Caches/pypoetry/virtualenvs/llama-app-backend-BtvjZBul-py3.11
Executable:     /Users/dguim/Library/Caches/pypoetry/virtualenvs/llama-app-backend-BtvjZBul-py3.11/bin/python
Valid:          True

Base
Platform:   darwin
OS:         posix
Python:     3.11.10
Path:       /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11
Executable: /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/bin/python3.11

I have the following dependencies in my pyproject.toml:

[tool.poetry.dependencies]
langchain-openai = "^0.1.1"
openai = "^1.16.2"
langchain-community = "^0.0.31"
langchain = "^0.1.14"

[dcguim]

I found a workaround through pyvenv, using the following function-hook, in my use-package :config section:

  :config
  ;; Automatically activate the Poetry virtual environment when starting ein:run
  (defun ein-activate-poetry-venv ()
    "Activate the Poetry virtual environment before running ein:run."
    (let ((venv-path (string-trim (shell-command-to-string "poetryenv"))))
      (when (and venv-path (file-exists-p (concat venv-path "/bin/activate")))
        (pyvenv-activate (concat venv-path "/bin/activate")))))

  ;; Hook to activate the Poetry environment before running ein
  (add-hook 'ein:notebooklist-login-hook 'ein-activate-poetry-venv)

I also created the following function in my .zshrc:

function poetryenv() {
    if [[ -f "pyproject.toml" ]]; then
        echo $(poetry env info --path)
    else
        echo "No pyproject.toml - current directory doesn't seem to contain a poetry project."
        return 1
    fi
}

This workaround is enough although I feel it could be better for poetry users if it was integrated directly into ein.

[ohad-ivix]

I implemented a different solution using the capabilities built into Jupyter itself: the kernel.json file specifies the full argv list for running the kernel process. It usually looks like this:

 "argv": [
  "python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ]

I changed that to activate the virtualenv before starting the process. In poetry this is done using the poetry run subcommand:

 "argv": [
  "poetry",
  "run",
  "python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ]

That should Just Work™ without needing any ELisp wrappers or EIN integration.