Unable to Import Packages that Installed with pip in JupyterLab module

[duchuyvp]

I use the jupyter-lab module in main.tf with default parameters. However, when I create a new Notebook, I encounter issues with importing packages installed using pip.

Steps to reproduce:

1. In a Notebook, run !pip install pandas --break-system-packages (using --break-system-packages to bypass the externally-managed-environment error). The package gets installed to ~/.local/lib/python3.12/site-packages.
2. Run import pandas as pd in the Notebook. This raises a ModuleNotFoundError.

Findings:

Output of !which pip in the Notebook: /usr/bin/pip
Output of sys.executable in the Notebook: /home/coder/.local/share/pipx/venvs/jupyterlab/bin/python
It seems that when JupyterLab is installed via pipx, it uses its own Python environment, separate from the system's pip. As a result, packages installed via pip are not accessible to JupyterLab.

Expected behavior:

Packages installed using !pip install in a Jupyter notebook should be available for import without additional configuration.

[duchuyvp]

I have a workaround solution is provide correct kernel at startup script and choose right kernel when create Notebook

resource "coder_agent" "main" {
  os             = "linux"
  arch           = "amd64"
  startup_script = <<-EOT
    set -e
    .....
    python3.12 -m pip install ipykernel --break-system-packages
    python3.12 -m ipykernel install --user --name="Python3.12.3"
    .....
  EOT

it works, but I think this is module's problem.

[matifali]

The module uses pipx to manage packages to avoid the limitation introduced in Ubuntu 24.04 that doesn't let you install packages at the system level.

[duchuyvp]

I've tried and it doesn't work

Did I do something wrong? Also tried for --include-deps and --force params

[matifali]

It's unfortunate that the change breaks your use case. pipx handles the installation of tools by creating an isolated environment.

[matifali]

Maybe using uv would be a good option in the future.
It is easier to install, does not require any dependencies, and can managethe Python versions too.

[ktrueda]

resource "coder_agent" "foo" {
  startup_script          = <<-EOT
    set -e
    # Add any commands that should be executed at workspace startup (e.g install requirements, start a program, etc) 
    sleep 100 # wait for jupyterlab install
    pipx inject jupyterlab pandas
  EOT
}

This is my work around. Sleep is so ugly. I hope that jupyterlab module handles tools like poetry natively.