Jupyter Notebooks: Kernels, Magics and Rich Output
Jupyter is an open-source environment for interactive computing. A notebook is a document that mixes live code, its output (tables, charts, HTML), equations and narrative text, which makes it the standard tool for data exploration, machine-learning experiments, teaching and reproducible research. The name refers to three languages it supported early on: Julia, Python and R. Today more than a hundred languages have kernels.
This page is the start of the Jupyter section. It explains how notebooks work and covers the day-to-day features: kernels, magic commands, shell commands, rich output and widgets. Jupyter best practices and tools continues with reproducibility, version control, automation and extensions, and Google Colab covers the hosted notebook service.
Versions at the time of writing (November 2026): JupyterLab 4.6, Notebook 7.6, IPython 9.18, ipykernel 7.4 and ipywidgets 8.1. The samples use one invented dataset, orders from an OrdersPortal shop, and were executed with Python 3.13 through nbclient; the outputs shown are from that run.
Topics on this page
Section titled “Topics on this page”| Topic | What it answers |
|---|---|
| How notebooks work | Server, kernel, .ipynb files and cell types |
| Installing JupyterLab and Notebook | Which interface to install and how to start it |
| Keyboard shortcuts | Command mode and edit mode |
| Kernels | Managing, adding and choosing kernels |
| Hidden state | Why out-of-order execution causes bugs |
| Magic commands | % and %% helpers for timing, files and the environment |
| Shell commands | Running ! commands and passing Python values |
| Rich output | HTML, Markdown, images, math and DataFrames |
| Interactive widgets | Sliders, dropdowns and callbacks with ipywidgets |
| Plotting | Matplotlib inline and interactive plotting libraries |
How notebooks work
Section titled “How notebooks work”A notebook file (.ipynb) is JSON: a list of cells with their source, their saved outputs and metadata such as the kernel name. When you open it, a Jupyter Server process serves the web interface and starts a kernel, a separate process that executes the code. The browser sends each cell to the kernel over a WebSocket and renders whatever comes back.
Browser (JupyterLab or Notebook) ⇄ Jupyter Server ⇄ Kernel (for example ipykernel running Python)Three cell types exist: Code cells run in the kernel; Markdown cells hold formatted text, headings, lists and LaTeX math; Raw cells are passed through unchanged, mainly for conversion tools. Outputs are saved inside the file, which is why notebooks are convenient to share and awkward in version control.
Installing JupyterLab and Notebook
Section titled “Installing JupyterLab and Notebook”There are two front ends, built on the same components since Notebook 7:
| JupyterLab 4 | Notebook 7 | |
|---|---|---|
| Layout | IDE-style: tabs, split panes, file browser, terminals, consoles | One document per browser tab, simple and focused |
| Extensions | JupyterLab extensions (prebuilt, installed with pip) | The same JupyterLab extensions |
| Start command | jupyter lab |
jupyter notebook |
python -m venv .venv && source .venv/bin/activate # or .venv\Scripts\Activate.ps1 on Windowspython -m pip install jupyterlab notebook pandasjupyter lab # opens http://localhost:8888/labInstall Jupyter into each project’s virtual environment (or register the environment as a kernel, below) so notebooks import the same package versions as the project.
Pitfalls:
- Old extensions after upgrading to Notebook 7. Extensions written for Notebook 6 and earlier (
nbextensions) do not run in Notebook 7. Look for a JupyterLab equivalent, or use thenbclassicpackage if you must keep the old interface for a while. pip installinto the wrong interpreter. Inside a notebook, use%pip install pandasrather than!pip install; the magic installs into the environment of the running kernel.
Keyboard shortcuts
Section titled “Keyboard shortcuts”Notebooks have two modes. Edit mode (Enter, or click into a cell) types into the cell. Command mode (Esc) acts on whole cells.
| Shortcut | Action |
|---|---|
Shift+Enter |
Run the cell and select the next one |
Ctrl+Enter |
Run the cell and stay on it |
Alt+Enter |
Run the cell and insert a new cell below |
Esc / Enter |
Switch to command mode / edit mode |
A / B (command mode) |
Insert a cell above / below |
D, D (command mode) |
Delete the cell |
M / Y (command mode) |
Change the cell to Markdown / Code |
0, 0 (command mode) |
Restart the kernel |
Ctrl+Shift+C (JupyterLab) |
Open the command palette |
Kernels
Section titled “Kernels”A kernel is the process that holds your variables. Each open notebook is connected to one kernel. From the Kernel menu you can Interrupt a long-running cell (like Ctrl+C), Restart to clear all state, or Restart Kernel and Run All Cells to prove the notebook runs top to bottom.
The most useful kernel operation is registering a project’s virtual environment so it appears in the kernel picker:
# inside the activated project environmentpython -m pip install ipykernelpython -m ipykernel install --user --name ordersportal --display-name "Python (OrdersPortal)"jupyter kernelspec list # shows installed kernels and their foldersOther languages install their own kernels: IRkernel for R, IJulia for Julia, and .NET Interactive for C# and F# (dotnet tool install -g Microsoft.dotnet-interactive, then dotnet interactive jupyter install).
Hidden state
Section titled “Hidden state”Cells can run in any order, and the kernel keeps whatever the last executed cell assigned. The execution counter ([5]) shows the order cells actually ran in, not their position.
# Cell 1discount = 0.10# Cell 2discount = 0.25# Cell 3print(f"Discount applied: {discount:.0%}")# Run 1, 2, 3, then edit and re-run only cell 1 and cell 3:# Cell 3 now prints 10%, but anyone running the saved notebook top to bottom gets 25%.This is the most common source of notebooks that “worked yesterday”. Restart and run all before you share, commit or trust a result.
Magic commands
Section titled “Magic commands”Magics are IPython commands, not Python. Line magics start with % and apply to one line; cell magics start with %%, must be the first line of a cell and apply to the whole cell. %lsmagic lists them all and %timeit? shows help for one.
The first cell builds the sample data used for the rest of the page:
import pandas as pd
orders = pd.DataFrame({ "order_id": ["ORD-1040", "ORD-1041", "ORD-1042", "ORD-1043"], "region": ["EU", "US", "EU", "APAC"], "total": [129.50, 89.00, 240.25, 56.75],})orders.groupby("region")["total"].sum()regionAPAC 56.75EU 369.75US 89.00Name: total, dtype: float64%timeit -n 1000 -r 5 orders["total"].sum()17.5 μs ± 860 ns per loop (mean ± std. dev. of 5 runs, 1,000 loops each)Your timings will differ; the point is the format: mean and spread over several runs.
%%capture capturedprint("loading 4 orders")print(captured.stdout.strip())loading 4 ordersFrequently used magics:
| Magic | Purpose |
|---|---|
%time, %%time |
Time one run of a statement or a cell |
%timeit, %%timeit |
Benchmark over many runs |
%pip, %conda |
Install packages into the kernel’s environment |
%run script.py |
Run a Python file in the notebook’s namespace |
%load file.py |
Paste a file’s contents into the cell |
%%writefile module.py |
Write the cell to a file |
%who, %whos, %reset |
List or clear variables (%who DataFrame lists only DataFrames) |
%env NAME=value |
Show or set environment variables |
%matplotlib inline / widget |
Static or interactive Matplotlib output (widget needs ipympl) |
%%capture name |
Capture stdout, stderr and display output into a variable |
%prun f() |
Profile a call with cProfile |
%load_ext autoreload then %autoreload 2 |
Reload edited modules before each cell runs |
%autoreload 2 is essential when you develop a package in an editor and exercise it from a notebook: edits to ordersportal/reports.py are picked up without restarting the kernel. It cannot update every object safely (for example instances created before a class changed), so restart when behavior looks inconsistent.
Shell commands
Section titled “Shell commands”A line starting with ! runs in the system shell. The output can be captured into a Python list, and Python variables are substituted with braces.
files = !ls *.pyfiles['build.py']region = "EU"!echo "Exporting orders for {region}"Exporting orders for EUEach ! line runs in a new shell, so !cd data has no lasting effect; use the %cd magic to change the notebook’s working directory. Shell commands also make a notebook platform-specific (ls is not available in Windows cmd), so prefer Python’s pathlib in notebooks other people will run.
Rich output
Section titled “Rich output”The last expression of a cell is displayed with its richest representation: a DataFrame becomes an HTML table, a Matplotlib figure an image. To display several things from one cell, or something other than the last value, call display() with an object from IPython.display.
from IPython.display import display, Markdowneu = orders.loc[orders.region == "EU", "total"].sum()display(Markdown(f"**EU revenue:** {eu:,.2f} EUR"))Rendered output: EU revenue: 369.75 EUR
Other display classes include HTML, Image(filename=...), JSON (a collapsible tree in JupyterLab), Math(r"\sum_i x_i") and Latex. DataFrames can be styled with orders.style.highlight_max(subset=["total"]).format({"total": "{:.2f}"}), and pd.set_option("display.max_columns", None) shows every column instead of an ellipsis.
Pitfall: huge outputs. Printing a 100,000-row DataFrame stores it in the .ipynb file, slows the browser and bloats commits. Show head() or a summary, and clear outputs before committing (see best practices).
Interactive widgets
Section titled “Interactive widgets”ipywidgets adds sliders, dropdowns, buttons and other controls whose values are synchronized between the browser and the kernel. JupyterLab 4 and Notebook 7 support ipywidgets 8 after pip install ipywidgets; no extra enable step is needed.
import ipywidgets as widgetsfrom IPython.display import display
region = widgets.Dropdown(options=sorted(orders.region.unique()), value="EU", description="Region")threshold = widgets.FloatSlider(value=50, min=0, max=250, step=10, description="Min total")out = widgets.Output()
def refresh(change=None): subset = orders[(orders.region == region.value) & (orders.total >= threshold.value)] out.clear_output() with out: print(f"{region.value}: {len(subset)} orders, {subset.total.sum():.2f}")
region.observe(refresh, names="value")threshold.observe(refresh, names="value")refresh()display(widgets.VBox([region, threshold, out]))Moving the slider to 200 shows EU: 1 orders, 240.25; switching the region to US then shows US: 0 orders, 0.00. (The callback logic was run headless with ipywidgets 8.1.9 by setting the widget values from code.)
For quick experiments, @widgets.interact(min_total=(0, 250, 10)) above a function builds the controls from its arguments. Layout containers (HBox, VBox, Tab, Accordion) group controls, and tqdm.notebook.tqdm gives a progress bar for loops. Widgets need a live kernel: in a static HTML export or on GitHub they appear as a placeholder or not at all.
Plotting
Section titled “Plotting”Matplotlib output appears inline by default in JupyterLab. For zooming and panning, install ipympl and use %matplotlib widget. Plotly, Altair and Bokeh produce interactive charts that render in JupyterLab without extra setup and stay interactive in HTML exports.
# fragment: needs matplotlibimport matplotlib.pyplot as plt
ax = orders.groupby("region")["total"].sum().plot(kind="bar", title="Revenue by region")ax.set_ylabel("EUR")plt.show()Interview questions
Section titled “Interview questions”1. What is the difference between the Jupyter Server and a kernel? The server serves the web interface, manages files and starts kernels; the kernel is a separate process that executes code and holds state. One server can run many kernels, in different languages, one per open notebook.
2. Why does a notebook work for the author and fail for a colleague? Usually hidden state: cells were run out of order, or a variable was defined in a cell that was later deleted. Other causes are packages installed into a different environment than the kernel’s, or absolute file paths. Restart and run all before sharing.
3. When would you use %pip install instead of !pip install?
Always inside a notebook. %pip installs into the environment of the running kernel; !pip runs whichever pip is first on the shell’s PATH, which may belong to another interpreter.
4. How do you make a project’s virtual environment available in JupyterLab?
Install ipykernel into that environment and run python -m ipykernel install --user --name <name>. The kernel then appears in the kernel picker of any Jupyter installation for that user.
5. Senior scenario: a data team’s notebooks have become the production reporting pipeline and break often. What do you change?
Move reusable logic into a tested Python package imported by the notebooks (with %autoreload during development), parameterize the notebooks and run them headless with papermill in a scheduler, pin the environment, strip outputs in version control, and add a CI job that executes each notebook. Anything that must be reliable and not exploratory should eventually become a script or a job, with the notebook kept as the readable report.
Related pages
Section titled “Related pages”- Jupyter best practices and tools — structure, reproducibility, version control, papermill, testing and extensions
- Google Colab — hosted notebooks with GPUs and Drive integration
- Python setup guide — virtual environments for notebook projects
- Python packages for AI and machine learning — libraries you will use from notebooks