Kernels & Magic Commands

Kernels & Magic Commands

Kernels

A kernel is the computational engine behind a notebook. Each notebook has exactly one kernel running.

Managing Kernels

Kernel menu → Restart          # clears all variables, fresh state
Kernel menu → Interrupt        # stops running cell (like Ctrl+C)
Kernel menu → Restart & Run All # clean run of entire notebook

Installing Additional Kernels

# R kernel
install.packages('IRkernel')
IRkernel::installspec()

# Julia kernel
# In Julia REPL:
using Pkg; Pkg.add("IJulia")

# .NET Interactive (C#, F#)
dotnet tool install -g Microsoft.dotnet-interactive
dotnet interactive jupyter install

# Bash kernel
pip install bash_kernel
python -m bash_kernel.install

# List installed kernels
jupyter kernelspec list

Kernel State Pitfall

# Cell 1 (run first)
x = 10

# Cell 2 (run second)
x = 20

# Cell 3 (run third)
print(x)  # 20

# If you re-run Cell 1 then Cell 3 without Cell 2:
print(x)  # still 20 — kernel remembers last assignment

Always use Restart & Run All before sharing a notebook to ensure reproducibility.

---

Magic Commands

Magic commands are IPython extensions prefixed with % (line magic) or %% (cell magic).

Line Magics (%)

# Timing
%time sum(range(1_000_000))
# CPU times: user 12.3 ms, sys: 0 ns, total: 12.3 ms

%timeit sum(range(1_000_000))
# 12.1 ms ± 234 µs per loop (mean ± std. dev. of 7 runs, 100 loops each)

# Current directory and file system
%pwd          # print working directory
%ls           # list files
%cd /path     # change directory

# Variable inspection
%who          # list all variables
%whos         # detailed variable info (name, type, value)
%reset        # delete all variables

# History
%history      # show command history
%history -n 10  # last 10 commands

# Load/save
%load script.py       # load file content into cell
%run script.py        # run a Python file
%save output.py 1-5   # save cells 1-5 to file

# Environment
%env                  # show environment variables
%env MY_VAR=hello     # set environment variable

# Matplotlib
%matplotlib inline    # render plots inline
%matplotlib widget    # interactive plots (requires ipympl)

Cell Magics (%%)

%%time
# Times the entire cell
import time
time.sleep(0.1)
result = sum(range(10_000_000))

%%timeit
# Benchmarks the entire cell
result = [x**2 for x in range(1000)]

%%bash
# Run bash commands in the cell
echo "Hello from bash"
ls -la
pip list | grep numpy

%%html
<h1 style="color: blue">Hello HTML</h1>
<p>This renders as HTML in the output</p>

%%javascript
console.log("Hello from JS");
element.textContent = "Rendered in notebook";

%%writefile mymodule.py
# Writes cell content to a file
def greet(name):
    return f"Hello, {name}!"

%%capture output
# Captures all output (stdout, stderr, display)
import warnings
warnings.warn("This won't show")
print("This is captured")
# Access via: output.stdout

%%latex
\begin{equation}
E = mc^2
\end{equation}

Shell Commands

Prefix with ! to run shell commands directly:

!pip install pandas
!git status
!ls -la
!python --version

# Capture output into Python variable
files = !ls *.csv
print(files)  # Python list of strings

# Pass Python variables to shell
name = "myfile"
!echo {name}

%autoreload — Auto-reload Modules

Essential when developing a module alongside a notebook:

%load_ext autoreload
%autoreload 2   # reload all modules before each cell execution

import mymodule
# Now edits to mymodule.py are picked up automatically

Profiling

%prun my_function()          # cProfile — function call stats
%lprun -f my_function my_function()  # line-by-line (requires line_profiler)
%memit my_function()         # memory usage (requires memory_profiler)

# Install profilers
!pip install line_profiler memory_profiler
%load_ext line_profiler
%load_ext memory_profiler