Skip to content

Python Standard Library: Key Modules and Popular Packages

  • The most useful standard library modules grouped by purpose.
  • When to reach for a third-party package instead of writing from scratch.
  • How to install packages safely with pip in a virtual environment.
  • The packages that appear in almost every professional Python project.

Python’s standard library covers an enormous range of tasks — HTTP requests, JSON handling, regular expressions, date/time arithmetic, testing, concurrency, and more — without installing anything. “Batteries included” is not a marketing phrase; it reflects the design philosophy that the language runtime should provide enough that you can get useful work done immediately.

The ecosystem of third-party packages (over 500,000 on PyPI) goes much further. Knowing which problems the standard library solves well, and which ones are better served by a battle-tested third-party package, is a core professional skill.

Save examples as .py and run with python examples.py (Python 3.9+).

from collections import defaultdict, Counter, deque, namedtuple
import heapq
# Counter: frequency counting
from collections import Counter
words = ["apple", "banana", "apple", "cherry", "banana", "apple"]
freq = Counter(words)
print(freq) # Counter({'apple': 3, 'banana': 2, 'cherry': 1})
print(freq.most_common(2)) # [('apple', 3), ('banana', 2)]
# defaultdict: auto-initialize missing keys
from collections import defaultdict
orders_by_customer = defaultdict(list)
for order in [("BensonCo", "ORD-001"), ("NovaCorp", "ORD-002"), ("BensonCo", "ORD-003")]:
orders_by_customer[order[0]].append(order[1])
print(dict(orders_by_customer))
# {'BensonCo': ['ORD-001', 'ORD-003'], 'NovaCorp': ['ORD-002']}
# deque: efficient queue / stack
from collections import deque
queue = deque(["ORD-001", "ORD-002"])
queue.appendleft("ORD-000") # O(1) prepend
queue.append("ORD-003") # O(1) append
queue.popleft() # O(1) dequeue: "ORD-000"
from datetime import date, datetime, timedelta, timezone
today = date.today()
now = datetime.now(tz=timezone.utc)
# Arithmetic
one_week_later = today + timedelta(weeks=1)
diff = datetime(2026, 12, 31) - datetime(2026, 11, 13)
print(diff.days) # 48
# Formatting and parsing
formatted = now.strftime("%Y-%m-%d %H:%M:%S UTC")
parsed = datetime.strptime("2026-11-13 09:00:00", "%Y-%m-%d %H:%M:%S")
print(parsed.isoformat()) # 2026-11-13T09:00:00
# ISO format parsing (Python 3.7+)
dt = datetime.fromisoformat("2026-11-13T09:00:00+00:00")

Always store and compare datetimes as UTC (timezone.utc). Convert to local time only for display. Naive datetimes (no timezone) cause bugs when servers are in different zones.

import re
text = "Order ORD-001 placed on 2026-11-13 for $149.99"
# Find all matches
dates = re.findall(r"\d{4}-\d{2}-\d{2}", text)
print(dates) # ['2026-11-13']
# Named groups
pattern = re.compile(r"(?P<order_id>ORD-\d+).*?\$(?P<amount>[\d.]+)")
m = pattern.search(text)
if m:
print(m.group("order_id")) # ORD-001
print(m.group("amount")) # 149.99
# Replace
clean = re.sub(r"\s+", " ", "Order ID: ORD-001")
print(clean) # Order ID: ORD-001
import itertools
import functools
# Chain multiple iterables into one
all_orders = list(itertools.chain(["ORD-001", "ORD-002"], ["ORD-003", "ORD-004"]))
# Group consecutive items by a key
data = [("2026-11", "ORD-001"), ("2026-11", "ORD-002"), ("2026-12", "ORD-003")]
for month, group in itertools.groupby(data, key=lambda x: x[0]):
print(month, list(group))
# Accumulate: running total
amounts = [100, 200, 150, 300]
running = list(itertools.accumulate(amounts))
print(running) # [100, 300, 450, 750]
# lru_cache: memoize expensive computations
@functools.lru_cache(maxsize=128)
def compute_tax(subtotal: float, rate: float) -> float:
return round(subtotal * rate, 2)
print(compute_tax(100.0, 0.08)) # 8.0 — computed
print(compute_tax(100.0, 0.08)) # 8.0 — from cache
# reduce
total = functools.reduce(lambda a, b: a + b, amounts)
print(total) # 750
import json
# Serialize to string
data = {"order_id": "ORD-001", "amount": 149.99}
json_str = json.dumps(data, indent=2)
# Deserialize
loaded = json.loads(json_str)
print(loaded["amount"]) # 149.99
# File round-trip
with open("order.json", "w", encoding="utf-8") as f:
json.dump(data, f, indent=2)
with open("order.json", encoding="utf-8") as f:
order = json.load(f)
import logging
# Configure once at application startup
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s %(levelname)s %(name)s — %(message)s",
datefmt="%Y-%m-%d %H:%M:%S",
)
logger = logging.getLogger("orders.service")
logger.debug("Processing order %s", "ORD-001") # not shown at INFO level
logger.info("Order %s confirmed, amount=%.2f", "ORD-001", 108.0)
logger.warning("Payment retry for %s", "ORD-002")
logger.error("Order %s failed: %s", "ORD-003", "card declined")

Never use print() for application output in production code. logging supports levels, formatters, handlers (file, rotating file, remote), and structured output — print() supports none of this.

import os
import subprocess
# Environment variables
api_key = os.environ.get("OPENAI_API_KEY") # returns None if missing
db_url = os.environ["DATABASE_URL"] # raises KeyError if missing
# Run external commands
result = subprocess.run(
["git", "log", "--oneline", "-5"],
capture_output=True,
text=True,
check=True, # raises CalledProcessError on non-zero exit
)
print(result.stdout)
import hashlib
import secrets
# Hash a string (for data integrity, not password storage)
digest = hashlib.sha256(b"ORD-001:BensonCo:149.99").hexdigest()
print(digest)
# Cryptographically secure random values (for tokens, not passwords)
token = secrets.token_hex(32) # 64-character hex string
print(token)

For password storage, use bcrypt (third-party) — never hashlib.

Package Use case Install
requests HTTP client — simpler than urllib pip install requests
httpx Async HTTP client, also sync pip install httpx
pydantic Data validation and settings with type hints pip install pydantic
python-dotenv Load .env files into os.environ pip install python-dotenv
pytest Testing framework pip install pytest
rich Terminal output: tables, progress bars, syntax highlighting pip install rich
pandas Data analysis and CSV/Excel handling pip install pandas
sqlalchemy SQL ORM and query builder pip install sqlalchemy
fastapi Async web API framework with automatic OpenAPI pip install fastapi uvicorn
celery Distributed task queue pip install celery

Standard library vs third-party: when to reach for a package

Section titled “Standard library vs third-party: when to reach for a package”
Task Standard library Use instead
HTTP requests urllib.request (verbose) requests or httpx
Data validation Manual if checks pydantic
Password hashing hashlib (wrong tool) bcrypt or argon2-cffi
Testing unittest (works, verbose) pytest
Environment variables os.environ (no types) python-dotenv + pydantic-settings
Date parsing from strings datetime.strptime (format required) dateparser or python-dateutil
CSV with large files csv (fine for most cases) pandas or polars
Async HTTP None in stdlib httpx (async)
  • Importing datetime and then calling datetime.now(). If you write import datetime, you need datetime.datetime.now(). Use from datetime import datetime to avoid the double name.
  • Using print() for diagnostics in production. Use logging from day one.
  • Not reading os.environ values at startup. Reading env vars deep inside functions makes them impossible to test. Read them at module load time or in a settings object initialized at startup.
  • Using hashlib for password storage. SHA-256 is fast — an attacker can try billions of guesses per second. Use bcrypt which is deliberately slow.
  • The standard library covers data collections (collections), time (datetime), text (re), functional tools (functools, itertools), I/O (json, csv, pathlib), and logging.
  • Use logging, not print(), for application output.
  • Always store and compare datetimes as UTC.
  • Third-party packages (requests, pydantic, pytest) complement but do not replace the standard library.