Python Standard Library: Key Modules and Popular Packages
What you’ll learn
Section titled “What you’ll learn”- The most useful standard library modules grouped by purpose.
- When to reach for a third-party package instead of writing from scratch.
- How to install packages safely with pip in a virtual environment.
- The packages that appear in almost every professional Python project.
Summary
Section titled “Summary”Python’s standard library covers an enormous range of tasks — HTTP requests, JSON handling, regular expressions, date/time arithmetic, testing, concurrency, and more — without installing anything. “Batteries included” is not a marketing phrase; it reflects the design philosophy that the language runtime should provide enough that you can get useful work done immediately.
The ecosystem of third-party packages (over 500,000 on PyPI) goes much further. Knowing which problems the standard library solves well, and which ones are better served by a battle-tested third-party package, is a core professional skill.
Save examples as .py and run with python examples.py (Python 3.9+).
Standard library: by category
Section titled “Standard library: by category”Data and collections
Section titled “Data and collections”from collections import defaultdict, Counter, deque, namedtupleimport heapq
# Counter: frequency countingfrom collections import Counterwords = ["apple", "banana", "apple", "cherry", "banana", "apple"]freq = Counter(words)print(freq) # Counter({'apple': 3, 'banana': 2, 'cherry': 1})print(freq.most_common(2)) # [('apple', 3), ('banana', 2)]
# defaultdict: auto-initialize missing keysfrom collections import defaultdictorders_by_customer = defaultdict(list)for order in [("BensonCo", "ORD-001"), ("NovaCorp", "ORD-002"), ("BensonCo", "ORD-003")]: orders_by_customer[order[0]].append(order[1])print(dict(orders_by_customer))# {'BensonCo': ['ORD-001', 'ORD-003'], 'NovaCorp': ['ORD-002']}
# deque: efficient queue / stackfrom collections import dequequeue = deque(["ORD-001", "ORD-002"])queue.appendleft("ORD-000") # O(1) prependqueue.append("ORD-003") # O(1) appendqueue.popleft() # O(1) dequeue: "ORD-000"Datetime
Section titled “Datetime”from datetime import date, datetime, timedelta, timezone
today = date.today()now = datetime.now(tz=timezone.utc)
# Arithmeticone_week_later = today + timedelta(weeks=1)diff = datetime(2026, 12, 31) - datetime(2026, 11, 13)print(diff.days) # 48
# Formatting and parsingformatted = now.strftime("%Y-%m-%d %H:%M:%S UTC")parsed = datetime.strptime("2026-11-13 09:00:00", "%Y-%m-%d %H:%M:%S")print(parsed.isoformat()) # 2026-11-13T09:00:00
# ISO format parsing (Python 3.7+)dt = datetime.fromisoformat("2026-11-13T09:00:00+00:00")Always store and compare datetimes as UTC (timezone.utc). Convert to local time only for display. Naive datetimes (no timezone) cause bugs when servers are in different zones.
Regular expressions
Section titled “Regular expressions”import re
text = "Order ORD-001 placed on 2026-11-13 for $149.99"
# Find all matchesdates = re.findall(r"\d{4}-\d{2}-\d{2}", text)print(dates) # ['2026-11-13']
# Named groupspattern = re.compile(r"(?P<order_id>ORD-\d+).*?\$(?P<amount>[\d.]+)")m = pattern.search(text)if m: print(m.group("order_id")) # ORD-001 print(m.group("amount")) # 149.99
# Replaceclean = re.sub(r"\s+", " ", "Order ID: ORD-001")print(clean) # Order ID: ORD-001Itertools and functools
Section titled “Itertools and functools”import itertoolsimport functools
# Chain multiple iterables into oneall_orders = list(itertools.chain(["ORD-001", "ORD-002"], ["ORD-003", "ORD-004"]))
# Group consecutive items by a keydata = [("2026-11", "ORD-001"), ("2026-11", "ORD-002"), ("2026-12", "ORD-003")]for month, group in itertools.groupby(data, key=lambda x: x[0]): print(month, list(group))
# Accumulate: running totalamounts = [100, 200, 150, 300]running = list(itertools.accumulate(amounts))print(running) # [100, 300, 450, 750]
# lru_cache: memoize expensive computations@functools.lru_cache(maxsize=128)def compute_tax(subtotal: float, rate: float) -> float: return round(subtotal * rate, 2)
print(compute_tax(100.0, 0.08)) # 8.0 — computedprint(compute_tax(100.0, 0.08)) # 8.0 — from cache
# reducetotal = functools.reduce(lambda a, b: a + b, amounts)print(total) # 750JSON and CSV
Section titled “JSON and CSV”import json
# Serialize to stringdata = {"order_id": "ORD-001", "amount": 149.99}json_str = json.dumps(data, indent=2)
# Deserializeloaded = json.loads(json_str)print(loaded["amount"]) # 149.99
# File round-tripwith open("order.json", "w", encoding="utf-8") as f: json.dump(data, f, indent=2)
with open("order.json", encoding="utf-8") as f: order = json.load(f)Logging
Section titled “Logging”import logging
# Configure once at application startuplogging.basicConfig( level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s — %(message)s", datefmt="%Y-%m-%d %H:%M:%S",)
logger = logging.getLogger("orders.service")
logger.debug("Processing order %s", "ORD-001") # not shown at INFO levellogger.info("Order %s confirmed, amount=%.2f", "ORD-001", 108.0)logger.warning("Payment retry for %s", "ORD-002")logger.error("Order %s failed: %s", "ORD-003", "card declined")Never use print() for application output in production code. logging supports levels, formatters, handlers (file, rotating file, remote), and structured output — print() supports none of this.
os and subprocess
Section titled “os and subprocess”import osimport subprocess
# Environment variablesapi_key = os.environ.get("OPENAI_API_KEY") # returns None if missingdb_url = os.environ["DATABASE_URL"] # raises KeyError if missing
# Run external commandsresult = subprocess.run( ["git", "log", "--oneline", "-5"], capture_output=True, text=True, check=True, # raises CalledProcessError on non-zero exit)print(result.stdout)hashlib and secrets
Section titled “hashlib and secrets”import hashlibimport secrets
# Hash a string (for data integrity, not password storage)digest = hashlib.sha256(b"ORD-001:BensonCo:149.99").hexdigest()print(digest)
# Cryptographically secure random values (for tokens, not passwords)token = secrets.token_hex(32) # 64-character hex stringprint(token)For password storage, use bcrypt (third-party) — never hashlib.
Essential third-party packages
Section titled “Essential third-party packages”| Package | Use case | Install |
|---|---|---|
requests |
HTTP client — simpler than urllib |
pip install requests |
httpx |
Async HTTP client, also sync | pip install httpx |
pydantic |
Data validation and settings with type hints | pip install pydantic |
python-dotenv |
Load .env files into os.environ |
pip install python-dotenv |
pytest |
Testing framework | pip install pytest |
rich |
Terminal output: tables, progress bars, syntax highlighting | pip install rich |
pandas |
Data analysis and CSV/Excel handling | pip install pandas |
sqlalchemy |
SQL ORM and query builder | pip install sqlalchemy |
fastapi |
Async web API framework with automatic OpenAPI | pip install fastapi uvicorn |
celery |
Distributed task queue | pip install celery |
Standard library vs third-party: when to reach for a package
Section titled “Standard library vs third-party: when to reach for a package”| Task | Standard library | Use instead |
|---|---|---|
| HTTP requests | urllib.request (verbose) |
requests or httpx |
| Data validation | Manual if checks |
pydantic |
| Password hashing | hashlib (wrong tool) |
bcrypt or argon2-cffi |
| Testing | unittest (works, verbose) |
pytest |
| Environment variables | os.environ (no types) |
python-dotenv + pydantic-settings |
| Date parsing from strings | datetime.strptime (format required) |
dateparser or python-dateutil |
| CSV with large files | csv (fine for most cases) |
pandas or polars |
| Async HTTP | None in stdlib | httpx (async) |
Common mistakes
Section titled “Common mistakes”- Importing
datetimeand then callingdatetime.now(). If you writeimport datetime, you needdatetime.datetime.now(). Usefrom datetime import datetimeto avoid the double name. - Using
print()for diagnostics in production. Useloggingfrom day one. - Not reading
os.environvalues at startup. Reading env vars deep inside functions makes them impossible to test. Read them at module load time or in a settings object initialized at startup. - Using
hashlibfor password storage. SHA-256 is fast — an attacker can try billions of guesses per second. Usebcryptwhich is deliberately slow.
Key takeaways
Section titled “Key takeaways”- The standard library covers data collections (
collections), time (datetime), text (re), functional tools (functools,itertools), I/O (json,csv,pathlib), and logging. - Use
logging, notprint(), for application output. - Always store and compare datetimes as UTC.
- Third-party packages (
requests,pydantic,pytest) complement but do not replace the standard library.
Related pages
Section titled “Related pages”- Python file I/O — the previous article in this series
- Python AI packages — PyTorch, transformers, LangChain, OpenAI SDK
- Python install guide — setting up pip and virtual environments
- Python tools overview — pip, uv, poetry, pyenv
Official references
Section titled “Official references”- Python docs: Standard Library — complete module index
- Python docs: collections — specialized container types
- Python docs: logging — logging HOWTO
- PyPI — Python Package Index