Skip to content

Python File Operations: Reading, Writing, and pathlib

  • Opening files with open() and the with statement for safe cleanup.
  • Read, write, append, and binary modes.
  • Using pathlib.Path for cross-platform path operations.
  • Reading and writing CSV and JSON files.
  • Python 3.9+ installed
  • Familiarity with strings and dictionaries

The recommended pattern for all file I/O is the with statement. It automatically closes the file even if an error occurs — you should almost never call .close() manually.

# Safe — closes automatically, even on error
with open("notes.txt", "w", encoding="utf-8") as f:
f.write("Hello, file!\n")
f.write("Second line\n")
# Risky — close() never runs if write() raises
f = open("notes.txt", "w")
f.write("data")
f.close()

Save as file_examples.py and run with python file_examples.py (Python 3.9+):

# Read entire file as a string
with open("notes.txt", "r", encoding="utf-8") as f:
contents = f.read()
print(contents)
# Read line by line (memory-efficient for large files)
with open("notes.txt", "r", encoding="utf-8") as f:
for line in f:
print(line.strip()) # strip trailing newline
# Read all lines into a list
with open("notes.txt", "r", encoding="utf-8") as f:
lines = f.readlines()
print(lines) # ['Hello, file!\n', 'Second line\n']

For large files, iterate with for line in f: rather than f.read() — it reads one line at a time instead of loading the entire file into memory.

# Write mode ("w") — overwrites if the file exists, creates if not
with open("report.txt", "w", encoding="utf-8") as f:
f.write("Order Report\n")
f.write("=============\n")
f.write("ORD-001: $149.99\n")
# Append mode ("a") — adds to the end without erasing existing content
with open("report.txt", "a", encoding="utf-8") as f:
f.write("ORD-002: $49.99\n")
# Exclusive create ("x") — fails if the file already exists
try:
with open("report.txt", "x", encoding="utf-8") as f:
f.write("This will fail if report.txt exists")
except FileExistsError:
print("File already exists, skipping")

Always specify encoding="utf-8" explicitly. The default encoding is platform-dependent (cp1252 on Windows, utf-8 on Linux/macOS), which causes silent corruption when files are shared across systems.

pathlib.Path is the modern, cross-platform way to work with file paths. It replaces os.path with a cleaner, composable API:

from pathlib import Path
# Build paths without os.path.join
base = Path("data")
report = base / "reports" / "2026-11-13.txt"
print(report) # data/reports/2026-11-13.txt (or with \ on Windows)
# Path properties
print(report.name) # 2026-11-13.txt
print(report.stem) # 2026-11-13
print(report.suffix) # .txt
print(report.parent) # data/reports
# Check existence
print(report.exists()) # False (not created yet)
print(report.is_file())
# Create directories
(base / "reports").mkdir(parents=True, exist_ok=True)
# Read and write through pathlib (no open() needed for simple cases)
report.write_text("Order Report\n", encoding="utf-8")
content = report.read_text(encoding="utf-8")
# Glob for files
for txt_file in Path(".").rglob("*.txt"):
print(txt_file)
import csv
from pathlib import Path
orders = [
{"order_id": "ORD-001", "customer": "BensonCo", "amount": "149.99"},
{"order_id": "ORD-002", "customer": "NovaCorp", "amount": "49.99"},
]
# Write CSV
out_path = Path("orders.csv")
with out_path.open("w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=["order_id", "customer", "amount"])
writer.writeheader()
writer.writerows(orders)
# Read CSV
with out_path.open(encoding="utf-8") as f:
reader = csv.DictReader(f)
for row in reader:
print(row["order_id"], row["amount"])
# ORD-001 149.99
# ORD-002 49.99

Pass newline="" when opening CSV files for writing — the csv module handles line endings itself. Without it, you get extra blank lines on Windows.

import json
from pathlib import Path
config = {
"app": "OrdersPortal",
"tax_rate": 0.08,
"features": ["invoicing", "shipping"],
}
config_path = Path("config.json")
# Write JSON
config_path.write_text(json.dumps(config, indent=2), encoding="utf-8")
# Read JSON
loaded = json.loads(config_path.read_text(encoding="utf-8"))
print(loaded["tax_rate"]) # 0.08
from pathlib import Path
# Read binary (images, PDFs, etc.)
pdf_bytes = Path("invoice.pdf").read_bytes()
# Write binary
Path("invoice_copy.pdf").write_bytes(pdf_bytes)
import csv
import json
from pathlib import Path
# Read orders from CSV, group by customer, write summary as JSON
input_csv = Path("orders.csv")
output_json = Path("customer_summary.json")
customers = {}
with input_csv.open(encoding="utf-8") as f:
reader = csv.DictReader(f)
for row in reader:
customer = row["customer"]
amount = float(row["amount"])
if customer not in customers:
customers[customer] = {"order_count": 0, "total": 0.0}
customers[customer]["order_count"] += 1
customers[customer]["total"] += amount
# Round totals
for c in customers.values():
c["total"] = round(c["total"], 2)
output_json.write_text(json.dumps(customers, indent=2), encoding="utf-8")
print(f"Wrote summary to {output_json}")

Run with python file_examples.py (Python 3.9+).

  • Not using with. A forgotten close() call causes resource leaks that manifest as Too many open files under load.
  • Missing encoding="utf-8". The default encoding is platform-dependent. Files written on Windows (cp1252) and read on Linux (utf-8) corrupt characters above ASCII 127.
  • Opening CSV with newline=None for writing. On Windows, this adds an extra \r before each \n.
  • Hardcoding path separators. "data\\reports" breaks on Linux/macOS. Use Path("data") / "reports".