Python File I/O: Reading, Writing, and pathlib
What you’ll learn
Section titled “What you’ll learn”- Opening and reading files with
open()and context managers. - Write modes, text vs binary, and encoding.
- Using
pathlib.Pathfor cross-platform path operations. - Reading and writing CSV and JSON — the two most common file formats in practice.
- Safe patterns for temporary files and atomic writes.
Summary
Section titled “Summary”Python provides simple built-in file I/O through open() and the pathlib module. The key rule: always use a with statement when opening files. It guarantees the file handle is closed even if an exception occurs mid-read.
Save examples as .py and run with python examples.py (Python 3.9+).
Opening files with open()
Section titled “Opening files with open()”# Reading a text filewith open("orders.txt", encoding="utf-8") as f: content = f.read() # read entire file as a string
with open("orders.txt", encoding="utf-8") as f: lines = f.readlines() # list of lines, each including \n
with open("orders.txt", encoding="utf-8") as f: for line in f: # most memory-efficient: iterate line by line print(line.rstrip()) # strip trailing newlineOpen modes
Section titled “Open modes”| Mode | Meaning |
|---|---|
"r" |
Read text (default) |
"w" |
Write text, truncate file if it exists |
"a" |
Append text, create if missing |
"x" |
Exclusive create — fail if file exists |
"r+" |
Read and write, file must exist |
"rb", "wb" |
Read/write binary (images, PDFs, etc.) |
# Writing a text file — creates or overwriteswith open("report.txt", "w", encoding="utf-8") as f: f.write("Order Report\n") f.write("=============\n")
# Appending — does not truncate existing contentwith open("report.txt", "a", encoding="utf-8") as f: f.write("ORD-001: BensonCo $108.00\n")Always specify encoding="utf-8" explicitly. The default encoding is platform-dependent (cp1252 on Windows, utf-8 on Linux/macOS), which causes silent data corruption when files are shared across systems.
pathlib: modern path handling
Section titled “pathlib: modern path handling”pathlib.Path is the right way to work with file paths in Python 3. It is cross-platform, composable, and more readable than string concatenation with os.path.
from pathlib import Path
# Build paths without os.path.joinbase = Path("/var/data/orders-portal")reports_dir = base / "reports"today_report = reports_dir / "2026-11-13.csv"
print(today_report.name) # 2026-11-13.csvprint(today_report.stem) # 2026-11-13print(today_report.suffix) # .csvprint(today_report.parent) # /var/data/orders-portal/reports
# Check existenceprint(today_report.exists()) # False (doesn't exist yet)print(today_report.is_file()) # Falseprint(reports_dir.is_dir()) # depends on the filesystem
# Create directoriesreports_dir.mkdir(parents=True, exist_ok=True) # like mkdir -p
# Read and write through pathlib (no open() needed for simple cases)today_report.write_text("order_id,amount\n", encoding="utf-8")content = today_report.read_text(encoding="utf-8")
# Glob for all CSV filesfor csv_file in reports_dir.glob("*.csv"): print(csv_file.name)
# Recursive globfor py_file in Path(".").rglob("*.py"): print(py_file)
# Rename and deletebackup = today_report.with_suffix(".bak")today_report.rename(backup)backup.unlink() # delete the fileCSV files
Section titled “CSV files”import csvfrom pathlib import Path
orders = [ {"order_id": "ORD-001", "customer": "BensonCo", "amount": "108.00"}, {"order_id": "ORD-002", "customer": "NovaCorp", "amount": "270.00"},]
# Write CSVout_path = Path("orders.csv")with out_path.open("w", newline="", encoding="utf-8") as f: writer = csv.DictWriter(f, fieldnames=["order_id", "customer", "amount"]) writer.writeheader() writer.writerows(orders)
# Read CSVwith out_path.open(encoding="utf-8") as f: reader = csv.DictReader(f) for row in reader: print(row["order_id"], row["amount"])# ORD-001 108.00# ORD-002 270.00Pass newline="" when opening CSV files for writing — the csv module handles line endings itself. Without it, you get extra blank lines on Windows.
JSON files
Section titled “JSON files”import jsonfrom pathlib import Path
config = { "app": "OrdersPortal", "tax_rate": 0.08, "features": ["invoicing", "shipping", "analytics"],}
config_path = Path("config.json")
# Write JSONconfig_path.write_text(json.dumps(config, indent=2), encoding="utf-8")
# Read JSONloaded = json.loads(config_path.read_text(encoding="utf-8"))print(loaded["tax_rate"]) # 0.08For large JSON files that don’t fit in memory, use the json module’s streaming alternatives from third-party libraries like ijson.
Binary files
Section titled “Binary files”# Read a PDF or image as bytespdf_path = Path("invoice.pdf")raw_bytes = pdf_path.read_bytes()
# Write binary dataoutput = Path("invoice_copy.pdf")output.write_bytes(raw_bytes)Atomic writes
Section titled “Atomic writes”A common production pattern: write to a temporary file first, then rename. The rename is atomic on most operating systems — a reader either sees the old version or the new one, never a half-written file.
import tempfilefrom pathlib import Path
def write_atomically(target: Path, content: str) -> None: """Write content to target path atomically via a temp file.""" target.parent.mkdir(parents=True, exist_ok=True) # Write to a temp file in the same directory with tempfile.NamedTemporaryFile( mode="w", encoding="utf-8", dir=target.parent, delete=False, suffix=".tmp", ) as tmp: tmp.write(content) tmp_path = Path(tmp.name) # Atomic rename tmp_path.replace(target) # replace() is cross-platform
write_atomically(Path("data/report.txt"), "Order Report\n")Common mistakes
Section titled “Common mistakes”- Missing
encoding="utf-8". The default encoding is platform-dependent. Files written on Windows (cp1252) and read on Linux (utf-8) will corrupt characters above ASCII 127. - Not using
with. If an exception occurs betweenopen()andclose(), the file handle leaks. Always usewith open(...) as f:. - Opening a CSV with
newline=Nonefor writing. On Windows, this adds an extra\rbefore each\n, producing double line breaks when the file is opened in a text editor. - Reading a whole large file into memory with
f.read(). For files that are larger than available RAM, iterate line by line withfor line in f:. - Hardcoding path separators.
"data\\reports"breaks on Linux/macOS. UsePath("data") / "reports"orPath("data/reports")— pathlib normalizes separators automatically.
Interview questions
Section titled “Interview questions”1. Why should you always use a with statement when opening files?
The with statement calls __enter__ on entry and guarantees __exit__ is called on exit, even if an exception is raised. For file objects, __exit__ closes the file handle. Without with, a forgotten close() call or an exception mid-read leaves the file descriptor open, which on systems with low file descriptor limits (e.g. Linux default 1024) causes OSError: [Errno 24] Too many open files under load.
2. What is the difference between Path.rename() and Path.replace()?
rename() raises FileExistsError on Windows if the destination exists. replace() atomically replaces the destination on both POSIX and Windows — it is the correct method to use for atomic file replacement and is safer in production.
3. Senior scenario: an OrdersPortal batch job writes a 500 MB CSV export to disk every hour. The file is served to users via a web endpoint. How do you prevent users from downloading a partially written file?
Write to a .tmp file in the same directory, then call tmp_path.replace(final_path). The rename is atomic — the web server either serves the previous complete file or the new complete file, never an incomplete write in progress. Additionally, serve the file from an object store (S3, Azure Blob) rather than the local filesystem for durability and horizontal scaling; the upload itself can be atomic using multipart upload with a final commit step.
Key takeaways
Section titled “Key takeaways”- Always use
with open(...) as f:— it guarantees the file is closed. - Always specify
encoding="utf-8"for text files. - Use
pathlib.Pathfor all path manipulation — it is cross-platform and composable. - Pass
newline=""when opening CSV files for writing. - Use temp file + atomic rename for files that are read while being written.
Related pages
Section titled “Related pages”- Python error handling — the previous article in this series
- Python standard library — the next article in this series
- Context managers feature guide —
withstatement deep dive
Official references
Section titled “Official references”- Python docs: Reading and Writing Files — official tutorial
- Python docs: pathlib — full Path reference
- Python docs: csv module — CSV reading and writing
- Python docs: json module — JSON encoding and decoding