Python Regular Expressions with the re Module
What you’ll learn
Section titled “What you’ll learn”- The difference between
re.match,re.search,re.findall, andre.finditer. - Substituting with
re.suband splitting withre.split. - Character classes, anchors, and quantifiers.
- Numbered and named capture groups.
- Flags such as
re.IGNORECASE,re.MULTILINE, andre.DOTALL. - Compiling patterns once with
re.compilefor reuse. - Common patterns: email, phone, date, and order IDs.
Prerequisites
Section titled “Prerequisites”- Python 3.9 or newer installed (
python --version). - Familiarity with strings and basic iteration.
Regular expressions describe text patterns compactly. They shine when the shape of the text varies — an order ID with three to six digits, a date in any month, an email at any domain — but they are not the right tool for every string job. Plain string methods like str.startswith, str.split, and the in operator are clearer and faster when the pattern is fixed, so reach for re only when you genuinely need to describe a pattern rather than match a literal string.
Step 1 — Searching vs matching
Section titled “Step 1 — Searching vs matching”The re module ships with Python, so there is nothing to install. The two functions people mix up first are re.match and re.search. re.match only matches at the start of the string; re.search scans the whole string and returns the first match it finds. Both return a match object when they succeed and None when they fail, so always check for None before reading groups.
import re
log = "Order ORD-001 created; Order ORD-002 created"
# re.match only looks at the start — misses both because the line starts with "Order"m = re.match(r"ORD-\d+", log)print(m) # None
# re.search scans the whole string and finds the first matchm = re.search(r"ORD-\d+", log)print(m.group()) # ORD-001print(m.start(), m.end()) # 6 12 — position of the matchVerify: Print m is None for the re.match call. It should be True, confirming match requires the pattern to start at the beginning.
Step 2 — Finding all matches with re.findall
Section titled “Step 2 — Finding all matches with re.findall”re.findall returns every non-overlapping match. When the pattern has no groups it returns a list of strings; with one group it returns a list of those group values; with multiple groups it returns a list of tuples. Use re.finditer when matches are large or many, because it yields match objects lazily instead of building a list.
log = "ORD-001 created; ORD-002 created; ORD-003 created"
ids = re.findall(r"ORD-\d+", log)print(ids) # ['ORD-001', 'ORD-002', 'ORD-003']
# One group returns the group contents, not the full matchjust_nums = re.findall(r"ORD-(\d+)", log)print(just_nums) # ['001', '002', '003']Verify: The plain version returns full ORD-### strings, while the grouped version returns only the digits. That shows how groups change what findall returns.
Step 3 — Character classes, anchors, and quantifiers
Section titled “Step 3 — Character classes, anchors, and quantifiers”Character classes match one character from a set. \d is a digit, \w is a word character, \s is whitespace, and their uppercase forms are the negations. Custom classes live in brackets: [A-Z] matches one uppercase letter and [^0-9] matches one non-digit. Anchors do not consume characters: ^ and $ match the start and end of the string. Quantifiers say how many times: * is zero or more, + is one or more, ? is zero or one, and {n,m} is between n and m.
# An order ID: ORD- followed by 3 to 6 digitsorder_pattern = r"ORD-\d{3,6}"
print(re.search(order_pattern, "ORD-001")) # matchprint(re.search(order_pattern, "ORD-123456")) # match — up to 6 digitsprint(re.search(order_pattern, "ORD-1")) # None — too few digits
# Character classesprint(re.findall(r"[A-Z]{3}-\d+", "ORD-001 and INV-002"))# ['ORD-001', 'INV-002']Verify: Search for ORD-\d{3,6} in "ORD-1234567". It should match ORD-123456 and leave the trailing 7 unmatched, confirming the upper bound is honored.
Step 4 — Numbered and named groups
Section titled “Step 4 — Numbered and named groups”Parentheses capture the part of the match inside them. Numbered groups start at 1, with group 0 being the whole match. Named groups use (?P<name>...) and make patterns self-documenting, which is worth the extra characters in any pattern you will revisit. groupdict() returns a dict of all named groups.
date_text = "2024-01-15"
# Numbered groupsm = re.search(r"(\d{4})-(\d{2})-(\d{2})", date_text)print(m.group(0)) # 2024-01-15 — full matchprint(m.group(1)) # 2024print(m.group(2)) # 01
# Named groups — far clearer when you read it laterm = re.search(r"(?P<year>\d{4})-(?P<month>\d{2})-(?P<day>\d{2})", date_text)print(m.group("year")) # 2024print(m.groupdict()) # {'year': '2024', 'month': '01', 'day': '15'}Verify: m.groupdict() should have exactly three keys — year, month, day — confirming every named group was captured.
Step 5 — Substitution and splitting
Section titled “Step 5 — Substitution and splitting”re.sub replaces matches with a string or the result of a function. A function replacement is powerful: it receives each match object and returns the text to substitute, so you can transform matches without a second pass. re.split splits a string wherever the pattern matches and discards the separators. Use re.escape when the separator contains regex metacharacters.
# Redact order IDs in a loglog = "ORD-001 shipped; ORD-002 pending"redacted = re.sub(r"ORD-\d+", "ORD-XXX", log)print(redacted) # ORD-XXX shipped; ORD-XXX pending
# Function replacement: uppercase the order IDsshouted = re.sub(r"ORD-\d+", lambda m: m.group().upper(), log)print(shouted) # ORD-001 shipped; ORD-002 pending (already uppercase)
# Split a CSV-ish line on semicolons, trimming whitespaceparts = re.split(r"\s*;\s*", "ORD-001 ; ORD-002 ; ORD-003")print(parts) # ['ORD-001', 'ORD-002', 'ORD-003']Verify: Replace the redacted pattern’s replacement with "[REDACTED]" and confirm every ORD-### becomes [REDACTED], showing the substitution is global by default.
Step 6 — Flags and compiling
Section titled “Step 6 — Flags and compiling”Flags change how the engine interprets the pattern. re.IGNORECASE makes matching case-insensitive, re.MULTILINE makes ^ and $ match the start and end of each line rather than the whole string, and re.DOTALL makes . match newlines too. When you reuse a pattern many times, compile it once with re.compile; the compiled object has the same methods as the module and avoids re-parsing the pattern on every call.
import re
# Case-insensitive: "order" or "ORDER" or "Order"print(re.findall(r"order", "Order ORDER order", re.IGNORECASE))# ['Order', 'ORDER', 'order']
# Multiline: match an order ID at the start of each linemultiline_log = "ORD-001\nORD-002\nORD-003"print(re.findall(r"^ORD-\d+", multiline_log, re.MULTILINE))# ['ORD-001', 'ORD-002', 'ORD-003']
# Compile once, use many timesorder_id = re.compile(r"ORD-\d+")for line in ["ORD-001", "bad", "ORD-002"]: if order_id.search(line): print(f"valid: {line}")# valid: ORD-001# valid: ORD-002Verify: Remove re.MULTILINE from the multiline findall. It should return only ['ORD-001'], because ^ then matches only the very start of the string.
Step 7 — Common patterns
Section titled “Step 7 — Common patterns”These patterns cover the shapes OrdersPortal encounters most. They are pragmatic, not exhaustive — for email validation that needs to follow the RFC, use a dedicated library. Always prefer raw strings (r"...") for patterns so backslashes are not interpreted by Python’s string escapes first.
import re
email = re.compile(r"[\w.+-]+@[\w-]+\.[A-Za-z]{2,}")print(email.search("reach us at support@ordersportal.com").group())# support@ordersportal.com
phone = re.compile(r"\(?\d{3}\)?[-.\s]?\d{3}[-.\s]?\d{4}")print(phone.search("Call 555-123-4567").group())# 555-123-4567
date = re.compile(r"\d{4}-\d{2}-\d{2}")print(date.findall("Due 2024-01-15 and 2024-02-20"))# ['2024-01-15', '2024-02-20']
order_id = re.compile(r"ORD-\d{3,6}")print(order_id.findall("ORD-001, ORD-002, ORD-999999"))# ['ORD-001', 'ORD-002', 'ORD-999999']Verify: Test email against admin@sub.ordersportal.co.uk. It should match admin@sub.ordersportal.co and stop at the first top-level segment — a reminder that a simple pattern is a quick filter, not full validation.
Complete working example
Section titled “Complete working example”Save as regex_demo.py and run with python regex_demo.py (Python 3.9+):
"""OrdersPortal: parse a log, extract order IDs and dates, and redact emails."""import re
log_lines = [ "2024-01-15 ORD-001 created by alice@ordersportal.com", "2024-01-16 ORD-002 created; contact support@ordersportal.com", "2024-01-17 ORD-003 created",]
date_re = re.compile(r"\d{4}-\d{2}-\d{2}")order_re = re.compile(r"ORD-\d{3,6}")email_re = re.compile(r"[\w.+-]+@[\w-]+\.[A-Za-z]{2,}")
for line in log_lines: date = date_re.search(line).group() orders = order_re.findall(line) redacted = email_re.sub("[REDACTED]", line) print(f"{date} | {orders} | {redacted}")2024-01-15 | ['ORD-001'] | 2024-01-15 ORD-001 created by [REDACTED]2024-01-16 | ['ORD-002'] | 2024-01-16 ORD-002 created; contact [REDACTED]2024-01-17 | ['ORD-003'] | 2024-01-17 ORD-003 createdCommon mistakes
Section titled “Common mistakes”- Using a plain string instead of a raw string.
"\d"is just a backslash-d to Python, which is fine here, but"\b"becomes a backspace. Always write patterns asr"...". - Calling methods on a
Nonematch.re.searchreturnsNoneon failure, sore.search(...).group()raisesAttributeErrorwhen nothing matches. Checkif m:first. - Confusing
re.matchwithre.search.matchanchors to the start;searchscans the whole string. Most of the time you wantsearch. - Greedy quantifiers eating too much.
.*matches as much as possible. To stop early, use the non-greedy.*?or anchor the pattern tightly. - Recompiling the same pattern in a loop. Build the pattern once with
re.compileoutside the loop.
Next steps
Section titled “Next steps”- Strings — string methods that often replace a regex
- Exceptions — handling errors when a match fails
- Iterators and generators —
re.finditeryields lazily
Related pages
Section titled “Related pages”- File operations — running regex over file contents
- Functions basics — wrapping regex in reusable functions
- Dictionaries — storing parsed fields by name