Python Tutorials

Python Regular Expressions (Regex)

8 min read

Have you ever tried to find all the email addresses hidden inside a 10,000-word document? Doing it manually would take hours. Writing standard Python loops and if statements to check for the “@” symbol and “.com” extensions would require dozens of lines of messy, error-prone code.

What if you could find every email, phone number, or specific date format in a massive document using just a single line of code?

Welcome to Regular Expressions (commonly called Regex). Regex is like a super-powered “CTRL+F” (Find) on steroids. It is a specialized micro-language used by professional developers to search, extract, and manipulate complex text patterns instantly. Learning Regex is a true superpower that will elevate your data processing skills to the senior level.

Regular Expressions (Regex)

A Regular Expression (Regex) is a sequence of characters that forms a search pattern. It is universally used in programming to check if a string contains a specified search pattern, to extract portions of text, or to replace specific characters.

In Python, Regex is not a built-in operator; it is accessed by importing the built-in re module, which contains all the necessary functions to compile and execute these powerful text searches.

Syntax & Basic Usage

Before we search for anything, we must import re. Furthermore, when writing Regex patterns in Python, we always prefix our strings with an r (e.g., r"pattern"). This creates a Raw String, telling Python to ignore special escape characters (like \n for newline) so the Regex engine can read the pattern exactly as written.

Here is the most basic usage using re.search(), which scans a string looking for the first location where a pattern produces a match.

import re

target_sentence = "The agent's secret code is 007 and his name is Bond."

# We want to find a sequence of exactly 3 digits. 
# In Regex, \d means "any digit" and {3} means "exactly 3 times".
search_pattern = r"\d{3}"

# re.search() returns a 'Match Object' if found, or 'None' if not found
match_result = re.search(search_pattern, target_sentence)

if match_result:
    # .group() extracts the actual text that was found
    print(f"Pattern found! The secret code is: {match_result.group()}")
else:
    print("Pattern not found.")

# Expected Output:
# Pattern found! The secret code is: 007
Code language: PHP (php)

Python Regex Methods and Function Arguments

The re module offers several different methods depending on what you want to achieve with your pattern. Let’s explore the core methods you will use every day.

1. re.findall() (Extracting Multiple Matches)

While re.search() stops after finding the very first match, re.findall() scans the entire text and returns a standard Python List containing every single match it found as strings.

import re

inventory_text = "We have 15 apples, 42 bananas, and 7 oranges in stock."

# \d+ means "one or more digits in a row"
number_pattern = r"\d+"

# Extract all numbers from the string
extracted_numbers = re.findall(number_pattern, inventory_text)

print(f"Extracted quantities: {extracted_numbers}")

# Expected Output:
# Extracted quantities: ['15', '42', '7']
Code language: PHP (php)

2. re.match() (Strict Beginning Match)

The re.match() method is very strict. It only checks if the pattern matches at the absolute beginning of the string. If the pattern exists later in the string, re.match() will return None.

import re

log_entry = "ERROR: Database connection failed at 10:45 PM."

# Check if the log starts exactly with the word "ERROR"
is_error = re.match(r"ERROR", log_entry)

# Check if it starts with "Database" (It doesn't, it starts with ERROR)
is_database = re.match(r"Database", log_entry)

print(f"Starts with ERROR? {'Yes' if is_error else 'No'}")
print(f"Starts with Database? {'Yes' if is_database else 'No'}")

# Expected Output:
# Starts with ERROR? Yes
# Starts with Database? No
Code language: PHP (php)

3. re.sub() (Search and Replace)

If you want to find a pattern and replace it with something else, use re.sub(pattern, replacement, string). This is incredibly useful for cleaning up messy user input or masking sensitive data.

import re

confidential_document = "Contact the CEO at 555-123-4567 for more information."

# Pattern for a standard US phone number (3 digits, hyphen, 3 digits, hyphen, 4 digits)
phone_pattern = r"\d{3}-\d{3}-\d{4}"

# Replace the found pattern with a masked string
redacted_document = re.sub(phone_pattern, "[REDACTED]", confidential_document)

print(f"Original: {confidential_document}")
print(f"Redacted: {redacted_document}")

# Expected Output:
# Original: Contact the CEO at 555-123-4567 for more information.
# Redacted: Contact the CEO at [REDACTED] for more information.
Code language: PHP (php)

4. re.split() (Advanced Splitting)

Standard Python strings have a .split() method, but it can only split by one specific character (like a comma). re.split() allows you to split a string using a complex Regex pattern.

import re

# Messy data separated by commas, semicolons, or dashes, with unpredictable spacing
messy_csv_data = "apple, banana; orange - grape;mango"

# The pattern breaks down as: match a comma, semicolon, OR dash ([-;,]) 
# followed by any amount of optional whitespace (\s*)
split_pattern = r"[-;,]\s*"

clean_list = re.split(split_pattern, messy_csv_data)

print(f"Cleaned List: {clean_list}")

# Expected Output:
# Cleaned List: ['apple', 'banana', 'orange', 'grape', 'mango']
Code language: PHP (php)

5. Match Objects and Grouping ()

When using re.search(), you often want to extract specific sub-parts of a pattern. By wrapping parts of your Regex in parentheses (), you create Groups.

import re

user_record = "Name: John Doe, Age: 34"

# We use parentheses to group the text we want to extract.
# (.*?) means "capture everything lazily"
# (\d+) means "capture one or more digits"
record_pattern = r"Name: (.*?), Age: (\d+)"

match_data = re.search(record_pattern, user_record)

if match_data:
    # group(0) is the entire matched string
    # group(1) is the first set of parentheses, group(2) is the second
    full_name = match_data.group(1)
    user_age = match_data.group(2)
    
    print(f"Extracted Name: {full_name}")
    print(f"Extracted Age: {user_age}")

# Expected Output:
# Extracted Name: John Doe
# Extracted Age: 34
Code language: PHP (php)

Real-World Practical Examples

Scenario 1: Validating and Extracting Email Addresses

In web scraping or user registration, you frequently need to validate if an input is a real email, and extract emails from large text blocks.

import re

# A large block of text scraped from a website
scraped_webpage_text = """
Please reach out to support@mycompany.com if you have issues. 
You can also contact the founder directly at founder.name_123@startup-domain.org. 
Invalid emails like user@.com or @website.com will be ignored.
"""

# A robust (but simplified) Regex pattern for email extraction
# \b       = Word boundary (ensures we don't start in the middle of a word)
# [\w.-]+  = 1 or more letters, numbers, underscores, dots, or hyphens
# @        = The literal '@' symbol
# [\w.-]+  = 1 or more domain characters
# \.       = The literal '.' symbol (escaped with a backslash)
# [a-zA-Z]{2,} = The domain extension (com, org, net) must be at least 2 letters
email_pattern = r"\b[\w.-]+@[\w.-]+\.[a-zA-Z]{2,}\b"

found_emails = re.findall(email_pattern, scraped_webpage_text)

print("--- Valid Emails Found ---")
for email in found_emails:
    print(email)

# Expected Output:
# --- Valid Emails Found ---
# support@mycompany.com
# founder.name_123@startup-domain.org
Code language: PHP (php)

Scenario 2: Data Cleaning (Standardizing Formats)

Imagine you are building a database, but users typed their phone numbers in wildly different formats. We can use re.sub() combined with capturing groups () to reorganize and standardize the data.

import re

raw_phone_numbers = [
    "(123) 456-7890",
    "123-456-7890",
    "123.456.7890",
    "1234567890"
]

# Pattern: Optional punctuation, 3 digits (Group 1), optional punctuation, 
# 3 digits (Group 2), optional punctuation, 4 digits (Group 3).
# \D* means "zero or more non-digits"
standardization_pattern = r"\D*(\d{3})\D*(\d{3})\D*(\d{4})\D*"

print("--- Cleaned Phone Numbers ---")
for raw_number in raw_phone_numbers:
    # We replace the messy string with our unified format using backreferences (\1, \2, \3)
    # \1 refers to what was captured in Group 1, \2 is Group 2, etc.
    clean_number = re.sub(standardization_pattern, r"(\1) \2-\3", raw_number)
    print(f"Raw: {raw_number:<15} -> Clean: {clean_number}")

# Expected Output:
# --- Cleaned Phone Numbers ---
# Raw: (123) 456-7890  -> Clean: (123) 456-7890
# Raw: 123-456-7890    -> Clean: (123) 456-7890
# Raw: 123.456.7890    -> Clean: (123) 456-7890
# Raw: 1234567890      -> Clean: (123) 456-7890
Code language: PHP (php)

Best Practices & Common Pitfalls

  • Always Use Raw Strings (r""): The biggest mistake beginners make is writing standard strings for Regex (e.g., "\d+"). In Python, a standard \ is an escape character. \b normally means “backspace,” but in Regex, it means “word boundary.” By using a raw string r"\b", you tell Python not to interpret the backslash, allowing the Regex engine to process it correctly.
  • Greedy vs. Lazy Matching: By default, Regex is “greedy.” A pattern like .* will match as much text as humanly possible, which often grabs way more than you intended. To make it “lazy” (stopping at the very first opportunity), add a question mark: .*?.
  • Precompile for Performance: If you are running the exact same Regex pattern thousands of times inside a loop, calling re.search() directly is inefficient because Python has to re-evaluate the pattern every loop. Instead, compile the pattern once before the loop using compiled_pattern = re.compile(r"pattern"), and then call compiled_pattern.search(text) inside the loop for a massive speed boost.

Summary

  • Regular Expressions (Regex) are powerful string-manipulation patterns used to search, extract, and clean text data.
  • Python handles Regex via the built-in re module.
  • Always use Raw Strings (r"...") to prevent Python from misinterpreting special Regex characters.
  • Use re.search() to find the first match, and re.findall() to return a list of all matches in a document.
  • Use re.sub() to cleanly search and replace complex patterns.
  • Parentheses () create Groups, allowing you to extract specific fragments of a matched pattern using .group(1).

Leave a Comment