Unit 6: Files and Exceptions; Regular Expressions

Persistence and robustness are the two concerns that separate a script from a program. A Python program that only uses variables loses all its data the moment the interpreter exits; a program that assumes every operation succeeds crashes on the first missing file or bad input. This unit covers file I/O (introduced in Python from its earliest versions through the built-in open()), object serialisation with the pickle module, structured error handling with try/except, and pattern matching with the re module.

Governing conventions the rest of the unit relies on:

• Everything is an object: a file is not a special language construct but an object returned by open(), carrying methods read(), write(), close().
• Text vs binary: text mode ('r', 'w', 'a') transfers str and applies encoding/newline translation; binary mode ('rb', 'wb') transfers bytes with no translation.
• Errors are exceptions, exceptions are classes: every runtime error raises an instance of a class descended from BaseException (ZeroDivisionError, FileNotFoundError, ValueError).
• EAFP over LBYL: Python style prefers "Easier to Ask Forgiveness than Permission" — attempt the operation and catch the exception — over checking preconditions first.
• Raw strings for patterns: regex patterns are written r'\d+' so that backslashes reach the regex engine untouched.
• The file pointer: each open file has a cursor; reading and writing advance it, so a file read to the end returns '' on the next read.

II. Text Files: Reading, Writing and the File Object

A. Text files

A text file is a sequence of characters, encoded to bytes on disk, conventionally divided into lines by a newline character.

• open() signature: open(filename, mode='r', encoding=None) returns a file object; filename may be relative ('data.txt') or absolute ('/home/u/data.txt').
• Modes: 'r' read (error if absent), 'w' write (truncates to zero length), 'a' append (writes at end), 'r+' read and write, 'x' exclusive creation.
• The with statement: the preferred idiom, because it closes the file even if an exception is raised mid-block.

with open('pi.txt') as f:      # mode 'r' is the default
    contents = f.read()
print(contents.rstrip())        # strip the trailing blank line

• Encoding matters: open('f.txt', encoding='utf-8') avoids a UnicodeDecodeError when a file written on one platform is read on another.

B. Reading from a file

Reading offers three granularities, chosen by how much memory you can spare.

• read(): returns the whole file as one string. f.read() on a 3-line file yields 'a\nb\nc\n' — note the trailing \n becomes an empty final line if you split().
• readlines(): returns a list of strings, each retaining its '\n'; len(f.readlines()) therefore counts lines.
• Iterating the object: for line in f: reads one line at a time — the memory-safe choice for large files.
• readline(): one line per call; returns '' at end of file, which is how loops detect EOF.

with open('pi_digits.txt') as f:
    lines = f.readlines()
pi_string = ''
for line in lines:
    pi_string += line.strip()
print(len(pi_string))

• Cursor control: f.seek(0) rewinds; f.tell() reports the byte offset.

C. Writing to a file

Writing requires an explicit mode, and write() never adds a newline for you.

• write(s): takes a single string and returns the number of characters written; f.write(42) raises TypeError.
• Newlines are manual: f.write('line 1\n') then f.write('line 2\n') produces two lines; without \n they concatenate.
• writelines(list): writes every string in an iterable, again adding nothing between them.
• 'w' destroys data: opening an existing file in 'w' truncates it before the first write — use 'a' to preserve content.

with open('log.txt', 'a') as f:
    f.write('user logged in\n')

D. Writing variables

Because write() accepts only strings, every non-string value must be converted first.

• Explicit conversion: f.write(str(count)) for an integer; f.write(str(3.14)) for a float.
• f-strings (Python 3.6+): the clearest route for mixed data — f.write(f'{name},{score},{avg:.2f}\n') embeds a formatted float.
• print to a file: print(name, score, sep=',', file=f) adds the newline automatically.
• Round-trip loss: everything read back is a str; int(f.readline()) is needed to restore a number, and '12\n' converts fine because int() tolerates whitespace.
• Limitation: lists and dictionaries written with str() come back as text, not structures — which is precisely the problem pickling solves.

E. Directories

File paths locate data relative to the working directory, and the os module manipulates them portably.

• Working directory: os.getcwd() returns it; os.chdir('data') changes it.
• Listing and testing: os.listdir('.') gives names; os.path.exists(p), os.path.isfile(p), os.path.isdir(p) test them.
• Creating and removing: os.mkdir('logs') (one level), os.makedirs('a/b/c') (nested), os.remove(f), os.rmdir(d).
• Portable joining: os.path.join('text_files', 'pi.txt') inserts / on Linux and \ on Windows; never hard-code the separator.
• Walking a tree: for root, dirs, files in os.walk('.') visits every subdirectory.
• Modern alternative: from pathlib import Path; Path('pi.txt').read_text() replaces open/read/close in one call.

III. Pickling: Serialising Python Objects

A. Pickling

Pickling converts an in-memory object graph into a byte stream that can be stored and later reconstructed exactly.

• Two calls: pickle.dump(obj, file) writes; pickle.load(file) reads back and returns an equal object.
• Binary mode is mandatory: the stream is bytes, so files must be opened 'wb' and 'rb'.
• What can be pickled: numbers, strings, lists, tuples, dicts, sets, and most user-defined class instances. Not file handles, sockets, lambdas or generators.
• In-memory variants: pickle.dumps(obj) returns a bytes object; pickle.loads(b) reverses it.

import pickle
scores = {'alice': 91, 'bob': 78}
with open('scores.pkl', 'wb') as f:
    pickle.dump(scores, f)
with open('scores.pkl', 'rb') as f:
    restored = pickle.load(f)
print(restored['alice'])   # 91, still an int

• Limitations: the format is Python-specific and version-sensitive, and unpickling untrusted data can execute arbitrary code — prefer json for data exchanged with other systems.

IV. Exception Handling: Controlled Failure

A. Handling the ZeroDivisionError exception

Division by zero is the canonical example because the error is unavoidable at runtime yet trivially recoverable.

• Unhandled behaviour: print(5/0) prints a traceback ending ZeroDivisionError: division by zero and terminates the program.
• Handled behaviour: the except block runs and execution continues.

try:
    answer = 5 / 0
except ZeroDivisionError:
    print("You can't divide by zero!")

• Integer vs float: 5 // 0 and 5 % 0 raise the same class; 5.0 / 0 also raises ZeroDivisionError, not inf.
• Capturing the object: except ZeroDivisionError as e: print(e) prints the message division by zero.

B. Using try-except blocks

The try block holds code that might fail; each except names the class it handles.

• Flow: if try succeeds, all except blocks are skipped; if it raises, Python matches the exception class against each except in order and runs the first that fits.
• Multiple handlers: except ValueError: then except TypeError: distinguishes causes; except (ValueError, TypeError): treats them alike.
• Class hierarchy: an except OSError catches FileNotFoundError because the latter is a subclass; order specific handlers first.
• finally: always executes — used to release resources, e.g. closing a connection whether or not an error occurred.
• Bad practice: bare except: swallows KeyboardInterrupt and typos alike; always name a class.
• Failing silently: pass is a legitimate body when the correct response is to do nothing.

C. The else block

The else clause holds code that should run only if the try block raised nothing.

• Purpose: keeps the try block minimal, so the handler cannot accidentally catch an error raised by follow-up code.

try:
    answer = int(a) / int(b)
except ZeroDivisionError:
    print('Cannot divide by zero.')
except ValueError:
    print('Please enter numbers only.')
else:
    print(f'Result: {answer}')
finally:
    print('Done.')

• Order is fixed: try → except(s) → else → finally.
• Contrast with putting the code in try: if print(f'Result: {answer}') sat inside try and itself raised, the message would be misattributed to the division.

D. Handling the FileNotFoundError exception

Missing files are an environmental fact, not a programming bug, so they are handled rather than prevented.

• Trigger: open('alice.txt') when the file is absent raises FileNotFoundError: [Errno 2] No such file or directory: 'alice.txt'.
• Handler: wrap the open and report or substitute a default.

filename = 'alice.txt'
try:
    with open(filename, encoding='utf-8') as f:
        contents = f.read()
except FileNotFoundError:
    print(f'Sorry, the file {filename} does not exist.')
else:
    words = contents.split()
    print(f'{filename} has about {len(words)} words.')

• Related classes: PermissionError (no access rights), IsADirectoryError, both siblings under OSError.
• Analysing many files: put the try inside a for filename in filenames: loop so one missing file does not abort the rest.

V. Regular Expressions and the re Module

A. Concept of regular expression

A regular expression is a compact string that describes a set of strings, matched by a finite-state engine rather than by literal comparison.

• Import and raw strings: import re; write patterns as r'\bcat\b' because '\b' in a normal string means backspace.
• Compilation: p = re.compile(r'\d{3}-\d{4}') builds a reusable pattern object; the module caches recent patterns anyway.
• Match objects: a successful search returns a Match; m.group(0) is the whole match, m.group(1) the first parenthesised group, m.start()/m.end() the offsets.
• Greedy by default: <.*> on '<a><b>' matches the whole string; <.*?> is lazy and matches '<a>'.

B. Various types of regular expressions

Pattern syntax divides into character classes, quantifiers, anchors and groups.

1. Character matching
   • Literals and .: . matches any character except newline.
   • Classes: [aeiou] any vowel; [^0-9] any non-digit; [a-z] a range.
   • Shorthands: \d digit, \w word character [A-Za-z0-9_], \s whitespace; uppercase \D \W \S negate them.
2. Structure and repetition
   • Quantifiers: * zero or more, + one or more, ? zero or one, {2,5} between two and five.
   • Anchors: ^ start of string, $ end, \b word boundary.
   • Alternation and grouping: (cat|dog)s? matches cat, dogs; (?P<year>\d{4}) names a group, retrieved by m.group('year').

Module functions:

• Flags: re.I case-insensitive, re.M makes ^/$ match per line, re.S lets . match newline.

C. Using match function

re.match() anchors the attempt at position 0 and returns None if the very beginning does not fit.

• Signature: re.match(pattern, string, flags=0).
• Truth testing: the result must be tested, because None is falsy and has no .group() — calling it raises AttributeError.

import re
m = re.match(r'(\d{2})/(\d{2})/(\d{4})', '25/12/2023 was a holiday')
if m:
    print(m.group(0))   # 25/12/2023
    print(m.group(3))   # 2023
    print(m.groups())   # ('25', '12', '2023')

• match vs search: re.match(r'holiday', '25/12/2023 was a holiday') returns None; re.search with the same pattern succeeds. Use match for validating whole fields, search for locating content.
• fullmatch(): requires the entire string to match — the strictest of the three, ideal for validating a PIN code.

D. Web Scraping by using Regular Expressions

Scraping extracts structured data from HTML fetched over HTTP; regular expressions serve as the extraction step on small, predictable markup.

• Fetch: import urllib.request; html = urllib.request.urlopen(url).read().decode('utf-8') — or requests.get(url).text.
• Extract with grouped patterns:

links = re.findall(r'href="(https?://[^"]+)"', html)
titles = re.findall(r'<h2[^>]*>(.*?)</h2>', html, re.S)
emails = re.findall(r'[\w.+-]+@[\w-]+\.[\w.]+', html)

• Clean up: re.sub(r'<[^>]+>', '', fragment) strips residual tags; .strip() removes whitespace.
• Store: write rows with f.write(f'{title},{link}\n'), or pickle.dump(records, f) to keep them as Python objects.
• Wrap in exception handling: network calls raise urllib.error.URLError or HTTPError, so the fetch belongs in a try block with the parsing in else.
• Limitations: HTML is not a regular language — nested tags, attributes in any order, unclosed elements and comments defeat regex, so libraries such as BeautifulSoup or lxml are correct for real documents; regex remains appropriate for pulling a well-formed field (a date, a price, an ID) out of already-isolated text.
• Etiquette: respect robots.txt, identify a User-Agent, and rate-limit requests.