Architecture of Large Python Applications

Every codebase is easy to change on day one. The question is whether it is still easy on day four hundred, with six people touching it. Architecture is not about elaborate patterns — it is almost entirely about which direction your dependencies point.

Layers, and the One Rule

Split the work into three kinds of code: the domain (your actual rules), the service layer (orchestration), and the interface layer (HTTP, CLI, database). Then apply one rule: dependencies point inward. The domain imports nothing.

Here is a domain object. Notice what is absent — no database, no framework, no imports at all:

# domain layer: pure rules, imports nothing
class Order:
    def __init__(self, items):
        self.items = items

    def total(self):
        return sum(price for _, price in self.items)

    def is_free_shipping(self):
        return self.total() >= 50

order = Order([('book', 20), ('pen', 35)])
print(order.total())
print(order.is_free_shipping())

55
True

“Free shipping over 50” is a business rule. It should be testable without a database, a web server, or a config file — and here it is.

Dependency Injection Is Just a Parameter

The phrase sounds like it needs a framework. In Python it is usually one argument. Instead of a function reaching out and constructing a database connection, you hand it one:

class InMemoryRepo:
    def __init__(self):
        self.saved = []

    def save(self, order):
        self.saved.append(order)

# service layer receives the repository instead of creating one
def place_order(order, repo):
    repo.save(order)
    return f'saved, {len(repo.saved)} total'

repo = InMemoryRepo()
print(place_order({'id': 1}, repo))
print(place_order({'id': 2}, repo))

saved, 1 total
saved, 2 total

place_order never names a database. In production you pass the real repository; in a test you pass this in-memory one — which is why the test above runs instantly, in your browser, with no database anywhere.

Boundaries Without Inheritance

To say “anything that can save” without forcing callers to inherit from your class, use a Protocol. It matches on shape: any object with the right methods qualifies.

from typing import Protocol

class Repository(Protocol):
    def save(self, item) -> None: ...

class SqlRepo:
    def save(self, item) -> None:
        print(f'INSERT {item}')

class FakeRepo:
    def save(self, item) -> None:
        print(f'pretend save {item}')

def run(repo: Repository):
    repo.save('order-1')

run(SqlRepo())
run(FakeRepo())

INSERT order-1
pretend save order-1

Look closely at SqlRepo and FakeRepo: neither inherits from Repository, or from each other. They satisfy the boundary purely by having a save() method. That is what makes Protocols usable with classes you did not write — including ones from the standard library.

Plugins: a Dict Is Enough

Extensibility has a reputation for complexity. The simplest working plugin system is a dictionary plus a decorator:

EXPORTERS = {}

def exporter(name):
    def register(fn):
        EXPORTERS[name] = fn
        return fn
    return register

@exporter('csv')
def to_csv(rows):
    return 'csv:' + str(len(rows))

@exporter('json')
def to_json(rows):
    return 'json:' + str(len(rows))

print(sorted(EXPORTERS))
print(EXPORTERS['csv']([1, 2, 3]))

['csv', 'json']
csv:3

Adding a third format means adding one decorated function. Nothing else in the program changes — no if/elif chain to extend, no central list to remember to update.

For classes, __init_subclass__ lets them enrol themselves the moment they are defined:

class Plugin:
    registry = {}

    def __init_subclass__(cls, name, **kwargs):
        super().__init_subclass__(**kwargs)
        Plugin.registry[name] = cls

class Markdown(Plugin, name='md'):
    pass

class Html(Plugin, name='html'):
    pass

print(sorted(Plugin.registry))
print(Plugin.registry['md'].__name__)

['html', 'md']
Markdown

This is the mechanism behind most “just subclass and it works” plugin systems — and it needs no metaclass.

Say What Is Public

Every module has a promise and an implementation. __all__ and a leading underscore mark the line:

# what a module chooses to expose
__all__ = ['public_api']

def public_api():
    return 'stable'

def _internal_helper():
    return 'may change without notice'

print(__all__)
print(public_api())

['public_api']
stable

_internal_helper is not enforced-private — Python trusts you — but the underscore is a clear signal that it may change without warning. __all__ additionally controls what a star-import pulls in.

Watch Out

Myth: Good architecture means lots of small files. Reality: Direction of dependencies matters far more than file count. Splitting a tangle across twenty files leaves you with a tangle in twenty files.

Myth: A circular import is an import problem, fixable with a lazy import inside a function. Reality: It is usually a design signal. Two modules importing each other means the boundary is in the wrong place — one of them belongs in a different layer.