Skip to content
  • Performance Tips · Python

Performance Tips

LearnViaNotes @learnvianotes· 4 min read

Fast Python comes from choosing the right algorithm and data structure, then letting C-implemented built-ins do the looping. Measure before and after every change.

1. Algorithms and data structures first

Python
emails = [f"user{i}@x.com" for i in range(5_000)]
blocked = [f"user{i}@x.com" for i in range(0, 5_000, 7)]

# O(n·m): "in list" scans the whole list every time
slow = [e for e in emails if e not in blocked]

# O(n + m): build a set once, then O(1) lookups
blocked_set = set(blocked)
fast = [e for e in emails if e not in blocked_set]

print(slow == fast, len(fast))       # True 4285
Instead ofUseWhy
x in list inside a loopx in set / dictO(n) → O(1)
list.pop(0) / insert(0, x)collections.dequeO(n) → O(1) at both ends
Repeated min() of a changing listheapqO(n) → O(log n)
Linear search in sorted databisectO(n) → O(log n)
Recomputing the same resultsfunctools.cacheExponential → linear for DP-style recursion
Counting with a manual dictcollections.CounterWritten in C, clearer

2. Let built-ins do the looping

Python
nums = list(range(1, 100_001))

total = 0
for n in nums:                        # interpreted, one bytecode loop per item
    total += n * n

fast_total = sum(n * n for n in nums)   # sum() runs in C
print(total == fast_total)              # True

words = ["b", "a", "c"] * 3
print("".join(sorted(set(words))))      # abc   ← set, sorted, join: all C
print(max(nums), any(n > 99_999 for n in nums))   # 100000 True

3. Reduce work inside hot loops

Python
import math

def distances_slow(points):
    result = []
    for x, y in points:
        result.append(math.sqrt(x * x + y * y))    # attribute lookups every iteration
    return result

def distances_fast(points):
    hypot = math.hypot                               # hoist the lookup into a local
    return [hypot(x, y) for x, y in points]          # comprehension + a specialised C function

pts = [(3, 4), (6, 8), (5, 12)]
print(distances_slow(pts) == distances_fast(pts), distances_fast(pts))   # True [5.0, 10.0, 13.0]
  • Local variables are faster than globals and attribute lookups (self.x, module.func) — hoist them out of tight loops.

  • Move invariant work (compiling a regex, building a set) outside the loop.

  • Avoid exceptions for normal control flow inside very hot loops — but try itself is free when nothing is raised (3.11+).

  • Use generators to stream large data instead of building huge lists: less memory means fewer cache misses and GC pauses.

4. Memory: slots, generators, arrays

Python
import sys
from array import array

class PointDict:
    def __init__(self, x, y):
        self.x, self.y = x, y

class PointSlots:
    __slots__ = ("x", "y")
    def __init__(self, x, y):
        self.x, self.y = x, y

a, b = PointDict(1, 2), PointSlots(1, 2)
print(hasattr(a, "__dict__"), hasattr(b, "__dict__"))   # True False

as_list = list(range(100_000))
as_array = array("i", range(100_000))                    # packed 4-byte C ints
print(sys.getsizeof(as_array) < sys.getsizeof(as_list))  # True

5. Cache and precompute

Python
from functools import cache

@cache
def ways(n):                  # count paths — exponential without the cache
    return 1 if n < 2 else ways(n - 1) + ways(n - 2)

print(ways(200))              # 453973694165307953197296969697410619233826
print(ways.cache_info().hits > 0)   # True

6. When pure Python isn't enough

ToolHow it helpsBest for
NumPy / pandasVectorised operations run in C over whole arraysNumeric and tabular data
multiprocessing / ProcessPoolExecutorUse every CPU coreCPU-bound, parallelisable work
asyncio / threadsOverlap waiting timeI/O-bound work
PyPyA JIT-compiled Python implementationLong-running pure-Python code
Cython / mypycCompile (typed) Python to C extensionsHot inner loops in libraries
NumbaJIT-compile numeric functions with a decoratorLoops over NumPy arrays
Rust / C extensions (PyO3)Write the hot core nativelyMaximum speed, bigger effort

Why is a list comprehension usually faster than a for loop with append?

How do you speed up CPU-bound Python code?

Free for students and teachers

Write notes like this one

Cards for every idea, quizzes and flashcards, read aloud — on the web and your phone.

Something wrong with this note? Open it to report