Skip to content
  • Debugging & Profiling · Python

Debugging & Profiling

LearnViaNotes @learnvianotes· 4 min read

Debugging finds why code is wrong; profiling finds where it is slow or hungry. Both start with reading carefully and measuring, not guessing.

Reading a traceback

Traceback
Traceback (most recent call last):
  File "report.py", line 12, in <module>
    print(average(scores))
          ~~~~~~~^^^^^^^^
  File "report.py", line 4, in average
    return sum(xs) / len(xs)
           ~~~~~~~~^~~~~~~~~
ZeroDivisionError: division by zero
  1. Read the last line first

    The exception type and message: what went wrong.

  2. Walk up the stack

    Frames are listed oldest → newest; the last frame is where it was raised.

  3. Find your code

    Skip library frames; the lowest frame in your file is usually the place to fix.

  4. Use the markers

    Since 3.11 ~~~^^^ points at the exact sub-expression that failed.

print, logging and assertions

Python
items = [3, 5]
total = sum(items)
print(f"{items=} {total=}")          # items=[3, 5] total=8   ← the "=" specifier labels values
print(f"{total / len(items)=:.2f}")  # total / len(items)=4.00

pdb — the built-in debugger

Put breakpoint() where you want to stop (3.7+). The program pauses there with an interactive (Pdb) prompt; you can inspect variables, step line by line and continue.

Python
def apply_discount(price, pct):
    breakpoint()              # execution pauses here
    return price - price * pct / 100
pdb session
> shop.py(3)apply_discount()
-> return price - price * pct / 100
(Pdb) p price, pct
(200, 15)
(Pdb) n
--Return--
(Pdb) c
CommandDoes
p expr / pp exprPrint / pretty-print an expression
n (next)Run the current line, stepping over calls
s (step)Step into a function call
c (continue)Run until the next breakpoint
r (return)Run until the current function returns
l / llList source around the current line / the whole function
w (where)Show the call stack
u / dMove up / down the stack to inspect callers
b file:line / b funcSet a breakpoint
qQuit
Terminal
$ python -m pdb script.py                 # start under the debugger
$ python -m pdb -c continue script.py     # run; drop into pdb post-mortem on a crash
$ PYTHONBREAKPOINT=0 python script.py     # disable every breakpoint() call

timeit — micro-benchmarks

Python
import timeit

setup = "data = list(range(10_000)); s = set(data)"
list_time = timeit.timeit("9_999 in data", setup=setup, number=1_000)
set_time = timeit.timeit("9_999 in s", setup=setup, number=1_000)
print(set_time < list_time)          # True   ← O(1) hash lookup beats an O(n) scan
Terminal
$ python -m timeit -s "xs = list(range(1000))" "sum(xs)"
20000 loops, best of 5: 11.2 usec per loop

cProfile — where does the time go?

Python
import cProfile, pstats, io

def slow_square(n):
    return sum(i * i for i in range(n))

def work():
    return [slow_square(20_000) for _ in range(20)]

profiler = cProfile.Profile()
profiler.enable()
work()
profiler.disable()

out = io.StringIO()
stats = pstats.Stats(profiler, stream=out).sort_stats("cumulative")
stats.print_stats(5)
print("slow_square" in out.getvalue())     # True
Typical report (timings vary)
      400063 function calls in 0.041 seconds

Ordered by: cumulative time

ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1    0.000    0.000    0.041    0.041 app.py:6(work)
    20    0.000    0.000    0.041    0.002 app.py:3(slow_square)
    20    0.019    0.001    0.041    0.002 {built-in method builtins.sum}
400020    0.022    0.000    0.022    0.000 app.py:4(<genexpr>)
ColumnMeaning
ncallsHow many times the function was called
tottimeTime spent in the function itself, excluding sub-calls
cumtimeTime including everything it called — sort by this to find hot paths
Terminal
$ python -m cProfile -s cumtime app.py      # profile a whole script
$ python -m cProfile -o out.prof app.py && snakeviz out.prof   # visualise

tracemalloc — where does the memory go?

Python
import tracemalloc

tracemalloc.start()
big = [str(i) * 10 for i in range(50_000)]
current, peak = tracemalloc.get_traced_memory()
top = tracemalloc.take_snapshot().statistics("lineno")[0]
tracemalloc.stop()
print(peak > 1_000_000, top.size > 1_000_000)   # True True
ToolKindGood for
cProfileDeterministic, built inWhich functions take the time
py-spySampling, attaches to a running processProduction processes, no code changes
line_profilerLine by lineThe slow line inside a known-slow function
scaleneCPU + memory + GPU samplingAn all-in-one view
memray / tracemallocMemory allocationsLeaks and peak usage

How would you find why a Python program is slow?

What does breakpoint() do?

Free for students and teachers

Write notes like this one

Cards for every idea, quizzes and flashcards, read aloud — on the web and your phone.

Something wrong with this note? Open it to report