What is the best way to perform line profiler in Python to achieve this?

General Discussions
1

I’ve been using cProfile to profile my code, and it’s been working great. I also use gprof2dot.py to visualize the results, which helps clarify things.

However, cProfile (and most other Python profilers I’ve seen so far) only profiles at the function-call level. This leads to confusion when certain functions are called from different places, as I can’t tell if call #1 or call #2 is taking up the majority of the time. This issue worsens when the function is deep in the call stack and called from multiple places.

How can I achieve line-by-line profiling?

Instead of this:

function #12, total time: 2.0s

I’d like to see something like this:

function #12 (called from somefile.py:102) 0.5s
function #12 (called from main.py:12) 1.5s

cProfile shows how much of the total time “transfers” to the parent, but this connection is lost when you have many layers and interconnected calls.

Ideally, I’d love a GUI that parses the data and shows me my source file with the total time given to each line. Something like this:

main.py:

a = 1 # 0.0s
result = func(a) # 0.4s
c = 1000 # 0.0s
result = func(c) # 5.0s

Then, I’d be able to click on the second “func(c)” call to see what’s happening.

2

The line_profiler module is specifically designed for line-by-line profiling of Python code. It gives detailed statistics about the time spent on each individual line within a function.

Install the line_profiler package:

pip install line_profiler

Use the @profile decorator to mark functions you want to profile: from line_profiler import LineProfiler

def my_function(a, b):
    result = a + b   # Line 1
    c = a * b        # Line 2
    return result + c

# Profile the function
profiler = LineProfiler()
profiler.add_function(my_function)
profiler.run('my_function(5, 10)')
profiler.print_stats()

The line_profiler will show the time spent on each line of my_function, similar to: Timer unit: 1e-06 s

Total time: 0.000234 s Function: my_function at line 3 0.000123 0.000123 1 1 result = a + b 0.000111 0.000111 1 1 c = a * b

3

cProfile can also be used to generate profiling data, which can then be visualized line-by-line using a tool like SnakeViz.

Profile your code with cProfile:

import cProfile

def my_function(a, b):
    result = a + b
    c = a * b
    return result + c

cProfile.run('my_function(5, 10)', 'profile_stats')

Use SnakeViz to visualize the profiling data:

pip install snakeviz

Run SnakeViz:

snakeviz profile_stats

This will launch a GUI that visualizes profiling results, helping you view which lines are consuming the most time. It may not be as granular as line_profiler, but it gives you a broader understanding of where time is spent.

4

py-spy is a sampling profiler for Python that can track the execution of Python programs, including visualizing call stacks and time spent on specific lines.

Install py-spy:

pip install py-spy

Run py-spy to attach to a running Python process and view the profiling information:

py-spy top --pid <PID>

Use the py-spy graphical interface to view the line-by-line breakdown of execution.

This solution provides a real-time, interactive way to monitor and profile Python code with a GUI, offering insight into execution time at a fine-grained level.