Thursday, July 19, 2012

Automatic Debug Shell in Python

I've been working on a few long-running Python scripts lately. I find debugging to be difficult when a script takes a long time to run, since you don't get any feedback until execution hits the point in the code you're working on.

Python's awesome pdb module lets you drop into an interactive shell by calling the set_trace function. This is really useful because you can inspect local variables from the interactive shell. I found it tedious, though, to keep inserting these statements into my code, hoping it was in the right place, and wasting time when an exception was thrown somewhere other than what I was expecting.

I found a nice recipe that lets you drop into interactive mode as soon as an unhandled exception occurs. I created a gist that incorporates some changes from the comments. It only enters the interactive shell if the script has an appropriate TTY session, and if the exception is not a SyntaxError.

Here's an example script showing how to use it:
#test.py
import debug

def what():
   x = 3
   raise NotImplementedError()

if __name__ == '__main__':
   what()

And an example of using it:
$ python test.py 
Traceback (most recent call last):
  File "test.py", line 8, in <module>
    what()
  File "test.py", line 5, in what
    raise NotImplementedError()
NotImplementedError

> /home/jterrace/python-exception-debug/test.py(5)what()
-> raise NotImplementedError()
(Pdb) x
3

This is really nice for debugging, because it drops you to a shell that has all the local variables when the exception occurred.

Update
Yang pointed out the ipdb module. It looks awesome as an alternative to pdb and it comes with a method for doing this automatically.