How to write error messages that don’t suck

I spend most of my life interacting with software I wrote, or know intimately. Most of this code is what you would consider ‘safety critical’, and for that reason, it’s written in a particularly stringent way of defensive programming. It also has to adhere to incredibly strict API contracts, so that its output has to be predictable. Error messages must be meticulous and detailed. But elsewhere, in the real world, with tough deadlines to keep, the best one can do is writing error messages that don’t suck. By that, I mean error messages that may not be works of art, but which do the job. Just how important this is I learned thanks to a recent incident.

A few days ago, I logged into my bank account, and saw that it looked like it had been raided by the Visigoth hordes. It turns out that I had an interminably long series of transactions –– not particularly big ones, but they do add up ––, for the same sum, to the same American airline, around the same time. I will name and shame them, because I want them to fix this issue: Delta, we’re talking about you.

You see, my wife has been spending Christmastide in the US, visiting friends and family (and giving me some much-needed alone time to finish the first half of my upcoming book), and for the transatlantic long haul segment of her homebound flight, we decided it would be prudent to upgrade her to a comfier seat. Except every time we tried to do so, it returned a cryptic error message – “Upgrade failed”, and a code: 4477. So we tried again. Even when my wife used Delta’s phone booking service, the booking clerk couldn’t provide her with better information as to where the problem may lie. Finally, using someone else’s card, it worked.

The problem was that each time, we were charged. The error message failed to convey one of the most important things an error message should tell you: what was changed, and what was not? Our assumption was that payment and seat upgrade was an atomic transaction: either you were charged and the seat upgrade succeeded, or the seat upgrade failed and you would not be charged. The fact that this was not so, and was not disclosed, led to what could have been quite vexing (such as our account being blocked for presumed fraudulent use).

And to think all of this could have been avoided by error messages that don’t suck.

The five rules of writing error messages that don’t suck

There are error messages that are a pure work of art. They’re something to aspire to, but in the real world, you often won’t have time to create such works of creative beauty, and have to create something acceptable, which is going to be an error message that does not suck. Such an error message really only needs to explain four things.

  • Why am I seeing this crap?
  • What caused this crap?
  • What, of all the things I wanted to happen, have happened before this crap hit?
  • What, owing to this crap, has not happened, despite me wanting them to?

Bonus points are awarded for “how do I avoid this crap from happening ever again?”, but that’s more the domain of the documentation than of error messages. Let’s take these one by one.

This is NOT what an informative error message looks like. Image courtesy of redditor u/BenkoUK.

Why am I seeing this crap?

Programming languages written for adults, such as Python, allow you to directly instantiate even base-level exception classes, such as IndexError or KeyError. The drawback of this is that most of these often don’t explain what actually happened. I encountered this recently in some code that, it turned out, I myself have committed (sometimes git blame really lives up to its name!).

Consider the following function:

def mean_gb(image, x, y):
    """Returns the mean of the green and blue values of the pixel at (x, y)."""
    mean_gb = (image[x][y][0] + image[x][y][1])/2
    return math.floor(mean_gb)

To explain in brief: in Python, it is a convention to represent images as a tensor of rank 3 (a cube of numbers) using an n-dimensional array type (ndarray) from numpy, where one dimension each represents the x and y coordinates of each pixel and one dimension represents the channel’s sequential identifier.1 The function above retrieves the average of the green and blue channel values of an image represented in this way. To access a value m_{i, j, k} of a tensor m of rank 3, numpy uses the chained accessors syntax m[i][j][k]. And as long as we provide a tensor of rank 3, this function works just fine.

What, however, if we simply provide a tensor of rank 2 (a two-dimensional matrix)? We get… an IndexError:

In  [5]: mean_gb(a, 1, 1) 
Out [5]: 
    --------------------------------------------------------------------------- IndexError 
    <ipython-input-4-743d4fa2a9dd> in <module> 
    ----> 1 mean_gb(a, 1, 1) 

    <ipython-input-3-fe96408de555> in mean_gb(image, x, y) 
          2 """Returns the mean of the green and blue values of the pixel at (x, y).""" 
          3 
    ----> 4 mean_gb = (image[x][y][0] + image[x][y][1])/2
IndexError: invalid index to scalar variable.

From the perspective of numpy, this is completely true: we did request a value that didn’t exist. In fact, we tried to index a single scalar (the first two accessors already yielded a single scalar, and a scalar cannot normally be indexed), which we are indeed being told, albeit in a slightly strange language (it’s not that the index to the scalar is invalid, it’s that the idea of indexing a scalar is invalid).

However, for the user, this makes no sense. It does not point out to the user that they submitted a tensor of rank 2 for a function that only makes sense with tensors of rank 3. The proper way to deal with this, of course, is to write a custom error message:

def mean_gb(image, x, y):     
    """Returns the mean of the green and blue values of the pixel at (x, y)."""          

    if len(image.shape) == 3:         
         mean_gb = (image[x][y][0] + image[x][y][1])/2
         return math.floor(mean_gb)     
    else:         
         raise IndexError(f"The image argument has incorrect dimensionality. The image you submitted has {len(image.shape)} dimensions, whereas this function requires the image to have 3 dimensions to work.")

Best practice: smart error messages
A good error message is as long as it needs to be and as short as it can be while not omitting any critical details.

The .shape property of a numpy.ndarray object provides a tuple containing the shape of the array. For instance, a 4-element vector has the shape (4,), a 3-by-5 matrix has shape (3, 5) and a tensor of rank 3 representing a 100×200 pixel image with three channels has shape (100, 200, 3). len(image.shape) therefore is a convenient way to determine the tensor dimensionality of an n-dimensional numpy array. In the above instance, if the dimensionality was anything other than three, an error message was raised. Note that the error message explained

  • what’s wrong (incorrect dimensionality),
  • why that dimensionality is wrong (it should be 3, and it’s something else), and
  • what the value is and what it should be instead.

An equally correct, and shorter, error message would have been

IndexError(“The image parameter must be a three-dimensional ndarray.”)

Short but sweet. You could also subclass IndexError:

class Expected3DError(IndexError):
     def __init__(self, *args, **kwargs):
         super(IndexError, self).__init__("Expected ndarray to have rank 3.", *args, **kwargs)

Best practice: exception/error classes
If a particular kind of error is prone to occur, it makes sense to give it its own class. The name itself can already hint at the more specific problem (e.g. that it’s the number of dimensions that’s at issue, not just some random index, in the case outlined above).

This could then be simply raised with no arguments (or, of course, you can build in additional fanciness, such as displaying the actual dimensions in the error message if the image or its shape is passed into the error):

def mean_gb(image, x, y):
    """Returns the mean of the green and blue values of the pixel at (x, y)."""
    
    if len(image.shape) == 3:
        mean_gb = (image[x][y][0] + image[x][y][1])/2
        return math.floor(mean_gb)
    else:
        raise Expected3DError()

This would yield the following output:

---------------------------------------------------------------------------
Expected3DError                           Traceback (most recent call last)
<ipython-input-49-743d4fa2a9dd> in <module>
----> 1 mean_gb(a, 1, 1)

<ipython-input-48-11cc8210b9d7> in mean_gb(image, x, y)
      5         mean_gb = (image[x][y][0] + image[x][y][1])/2
      6     else:
----> 7         raise Expected3DError()

Expected3DError: Expected ndarray to have rank 3.

As you can see, there are many ways to write error messages that tell you why the error has occurred. They do not need to replace a traceback, but they need to explain the initial (root) cause, not the proximate cause, of the error.

What caused this crap?

Yes, this is a single Java stack trace. It is worse than uninformative, and superbly confusing, despite valiant efforts to the contrary. Image courtesy of Tomasz Nurkiewicz, who contributed the root-cause-first patch to Java.

Best practice: tracebacks/stack traces
Always ensure your stack traces are meaningful and root-cause-first (the first function to be mentioned is the innermost function).

Here, the root cause is explained… badly. 58,220 is a number in the human sense of the word but to the computer, it contains a non-numeric character (the ,).
Good solution: rephrase error message to explain numbers must only consist of digits.
Better solution: simply prune the comma.
Image courtesy of r/softwaregore.

Closely related to the previous, your error message should provide some explanation of root causes. For instance, in the above example, explaining that the function expected a 3-dimensional ndarray described not only the reason the error message appeared but also why the value provided was erroneous. How much detail this requires depends strongly on the overall context of the API. However, users will want to know what actually happened.There is a way to overdo this, of course, as anyone who has seen the horror that is a multi-page Java traceback. But in general, error messages that explain what is expected, and possibly also what was provided instead, spare long hours of debugging. In user-facing interfaces, this is often the only way for the user to understand what the issue is and try to fix it. For instance, a form checker that rejects an e-mail address should at least try to explain which of the relatively few categories (throwaway domain, barred e-mail address, missing @, domain name is missing at least one ., etc.) the issue falls under. Believe it or not, users do not like to play ‘let’s figure out what the issue is’! For instance, I often use e-mail subaddresses that help me filter stuff, so many of my subscriptions are under [email protected]. Not all e-mail field checkers allow the + symbol, even though it is part of a valid e-mail address2and so are some far more exotic things, such as escaped @-signs! To cut a long story short: tell your users where the issue is.

Best practice: error numbers
Error numbers are fine in larger projects, e.g. Django has a great system for it. However, if you do use error numbers, make sure you have 1) a consistent error number nomenclature, 2) a widely available explanation of what each error code means and 3) if possible, a link to the online error code resolver in the error body

There are multiple ways to accomplish this. Consider my failing channel averager function from the previous subsection. You could phrase the IndexError‘s message string in multiple ways. Here are two examples I considered:

# Version 1
IndexError("Dimension mismatch.")

# Version 2
IndexError("This function requires a 3-dimensional ndarray.")
It helps to anticipate what users think is illogical, including minor things such as negative balances. Especially where money is at issue, it would have been prudent to explain to the user how adding $15.00 of credit got them a $2.59 balance. Image courtesy of r/softwaregore.

If you have seen the source code, or spend some time thinking, both make sense. However, Version 2 is far superior: it explains what is expected, and while Dimension Mismatch sounds like a great name for an synthpop band, it isn’t exactly the most enlightening error message.

Keep in mind that users may insert a breakpoint and run a traceback on your code, user interfaces do not afford them this option. Therefore, errors on user-facing interfaces should make it very clear what was expected and what is being provided. While a technical user of your code, faced with the better Version 2 error message from above could simply examine the dimensionality of the ndarray you provided to the function, this is not an option for someone interacting with your code via, say, a web-based environment. Therefore, these environments should be very clear about why they rejected particular values, and try to identify what the user input should have looked like versus what it did look like.

What, of all the things I wanted to happen, have happened before this crap hit?

The assumption by most people is that transactions are atomic: either the entire transaction goes through, or the entire transaction is rolled back. Therefore, if a transaction is not atomic, and permanently changes state, it should make this clear. The general human reaction to an error is to try it again, so it is helpful (Delta web devs, please listen carefully) to point out which part of the transaction has occurred.

Best practice: non-atomic transactions
Non-atomic transactions can fake atomicity by built-in rollback provisions. Always perform the steps that are furthest from your system (e.g. payment processing) last, because those will be harder to roll back than any steps or changes in your own system, over which you have more total and more immediate control.

This is especially important for iterator processors. Iterator processors go through a list of things – numbers, files, tables, whatever – and perform a task on them. Often, these lists can be quite long, and it is helpful if the user knows which, if any, of the items have been successfully processed and which have not. One of the worst culprits in this field that I have ever come across was a software that took in BLAST queries, each in a separate file, and rendered visualisations. It would then proudly proclaim at the end that it has successfully rendered 995 of 1,000 files –– without, of course, telling you which five files did not render, why they did not render, and least of all why the whole application did not have a verbose mode or a logging facility. Don’t be the guy who develops eldritch horrors like this.

What, owing to this crap, has not happened, despite me wanting them to?

The reverse of the above is that it’s useful for users to know what has not been changed. This is crucial, since users need to have an awareness of what still needs to be done for a re-run. This occurs not only in the context of error messages. One of my persistent bugbears is the Unix adduser command which, after you have provided it with some details about the new user, asks you if that’s all ok, and notes the default answer is ‘yes’. However, you never get any confirmation as to whether the user has been created successfully. This is particularly annoying when batch creating users, as you cannot get a log of which user has been created successfully on stdout unless you write your own little hack that echoes the corresponding message if the exit code of adduser is 0.

Best practice: tracking sub-transaction status
Non-atomic transactions that consist of multiple steps should track the status of each of these, and be able to account for what steps have, and what steps have not, been performed.

Where a particular transaction is not atomic, it is not sufficient to simply point out its failure. Unless the transaction can ‘fake atomicity’ with rollbacks (see best practice note above), it must detail what has, and what has not, occurred. It is important for the code itself to support this, by keeping track of which steps have been successfully carried out, especially if the function consists of asynchronous calls.

Conclusion

Writing great error messages is an art for in and of itself, and it takes time to master. Unfortunately, it is a vanishing art, usually under the pressures of developing software as fast as possible with various agile methodologies that then end up pushing these into a relatively unobtrusive backlog. Just because error messages don’t make a smidgen of sense does not mean they don’t pass tests –– but they are the things that can turn users off your product for good.

The good news is that your error messages don’t have to be perfect. They don’t even have to be good. It’s enough if they don’t suck. And my hope is that with this short guide, I have helped point out some ways in which you, too, can write error messages that don’t suck. With that in mind, you will be able to build user interfaces and APIs that users will love to use because when they get something wrong (and they will get things wrong, because that’s what we humans do), they will have a less frustrating experience. We have been letting ourselves off the hook for horrid error messages and worst practices like error codes that are not adequately resolved. It is time for all of us to step up and write code that doesn’t add insult to injury by frustrating error messages after failure.

References   [ + ]

1.Typically ordered in reverse (blue, green, red) due to a quirk of OpenCV.
2.See RFC 822, Section 6.1.

On the challenge of building HTTP REST APIs that don’t suck

Here’s a harsh truth: most RESTful HTTP APIs (in the following, APIs) suck, to some degree or another. Including the ones I’ve written.

Now, to an extent, this is not my fault, your fault or indeed anyone’s fault. APIs occupy a strange no man’s land between stuff designed for machines and stuff designed for humans. On one hand, APIs are intended to allow applications and services to communicate with each other. If humans want to interact with some service, they will do so via some wrapper around an API, be it an iOS app, a web application or a desktop client. Indeed, the tools you need to interact with APIs – a HTTP client – are orders of magnitude less well known and less ubiquitous than web browsers. Everybody has a web browser and knows how to use one. Few people have a dedicated desktop HTTP browser like Paw or know how to use something like curl. Quick, how do you do a token auth header in curl?

At the same time, even if the end user of the API is the under-the-hood part of a client rather than a human end user, humans have to deal with the API at some point, when they’re building whatever connects to the API. Tools like Swagger/OpenAPI were intended to somewhat simplify this process, and the idea was good – let’s have APIs generate a schema that they also serve up from which a generic client can then build a specific client. Except that’s not how it ended up working in practice, and in the overwhelming majority of cases, the way an API handler is written involves Dexedrine, coffee and long hours spent poring over the API documentation.

Which is why your API can’t suck completely. There’s no reason why your API can’t be a jumbled mess of methods from the perspective of your end user, who will interact without your API without needing to know what an API even is. That’s the beauty of it all. But if you want people to use your service – which you should very much want! -, you’ll have to have an API that people can get some use out of.

Now, the web is awash with stuff about best practices in developing REST APIs. And, quite frankly, most of these are chock-full of good advice. Yes, use plural nouns. Use HATEOAS. Use the right methods. Don’t create GET methods that can change state. And so on.

But the most important thing to know about rules, as a former CO of mine used to say, is to know when to break them, and what the consequences will be when you do. There’s a philosophy of RESTful API design called pragmatic REST that acknowledges this to an extent, and uses the ideas underlying REST as a guideline, rather than strict, immutable rules. So, the first step of building APIs that don’t suck is knowing the consequences of everything you do. The problem with adhering to doctrine or rules or best practices is that none of that tells you what the consequences of your actions are, whether you follow them or not. That’s especially important when considering the consequences of not following the rules: not pluralizing your nouns and using GET to alter state have vastly different consequences. The former will piss off your colleagues (rightly so), the latter will possibly endanger the safety of your API and lead to what is sometimes referred to in the industry as Some Time Spent Updating Your LinkedIn & Resume.

Secondly, and you can take this from me – no rules are self-explanatory. Even with all the guidance in the world in your hand, there’s a decent chance I’ll have no idea why most of your code is the way it is. The bottom line being: document everything. I’d much rather have an API breaking fifteen rules and giving doctrinaire rule-followers an apoplectic fit but which is well-documented over a super-tidy bit of best practices incarnate (wouldn’t that be incodeate, given that code is not strictly made of meat?) that’s missing any useful documentation, any day of the week. There are several ways to document APIs, and no strictly right one – in fact, I would use several different methods within the same project for different endpoints. So for instance a totally run-of-the-mill DELETE endpoint that takes an object UUID as an argument requires much less documentation than a complex filtering interface that takes fifty different arguments, some of which may be mandatory. A few general principles have served me well in the past when it comes to documenting APIs:

  • Keep as much of the documentation as you can out of the code and in the parts that make it into the documentation. For instance, in Python, this is the docstring.
  • If your documentation allows, put examples into the docstring. An example can easily be drawn from the tests, by the way, which makes it a twofer.
  • Don’t document for documentation’s sake. Document to help people understand. Avoid tedious, wordy explanation for a method that’s blindingly obvious to everyone.
  • Eschew the concept of ‘required’ fields, values, query parameters, and so on. Nothing is ‘required’ – the world will not end if a query parameter is not provided, and you will be able to make the request at the very least. Rather, make it clear what the consequences of not providing the information will be. What happens if you do not enter a ‘required’ parameter? Merely calling it ‘required’ does not really tell me if it will crash, yield a cryptic error message or simply fail silent (which is something you also should try to avoid).
  • Where something must have a particular type of value (e.g. an integer), where a value will have to be provided in a particular way (e.g. a Boolean encoded as o/1 or True/False) or has a limited set of possible values (e.g. in an application tracking high school students, the year query parameter may only take the values ['freshman', 'sophomore', 'junior', 'senior']), make sure this is clearly documented, and make it clear whether the values are case sensitive or not.
  • Only put things into inline comments that would only be required for someone who is reading your code. Anything a user of your methods/endpoints ought to know about them should be in the docstring or otherwise end up in your documentation – your docstring, of course, has the added benefit that it will be visible not only for people reading the documentation but also for whoever is reading your code.
  • If you envisage even the most remote possibility that your API will have to handle Unicode, emojis or other fancy things (basically, anything beyond ASCII), make sure you explain how your API handles such values.

Finally, eat your own dog food. Writing a wrapper for your API is not only a commercially sound idea (it is much more fun for other developers to just grab an API wrapper for their language of choice than having to homebrew one), it’s also a great way to gauge how painful it is to work with your API. Unless it’s anything above a 6 on the 1-10 Visual Equivalent Scale of Painful and Grumpy Faces, you’ll be fine. And if you need to make changes, make any breaking change part of a new version. An API version string doesn’t necessarily mean the API cannot change at all, but it does mean you may not make breaking changes – this means any method, any endpoint and any argument that worked on day 0 of releasing v1 will have to work on v1, forever.

Following these rules won’t ensure your API won’t suck. But they’ll make sucking much more difficult, which is half the victory. A great marksmanship instructor I used to know said that the essence of ‘technique’, be it in handling a weapon or writing an API, is to reduce the opportunity of making avoidable mistakes. Correct running technique will force you to run in a way that doesn’t even let you injure your ankle unless you deviate from the form. Correct shooting technique eliminates the risk of elevation divergences due to discrepancies in how much air remains in the lungs by simply making you squeeze the trigger at the very end of your expiration. Good API development technique keeps you from creating APIs that suck by restricting you to practices that won’t allow you to commit some of the more egregious sins of writing APIs. And the more you can see beyond the rules and synthesise them into a body of technique that keeps you from making mistakes, the better your code will be, without cramping your creativity.

Fixing the mysterious Jupyter Tensorflow import bug

There’s a weird bug afoot that you might encounter when setting up a ‘lily white’ (brand new) development environment to play around with Tensorflow.  As it seems to have vexed quite a few people, I thought I’ll put my solution here to help future  tensorflowers find their way.  The problem presents after you have set up your new  virtualenv . You install Jupyter and Tensorflow, and  when importing, you get this:

In [1]:   import tensorflow as tf

---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
in ()
----> 1 import tensorflow as tf

ModuleNotFoundError: No module named 'tensorflow'

Oh.

Added perplexion

Say you are a dogged pursuer of bugs, and wish to check if you might have installed Tensorflow and Jupyter into different virtualenvs. One way to do that is to simply activate your virtualenv (using activate or source activate, depending on whether you use virtualenvwrapper), and starting a Python shell. Perplexingly, importing Tensorflow here will work just fine.

The solution


Caution


At this time, this works only for CPython aka ‘regular Python’ (if you don’t know what kind of Python you are running, it is in all likelihood CPython).


Note

In general, it is advisable to start fixing these issues by destroying your virtualenv and starting anew, although that’s not strictly necessary. Create a virtualenv, and note the base Python executable’s version (it has to be a version for which there is a Tensorflow wheel for your platform, i.e. 2.7 or 3.3-3.6).


Step 1

Go to the PyPI website to find the Tensorflow installation appropriate to your system and your Python version (e.g. cp36 for Python 3.6). Copy the path of the correct version, then open up a terminal window and declare it as the environment variable TF_BINARY_URL. Use pip to install from the URL you set as the environment variable, then install Jupyter.

[email protected] ~ $ export TF_BINARY_URL=https://pypi.python.org/packages/b1/74/873a5fc04f1aa8d275ef1349b25c75dd87cbd7fb84fe41fc8c0a1d9afbe9/tensorflow-1.1.0rc2-cp36-cp36m-macosx_10_11_x86_64.whl#md5=c9b6f7741d955d1d3b4991a7942f48b9
[email protected] ~ $ pip install --upgrade $TF_BINARY_URL jupyter                 
Collecting tensorflow==1.1.0rc2 from https://pypi.python.org/packages/b1/74/873a5fc04f1aa8d275ef1349b25c75dd87cbd7fb84fe41fc8c0a1d9afbe9/tensorflow-1.1.0rc2-cp36-cp36m-macosx_10_11_x86_64.whl#md5=c9b6f7741d955d1d3b4991a7942f48b9
  Using cached tensorflow-1.1.0rc2-cp36-cp36m-macosx_10_11_x86_64.whl
Collecting jupyter
  Using cached jupyter-1.0.0-py2.py3-none-any.whl

(... lots more installation steps to follow ...)

Successfully installed ipykernel-4.6.1 ipython-6.0.0 jedi-0.10.2 jinja2-2.9.6 jupyter-1.0.0 jupyter-client-5.0.1 jupyter-console-5.1.0 notebook-5.0.0 prompt-toolkit-1.0.14 protobuf-3.2.0 qtconsole-4.3.0 setuptools-35.0.1 tensorflow-1.1.0rc2 tornado-4.5.1 webencodings-0.5.1 werkzeug-0.12.1

Step 2


Now for some magic. If you launch Jupyter now, there’s a good chance it won’t find Tensorflow. Why? Because you just installed Jupyter, your shell might not have updated the jupyter alias to point to that in the virtualenv, rather than your system Python installation.

Enter which jupyter to find out where the Jupyter link is pointing. If it is pointing to a path within your virtualenvs folder, you’re good to go. Otherwise, open a new terminal window and activate your virtualenv. Check where the jupyter command is pointing now – it should point to the virtualenv.

Step 3


Fire up Jupyter, and import tensorflow. Voila – you have a fully working Tensorflow environment!

As always, let me know if it works for you in the comments, or if you’ve found some alternative ways to fix this issue. Hopefully, this helps you on your way to delve into Tensorflow and explore this fantastic deep learning framework!

Header image: courtesy of Jeff Dean, Large Scale Deep Learning for Intelligent Computer Systems, adapted from Untangling invariant object recognition by DiCarlo and Cox (2007).

Give your Twitter account a memory wipe… for free.

The other day, my wife has decided to get rid of all the tweets on one of her twitter accounts, while of course retaining all the followers. But bulk deleting tweets is far from easy. There are, fortunately, plenty of tools that offer you the service of bulk deleting your tweets… for a cost, of course. One had a freemium model that allowed three free deletes per day. I quickly calculated that it would have taken my wife something on the order of twelve years to get rid of all her tweets. No, seriously. That’s silly. I can write some Python code to do that faster, can’t I?

Turns out you can. First, of course, you’ll need to create a Twitter app from the account you wish to wipe and generate an access token, since we’ll also be performing actions on behalf of the account.

import tweepy
import time

CONSUMER_KEY=<your consumer key>
CONSUMER_SECRET=<your consumer secret>
ACCESS_TOKEN=<your access token>
ACCESS_TOKEN_SECRET=<your access token secret>
SCREEN_NAME=<your screen name, without the @>

Time to use tweepy’s OAuth handler to connect to the Twitter API:

auth = tweepy.OAuthHandler(CONSUMER_KEY, CONSUMER_SECRET)
auth.set_access_token(ACCESS_TOKEN, ACCESS_TOKEN_SECRET)

api = tweepy.API(auth)

Now, we could technically write an extremely sophisticated script, which looks at the returned headers to determine when we will be cut off by the API throttle… but we’ll use the easy and brutish route of holding off for a whole hour if we get cut off. At 350 requests per hour, each capable of deleting 100 tweets, we can get rid of a 35,000 tweet account in a single hour with no waiting time, which is fairly decent.

The approach will be simple: we ask for batches of 100 tweets, then call the .destroy() method on each of them, which thanks to tweepy is now bound into the object representing every tweet we receive. If we encounter errors, we respond accordingly: if it’s a RateLimitError, an error object from tweepy that – as its name suggests – shows that the rate limit has been exceeded, we’ll hold off for an hour (we could elicit the reset time from headers, but this is much simpler… and we’ve got time!), if it can’t find the status we simply leap over it (sometimes that happens, especially when someone is doing some manual deleting at the same time) and otherwise, we break the loops.

def destroy():
    while True:
        q = api.user_timeline(screen_name=SCREEN_NAME,
                              count=100)
        for each in q:
            try:
                each.destroy()
            except tweepy.RateLimitError as e:
                print (u"Rate limit exceeded: {0:s}".format(e.message))
                time.sleep(3600)
            except tweepy.TweepError as e:
                if e.message == "No status found with that ID.":
                    continue
            except Exception as e:
                print (u"Encountered undefined error: {0:s}".format(e.message))
                break
        break

Finally, we’ll make sure this is called as the module default:

if __name__ == '__main__':
    destroy()

Happy destruction!