Better exception handling

Exception handling seems to be a solved problem. Just use some kind of try/case/finally constructs and we're done. Are we? Exception handling creates several flaws and severe holes in the static checking of a program. While lots of people are researching type systems, exceptions open an ugly hole in the back of most of those systems. But if you try to do it right and type-check exceptions (like Java does) we get horrid and sometimes even hard to write constructs which blow up our code with lots of boilerplate. I tried to find a better approach to the problem.

Whats wrong with checked exceptions? Lets look at a simple example in Java (I use Java here because most languages use unchecked 'dynamic' exceptions). We want to open a file and read some data from it. If we are naive, we could think it could be implemented as

String readData(File file) {
  BufferedReader in = new BufferedReader(new FileReader(file));
  String data = in.readLine();
  in.close();
  return data;
}

Of course it's not so simple. The 'new FileReader(file)' can throw a FileNotFoundException. And since FileNotFoundException is a 'checked exception' the above code wouldn't compile - which is kind of a good thing because it prevents us to overlook the fact that the reading-process can fail and makes us think of a more sound solution.

So we have to handle this exception somehow. We may decide that this function should return a default value if there is an error opening the file. So we update our code accordingly:

String readData(File file, String default_value) {
  String data;
  BufferedReader in = null;
  try {
    in = new BufferedReader(new FileReader(file));
    data = in.readLine();
    in.close();
  }
  catch(FileNotFoundException e) {
    return default_value;
  }
  return data;
}

Looks fine. But it still won't compile yet. First our 'readLine' could throw an IOException, so we replace our FileNotFoundException with a IOException to catch both kind of exceptions. Now it would compile - but it's still not correct. Look at the 'in.close()'. If an exception occurs in 'in.readLine()' the program will skip 'in.close()' and we may have a resource leak (sure, it will collected by the GC some time later but we want to be really correct here).

So we change our code into

String readData(File file, String default_value) {
  String data;
  BufferedReader in = null;
  try {
    in = new BufferedReader(new FileReader(file));
    data = in.readLine();
  }
  catch(IOException e) {
    return default_value;
  }
  finally {
    if (in != null) in.close();
  }
  return data;
}

to make sure that in.close() will be called if an exception occurs. But again this won't compile because in.close() can throw again a checked exception.

String readData(File file, String default_value) {
  String data;
  BufferedReader in = null;
  try {
    in = new BufferedReader(new FileReader(file));
    data = in.readLine();
  }
  catch(IOException e) {
    return default_value;
  }
  finally {
    if (in != null) {
      try {
        in.close();
      }
      catch(IOException e) {
        return default_value;
      }
    }
  }
  return data;
}

Just look at the size of this code. Is it correct now? It will compile, but the behavior may be still wrong. If our 'in.readLine()' succeeded and then an error happens in 'in.close()' we get out default value even if we have some valid data. Of course this depends on the specification but to make sure that we get data in the moment even in this case we create a final version of the code:

String readData(File file, String default_value) {
  String data = default_value;
  BufferedReader in = null;
  try {
    in = new BufferedReader(new FileReader(file));
    data = in.readLine();
  }
  catch(IOException e) {}
  finally {
    if (in != null) {
      try {
        in.close();
      }
      catch(IOException e) {}
    }
  }
  return data;
}

Thats finally it. This code seems to work - and funny enough the 'catch' blocks are both empty now. Its all boilerplate. And it has required more thinking than one really wants to invest in such a simple piece of code.

So while checked exceptions give some safety because they don't let you miss a exception in your code so easy, they do create lots of hard to read and write code. Thats the reason the Java creators obviously started to question themselfs if its really a good idea and added unchecked exceptions to the language too. The most famous one (also available in most other OOP-languages) is the quite frequent 'NPE', the 'null pointer exception'. In principle every method call or field access in the language can create such an exception so Java don't require to check those explicitly. But is this really a sensible solution?

First: Every time we write code which could throw an exception we should be aware of this. Simply doing a core dump if we call a method on a null pointer would be much worse.

And for less frequent situations like opening files it may be a good idea to force programmers to think about handling of the possible error cases. The main reason someone uses a 'static typed' language is that the compiler can find runtime errors by using some typing rules. But if the compiler can only prevent 'no-method-found exceptions' while lots of other kinds of exceptions slip thru, we wonder if this is worth all the type-annotations and harder to understand language semantics. Java tried to solve this problem at least partially - but in the end the result wasn't that impressive.

But could we do it better? Or have we simply to accept the fact that exceptions can happen all over the place and we go happily back to unchecked exceptions with try/catch using lots of unit tests instead of relying on compiler checks. While some people seem to draw this conclusion, I think there is a better way: Create a language which can't throw any exceptions anymore!

Lets for example look at one of the most common exceptions, the NPE. The only reason it's possible to have such an exception is that we allow references to be 'null'. If we require that all references have to be non-null, we also don't need a NPE anymore. Ok, you may object that this is a bit to much, at least for a normal OOP language like Java, so lets tackle the problem a bit more cautiously and require that references which can be 'null' have to be explicitly declared as 'nullable' and that accesses via those references are only possible if the compiler can prove that it will be never null at the points where it matters.

In other words: If we want to call a method on a nullable reference 'ref' we have to use some if(ref != null)-expression first and handle the other case explicitly.

This isn't a new idea and would be a great extension to many static typed OOP-languages (I know that it's kind of possible in Java now by using annotations, but its far from comprehensive enough in the moment). It makes the NPE unnecessary and we would be able to remove it from the language completely. Why not think about concepts to do similar things with all kinds of exceptions?

This is of course hard and I suppose it won't work for most of the current languages. But lets start to think about it. One of the second most frequent exceptions is the "Array index out of range" exception. Can we eliminate this one, too? If we look at code which uses arrays or array-like collections, we notice that indexed access is in fact quite seldom really necessary. Most often data is appended or the array is simple iterated over. Those operations are 'index safe' because they simply don't need an explicit index. And if we look at list-handling functions in most functional languages we see that direct array access isn't seldom necessary there, so it seems possible to get around it in most cases.

But what if we really need some direct access via some index in one of the remaining cases? We can now simple require the same thing as from the above concept to remove NPEs from a language: If we want to access an array by providing an index we first have to make sure that the compiler can prove that the index is valid. Which again means that we may need to add a 'if (index < array.length) ...' check. This may look ugly but we have to check for bounds somehow anyway, so why not make it provable by the compiler?

To make this a bit more easy we can also add special annotations to method/function declarations. For example if we have a function with signature

    int addTwoElements(int[] array, int index1, int index2)

and we require that both indexes are in the range of the array length, we add annotations to let the compiler know that 'index1' and 'index2' are indexes into 'array'. This could look like

  int addTwoElements(int[] array, @index(array) int index1, @index(array) int index2)

and would allow the compiler to make sure that addTwoElements is called with valid parameters and we won't need to check it the function. This is a Java-esque solution and could be improved. But remember: The main idea is to write code in a way that indexed access isn't even necessary. So those annotations are only necessary for the remaining cases.

This way we can start to write programs in a way exceptions aren't possible. In most cases where a method or function can throw an exception means that we haven't thought enough about the problem. If we pull the ways a function can go wrong out of the code and put it into the type of the function we can make the compiler prove that our code has to succeed and that all necessary checking occurs before a function is called.

This is slightly similar to the concept of dependent types but it's more simple to do because the compiler only has to to the checks while the programmer creates the annotations himself. We don't require the compiler to prove everything. Where it fails to do so, we simply add a check or an annotation. Such a type system is much easier to create, to implement and to understand than a real, useful 'dependently typed' type system.

But what about the file-access problem like in the example above. File access can go wrong in any part of the operation (even in a 'close') and we somehow have to react to it. And since the success of an operation depends on the external state of the program it can't thus be proved or disproved by the compiler. So how to tackle this problem?

First idea: Just give up and add checked exceptions just for those kinds of cases where access to some some external resource can go wrong. By adding some language support (like C#'s 'using' construct or something similar) and we can even avoid most of the boilerplate code. But aren't there better solutions?

There are, but it's hard/impossible to do in a language like Java because it requires to write side-effect-free code. For example for the file access we can pull the file access 'out' of the body of a function and use lazy evaluation to process the data. We would now have a model where all I/O read all the data at once and if it fails, it fails in the read operation before the operation which uses the data. This way we would simply handle the check once here instead of distributing it to each and every 'readData' call alls over the code. But its impossible to implement it this way, first for performance reasons and second because input and output may happen simultaneously and may even interact to each other. So we do the reading and writing lazily but provide a model where it looks like it happens at once.

If now something goes wrong, the whole process is aborted and the operation is undone. The 'outer'-layer of the code detects the error and uses some ordinary if-then construct to do error handling. Since 'bubbling' of exceptions from a 'throw' to a 'catch' isn't necessary now, we can use simple and explicit ways to handle failure.

The whole code which does the operation (a complex parser for example) won't even need to know about the abortion because this is only of interest of the caller. Why is this impossible in a language like Java? Because Java could do operations with unlimited side-effects which aren't 'undoable' anymore. So at least the code which does the operation has to be pure functional or there has to be some kind of transaction-support.

This will work for many situations where we have a 'all or nothing' requirement. But what if we also want a partial result if an error occurs somewhere in between? Just add some 'error-token' to the data stream and let the operation handle it. This way we have two different kinds of IO-streams: One which will undo the whole operation if a failure happens and one which will simply provide an 'ERROR' token from the moment where some error happens but which won't undo the operation. By adding this behavior into the type of the IO-facility we have compile-time safety and still lots of flexibility.

Of course those three examples are only a little part of the huge number of exceptional situations a program can run into. To solve all the other ones we have to think about those and have to develop new idioms and patterns. But in the end it will always require that a programmer thinks a problem thru instead of relying on tests.

Bugs in programs will of course still exists. But by writing programs in the way described above (and having programming languages which support this style) we could get rid of lots of the most annoying bugs and write better and more stable programs. And by the way we could get rid of lots of ugly exception handling boilerplate code.