OOP is dead (part 3)

Before I take an in depth look at another severe problem with object-oriented-programming I want to talk about why I believe that OOP its really dead. Hey, Java is alive and kicking, C# (.NET/mono) also and still there's lot's of C++ programming. So how could OOP be dead then?

Sure, I used exaggeration to make my point more obvious. But there is more: Have you looked at the actual development in recent program language design? Some well known Java-guys propose to add closures. C# gets closures too, and there is this LINQ thing which is also totally un-OOP. And C++? Look at the recent trends in template-meta-programming (boost++) which is heading directly away from OOP too. And we have those 'new' languages, like Ruby and Python. Sure they are OOP to a big degree, but they also incorporate lots of functional aspects into their design. Same for really new languages like Scala which is only OOP to a certain degree anymore.

Another trend is meta-programming. While I think that meta-programming is a bad idea, it shows that there are limitations in a language one can only overcome with meta-programming. In Java for example we have annotations and reflection which allows to change then semantics of the language. Other languages have other means to accomplish the same. But while meta-programming is quite natural in a language like Lisp, its hard and cumbersome in a language like Java. So if people really go the way to do it, there have to be some obvious problems which aren't solvable with the language itself.

So why would people make those decisions if OOP is really alive? It's not that people abandon languages like C++ or Java, but they start to use them in an increasing non-OO way. And this means that OOP starts to lose importance.

And on the other hand we look at the ever increasing number of frameworks for Java which only try to solve problems which only exists because of the limitations of OOP. Frameworks of sometimes absurd size and complexity considering what problems they try to solve. Ok, maybe that's not a problem with OOP but only with Java. But I really doubt it. I've designed several OOP-languages over the last 20 years and each and every time I wasn't able to solve certain severe problems. Sure, you can make things more easy than in Java, but it's always only a gradual increase not a real big one.

what's the most important point in programming? Creating compact code? Creating easy maintainable programs? Having a very expressive language which allows elegant solutions to common problems? Sure all those are nice to haves, but the paramount point simply is one thing: Code reuse. If you can reuse existing code, you don't have to write it, to debug it, to maintain it. It's the optimal productivity boost and always faster then to re-implement something, even if it's in the most elegant, compact and easy way.

So what's the problem with code reuse? Why is it so difficult? And why especially in OOP? And isn't one of the core ideas of OOP to make code-reuse much more easy? I thought so myself, but for some I doubt the validity of this claim. It simply hasn't worked. But why?

First: Dependencies.
In OOP you have to create lots of unnecessary dependencies just for performance reasons. Think of the cache-example from the last article: Adding simple caching created lots of new potential problems and pitfalls. And in fact it's even totally useless for the function of the program, it's only needed to improve performance. But because of the cache, you need more dependencies to notify the cache of changes in data structures on which the cached value depends. But each dependencies breaks opportunities for code reuse, because you don't only have to capture the 'code itself' but also all the dependencies if you want to make a reusable piece of code. That's the reason, most frameworks grow that fast: They simply have to provide all those dependencies.

Second: Extensibility.
Isn't that something in which OOP should really shine? With inheritance, which allows the creation of slightly extended classes from existing ones, OOP should be able to give lots of extensibility. That's the theory. In practice there are several reason why it often won't work:

• The function-composition-problem I mentioned in part one.
  
  
• In OOP the type of an object is determined at creation time. So if you have an extended version of some class you can only have the advantages if you also have control over the creations of objects. If object-creation isn't in your reach, you will only get objects of you e 'old uninteresting' base-class instead of your 'shiny-new' one with enhanced functionality. To overcome that problem the factory-pattern has been invented, but it creates additional complexity and isn't of any use if the object you want to extend isn't created by a factory. There's also the idea to make object-types changeable at runtime, but this approach has drawbacks which easily outweights the advantages.
  
  
• The limitations of inheritance. Inheritance is a very simple thing, it can only describe 'is-a' relations. While this is perhaps the most important relation, it still only one. And if we want to have multiple 'is-a' relations ('multiple inheritance') we run into lots of difficult specification problems, the reason why some languages simply avoided multiple-inheritance or limited it to interface-inheritance. Also inheritance proposes a simply hierarchical structure of a program, where you can only add leaves to the tree. And you get lots of additional dependencies.
  
  
• the limitations of overloading: Overloading only works as good the design of a class. To allow flexibility a class has to consists of very small methods which call each other in a way that you can add as much functionality as possible by overriding them. But in practice that's much more difficult then in theory. If you create more small methods, the code gets more complicated even if the added complexity is maybe never needed, because nobody will ever extend a class in those ways. And if you are to conservative in splitting up the class into small methods, then you could block extensibility later and have to refactor the base class. While refactoring is always possible, it has a big disadvantages: It can break contracts and thus existing code. Also it's against the idea of code reuse where you simply use existing code without having to modify it.
  
  
• explicit data paths: This is a topic I will look at in detail in a later article. In short: In OOP you have to specify all data access-paths explicitly. If you have some data-structure you always have to specify some concrete structure how the data is structured. If this structure changes, often lots of code which access data has to be changed too. And a concrete structure which is usable for problem A isn't necessarily usable for problem B. So if you want to reuse the solution for problem A for problem B that won't work, because the data structures have changed, even if A and B otherwise have much in common.
  

For all those points there exists solutions. But in practice they often simply don't work. It's hard to tell why without looking at concrete examples, but I think the prime reason for it is over-specification: In OOP you have to write much more explicitly as is really useful. Often only for performance reasons. And over-specification leads to additional constraints and complexities which makes reuse much more improbable, because to fit into a new area, existing code don't only have to match the problem but also all those optimizations and tweaks which are necessary in OOP.

I think one of the reasons for that stems from the origins of OOP: Simulation. Each object there is some independent 'living thing' which has to be specified on it's own. Because of this OOP is very local, very 'to-the-metal' and it's hard to look at the overall structure of a program. Thus we have to code lots of things explicitly - which then leads to the described problems. So instead of OOP we need more abstract ways of programming, because abstraction allows the compiler to do much things automatically the programmer has to do now by himself. And by this the code is freed from unneccessary complexity which would not only make programming more easy but would also create more opportunities for code-reuse.

Java enhancements I would like to see

After some critique of the new closure proposal (because of it's redundancy) I want to talk about some Java extensions I would like to see. Those extensions complement Java in useful ways to make some often used idioms a bit more easy to use without promoting a totally different programming style.

Counter for the for-each-loop:

I often write something like this:

int i = 0;
for(Something s: data) {
   if (i++ > 0) out.print(", ");
   out.print(s);
}

While this works, it would be more easy that a for-each loop creates it's own counter on demand:

loop: for(Something s: data) {
   if (loop.count > 0) out.print(", ");
   out.print(s);
}

We can also extend this idea to supplying 'isLast' and 'isFirst' variables to make the above code more readable:

loop: for(Something s: data) {
   if (!loop.isFirst) out.print(", ");
   out.print(s);
}

A 'isLast' is much more seldom used but has it's benefits sometimes. While it's not so important, it's a bit more difficult to implement because we have to delay the evaluation of the loop-body by one element to know if the actual element is really the last one. But if it's required it's quite useful and could make loops much easier readable.

Parallel iteration

Sometimes you have to compare two collections, add them element-wise etc. So the new for-each loop is useless and you need to use the old explicit one. This could be easily changed with an additional modification to the for-each loop:

for(Something s1: data1; Something s2: data2) {
   ...
}

This loop will loop until if one of the iterators has no elements left. It's a really straight forward extensions and should be quite easy to implement.

'on-break' extension for loops

This one is to solve constant annoyance: Figuring out if a loop terminated normally or by a break.

Imagine you have something like

boolean has_error = false;
for(Something v: data) {
  if (v.containsError()) {
     has_error = true;
     break;
  }
}
if (has_error) {
  ...
}

This is a very common idiom for handling breaks: Simply setting a flag to know if a loop breaks. But you have to do it by hand and thus it's error prone (imagine, setting has_error = false by accident). Sometimes it's possible to do the stuff in if (has_error) { ... } in the loop-body, but there are lots of cases where this won't work. Also it won't work if the break is created by the compiler like above in parallel-iteration.

So I propose the following syntax:

for(Something v: data) {
  if (v.containsError()) break;
}
catch() {
   ... // executed on break only
}

This is really straightforward and don't use any new keyword.

Allow for each top iterate if supplied with an Iterator and not only an Iterable

Sometimes is useful to provide different kinds of iterators per collection. Maybe:

    Iterator forwardIterator();
    Iterator backwardIterator();
    Iterator depthFirstIterator();
    Iterator filterIterator(Filter filter);

In the moment we have to make all those Iterators self Iterable and add a

    Iterator iterator() { return this; }

method to the iterator. It would be better if you could use iterators directly in a for-each loop and write for example:

   for(Element e: tree.depthFirstIterator()) {
      ...
   }

even if depthFirstIterator() only gives a normal Iterator.

Make iterators 'Closable' and close them in the for-loop automatically.

This is very important if you use for example iterators to iterate over a result set from an SQL query. In the moment it often prevents the usage of the for-each loop, because you have no access to the iterator object there. The problem: You have to create a new version of the iterator interface to prevent breaking existing code. So we have

interface Iterator2 extends Iterator, Closeable { 
    void close();
}

and a matching Iterable2 definition. Also the for-each loop has to be aware of this and generates a call to close() for Iterator2 iterators. The Iterator2 thing could be avoided if the below 'default-methods for interfaces' extension is implemented.

A 'using' statement like in C#

We all know how annoying correct exception handling and proper closing is in some cases. To make this easier I propose using 'try' for a 'using-like' construct:

try [type] [var] = [expr] { [body] }

catch blocks can optionally added to dispatch on exceptions like usual, but the 'try'-block will close [var] properly without requiring all those nested try-catch-finally mumbo-jumbo.

A 'instanceof' with autocasting

While it's often bad style to write something like

if (x instanceof SomeClass) {
   SomeClass x1 = (SomeClass)x;
   ...
}

it's often unavoidable. This idiom has two problems: First you have to invent a new name for 'x' and second the compiler won't check if the cast is correct. Also it's bloat. So I propose this syntax:

for(x instanceof SomeClass) {
   ... // x is casted here to 'SomeClass' properly
}

This is very easy to implement, don't requires new keywords and solves the above problems.


And after all those (nice and easy) 'syntactic sugar' now to the more interesting, but also less easy ones:

Default implementations for interfaces

This is a bit controversial, but I think the benefits are bigger then the risks. Interfaces remains field-less, but the method can contain default implementations. If you create a class C which implements an interface I, all default method-implementations in I will be copied to the body of C unless C implements those method itself.

With this feature interfaces could be more rich without requiring too implement each and every method in each class which implements the interface. This would lead to better code reuse, because interfaces could be made more customizable. Also you get less legacy issues, it's no more problem to extend an interface without the need to update all classes who implements this interface.

Consumer/provider reversal ('yielding')

If you have to write a more complex iterator which iterates over recursive data structures you know the problem: You have to do many things by hand, the compiler does normally itself. You need to provide your own stack, store the state of the iteration etc. This is very cumbersome and error-prone.

This problem is most often avoided by using closures for iteration. With this the state of iteration is handled as usual and the code becomes much clearer. Lets look at a simple example using closures:

class Tree {
   E value;
   Tree left, right;

   interface ForEach {
      void eval(E value);
   }
   
   void forEach(ForEach func) {
      func.eval(value);
      if (left != null) left.forEach(func);
      if (right != null) right.forEach(func);
   }

   void forEachDepthFirst(ForEach func) {
      if (left != null) left.forEach(func);
      if (right != null) right.forEach(func);
      func.eval(value);
   }
   
   void print(final PrintWriter out) {
      final boolean[] is_first = new boolean[] { true };
      
      forEach(new ForEach() { public void eval(E value) {
         if (is_first[0]) is_first[0] = false;
         else out.print(", ");
         out.print(value);
      }});
   }
}

With this implementation we can easily iterate over the tree in two different ways (depth first and depth last). But it shows two problems:
- First we have to maintain a 'is_first' state to place the ", " correctly.
- But really worse: It won't work with parallel iteration

If we have had an iterator implementation for 'forEach' and 'forEachDepthFirst' we could for example write

void isEqual(Tree t1, Tree t2) {
   for(E v1: t1; E v2: t2) {
      if (v1.equals(v2)) break;
   }
   catch {
      return false;
   }
   return true;
}

The advantage with this approach is that it would work for every Iterable, but how would you solve this problem with using closures and 'forEach'? You have to provide additional implementations for two trees, three trees etc. And what if you want to compare the content of a List and a Tree element-wise? Here Iterators are much more powerful. And because they are a part of Java for some time now, I propose to make the implementation easier instead of switching to a different and even less powerful abstraction.

The idea is well known as 'yielding'. It simply allows to suspend the execution while saving the state for later resume. While this can implemented with threading, its a very expensive way to do it. So its better to do it by code rewriting directly in the compiler.

How would a iterator with yielding look? Consider the below method as part of the class above:

Iterator iterator() {
  yield value;
  if (left != null) for(E v: left) yield v;
  if (right != null) for(E v: right) yield v;
}

That's it. As easy as it gets. 'yield' will suspend the execution and provide the result via the 'next()' method of the Iterator to the consumer. The problem here is the use of a new keyword. This I propose using 'throw return' for this. While this isn't totally nice, it somehow captures the idea of yielding and is short enough. To make the method as yielding the compiler could interfere this himself, or we could use the 'throw' keyword to mark those methods explicitly:

public throw Iterator depthFirst() {
  if (left != null) for(E v: left) throw return v;
  if (right != null) for(E v: right) throw return v;
  throw return value;
}

With this extension, iterators are quite easy to implement, but I know that this extension is a bit difficult implement. But it could provide a huge gain in productivity and more people would provide useful implementations of iterators.

Including inheritance

This is a bit more complex but could bring lots of benefits in code reuse and maintenance. It's a bit similar to traits, and has quite simple semantics.

Lets have an example:

class MyListModel implements ListModel {
 public int getSize() { ... }
 public Object getElementAt(int index) { ... }

 public void addListDataListener(ListDataListener l) { ... }
 public void removeListDataListener(ListDataListener l) { ... }
}

The problem are those two method at the bottom. Often you want to use a standard implementation, and only want to implement the getSize and getElementAt method. To prevent this reimplementation there is a AbstractListModel class in the libs which have a standard implementation for addListDataListener and removeListDataListener. While this works often, it's problematic if MyListModel also should extends another class, or if you want a different standard implementation.

So I propose to simply include standard implementations for interfaces:

class DataListenerImpl implements ListModel {
 public void addListDataListener(ListDataListener l) { 
    listenerList.add(ListDataListener.class, l);
 }
 public void removeListDataListener(ListDataListener l) {
    listenerList.remove(ListDataListener.class, l);
 }
 
 private EventListenerList listenerList = new EventListenerList();
}

This class won't work on it's own, but with the new extenson you could include it in another one:

class MyListModel implements ListModel {
 import DataListenerImpl;

 public int getSize() { ... }
 public Object getElementAt(int index) { ... }
}

that's it, MyListModel now have the methods and fields from DataListenerImpl which satisfy the ListModel interface. The fun part of this extension is that it's possible to split up implementations into several topics and the include them where you want them. So maybe you write a class

class MapModelImpl implements ListModel {
 public int getSize() { map.size(); }
 public Object getElementAt(int index) { ... }

 private Map map;
}

you can then simply synthesize a new ListModel by writing

class MyListModel implements ListModel {
 import DataListenerImpl;
 import MapModelImpl;
}

This would work for much more complex cases then described above and the example is far from complete, but the idea should be clear. I can won't elaborate here on the details (constructor handling, collisions), maybe in a later post.

OOP is dead (part 2)

A second and even more severe reason why OOP has reached it's peak and more and more people are looking for alternatives is mutable state (I'll write only 'state' from now on). Most programmers don't even think much about state, they simply take it as common part of programming, but in fact it's the source of most bugs and difficulties in programming. So what's 'state'? It's the current value of all data which changes while the program executes. The code of the program and constants don't change and are thus not a part of state. But in OOP each object has local data which often is mutated by calling a method, so each object has its own state. All objects together plus the values of local variables form the state of the program.

What's the problem with state? Simple: It can change. And if it changes and the change isn't reflected accordingly in other parts of the state, we have a bug. Lets look at a simple example:

void calcSum(List<Integer> data) {
   int n = data.size();
   int sum = 0;
   for(int i = 0; i < n; i++) {
       sum += data.get(i);
   }
   return sum;
}

This code looks perfectly valid, but in fact contains a hard to find bug. Imagine while executing the above method, an other thread is removing one element from 'data'. The result will be a runtime error, because n is now bigger then data.size() and for i = n - 1 we get an array range overflow.

Of course every Java programmer knows how to prevent this kind of problem (putting the method into a synchronized(data)-block), but if shows how easy it is to make mistakes which will happen nearly never - but then sometimes in a critical moment. The reason is mutable state: The above method depends on the state of 'data' and if this object is changed somewhere else while the method is executing, we have a potential problem.

But that's not all. Lets look at another example: Caching, which is often required to get a better runtime performance.

class CustomerList {
   private List<Customer> customers;
   float average_age = -1;
   
   public void notifyChanged() {
       average_age = -1;
   }
   
   public float getAverageAge() {
      if (average_age < 0) average_age = calcAverageAge();
      return average_age;
   }

   private float calcAverageAge() {
      if (customers.size() > 0) {
          float sum = 0;
          for(Customer c: customers) sum += c.getAge();
          return sum/customers.size();
      }
      return 0;
   }
   
   public void addCustomer(Customer c) {
       customers.add(c);
       notifyChanged();
   }
}

The calculation of the average age of the customers is cached here in a field. If you add a customer the cache is cleared an will be calculated next time it's required. This idiom is quite useful and common, but it has it's dangers. The danger is that the state of the list isn't automatically reflected in the state of the 'average_age' field. If someone changes the age of a customer externally, the whole caching will fail because this class has no idea that a changed happened. There are several methods to avoid this problem, one is to simply add a reference to the containing CustomerList-object to every Customer-object and then add 'notifyChanged()' calls to each method which changes relevant data of the Customer-object.

But you see how this seemingly simple optimization increases the complexity of the program. First we only have the idea with the cache, then we discover that we need notifications, then we have to add some registration mechanism, that every Customer-object knows it's container. And if Customer can be part of multiple CustomerList-objects we need the full fledged observer pattern, have to identify each and every position in the code which can changed a cached value etc. Imagine you have to maintain a complex program which uses several of those mechanisms and also optimizations to have multiple notification messages (so that the average_age cache isn't cleared if only the address of a customer is changing). You see how complex our once simple program is getting.

All this trouble only has one reason: Mutable state. If you change one part of the state (the age of a customer) all depending parts of the state (here average_age) have to be updated too. In complex programs we can even get circular dependencies and also cross-linking. Using caches is only one part of the problem, the problem happens with each and every dependency in the program, for example consider a GUI-view which has to be updated each time the underlying data changes. And if the whole thing has to be multithreaded, you have to additional sprinkle lots of 'synchronized' blocks all over the program.

But I think most Java programmers know that for long. And also programmers who are using Python, Ruby, Smalltalk etc. It's not a problem with a certain language, it's the programming paradigm, it's object orientation. Object orientation means that you have to do much 'by hand'. Each object, each dependency has to be maintained explicitly by the programmer. Sure, with clever libraries and frameworks, all this is solvable, but you still have to do much per hand. Is this complexity really necessary?

The answer is no. But we need a different paradigm. Most programmers already now one: Relational database management systems. Those systems do lots of caching and index updating totally automatically in the background. You only has to give a SQL query and the system gets the answer. It's using caches, it's updating indices, optimizing the order of table queries and lots more. But the programmer simply has to provide a simple query. Sure, even the best RDBMS have certain limitations, but it shows a way to go.

But there are other ways to reduce the dangers of mutable state. One is to simply have none. But is it possible? Can we program without having mutable state? Yes, and many already know: I'm talking about referential transparency (most often used in functional programming). This simply means that there are no 'side-effects' in a program, thus there can't be any mutation of data. How does it works? We require that each and every call to a function always give the same result if we provide the same parameters to the call. A language with referential transparency solves most of the problems mentions here. Since data isn't mutable there can't be concurrent modifications and multithreading is as simple as it gets. And without modification the compiler can do caching on it's own, he simply has to look at the input parameters of a function and if those are the same as before, he also can return the same value as before.

Of course referential transparency has it's disadvantages: It's a bit hard to program with it, because you can't change data. But how do you do any interesting things with it? Imagine a simple interactive application which reads some data from the user, calculates something and writes the result on the screen. How could we write this in a language with referential transparency? (I use a Java-like syntax)

function int readData(IOState io);
function void writeData(IOState io, int data);
function int calc(int value);

function void littleApp(IOState io) {
   (int val, IOState new_io) = readData(io);
   if (val != 0) {
      int res = calc(val);
      IOState new_io2 = writeData(new_io, res);
      myLittleApp(new_io2);
   }
}

Those three functions above are defined somewhere else. The interesting part here is the 'IOState' parameter. It holds the state of the IO-subsystem which is used to read and write data from and to the terminal. Without this IOState, the 'readData' function would return always the same value, because with the same input each function in a referential transparent always has to return the same result. The same goes for the 'writeData' function, which uses IOState to chain the writing in the right order. The last call 'myLittleApp(new_io2)' loops the application until someone enters a '0'. Recursion is necessary (but cheap here, because the function it tail recursive and can be optimized away by a good compiler), because we can't change a 'variable' (this would be mutation and could lead to side-effects which are prohibited). Each value is only assigned once, so in fact we can only copy something and never change it.

But you see that it's really possible to write interactive programs without having a single mutation of data. Everything is happening by creating a (sometimes modified) copy of existing data. And if the user enters the same value twice, the compiler could safely return the earlier result, because a call to 'calc' only depends on a single parameter.

The above is the real appeal of functional programming, not the existence of closures or continuations. With referential transparency it's not only possible to remove lots of dangerous constructs, concurrency issues and have automatic caching, we also get better ways for testing (if each function only depends on it's parameters, testing is much easier, because we don't need to set up a complex background state) and the compiler can make lots of optimizations and even make the program multithreaded by itself.

All those advantages are paid with a different way to write programs, and some new difficulties which also have new means to solve them (google for 'Monads Haskell' if you want to know more). But it shows that the intrinsic problems of object oriented programming are in fact solvable if we use a different paradigm. Relational databases and functional programming are only two examples, there are more. But before I delve deeper into this, I will try to list other reasons why OOP is dead soon.


Also have look at Part 1 and Part 3 of this series.

Some more remarks on closures in Java (part 2)

One feature they proposed are non-local returns. This feature is well known from Smalltalk and allows a closure to change the control-flow of the enclosing method. In Smalltalk there was a good reason for this feature: Smalltalk had no control-constructs like if-then, for, while etc.

For example let's look at 'if-then' in Smalltalk. It's implemented as a method of the true and false classes. I've translated it her into a Java-like syntax:

abstract class Boolean {
   abstract void ifThenElse(void() on_true, void() on_false);
}
 
class True extends Boolean {
   void ifThenElse(void() on_true, void() on_false) {
      on_true();
   }
}
 
class False extends Boolean {
   void ifThenElse(void() on_true, void() on_false) {
      on_false();
   }
}
 
void testIt() {
   Boolean ok = new False(); // or True
   
   ok.ifThenElse(() {
       ... // called if True
   },
   () {
       ... // called if False
   });
}

While this looks a bit complicated in Java, the Syntax in Smalltalk is a bit different and there you would write

ok ifTrue: [ ... ] ifFalse: [ ... ].

Without some kind of non-local return it would be impossible to return from a method if some condition is true. In most language those non-local returns are unnecessary, simply because they have special control constructs directly defined in the language, but the designers of Smalltalk decided to make it otherwise. And it really had it's advantages... BUT: In Java there already is a 'if-then' construct. And a 'for' and 'while'.

So why add non-local returns to closures? I don't know. Maybe because the proponents are fascinated with Smalltalk and want to have a exact clone? But non-local returns are also not common in functional programming and have nothing to do with closures per se. Languages like Ocaml or Haskell don't have it - because it's simply unnecessary. And if you really need it, you have it: It's called exception handling. You simply throw an 'Return' and catch it at the point you want to return to. This works in Java too.

So non-local returns are:
- mostly unnecessary, because Java already has control constructs
- available if necessary by using throw/catch

You may remark now that using Exceptions requires more writing than a simple return. Sure, but who cares, Java is full of construct which requires more typing then the same constructs in other languages. Also non-local returns aren't 'good practice' because it can create obscure errors especially if it's used to easy. For certain 'dangerous' features it's better if they require a bit more typing so that the programmer have to think if it's really a good idea to do. And a reader of the code would less probable overlook it.

Some more remarks on closures in Java

First: They already exists. Inner classes ARE closures. Really. I can't say that often enough, but everybody who don't recognize anonymous-inner-classes as closures simply hasn't understood the concept of either anonymous-inner-classes or closures.

But inner classes have an additional feature: Multiple entry-points.

Lets have a little, somewhat silly example (I use the proposed closure-syntax):

void showIncr(int(int) incr) {
   int val = 1;
   for(int i = 0; i < 4; i++) {
    if (i > 0) System.out.print(", "); 
    System.out.print(val); 
    val = incr(val);
   }
}

void useIt() {
  int(int) add2 = (int x) { return x + 2; }
  int(int) mul2 = (int x) { return x * 2; }

  showIncr(add2); // will print: 1, 3, 5, 7
  showIncr(mul2); // will print: 1, 2, 4, 8
}

So we have a method 'showIncr' which shows the effect of a 'incr'-function we supply as a parameter. In a real example the function would of course do something useful, like calculating the numeric derivative or building some kind of data structure etc.

While it looks interesting, lets try it in Java 5. In fact it's nearly as easy:

interface IncrFunc {
  int eval(int x);
}

void showIncr(IncrFunc incr) {
   int val = 1;
   for(int i = 0; i < 4; i++) {
    if (i > 0) System.out.print(", "); 
    System.out.print(val); 
    val = incr.eval(val);
   }
}

void useIt() {
  IncrFunc add2 = new IncrFunc() { int eval(int x) { return x + 2; }}
  IncrFunc mul2 = new IncrFunc() { int eval(int x) { return x * 2; }}

  showIncr(add2); // will print: 1, 3, 5, 7
  showIncr(mul2); // will print: 1, 2, 4, 8
}

Not a big difference. Sure you have to type some more characters, but in principle it's identical. Also if you use a modern IDE it can do most of the work itself. In IDEA for example you can type:

    IncrFunc add2 = new

and then hit shift+ctrl+space, the IDE generates

IncrFunc add2 = new IncrFunc() { 
  int eval(int x) { 
    return 0;
  }
}

automatically and you only have to fill out the body and replace the '0' by 'x + 2'. So closures are really well supported for some time now.

But can the new closure syntax do this:

Imagine after a year in production your nice little program from above should get a new feature: It should print out the name of the increment-operation. So I simply modify the latter code to:

interface IncrFunc {
  int eval(int x);
  String getName();
}

void showIncr(IncrFunc incr) {
   int val = 1;
   System.out.print(incr.getName() + ": "); 
   for(int i = 0; i < 4; i++) {
    if (i > 0) System.out.print(", "); 
    System.out.print(val); 
    val = incr.eval(val);
   }
}

void useIt() {
  IncrFunc add2 = new IncrFunc() { 
   int eval(int x) { return x + 2; } 
   String getName() { return "add 2"; }
  }
  IncrFunc mul2 = new IncrFunc() { 
   int eval(int x) { return x * 2; }
   String getName() { return "mul 2"; }
  }

  showIncr(add2); // will print: "add 2: 1, 3, 5, 7" now
  showIncr(mul2); // will print: "mul 2: 1, 2, 4, 8" now
}

It's only logical to let the operation itself store it's name, so the above is the most logical extension of the existing program. But how do you modify the version with the new closure proposal in this way? Closures only have one entry-point, like the old 'IncrFunc' has only one 'eval' method. But because of this, it's not possible to update the first program the same way it was possible in the second one. Instead the first program would require a complete rewrite.

Now imagine you used those 'new closures' extensive in your huge application and decide to change it in the way similar to the above. Do you still think that those 'new closures' are a good idea?

But what's the point of the proposal then? While I'm a bit unsure myself I suspect that the proponents want to make closures more easy to use as it's possible now. And with more easy use, they hope it will be used more often and could lead to more easy programming. But: Certain kinds of refactoring would in fact get more complicating (look above). And as a programmer you would always have to choose and decide which kind of closures you want to use, the new ones (with some limitations) or the older (with a little bit more typing).

Also Java simply isn't a functional programming language. In functional programming there are other means to accomplish the same as in Java, but they only work if they are used with knowledge and all over the program in a consistent way. But Java is a OO-language, and while I have my doubts about OO in general, I have much bigger doubts in multi-paradigm programming. Java has lots of success in building large applications and there is now a 'Java-way' of programming which works fine and has it's own advantages. Trying to change this way to a little bit more functional one simply wouldn't work. You have to do it fully or stay on the old track. If you mix two of your favorite foods with a blender you wouldn't expect that it would really taste better, would you?

Don't get me wrong: Java can use lots of little tweaks and extensions to the language, but those extensions should complement the 'Java-way' of programming instead of trying to force Java programmers to move over to some kind of half-way functional programming.

Closures for Java

Recently there was a proposal to add closures to Java. While this sounds like a good idea to many people who are used to have closures in their favorite language, it's in fact really useless, because Java already has closures for some time now.

In Java closures are called anonymous inner classes. Those are in fact full featured closures, because they really capture the environment of the method the closure was created in. This capturing is limited to variables declared as 'final' because to allow capturing arbitrary variables there would be the need to allocate them on the heap. This was a no-no a few years ago when Java was relatively new and slow and memory and runtime efficiency was really an issue.

Because of this, adding another syntax for closures isn't a very good idea because it only makes the language more complicated without getting something really new. Instead of this I would propose two additions to the language which could server the same purpose: Make the usage of closures more concise.

First we need a syntax to make non-final variables accessible from the closure. I propose using the keyword transient, because it's available and has no use other then on field declarations. To make a local accessible to a closure one can use this idiom:

final int[] val = new int[] { 1 };

val is now a final variable and can be accessed from within the closure. To get it's value you have of course to write val[0] everywhere. To make this more simple, I propose the following syntax:

transient int val = 1;

The compiler can then create the above code and use val[0] for each access to the variable. This little extension it's also quite easy useful for multiple return values. You can declare a method

void methodWithTwoReturnValues(transient int x, 
    transient int y) {
   x = 10;
   y = 20;
}

and use it this way:

transient int x, y;
methodWithTwoReturnValues(x, y);
// access x and y here.

So this extension serves two purposes: Make use of non-final variables with closures easier and have call-by-reference for arbitrary values.

The next addition is an easier syntax for the callsite. In the moment to use a closure somewhere you have to write something like

new ClosureType() {
 int eval(int val)  {
    return val*2;
 }
}

If the class 'ClosureType' has only a single method, this could be simplified with an easy compiler extension to:

   new ClosureType()(int val)  { return val*2; }

It's still not as short as possible, but it still contains all information necessary. The compiler has to look if 'ClosureType' has a single method with a single int parameter, then get the return type of this method and use this to synthesize the full method.

OOP is dead

Sure, that sounds dramatic, but I really think that OOP reached it's peak and is on the decline. In the last time we saw increasing interest in functional programming languages and concepts from functional programming (like closures, continuations etc). Many people want to see those concepts available in well known OOP-languages like Java, C#, C++ etc. And the language designers reacts and are starting to integrate those features.

But will this really ease software development? I doubt it. Feature overkill never was a solution - in nearly every are of life. Only adding feature after feature into something don't makes it better. It much more important that someone creates a homogeneous, useful and practical combination of features. To make programming as easy as possible, the programming language should guide the programmer in how to tackle a certain problem. If you have two different paradigm's available in the language how can you decide which paradigm you should use? And if you have to use 3rd party code, how probable is it that they used the same paradigm you used in your application. Fitting code in different paradigms together creates a so called 'impedance mismatch' which often leads do error prone interface-code (think of OR-mappers to fit a declarative relational language like SQL to a OOP-language like Java).

But if 'multi-paradigm' languages are bad (because they don't guide programmers and make code reuse more difficult) what's the appeal of those features from the world of functional programming? It's simply because OOP has it's limitations and those limitations are starting to become more and more obvious.

I will try to look at some of the reasons why OOP failed to live up to its promises and why I think that OOP will be superseded in the future.

One of this limitations is the 'specialty' of the 'this'/'self' parameter. A method call has a specific parameter which is used for method-dispatch: Determining the method to call at runtime. But what if you need two parameters to determine this method? In OOP you can use the visitor pattern, but it requires lots of boiler-plate code. And if you have 3 or more parameter or even want to decide on more complex conditions (like value ranges, structural constraints etc) it's totally useless.

In functional programming it's different: The central element is the function and all parameters are similar. To dispatch between different implementations at runtime most fp-languages support pattern matching which is much more powerful even then multiple-dispatch because it allows dispatch on the deep structure of the data and not only on the type. But even much more simple problems show the problem with using methods:

Lets look at the Java-runtime-libs. Imagine you want to add a new method to a class from those libs. Maybe you want to have a method 'trimLeft' for Strings which works like the standard 'trim' but only trims the left side of the String. It's not possible. You can't extend String because it's final (for otherwise good reasons), and even if you could, all the String you don't create by yourself are 'standard-strings' without a trimLeft method. So the only way to do it is to invent a new class, maybe 'StringUtils' and put your 'trimLeft' method as static method in it. So every time you have want to trim a string you have to remember that 'String.trimLeft' doesn't exists and that you have to call StringUtils.trimLeft(string) instead.

In a functional (and even procedural) language that's not a problem: Because all operations are implemented via functions, you simply define

function trimLeft(s: String);

and that's it. Totally equal to a function which resides in a lib, and you can import your own extension module wherever you want and use it for every String.

Ok, the Python/Ruby/Smalltalk/etc. user will point out, how easy it is in their favorite language to extend a given class later. But even this has it's disadvantages, because it only works in dynamically typed languages and there are some very valid reasons why dynamical (or latent) typing is a bit problematic. Also it don't solves the problem that 'self' isn't equal to the other parameters.

Look at a simple operation like the addition. In OOP you write something like v1.add(v2). This is totally unsymmetric and don't reflects the structure of the addition. Compare this to the procedural/functional solution:

function add(v1: Int, v2: Int):Int;
function add(v1: Float, v2: Int):Float;
function add(v1: Int, v2: Float):Float;
function add(v1: Float, v2: Float):Float;

As easy as it gets and all adds are defined together at one point in your program. And if you decide to extend 'add' with your own type, for example a new type 'Complex', you simply add some new definitions in your library. In a typical OOP-Language you have to modify Int.add instead, and Float.add, and BigInt.add etc. Even if it's possible, it's still lots of work and it's non-local, the changes are in several classes instead of being bundled in one single module.

Because of this OOP is in fact less extensible as functional and even procedural programming. Sure, there are ways to solve this problem (think of C++'s friend functions for example), but those turn out quite often as borrowed from fp-land and are often conflicting with other parts of the language.


Also have look at Part 2 and Part 3 of this series.