Top oop Questions

List of Tags

I've mastered almost all the Python concepts (well, let's say they're just OO concepts :-)) but this one is tricky.

I know it has something to do with introspection but it's still unclear to me.

So what are metaclasses? What do you use them for?

Concrete examples, including snippets, much appreciated!

Answered By: e-satis ( 1730)

Now that I know better, and due to demands in the answers on the yield keyword and the decorators, I'm going to answer this question with a lot of detail.

Disclaimer: very long post.

Classes as objects

Before understanding metaclasses, you need to master classes in Python. And Python has a very peculiar idea of what classes are, borrowed from the Smalltalk language.

In most languages, classes are just pieces of code that describe how to produce an object. That's kinda true in Python too:

  >>> class ObjectCreator(object):
  ...       pass
  ... 

  >>> my_object = ObjectCreator()
  >>> print my_object
  <__main__.ObjectCreator object at 0x8974f2c>

But classes are more than that in Python. Classes are objects too.

Yes, objects.

As soon as you use the keyword class, Python executes it and creates an OBJECT. The instruction

  >>> class ObjectCreator(object):
  ...       pass
  ... 

creates in memory an object with the name ObjectCreator.

This object (the class) is itself capable of creating objects (the instances), and this is why it's a class.

But still, it's an object, and therefore:

  • you can assign it to a variable
  • you can copy it
  • you can add attributes to it
  • you can pass it as a function parameter

e.g.:

  >>> print ObjectCreator # you can print a class because it's an object
  <class '__main__.ObjectCreator'>
  >>> def echo(o):
  ...       print o
  ... 
  >>> echo(ObjectCreator) # you can pass a class as a parameter
  <class '__main__.ObjectCreator'>
  >>> print hasattr(ObjectCreator, 'new_attribute')
  False
  >>> ObjectCreator.new_attribute = 'foo' # you can add attributes to a class
  >>> print hasattr(ObjectCreator, 'new_attribute')
  True
  >>> print ObjectCreator.new_attribute
  foo
  >>> ObjectCreatorMirror = ObjectCreator # you can assign a class to a variable
  >>> print ObjectCreatorMirror.new_attribute
  foo
  >>> print ObjectCreatorMirror()
  <__main__.ObjectCreator object at 0x8997b4c>

Creating classes dynamically

Since classes are objects, you can create them on the fly, like any object.

First, you can create a class in a function using class:

  >>> def choose_class(name):
  ...     if name == 'foo':
  ...         class Foo(object):
  ...             pass
  ...         return Foo # return the class, not an instance
  ...     else:
  ...         class Bar(object):
  ...             pass
  ...         return Bar
  ...     
  >>> MyClass = choose_class('foo') 
  >>> print MyClass # the function returns a class, not an instance
  <class '__main__.Foo'>
  >>> print MyClass() # you can create an object from this class
  <__main__.Foo object at 0x89c6d4c>

But it's not so dynamic, since you still have to write the whole class yourself.

Since classes are objects, they must be generated by something.

When you use the class keyword, Python creates this object automatically. But as with most things in Python, it gives you a way to do it manually.

Remember the function type? The good old function that lets you know what type an object is:

>>> print type(1)
<type 'int'>
>>> print type("1")
<type 'str'>
>>> print type(ObjectCreator)
<type 'type'>
>>> print type(ObjectCreator())
<class '__main__.ObjectCreator'>

Well, type has a completely different ability, it can also create classes on the fly. type can take the description of a class as parameters, and return a class.

(I know, it's silly that the same function can have two completely different uses according to the parameters you pass to it. It's an issue due to backwards compatibility in Python)

type works this way:

  type(name of the class, 
       tuple of the parent class (for inheritance, can be empty), 
       dictionary containing attributes names and values)

e.g.:

>>> class MyShinyClass(object):
...       pass

can be created manually this way:

  >>> MyShinyClass = type('MyShinyClass', (), {}) # returns a class object
  >>> print MyShinyClass
  <class '__main__.MyShinyClass'>
  >>> print MyShinyClass() # create an instance with the class
  <__main__.MyShinyClass object at 0x8997cec>

You'll notice that we use "MyShinyClass" as the name of the class and as the variable to hold the class reference. They can be different, but there is no reason to complicate things.

type accepts a dictionary to define the attributes of the class. So:

>>> class Foo(object):
...       bar = True

Can be translated to:

  >>> Foo = type('Foo', (), {'bar':True})

And used as a normal class:

  >>> print Foo
  <class '__main__.Foo'>
  >>> print Foo.bar
  True
  >>> f = Foo()
  >>> print f
  <__main__.Foo object at 0x8a9b84c>
  >>> print f.bar
  True

And of course, you can inherit from it, so:

  >>>   class FooChild(Foo):
  ...         pass

would be:

  >>> FooChild = type('FooChild', (Foo,), {})
  >>> print FooChild
  <class '__main__.FooChild'>
  >>> print FooChild.bar # bar is inherited from Foo
  True

Eventually you'll want to add methods to your class. Just define a function with the proper signature and assign it as an attribute.

>>> def echo_bar(self):
...       print self.bar
... 
>>> FooChild = type('FooChild', (Foo,), {'echo_bar': echo_bar})
>>> hasattr(Foo, 'echo_bar')
>>> hasattr(FooChild, 'echo_bar')
True
>>> my_foo = FooChild()
>>> my_foo.echo_bar()
True

You see where we are going: in Python, classes are objects, and you can create a class on the fly, dynamically.

This is what Python does when you use the keyword class, and it does so by using a metaclass.

What are metaclasses (finally)

Metaclasses are the 'stuff' that creates classes.

You define classes in order to create objects, right?

But we learned that Python classes are objects.

Well, metaclasses are what create these objects. They are the classes' classes, you can picture them this way:

  MyClass = MetaClass()
  MyObject = MyClass()

You've seen that type lets you do something like this:

  MyClass = type('MyClass', (), {})

It's because the function type is in fact a metaclass. type is the metaclass Python uses to create all classes behind the scenes.

Now you wonder why the heck is it written in lowercase, and not Type?

Well, I guess it's a matter of consistency with str, the class that creates strings objects, and int the class that creates integer objects. type is just the class that creates class objects.

You see that by checking the __class__ attribute.

Everything, and I mean everything, is an object in Python. That includes ints, strings, functions and classes. All of them are objects. And all of them have been created from a class:

  >>> age = 35
  >>> age.__class__
  <type 'int'>
  >>> name = 'bob'
  >>> name.__class__
  <type 'str'>
  >>> def foo(): pass
  >>> foo.__class__
  <type 'function'>
  >>> class Bar(object): pass
  >>> b = Bar()
  >>> b.__class__
  <class '__main__.Bar'>

Now, what is the __class__ of any __class__ ?

  >>> a.__class__.__class__
  <type 'type'>
  >>> age.__class__.__class__
  <type 'type'>
  >>> foo.__class__.__class__
  <type 'type'>
  >>> b.__class__.__class__
  <type 'type'>

So, a metaclass is just the stuff that creates class objects.

You can call it a 'class factory' if you wish.

type is the built-in metaclass Python uses, but of course, you can create your own metaclass.

The __metaclass__ attribute

You can add a __metaclass__ attribute when you write a class:

class Foo(object):
  __metaclass__ = something...
  [...]

If you do so, Python will use the metaclass to create the class Foo.

Careful, it's tricky.

You write class Foo(object) first, but the class object Foo is not created in memory yet.

Python will look for __metaclass__ in the class definition. If it finds it, it will use it to create the object class Foo. If it doesn't, it will use type to create the class.

Read that several times.

When you do:

class Foo(Bar):
  pass

Python does the following:

Is there a __metaclass__ attribute in Foo?

If yes, create in memory a class object (I said a class object, stay with me here), with the name Foo by using what is in __metaclass__.

If Python can't find __metaclass__, it will look for a __metaclass__ in Bar (the parent class), and try to do the same.

If Python can't find __metaclass__ in any parent, it will look for a __metaclass__ at the MODULE level, and try to do the same.

Then if it can't find any __metaclass__ at all, it will use type to create the class object.

Now the big question is, what can you put in __metaclass__ ?

The answer is: something that can create a class.

And what can create a class? type, or anything that subclasses or uses it.

Custom metaclasses

The main purpose of a metaclass is to change the class automatically, when it's created.

You usually do this for APIs, where you want to create classes matching the current context.

Imagine a stupid example, where you decide that all classes in your module should have their attributes written in uppercase. There are several ways to do this, but one way is to set __metaclass__ at the module level.

This way, all classes of this module will be created using this metaclass, and we just have to tell the metaclass to turn all attributes to uppercase.

Luckily, __metaclass__ can actually be any callable, it doesn't need to be a formal class (I know, something with 'class' in its name doesn't need to be a class, go figure... but it's helpful).

So we will start with a simple example, by using a function.

# the metaclass will automatically get passed the same argument
# that you usually pass to `type`
def upper_attr(future_class_name, future_class_parents, future_class_attr):
  """
    Return a class object, with the list of its attribute turned 
    into uppercase.
  """

  # pick up any attribute that doesn't start with '__'
  attrs = ((name, value) for name, value in future_class_attr.items() if not name.startswith('__'))
  # turn them into uppercase
  uppercase_attr = dict((name.upper(), value) for name, value in attrs)

  # let `type` do the class creation
  return type(future_class_name, future_class_parents, uppercase_attr)

__metaclass__ = upper_attr # this will affect all classes in the module

class Foo(): # global __metaclass__ won't work with "object" though
  # but we can define __metaclass__ here instead to affect only this class
  # and this will work with "object" children
  bar = 'bip'

print hasattr(Foo, 'bar')
# Out: False
print hasattr(Foo, 'BAR')
# Out: True

f = Foo()
print f.BAR
# Out: 'bip'

Now, let's do exactly the same, but using a real class for a metaclass:

# remember that `type` is actually a class like `str` and `int`
# so you can inherit from it
class UpperAttrMetaclass(type): 
    # __new__ is the method called before __init__
    # it's the method that creates the object and returns it
    # while __init__ just initializes the object passed as parameter
    # you rarely use __new__, except when you want to control how the object
    # is created.
    # here the created object is the class, and we want to customize it
    # so we override __new__
    # you can do some stuff in __init__ too if you wish
    # some advanced use involves overriding __call__ as well, but we won't
    # see this
    def __new__(upperattr_metaclass, future_class_name, 
                future_class_parents, future_class_attr):

        attrs = ((name, value) for name, value in future_class_attr.items() if not name.startswith('__'))
        uppercase_attr = dict((name.upper(), value) for name, value in attrs)

        return type(future_class_name, future_class_parents, uppercase_attr)

But this is not really OOP. We call type directly and we don't override call the parent __new__. Let's do it:

class UpperAttrMetaclass(type): 

    def __new__(upperattr_metaclass, future_class_name, 
                future_class_parents, future_class_attr):

        attrs = ((name, value) for name, value in future_class_attr.items() if not name.startswith('__'))
        uppercase_attr = dict((name.upper(), value) for name, value in attrs)

        # reuse the type.__new__ method
        # this is basic OOP, nothing magic in there
        return type.__new__(upperattr_metaclass, future_class_name, 
                            future_class_parents, uppercase_attr)

You may have noticed the extra argument upperattr_metaclass. There is nothing special about it: a method always receives the current instance as first parameter. Just like you have self for ordinary methods.

Of course, the names I used here are long for the sake of clarity, but like for self, all the arguments have conventional names. So a real production metaclass would look like this:

class UpperAttrMetaclass(type): 

    def __new__(cls, name, bases, dct):

        attrs = ((name, value) for name, value in dct.items() if not name.startswith('__'))
        uppercase_attr = dict((name.upper(), value) for name, value in attrs)

        return type.__new__(cls, name, bases, uppercase_attr)

We can make it even cleaner by using super, which will ease inheritance (because yes, you can have metaclasses, inheriting from metaclasses, inheriting from type):

class UpperAttrMetaclass(type): 

    def __new__(cls, name, bases, dct):

        attrs = ((name, value) for name, value in dct.items() if not name.startswith('__'))
        uppercase_attr = dict((name.upper(), value) for name, value in attrs)

        return super(UpperAttrMetaclass, cls).__new__(cls, name, bases, uppercase_attr)

That's it. There is really nothing more about metaclasses.

The reason behind the complexity of the code using metaclasses is not because of metaclasses, it's because you usually use metaclasses to do twisted stuff relying on introspection, manipulating inheritance, vars such as __dict__, etc.

Indeed, metaclasses are especially useful to do black magic, and therefore complicated stuff. But by themselves, they are simple:

  • intercept a class creation
  • modify the class
  • return the modified class

Why would you use metaclasses classes instead of functions?

Since __metaclass__ can accept any callable, why would you use a class since it's obviously more complicated?

There are several reasons to do so:

  • The intention is clear. When you read UpperAttrMetaclass(type), you know what's going to follow
  • You can use OOP. Metaclass can inherit from metaclass, override parent methods. Metaclasses can even use metaclasses.
  • You can structure your code better. You never use metaclasses for something as trivial as the above example. It's usually for something complicated. Having the ability to make several methods and group them in one class is very useful to make the code easier to read.
  • You can hook on __new__, __init__ and __call__. Which will allow you to do different stuff. Even if usually you can do it all in __new__, some people are just more comfortable using __init__.
  • These are called metaclasses, damn it! It must mean something!

Why the hell would you use metaclasses?

Now the big question. Why would you use some obscure error prone feature?

Well, usually you don't:

Metaclasses are deeper magic than 99% of users should ever worry about. If you wonder whether you need them, you don't (the people who actually need them know with certainty that they need them, and don't need an explanation about why).

Python Guru Tim Peters

The main use case for a metaclass is creating an API. A typical example of this is the Django ORM.

It allows you to define something like this:

  class Person(models.Model):
    name = models.CharField(max_length=30)
    age = models.IntegerField()

But if you do this:

  guy = Person(name='bob', age='35')
  print guy.age

It won't return an IntegerField object. It will return an int, and can even take it directly from the database.

This is possible because models.Model defines __metaclass__ and it uses some magic that will turn the Person you just defined with simple statements into a complex hook to a database field.

Django makes something complex look simple by exposing a simple API and using metaclasses, recreating code from this API to do the real job behind the scenes.

The last word

First, you know that classes are objects that can create instances.

Well in fact, classes are themselves instances. Of metaclasses.

  >>> class Foo(object): pass
  >>> id(Foo)
  142630324

Everything is an object in Python, and they are all either instances of classes or instances of metaclasses.

Except for type.

type is actually its own metaclass. This is not something you could reproduce in pure Python, and is done by cheating a little bit at the implementation level.

Secondly, metaclasses are complicated. You may not want to use them for very simple class alterations. You can change classes by using two different techniques:

  • monkey patching
  • class decorators

99% of the time you need class alteration, you are better off using these.

But 99% of the time, you don't need class alteration at all :-)

568
kunj2aan

I am learning GoF Java Design Patterns and I want to see some real life examples of them. Can you guys point to some good usage of these Design Patterns, preferably in Java's core libraries?

Thank you!

Answered By: BalusC ( 1309)

You can find an overview of a lot design patterns in Wikipedia. It also mentions which patterns are mentioned by GoF. I'll sum them up here and try to assign as much as possible pattern implementations found in both the Java SE and Java EE API's.


Creational patterns

Abstract factory (recognizeable by creational methods returning the factory itself which in turn can be used to create another abstract/interface type)

Builder (recognizeable by creational methods returning the instance itself)

Factory method (recognizeable by creational methods returning an implementation of an abstract/interface type)

Prototype (recognizeable by creational methods returning a different instance of itself with the same properties)

Singleton (recognizeable by creational methods returning the same instance (usually of itself) everytime)


Structural patterns

Adapter (recognizeable by creational methods taking an instance of different abstract/interface type and returning an implementation of own/another abstract/interface type which decorates/overrides the given instance)

Bridge (recognizeable by creational methods taking an instance of different abstract/interface type and returning an implementation of own abstract/interface type which delegates/uses the given instance)

  • None comes to mind yet. A fictive example would be new LinkedHashMap(LinkedHashSet<K>, List<V>) which returns an unmodifiable linked map which doesn't clone the items, but uses them. The java.util.Collections#newSetFromMap() and singletonXXX() methods however comes close.

Composite (recognizeable by behavioral methods taking an instance of same abstract/interface type into a tree structure)

Decorator (recognizeable by creational methods taking an instance of same abstract/interface type which adds additional behaviour)

Facade (recognizeable by behavioral methods which internally uses instances of different independent abstract/interface types)

Flyweight (recognizeable by creational methods returning a cached instance, a bit the "multiton" idea)

Proxy (recognizeable by creational methods which returns an implementation of given abstract/interface type which in turn delegates/uses a different implementation of given abstract/interface type)

The Wikipedia example is IMHO a bit poor, lazy loading has actually completely nothing to do with the proxy pattern at all.


Behavioral patterns

Chain of responsibility (recognizeable by behavioral methods which (indirectly) invokes the same method in another implementation of same abstract/interface type in a queue)

Command (recognizeable by behavioral methods in an abstract/interface type which invokes a method in an implementation of a different abstract/interface type which has been encapsulated by the command implementation during its creation)

Interpreter (recognizeable by behavioral methods returning a structurally different instance/type of the given instance/type; note that parsing/formatting is not part of the pattern, determining the pattern and how to apply it is)

Iterator (recognizeable by behavioral methods sequentially returning instances of a different type from a queue)

Mediator (recognizeable by behavioral methods taking an instance of different abstract/interface type (usually using the command pattern) which delegates/uses the given instance)

Memento (recognizeable by behavioral methods which internally changes the state of the whole instance)

Observer (or Publish/Subscribe) (recognizeable by behavioral methods which invokes a method on an instance of another abstract/interface type, depending on own state)

State (recognizeable by behavioral methods which changes its behaviour depending on the instance's state which can be controlled externally)

Strategy (recognizeable by behavioral methods in an abstract/interface type which invokes a method in an implementation of a different abstract/interface type which has been passed-in as method argument into the strategy implementation)

Template method (recognizeable by behavioral methods which already have a "default" behaviour definied by an abstract type)

Visitor (recognizeable by two different abstract/interface types which has methods definied which takes each the other abstract/interface type; the one actually calls the method of the other and the other executes the desired strategy on it)

396
Mike Stone

Ok, I may resort to a tad ranting here, so let me apologize in advance, but I'm really curious if others find this pattern annoying too (and I wonder if it is a justifiable pattern)…

So, after just looking at a particular question, I noticed that almost all of the responses suggested creating an interface for injecting a mock in some test code.

I don't mind using interfaces, and sometimes they can really help in static typed languages like C# and Java… but I do mind seeing interfaces for almost every class in a system(or in general being used where they aren't really needed).

I have 2 major problems with using an interface when it isn't called for:

  • You abstract away where the implementation is coming from. This problem has a couple consequences… in an IDE, it means that when I try to browse to the source of this method being called… I get taken to an interface instead of some code that I can look at and see what is going on. This bothers me a lot, but also this is a real problem to me to hide where the implementation is coming from (sometimes it can be in non-obvious locations).
  • It adds ANOTHER file to the system. I tend to be a minimalist in my programming… if I don't really need another method, or another class, or even another file… not unless that extra thing is justified (flexibility that is going to be used, or makes the design cleaner, or provides some real benefit).

Now… if you are testing something, and you create an interface JUST TO ALLOW MOCKING… this seems to be adding a layer of minor headaches for no real benefit. What does creating the interface do that just overriding the class won't do? What is so bad about having a mock that merely overrides some methods of the single implementation class?

I guess it should be no surprise then that I much prefer Java's default virtual methods (ie requiring a final keyword to have a method that CAN'T be overriden) to C#'s default final methods… and I also tend to avoid the final keyword on methods and classes too.

So is there something to using interfaces that I am missing? Is there some hidden benefit of using an interface when you have 1 version of a class and no immediate need to create an interface?

Answered By: Hallgrim ( 160)

A big headache with interfaces in Java and C# is that they give you so few options to evolve your contract. E.g.:

interface ILogSink {
  Log(DateTime timestamp, string message);
}

If you define this in a framework and have a lot of different external consumers that all implement this interface you cannot change it without breaking their implementation. If you need to change it in the next version, you end up with something like:

interface ILogSinkVersion2 {
  Log(DateTime timestamp, string message, CultureInfo culture);
}

If you use a base class on the other hand, you can start out with:

class LogSink {
  void Log(DateTime timestamp, string message);
}

And evolve it to:

class LogSink {
  [Obsolete("Please include CultureInfo using Log(DateTime, string, CultureInfo)")]
  virtual void Log(DateTime timestamp, string message) {
  }

  virtual void Log(DateTime timestamp, string message, CultureInfo culture) {
    // Call old method for old implementations:
    Log(timestamp, message);
  }
}

This way you can evolve your code without forcing every implementation of the contract to consume your changes immediately.

Update: This is not a big deal if you sit on your own island and own all your code. However if you maintain a framework interface like log4net's ILog interface or .NET frameworks IDisposable interface, you need a quite good reason to add or change methods, since it will affect a few thousand implementations throughout the world.

Since I started learning F# and OCaml last year, I've read a huge number of articles which insist that design patterns (especially in Java) are workarounds for the missing features in imperative languages. One article I found makes a fairly strong claim:

Most people I've met have read the Design Patterns book by the Gang of Four. Any self respecting programmer will tell you that the book is language agnostic and the patterns apply to software engineering in general, regardless of which language you use. This is a noble claim. Unfortunately it is far removed from the truth.

Functional languages are extremely expressive. In a functional language one does not need design patterns because the language is likely so high level, you end up programming in concepts that eliminate design patterns all together.

The main features of functional programming include functions as first-class values, currying, immutable values, etc. It doesn't seem obvious to me that OO design patterns are approximating any of those features.

Additionally, in functional languages which support OOP (such as F# and OCaml), it seems obvious to me that programmers using these languages would use the same design patterns found available to every other OOP language. In fact, right now I use F# and OCaml everyday, and there are no striking differences between the patterns I use in these languages vs the patterns I use when I write in Java.

Is there any truth to the claim that functional programming eliminates the need for OOP design patterns? If so, could you post or link to an example of a typical OOP design pattern and its functional equivalent?

Answered By: jalf ( 376)

The blog post you quoted overstates its claim a bit. FP doesn't eliminate the need for design patterns. The term "design patterns" just isn't widely used to describe the same thing in FP languages. But they exist. Functional languages have plenty of best practice rules of the form "when you encounter problem X, use code that looks like Y", which is basically what a design pattern is.

However, it's correct that most OOP-specific design patterns are pretty much irrelevant in functional languages.

I don't think it should be particularly controversial to say that design patterns in general only exist to patch up shortcomings in the language. And if another language can solve the same problem trivially, that other language won't have need of a design pattern for it. Users of that language may not even be aware that the problem exists, because, well, it's not a problem in that language.

The main features of functional programming include functions as first-class values, currying, immutable values, etc. It doesn't seem obvious to me that OO design patterns are approximating any of those features.

What is the command pattern, if not an approximation of first-class functions? :) In a FP language, you'd simply pass a function as the argument to another function. In an OOP language, you have to wrap up the function in a class, which you can instantiate and then pass that object to the other function. The effect is the same, but in OOP it's called a design pattern, and it takes a whole lot more code. And what is the abstract factory pattern, if not currying? Pass parameters to a function a bit at a time, to configure what kind of value it spits out when you finally call it.

So yes, several GoF design patterns are rendered redundant in FP languages, because more powerful and easier to use alternatives exist.

But of course there are still design patterns which are not solved by FP languages. What is the FP equivalent of a singleton? (Disregarding for a moment that singletons are generally a terrible pattern to use)

And it works both ways too. As I said, FP has its design patterns too, people just don't usually think of them as such.

But you may have run across monads. What are they, if not a design pattern for "dealing with global state"? That's a problem that's so simple in OOP languages that no equivalent design pattern exists there.

We don't need a design pattern for "increment a static variable", or "read from that socket", because it's just what you do.

In (pure) functional languages, side effects and mutable state are impossible, unless you work around it with the monad "design pattern", or any of the other methods for allowing the same thing.

Additionally, in functional languages which support OOP (such as F# and OCaml), it seems obvious to me that programmers using these languages would use the same design patterns found available to every other OOP language. In fact, right now I use F# and OCaml everyday, and there are no striking differences between the patterns I use in these languages vs the patterns I use when I write in Java.

Perhaps because you're still thinking imperatively? A lot of people, after dealing with imperative languages all their lives, have a hard time giving up on that habit when they try a functional language. (I've seen some pretty funny attempts at F#, where literally every function was just a string of 'let' statements, basically as if you'd taken a C program, and replaced all semicolons with 'let'. :))

But another possibility might be that you just haven't realized that you're solving problems trivially which would require design patterns in an OOP language.

When you use currying, or pass a function as an argument to another, stop and think about how you'd do that in an OOP language.

Is there any truth to the claim that functional programming eliminates the need for OOP design patterns?

Yep. :) When you work in a FP language, you no longer need the OOP-specific design patterns. But you still need some general design patterns, like MVC or other non-OOP specific stuff, and you need a couple of new FP-specific "design patterns" instead. All languages have their shortcomings, and design patterns are usually how we work around them.

Anyway, you may find it interesting to try your hand at "cleaner" FP languages, like ML (my personal favorite, at least for learning purposes), or Haskell, where you don't have the OOP crutch to fall back on when you're faced with something new.

Edit: As expected, a few people objected to my definition of design patterns as "patching up shortcomings in a language", so here's my justification: As already said, most design patterns are specific to one programming paradigm, or sometimes even one specific language. Often, they solve problems that only exist in that paradigm (See monads for FP, or abstract factories for OOP). Why doesn't the abstract factory pattern exist in FP? Because the problem it tries to solve does not exist there. So, if a problem exists in OOP languages, which does not exist in FP languages, then clearly that is a shortcoming of OOP languages. The problem can be solved, but your language does not do so, but requires a bunch of boilerplate code from you to work around it. Ideally, we'd like our programming language to magically make all problems go away. Any problem that is still there is in principle a shortcoming of the language. ;)

When should I use an interface and when should I use a base class?

Should it always be an interface if I don't want to actually define a base implementation of the methods?

If I have a Dog and Cat class. Why would I want to implement IPet instead of PetBase? I can understand having interfaces for ISheds or IBarks (IMakesNoise?), because those can be placed on a pet by pet basis, but I don't understand which to use for a generic Pet.

Answered By: Jon Limjap ( 199)

Let's take your example of a Dog and a Cat class, and let's illustrate using C#:

Both a dog and a cat are animals, specifically, quadruped mammals (animals are waaay too general). Let us assume that you have an abstract class Mammal, for both of them:

public abstract class Mammal

This base class will probably have default methods such as:

  • Hunt
  • Feed
  • Mate

All of which are behavior that have more or less the same implementation between either species. To define this you will have:

public class Dog : Mammal
public class Cat : Mammal

Now let's suppose there are other mammals, which we will usually see in a zoo:

public class Giraffe : Mammal
public class Rhinoceros : Mammal
public class Hippopotamus : Mammal

This will still be valid because at the core of the functionality, Hunt(), Feed() and Mate() will still be the same.

However, giraffes, rhinoceros, and hippos are not exactly animals that you can make pets out of. That's where an interface will be useful:

public interface IPettable
{
    IList<Trick> Tricks{get; set;}
    void Bathe();
    void Train(Trick t);
}

The implementation for the above contract will not be the same between a cat and dog; putting their implementations in an abstract class to inherit will be a bad idea.

Your Dog and Cat definitions should now look like:

public class Dog : Mammal, IPettable
public class Cat : Mammal, IPettable

Theoretically you can override them from a higher base class, but essentially an interface allows you to add on only the things you need into a class without the need for inheritance.

Consequently, because you can usually only inherit from one abstract class (in most statically typed OO languages that is... exceptions include C++) but be able to implement multiple interfaces, it allows you to construct objects in a strictly as required basis.

Why prefer composition over inheritance? What trade-offs are there for each approach? When should you choose inheritance over composition?

Answered By: Gishu ( 316)

Prefer composition over inheritance as it is more malleable / easy to modify later, but do not use a compose-always approach. With composition, it's easy to change behavior on the fly with Dependency Injection / Setters. Inheritance is more rigid as most languages do not allow you to derive from more than one type.. So the goose is more or less cooked once you derive from Class A.
My acid test for the above is:

  • Does TypeB want to expose the complete interface (all public methods no less) of TypeA such that TypeB can be used where TypeA is expected? Indicates Inheritance.

e.g. A Cessna biplane will expose the complete interface of an airplane, if not more. So that makes it fit to derive from Airplane.

  • Does TypeB only want only some/part of the behavior exposed by TypeA? Indicates need for Composition.

e.g. A Bird may need only the fly behavior of an Airplane. In this case, it makes sense to extract it out as an interface / class / both and make it a member of both classes.

Update: Just came back to my answer and it seems now that it is incomplete without a specific mention of Barbara Liskov's Liskov Substitution Principle as a test for 'Should I be inheriting from this type?'

243
Dean J

What's the advantage of using getters and setters - that only get and set - instead of simply using public fields for those variables?

If getters and setters are ever doing more than just the simple get/set, I can figure this one out very quickly, but I'm not 100% clear on how:

public String foo;

is any worse than:

private String foo;
public void setFoo(String foo) { this.foo = foo; }
public String getFoo() { return foo; }

Whereas the former takes a lot less boilerplate code.


Compiling the list up here at the top of what seemed winners to me, from the viewpoint of a Java web dev:

  1. When you realize you need to do more than just set and get the value, you don't have to change every file in the codebase.
  2. You can perform validation here.
  3. You can change the value being set.
  4. You can hide the internal representation. getAddress() could actually be getting several fields for you.
  5. You've insulated your public interface from changes under the sheets.
  6. Some libraries expect this. Reflection, serialization, mock objects.
  7. Inheriting this class, you can override default functionality.
  8. You can have different access levels for getter and setter.
  9. Lazy loading.
  10. People can easily tell you didn't use Python.
Answered By: LBushkin ( 161)

There are actually many good reasons to consider using accessors rather than directly exposing fields of a class - beyond just the argument of encapsulation and making future changes easier.

Here are the some of the reasons I am aware of:

  • Encapsulation of behavior associated with getting or setting the property - this allows additional functionality (like validation) to be added more easily later.
  • Hiding the internal representation of the property while exposing a property using an alternative representation.
  • Insulating your public interface from change - allowing the public interface to remain constant while the implementation changes without effecting existing consumers.
  • Controlling the lifetime and memory management (disposal) semantics of the property - particularly important in non-managed memory environments (like C++ or Objective-C).
  • Providing a debugging interception point for when a property changes at runtime - debugging when and where a property changed to a particular value can be quite difficult without this in some languages.
  • Improved interoperability with libraries that are designed to operate against property getter/setters - Mocking, Serialization, and WPF come to mind.
  • Allowing inheritors to change the semantics of how the property behaves and is exposed by overriding the getter/setter methods.
  • Allowing the getter/setter to be passed around as lambda expressions rather than values.
  • Getters and setters can allow different access levels - for example the get may be public, but the set could be protected.

In terms that an OOP programmer would understand (without any functional programming background), what is a monad?

What problem does it solve and what are the most common places it's used?

EDIT:

To clarify the kind of understanding I was looking for, let's say you were converting an FP application that had monads into an OOP application. What would you do to port the responsibilities of the monads to the OOP app?

Answered By: Eric Lippert ( 245)

In terms that an OOP programmer would understand (without any functional programming background), what is a monad?

A monad is an "amplifier" of types that obeys certain rules and which has certain operations provided.

First, what is an "amplifier of types"? By that I mean some system which lets you take a type and turn it into a more special type. For example, in C# consider Nullable<T>. This is an amplifier of types. It lets you take a type, say int, and add a new capability to that type, namely, that now it can be null when it couldn't before.

As a second example, consider IEnumerable<T>. It is an amplifier of types. It lets you take a type, say, string, and add a new capability to that type, namely, that you can now make a sequence of strings out of any number of single strings.

What are the "certain rules"? Briefly, that there is a sensible way for functions on the underlying type to work on the amplified type such that they follow the normal rules of functional composition. For example, if you have a function on integers, say

int M(int x) { return x + N(x * 2); }

then the corresponding function on Nullable<int> can make all the operators and calls in there work together "in the same way" that they did before.

(That is incredibly vague and imprecise; you asked for an explanation that didn't assume anything about knowledge of functional composition.)

What are the "operations"?

  1. That there is a way to take a value of an unamplified type and turn it into a value of the amplified type.
  2. That there is a way to transform operations on the unamplified type into operations on the amplified type that obeys the rules of functional composition mentioned before
  3. That there is usually a way to get the unamplified type back out of the amplified type. (This last point isn't strictly necessary for a monad but it is frequently the case that such an operation exists.)

Again, take Nullable<T> as an example. You can turn an int into a Nullable<int> with the constructor. The C# compiler takes care of most nullable "lifting" for you, but if it didn't, the lifting transformation is straightforward: an operation, say,

int M(int x) { whatever }

is transformed into

Nullable<int> M(Nullable<int> x) 
{ 
    if (x == null) 
        return null; 
    else 
        return new Nullable<int>(whatever);
}

And turning a nullable int back into an int is done with the Value property.

It's the function transformation that is the key bit. Notice how the actual semantics of the nullable operation -- that an operation on a null propagates the null -- is captured in the transformation. We can generalize this. Suppose you have a function from int to int, like our original M. You can easily make that into a function that takes an int and returns a Nullable<int> because you can just run the result through the nullable constructor. Now suppose you have this higher-order method:

Nullable<T> Bind<T>(Nullable<T> amplified, Func<T, Nullable<T>> func)
{
    if (amplified == null) 
        return null;
    else
        return func(amplified.Value);
}

See what you can do with that? Any method that takes an int and returns an int, or takes an int and returns a nullable int can now have the nullable semantics applied to it.

Furthermore: suppose you have two methods

Nullable<int> X(int q) { ... }
Nullable<int> Y(int r) { ... }

and you want to compose them:

Nullable<int> Z(int s) { return X(Y(s)); }

That is, Z is the composition of X and Y. But you cannot do that because X takes an int, and Y returns a nullable int. But since you have the "bind" operation, you can make this work:

Nullable<int> Z(int s) { return Bind(Y(s), X); }

The bind operation on a monad is what makes composition of functions on amplified types work. The "rules" I handwaved about above are that the monad preserves the rules of normal function composition; that composing with identity functions results in the original function, that composition is associative, and so on.

In C#, "Bind" is called "SelectMany". Take a look at how it works on the sequence monad. We need to have three things: turn a value into a sequence, turn a sequence into a value, and bind operations on sequences. Those operations are:

IEnumerable<T> MakeSequence<T>(T item)
{
    yield return item;
}
T Single<T>(IEnumerable<T> sequence)
{
    // let's just take the first one
    foreach(T item in sequence) return item; 
}
IEnumerable<T> SelectMany<T>(IEnumerable<T> seq, Func<T, IEnumerable<T>> func)
{
    foreach(T item in seq)
        foreach(T result in func(item))
            yield return result;            
}

The nullable monad rule was "to combine two functions that produce nullables together, check to see if the inner one results in null; if it does, produce null, if it does not, then call the outer one with the result". That's the desired semantics of nullable. The sequence monad rule is "to combine two functions that produce sequences together, apply the outer function to every element produced by the inner function, and then concatenate all the resulting sequences together". The fundamental semantics of the monads are captured in the Bind/SelectMany methods; this is the method that tells you what the monad really means.

We can do even better. Suppose you have a sequences of ints, and a method that takes ints and results in sequences of strings. We could generalize the binding operation to allow composition of functions that take and return different amplified types, so long as the inputs of one match the outputs of the other:

IEnumerable<U> SelectMany<T,U>(IEnumerable<T> seq, Func<T, IEnumerable<U>> func)
{
    foreach(T item in seq)
        foreach(U result in func(item))
            yield return result;            
}

So now we can say "amplify this bunch of individual integers into a sequence of integers. Transform this particular integer into a bunch of strings, amplified to a sequence of strings. Now put both operations together: amplify this bunch of integers into the concatenation of all the sequences of strings." Monads allow you to compose your amplifications.

What problem does it solve and what are the most common places it's used?

That's rather like asking "what problems does the singleton pattern solve?", but I'll give it a shot.

Monads are typically used to solve problems like:

  • I need to make new capabilities for this type and still combine old functions on this type to use the new capabilities.
  • I need to capture a bunch of operations on types and represent those operations as composable objects, building up larger and larger compositions until I have just the right series of operations represented, and then I need to start getting results out of the thing
  • I need to represent side-effecting operations cleanly in a language that hates side effects

(Note that these are basically three ways of saying the same thing.)

C# uses monads in its design. As already mentioned, the nullable pattern is highly akin to the "maybe monad". LINQ is entirely built out of monads; the "SelectMany" method is what does the semantic work of composition of operations. (Erik Meijer is fond of pointing out that every LINQ function could actually be implemented by SelectMany; everything else is just a convenience.)

To clarify the kind of understanding I was looking for, let's say you were converting an FP application that had monads into an OOP application. What would you do to port the responsibilities of the monads into the OOP app?

Most OOP languages do not have a rich enough type system to represent the monad pattern itself directly; you need a type system that supports types that are higher types than generic types. So I wouldn't try to do that. Rather, I would implement generic types that represent each monad, and implement methods that represent the three operations you need: turning a value into an amplified value, turning an amplified value into a value, and transforming a function on unamplified values into a function on amplified values.

A good place to start is how we implemented LINQ in C#. Study the SelectMany method; it is the key to understanding how the sequence monad works in C#. It is a very simple method, but very powerful!

For a more in-depth and theoretically sound explanation of monads in C#, I highly recommend my colleague Wes Dyer's article on the subject. This article is what explained monads to me when they finally "clicked" for me.

http://blogs.msdn.com/wesdyer/archive/2008/01/11/the-marvels-of-monads.aspx

I am a long-time Applescript user and new shell scripter who wants to learn a more general scripting language like Javascript or Python for performance reasons.

I am having trouble getting my head around concepts like object orientation, classes and instantiation.

If someone could point me to a pithy explanation of methods vs. functions it might help me get over the "hump". The explanations I found using google are just barely over my head.

Thanks.

Answered By: Andrew Edgecombe ( 163)

A function is a piece of code that is called by name. It can be passed data to operate on (ie. the parameters) and can optionally return data (the return value).

All data that is passed to a function is explicitly passed.

A method is a piece of code that is called by name that is associated with an object. In most respects it is identical to a function except for two key differences.

  1. It is implicitly passed the object for which it was called
  2. It is able to operate on data that is contained within the class (remembering that an object is an instance of a class - the class is the definition, the object is an instance of that data)

(this is a simplified explanation, ignoring issues of scope etc.)

I have had recently two telephone interviews where I've been asked about the differences between an Interface and an Abstract class. I have explained every aspect of them I could think of, but it seems they are waiting for me to mention something specific, and I don't know what it is.

From my experience I think the following is true. If I am missing a major point please let me know.

Interface:

Every single Method declared in an Interface will have to be implemented in the subclass. Only Events, Delegates, Properties (C#) and Methods can exist in a Interface. A class can implement multiple Interfaces.

Abstract Class:

Only Abstract methods have to be implemented by the subclass. An Abstract class can have normal methods with implementations. Abstract class can also have class variables beside Events, Delegates, Properties and Methods. A class can only implement one abstract class only due non-existence of Multi-inheritance in C#.

  1. After all that, the interviewer came up with the question "What if you had an Abstract class with only abstract methods? How would that be different from an interface?" I didn't know the answer but I think it's the inheritance as mentioned above right?

  2. An another interviewer asked me what if you had a Public variable inside the interface, how would that be different than in Abstract Class? I insisted you can't have a public variable inside an interface. I didn't know what he wanted to hear but he wasn't satisfied either.

See Also:

Answered By: Michael Burr ( 87)

While your question indicates it's for "general OO", it really seems to be focusing on .NET use of these terms.

In .NET (similar for Java):

  • interfaces can have no state or implementation
  • a class that implements an interface must provide an implementation of all the methods of that interface
  • abstract classes may contain state (data members) and/or implementation (methods)
  • abstract classes can be inherited without implementing the abstract methods (though such a derived class is abstract itslef)
  • interfaces may be multiple-inherited, abstract classes may not (this is probably the key concrete reason for interfaces to exist separately from abtract classes - they permit an implementation of multiple inheritance that removes many of the problems of general MI).

As general OO terms, the differences are not necessarily well-defined. For example, there are C++ programmers who may hold similar rigid definitions (interfaces are a strict subset of abstract classes that cannot contain implementation), while some may say that an abstract class with some default implementations is still an interface or that a non-abstract class can still define an interface.

Indeed, there is a C++ idiom called the Non-Virtual Interface (NVI) where the public methods are non-virtual methods that 'thunk' to private virtual methods: