About complexity

This picture is from a presentation from Anton Böhm of IT Serve, given at a guest lecture at SWS.

I think this picture can be interpreted in many ways. The thought that jumped into my head, was the following:

For a long time software development fought with performance. Performance was the limiting factor for a lot of software systems. The picture shows that complexity of IT-systems is growing much faster than performance. So nowadays the limiting factor for the development of an IT-system is the complexity and not performance any more.

So managing complexity should become the primary focus of software development. There is a need for concepts and technologies/tools/frameworks that help us to get a grip on complexity-management. I think Object-Relational-Mapping, Domain Driven Design, Dependency Injection and Aspect Oriented Programming are steps in this direction. These concepts battle complexity by applying Separation of Concerns, intuitive Modeling and abstraction.

Another trend is Model Driven Architecture and Model Driven Design. I am not yet convinced by these methodologies, since in my opinion they rely too much on tools, which leads just to another dimension of complexity… but I am looking forward to get enlighted ;-)

Classic Mistakes in Software Development

Pause for a moment and look at this List. Can you honestly say that your project is not affected by any of these? Yes? Then hire me, here is my CV ;-)

Early Access Program: NHibernate in Action ... and some DataSet-bashing for my part.

According to Amazon NHibernate in Action is going to be released in July. I am looking forward to get a copy.

But you can already start reading the book today through the Manning Early Access Program.

The first chapter is free. Chapters 2–4 are available if you are willing to pay for the Early Access Program.

If OR-Mapping is not completely new territory for you, the first chapter does not offer much. It is basically an introduction to OR-Mapping and the concepts around OR-Mapping (Domain-Model, Layered-Architecture, some Patterns and the Impedance Mismatch). You can find the same information in several other books like Java Persistence with Hibernate, Hibernate in Action, Applying Domain-Driven Design and Patterns and POJOs in Action.

However there is one interesting difference to the books above: It discusses OR-Mapping as alternative to DataSets.

The book points out, that DataSets are a very efficient way to implement an initial Data-Access-Layer and for displaying (tabular) data in a GUI. But DataSets do heavily restrict your freedom in modeling the business domain and applying OO-Analysis and OO-Design.

These summarizes the feeling I have about datasets. I hope I will find the time to explore the reasons behind this, and write a post about it.

But for now lets look at the good and the bad of DataSets

• good: helps with DB-access and Data binding -> these are both technical/infrastructure-concerns
• bad: limits modeling in your business logic -> for non-trivial applications this should be your main concern!

So if your application has no business logic, then Dataset are optimal: The GUI is basically directly bound to the database, although DataSets make it possible to have several layers/tears between database an GUI, you basically don't do anything of business value (security, distribution, transactions, basic validation do not count as business value they are technical/infrastructure-concerns).

But if you have non-trivial business logic, you have to decide where your priorities lie. Is it more important to easily solve the technical/infrastructural-concerns or is it more important to develop an elegant, maintainable and extensible business-model?
You don't have to answer right now ;-) but put into consideration that DB-access and Databinding without DataSets is not as hard as it used to be some years ago (thinking of NHibernate, Genome, OpenAccess and Object-DataBinding) … well, can you guess my answer to the above question? ;-)

I think DataSets are still very popular, because for non-experienced programmers they allow to quickly develop a vertical prototype for an application. The goal of the vertical prototype is often to prove that the technical concerns can be solved. So the prototype is mostly about infrastructure. The prototype seldom provides a lot of the business functionality (well after all its a prototype ;-), and it certainly does not have to manage complexity, maintainability and extensibility.

Problems arise only later when the application should evolve horizontally and provide the real business functionality. But at this point the architecture is often fixed, and the door is closed: Welcome in a world of pain!

So here my final blow for today: DataSets are misleading, they promise a multi layered/multi tiered architecture but basically the programming model is the same as in a client-server app with the business logic in the database. With DataSets the code is just not executed in the database but in some other layer/tier but even that is questionable a good thing…

 

 

Wisdom from the Master

From the article about Application Facades:

• “Don’t try to come up with a clever way to do something if a simple way works fine. So this approach has limitations. Always ask yourself if you can live with the limitations — if you can you should. You can always make it more complicated later.”
• “A simple testing framework makes life easier. As you go on you can add more tests, but you never throw tests away. Not unless you like driving at night with your lights off.”
• “I like it when bits of programs don’t know what other bits are doing.”

Excellent Article from Martin Fowler: Application Facades

There is a lot of good content up at Martin Fowler: Articles. (I definitely have to improve my reading speed …)

Today I read the Article about Application Facades. This is an excellent article!

The main goal of the article is to provide an example how to present a domain model in a GUI. Fowler uses an Application Facade to expose the domain model. The application facade then also defines a clear interface to test the business logic.

The article seems to be quite old. Fowler mentions problems with Café’s debugger (I vaguely remember something called Café, but I think I was playing SimCity in monochrome mode at that time…). Also Unit-Testing-Frameworks seemed not to exist at the time the article was written, since Fowler shows how to create a crude Framework by hand.

Nevertheless the article is very interesting and the Application Facade is a Pattern that can be used in modern business applications.

But the main value of the article is not the explanation of the pattern nor the code examples. What makes this article a real must read is how Fowler shows the process of developing a domain model. It is a real pleasure to see how the model is evolving in several iterations, and how OO-Techniques are applied to make the model more elegant. Fowler also shows the typical TDD-cycles of implementing-testing-extending-testing-refactoring… and it really makes sense!

I can really recommend this article for anybody who wants to see a beautiful example of DDD and applying OO-Techniques to a domain model.

Rory about the pseudo-geekness of 24!

This is funny:

Neopoleon : The Most Disgusting Thing in the Universe.

If you are in a hurry, just skip the first third of the post to where he starts about 24 and the things a geek just cant accept. He hits the nail on the head again!

I you want more and still have time, look at this.

Creativity and competent People

I listened to an interview with Eric Evans (Author of Domain Driven Design, see also DomainDrivenDesign.org). The Podcast with the interview is available from Parleys.

He brings up an interesting observation about how software is developed today:

• Object oriented software development is a creative process. Designing and modeling needs a lot of creativity. Only with creativity we can develop a model that really “crunches” the domain-problems in an elegant, maintainable and extensible way. This creative process is a heavy work and needs a lot of experimenting and freedom.
• A lot of companies/shops have the goal to establish a very rigid way how software is developed. This can be with a heavy, constricting framework and/or with a heavyweight process.
• The goal of establishing a rigidly standardized way of software development is to minimize risk: If everything is predefined nothing can be done wrong, if everything is done the same way, it is easier to maintain. The developer who uses (has to use) the framework should not have to nor be able to use creativity, because if he is creative ha can do something wrong.
• This kind of thinking also limits the potential of real good object oriented modeling: If the framework heavily constrains the creativity of the developers no real “domain-crunching” can be applied. As a consequence, the resulting model often is not optimal, clumsy or even wrong.

I'm currently experiencing these observations on the project I am working on.

I think if there are good people working with a heavily constraining framework, a lot of potential is wasted. This is also a big risk, because if the model is flawed, maintainability and extensibility will be suffering. Also if the framework does not provide (elegant) ways to do something, developers will have to “hack around” the framework. The result will certainly not be maintainable. So in these cases using a heavily constraining framework totally missed its goals!
On the other hand if the people are not good, the framework has to be very good, to guarantee that nothing can be done wrong (but everything necessary can be done easily). I don't think a framework like this can exist… so you need good people anyway.

Well, thinking about it: If your business-problem is not trivial, you need good people to come up with a good solution… so why constrain their ability to model the problem in an optimal way?

Eric makes another interesting observation in the interview:

• In many companies/shops the good people are working on developing the technology/infrastructure/framework, but are not really working on business-problems.
• This is a problem, because this leaves the mediocre people to solve the business-problems and to perform the “domain-crunching”. But finally this is the real value that is provided to the customer…

I think this is really a big problem today, and companies/shops that do not change already have or will soon have real problems. Management and developers have to realize that solving the concrete business problems is the first and most important challenge. Technical concerns should only be of relevance if they are a real obstacle to solve the business problems. But technical concerns should certainly not drive your development process/infrastructure! I even go a step further in arguing that technology is not an issue any more today for most business applications. There exist standardized and proven technologies for most technical problems in enterprise development. You just have to understand and use them, but you certainly don't have to reinvent and develop them!

And now for finishing, imagine the worst case scenario: The mediocre people develop the heavily constraining framework that has to be used by the good people… where does this lead?

Hello world!

From Anton Böhm of IT Serve, at a guest lecture at SWS:

• Life is more complex than Hello Wold!

About salary...

From Anton Böhm of IT Serve, at a guest lecture at SWS:

• High salaries are no long-term motivation.
• Low salaries are an obstacle.

Learning by...

I heard a new one lately:

“Learning by Suffering!”

The Zen Of Python

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

From Ayende.

Microsoft on the way to DDD?

I just listened to the newest episode of Dot Net Rocks!

Jesse Liberty and Alex Horovitz make some interesting statements at the beginning of the show:

• With .Net 3.0 it is now sometimes possible to start a project from the business-model and work up to the GUI and down to the DB
• On the .Net-Platform it should be easier for the developers to focus on he real business problems.
• Right now we are stuck with DataSets, which do not allow to implement business-logic in an intuitive way (think of: Customer.Address.PhoneNumber.AreaCode = 097; )
• Microsoft is working on technologies (e.g. LINQ) that will make it easier in the future to solve the real business problems.

Of course this is fuel for the flames of my current crusade against DataSets and for DDD!

My comments:

• Every project that has non-trivial business-logic should start from the business-model! Thats how domain-crunching is applied! Thats the only way all the insights from the OO-Analysis and Design can be applied. All the rest is infrastructure (persistence, distribution, deployment …) and nice wrapping (GUI).
• If even Microsoft is moving away from DataSets you should get rid of them as fast as possible! There are technologies available right now (NHibernate, Genome, OpenAccess, CSLA …), you dont have to wait for Microsoft!
• It seems that Microsoft is moving towards DDD and ORM. In Java EE this has been State of the Art for some time! I am glad Microsoft seems to accept, that it is the way to go.

Sunday Morning on InfoQ, Part 2

Well it is still Sunday morning, and the weather is still bad … so I am staying on InfoQ

I watched an interview with Jimmy Nillson on Domain Driven Design

The interview is about Domain Driven Design (DDD) (see also my presentation).
If you have read DDD of Eric Evans and ADDDAP of Jimmy Nillson, the Interview does not offer much interesting content.

However at the end of the interview Jimmy talks about MDA in relation to DDD. He states that in DDD the code is the model while in MDA the model defines the code. At the time being the two notions stand in conflict with each other. Jimmy states that this does not have to be, but with the current state of the tools and implementations of MDA (“executable UML”) the conflict exists.

This reflects also my current opinion. I basically think it is not a good idea to use code-generation in your business-logic. The business-logic is the the core of your application. It is the value that your application offers to the user, its where the knowledge of the domain is buried where and all the insights extracted from analysis and design are applied.

Code-generation cannot help you with creating this business-logic, because the knowhow and the insights about the domain cannot be generated automatically.

Code-generation can only help you with infrastructural concerns like persistence, gui-binding, distribution … but not with implementing the core business-logic.

So again it is important to apply strict Separation of Concerns, so that the core business-logic can be developed independently of infrastructural problems. The more the infrastructural problems leak into the business logic, the more moves the concrete implementation of the business-logic away from the core-domain. Then again people will think that code-generation solves their problem. But in reality this would only be fighting symptoms of a bad architecture and not seizing the root of the problem, which is leaking concerns.

Sunday Morning on InfoQ, Part 1

Its Sunday morning and the weather is really bad … so I went to InfoQ …

I found a presentation called The Principle of Agile Design by Bob Martin (“Uncle Bob” from Object Mentor and butUncleBob).

This is an excellent talk, really insightful but funny at the same time. I had to laugh out loud several times. Uncle Bob is is a amazing presenter, he manages to immediately get your attention, and makes sure that you stay tuned.

The talk is about agile design principles in OO-Programming.

The main point of the talk is the following: OO is about managing dependencies. Abstraction is a tool to manage dependencies.

In OO-Design high-level policy/components do not depend on low-level details/components. Abstraction of the low-level details removes the dependency from the high-level components. It is a characteristic of procedural programs that they do depend on low-level details.

Bob Martin explains these concepts with a very illustrative and funny example of a copy module, that copies chars from the keyboard to the printer. Later on the module has to be extended and begins to rot… Watch it, you will laugh…

Bob Martin then moves on to some class design principles every OO-designer should know about:

These are the SOLID-Principles:

• SRP: Single Responsibility Principle
• OCP: The Open/Close Principle
• LSP: Liskov Substitution Principle
• ISP: Interface Segregation Principle
• DIP: Dependency Inversion Principle

SRP: A class should have one, and only one, reason to change. If a class has many reason to change it becomes rigid, it becomes a dependency magnet. That means if you change any part of the system, you have to change the given class too. Strict Separation of Concerns and Decoupling helps to ensure the SRP. Technically this often means to split one class in several classes that have exactly one, well defined responsibility.

OCP: Modules should be open for extension, but closed for modification (Bertrand Meyer). Change should be applied with addition and not with modification.

Unfortunately the recording ends here suddenly, because the presentation shifted into “tutorial mode”, which could not be recorded anymore!?!

This is quite disappointment, but the talk is really worth to watch it anyway.

Handling a dead horse

[from hammets blog]

Dakota tribal wisdom says that when you discover you are riding a dead horse, the best strategy is to dismount. However, in business we often try other strategies with dead horses, including the following:

• Buying a stronger whip.
• Changing riders.
• Say things like, “This is the way we have always ridden this horse.”
• Appointing a committee to study the horse.
• Arranging to visit other sites to see how they ride dead horses.
• Increasing the standards to ride dead horses.
• Appointing a tiger team to revive the dead horse.
• Creating a training session to increase our riding ability.
• Comparing the state of dead horses in todays environment.
• Change the requirements declaring that “This horse is not dead.”
• Hire contractors to ride the dead horse.
• Harnessing several dead horses together for increased speed.
• Declaring that “No horse is too dead to beat.”
• Providing additional funding to increase the horse’s performance.
• Do a Cost Analysis study to see if contractors can ride it cheaper.
• Purchase a product to make dead horses run faster.
• Declare the horse is “better, faster and cheaper” dead.
• Form a quality circle to find uses for dead horses.
• Revisit the performance requirements for horses.
• Say this horse was procured with cost as an independent variable.
• Promote the dead horse to a supervisory position.

About DB-Design

Here just another insight from the Lookup Table Madness article:

Remember that the primary reason to use a database is not to "persist data"; that can be done more efficiently with file-based systems. The purpose of a relational database is to enforce the rules that govern how data is to be created, maintained and used; in other words the database enforces the rules that give the data meaning.Without those rules your data becomes a meaningless tangle of 1's and 0's. In database design, the first and foremost consideration should always be logical correctness; all other concerns are secondary.

I repeat:

In database design, the first and foremost consideration should always be logical correctness; all other concerns are secondary.

This means:

• Do not think about performance, when you design your DB-model. Changes to improve performance should be made  in a second iteration and consciously declared as deviation from the optimal model.
• Introduce constraints as early as possible. It may be tedious in the beginning to work with a highly constrained data-store, but it forces your application to behave correctly. This is according to the Fail Fast Principle. Introducing constraints late will certainly force you to test the whole application again and probably force you to rework a lot of the application. Also introducing new constraints on an existing set of data can be painful.

Generalization - Another Gap between DB- and OO-Models

I was just reading a moderately interesting article about MUCKs (Massively Unified Code-Key tables).

Here is the article:

Lookup Table Madness

(you have to register…)

However there are two interesting thoughts in the article which I want to repeat here.

The first point is a very general statement:

Be careful that you don't generalize your database into irrelevance, building in so much flexibility that your system degenerates into chaos (chaos being the ultimate in flexibility).

This statement is certainly valid for any part of a system not just for the database. Generalization is a two edged blade and it is the characteristic of a good design to apply as much generalization as possible while keeping the model still usable.

The second point elaborates on Generalization in respect to the differences between the OO-model and the DB-model:

Generalization is a good thing, in the right context, but it is a tool and should not be used indiscriminately. Generalization is something that can be very useful in the Object Oriented (OO) programming world where you are concerned with what things do, or how they act. If two things act the same way they can be brought together under a single class. Even if they differ significantly, they might be able to inherit their common characteristics from a parent class, relying on subclasses to provide their unique behaviors. In the world of relational database design we really don't care so much about how things act, we care about what they are. In that context, generalization is very seldom useful. In the world of OO programming generalization helps improve code reuse, modularity, and maintainability. By contrast, generalization in a database leads to ambiguity, or loss of meaning.

This once more shows that there are fundamental differences between concept of a relational data store and a object oriented application. If the business logic of the application is non-trivial, the behavior of our business entities is getting very important. Generalization then is a useful instrument to introduce abstractions and therefore to reduce complexity. In the Database on the other hand we are not interested in behavior, so in most cases we don't want abstraction!

This leads us back to the good old object-relational-mismatch. And for me its another sign, that you should definitely not build your business logic around a relational model. Because sooner or later you will have the need for abstraction and your relational model will not be able to deliver this abstraction, nor can it be changed, because it is tied to the DB-schema.

Personally I believe the only real solution is to design a Domain Model for the business logic separately from the DB-schema. Ideally the two tasks are performed by different persons/teams. When both models are completed OR-Mapping is applied to bridge the gap between them. Almost certainly some compromises on both models will be necessary to make the mapping feasible, but those compromises will be made very conscious, and the different purposes of the two models will be kept intact,

test2

Testing

Test-Entry