Illustrative Programming: I am a skeptic! (and I am not alone)

I have stated several times, that I don't believe in visual modeling and cannot share the enthusiasm that a certain branch of MDSD people have for visual modeling:

Visual modeling: Is it worth it? An example...

Musing about Code Generation, MDSD and DSLs

Now Martin Fowler is coining the Term "Illustrative Programming" in the context of Language Workbenches like IntentionalSoftware, which are tools to build "illustrative DSLs".

For a lot of people coming from the UML camp, the notion of a "DSL" is what Fowler is calling an "illustrative DSL" here. Visual Studio Domain-Specific Language Tools are a striking example of this.

In his article Martin Fowler quotes some opinions of Neal Ford about the concepts of Illustrative Programming:

I think these tools work best for lay people (thus, your link to LayProgrammers). However, in general, tools like this slow down experienced/power users. [...] I would much prefer a markup language I could use to directly define stuff, with macros, snippets, and all the other things I'm accustomed to as a developer.

As these tools grow, they get unwieldy (perhaps because they are ceasing to be domain specific enough?). [...] APIs in programming languages scale much better, with several orders of magnitude more density before they become hard to navigate.

All the best-practices and tools don't exist there: refactoring, levels of testing, etc. Also, you loose the connection to text, meaning that macro facilities either don't exist or complex one-offs. [...]

I share your bullishness around these types of tools, but they are a long time from being useful for full-bore Agile development. I hope they mature fast.

Reading these comments, I am having a deja-vu!

This is really expressing what I was feeling when I was involved in MDSD attempts in recent projects. I don't generally doubt the noble goals that the visual modeling camp believe in.

But the tools and environment are just not ready to provide value. At least not in that kind of mainstream real world enterprise projects I have been involved in.

Also in my experience, the target of illustrative programming is the developers, which is wrong!

As Fowler is pointing out, the goal of illustrative programming is to engage lay-programmers.
Developers need other tools than lay-programmers! So if there are no lay-programmers in your project, there is arguably need for illustrative programming...

This insight was always lacking when I was confronted with attempts at visual modeling.

It is a common mistake in a lot of projects to establish some kind of two-class society among the developers, in this case it could be tempting to degrade the second class to lay-programmers. But that is a sick environment to begin with (and the topic for another post)...

of cource you should follow me on twitter

Visual modeling: Is it worth it? An example...

The following link points to an interesting article (in German):

Groovy-DSLs mit Eclipse-GMF visualisieren

The article perfectly shows some of the issues I have with the current state of visual modeling and MDSD.

The first part of the article is about implementing a domain concept. This is directly connected to the business problem which should be solved by the software at hand.

The second part of the article is about making the solution of the first part accessible for visual modeling.

I did quite easily understand the first part. The second part however seems a lot like rocket science to me (or some secret ancient ritual). There are a lot of steps and different concepts involved (code, configuration, libraries, guis, wizards, repetitive reflection of the actual domain concept, mapping ...).

While I feel confident, that I learned something from the first part (creating a simple textual DSL with Groovy Builder), I do not have the feeling that I learned something useful from the second part, except that it is very complicated to use GMF...

In my opinion, as long as setting up (and maintaining) visual modeling for a certain concept is so much more complicated than the underlying concept itself, visual modeling hardly pays off!

You really have to evaluate costs versus benefits. And in my experience this is almost never done properly.
In the given article: What is the real benefit of visually modeling the process-chain?
Who is the target? Who will be using the visual modeling capabilities?
Are stake holders really able and willing to grasp the visual models? Or are you still going to communicate with them by Visio and Word?
If it is not for the stakeholders, do developers really like to work with the visual models, or is the simple textual DSL much more preferred by them (easier handling with current tools (refactoring, diffing, versioning ...))?

If you think about these questions, is the effort put into the infrastructure really worth it? The article says itself:

Indeed the learning curve [of GMF] is very steep. [...] To gain a better understanding there is often no other way than explore the GMF-Code with the help of a debugger.

This quote alone would rise a big questionmark if I were the project leader of a project that is actually supposed to provide business value...

In my opinion visual modeling is vastly overrated, and especially in the current state is hardly worth the effort.

In my experience any large-scale modeling efforts often turn very fast into a completely disconnected reality-loop. People are wasting a lot of effort just to conquer the accidental complexity that is introduced by the modeling requirement. The actual business requirement then comes only at second priority.

The metaphor that comes to my mind is this:
At the beginning you have a business goal, that is like the requirement to cross a river. But once you jumped into the river, you notice, that you vastly underestimated the current.
From now on you are fully occupied to keep your head over the water. Reaching the other side (not to mention at the specified spot) is of no priority at all any more.

follow me on twitter, I need some friends :-)

Raising the Level of Abstraction

Abstraction is a central concept in software development.
Abstraction is probably our only hope to conquer the ever rising complexity and requirements in current and future systems.

When designing a software system today often there are no physical constraints. The only constraints are intellectual: Is the design understandable, maintainable, what are the effects of future changes ...

Abstraction is supposed to be a tool to do the right choices when designing.
But what is abstraction really? How do you characterize it? How do you measure it?

There seem to be a lot of different notions about how to raise the abstraction level.

Some people talk about nonterminal symbols:

<integer> ::= ['-'] <digit> {<digit>}
<digit> ::= '0' | '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9'

Some people believe in pictures and visuals:

Some people see hope in reducing the signal-to-noise ratio of code:

[...] a lot of Java programs are the boiler plate of controlled structures with very little business logic. It's probably a four or five to one ratio between business logic and boiler plate control structures. So Ruby gives you that conciseness.
You are saying: "Here is the business logic, apply this business logic [...]

-- David Pollak, InfoQ

For me, the third approach is currently the most promising. That's where DDD tries to pull the lever. That's where concepts like the Ubiquitous Language and Domain Specific Languages try to provide value.


And finally remember Joel Spolsky's Law of Leaky Abstractions:

All non-trivial abstractions, to some degree, are leaky.

follow me on twitter, I need some friends :-)

Provocative: MDSD vs. Platform

Two controversial statements about MDSD from the 'Zühlke Late Afternoon Talk: Quo vadis software modeling?':

If you have a good platform then you don't need a modeling tool.

If you have a good modeling tool, your platform becomes irrelevant.

So where lies the truth? Obviously the boundary is flowing ...

I my opinion 'modeling' is a very broad concept, so I would not necessary agree that you can trade between modeling and platform.
But when you replace 'modeling' with 'code generation' I am pretty convinced, that both quotes apply, and they represent quite opposite approaches to software development.

As I stated in Musing about Code Generation, MDSD and DSLs, I think the current trends in enterprise platforms lead to the first statement.

In my opinion code generation is less and less needed on modern application platforms, and since MDSD currently is so tightly coupled with code generation (at least in my experience) I think MDSD in its current manifestation is getting more and more irrelevant.

I think MDSD should somehow better adjust to the notion that "the code is the model" and provide additional abstractions (for different intentions) on top of that.
I think the current DSL-movement is an effort in this direction.
Visualisation would be another promising aspect.

I know this is a provocative statement. Many MDSD evangelists are convinced by their approach... but I just have not seen the light until now, that does not mean that I am not open for an epiphany.
So please leave a comment...

follow me on twitter, I need some friends :-)

Musing about Code Generation, MDSD and DSLs

Disclaimer: I am a developer of business applications. The following view is certainly shaped by my experiences in trenches of enterprise IT. I am aware, that there are other domains of software development. Especially in industrial and/or embedded systems development the world might look different.

I have blogged about MDSD and code generation before: Code Generation vs. Code Synthesis, Sunday Morning on InfoQ, Part 2 and Generic and generated - an oxymoron?.

I think MDSD is still too much tied to code generation.

Code generation is only useful, when your code contains a lot of noise and/or is very repetitive. In other words, when your code can not mainly focus on the essential complexity, but is polluted with accidental complexity.

In my opinion the trend in current platforms and frameworks is exactly to remove accidental complexity and letting the application code focus on essential complexity.

So these new platforms/frameworks take a lot of the wind out of the sails of the MDSD movement.

DSLs (textual ones) are another trend that tries to enable the sole focus on essential complexity. While DSLs can be used as source for code generation, they are not primarily tied to code generation. DSLs (especially internal DSLs) basically let it open to be interpreted/executed at runtime. [runtime code generation is somewhere in between].

So even if DSLs are often mentioned together with MDSD, I think they are really an alternative to MDSD. And given the current hype, they also take the momentum out of the MDSD movement.

If you have reached the goal of ultimately removing accidental complexity out of your code (I know this is fictitious), then you have arrived at a textual model of your business problem. This cannot be further simplified!

Nice boxes and diagrams do not offer another simplification or elevation of the abstraction level! They may be good for visualization, but even that is questionable. A picture is not always the best representation to grasp complexity!

Further more, the current state of the tooling involved in MDSD adds another level of accidental complexity (think: synchronization, diffing/merging, refactoring ...) which basically nullifies the ultimate goal of simplification!

Look at the following ActiveRecord model, as an illustration of my theory:

class Firm < ActiveRecord::Base
   has_many   :clients
   has_one    :account
   belongs_to :conglomorate
end

I dont think that this can be any more simplified. Also any graphical representation with boxes and lines would be no simplification, and would not offer any more convenience (think about layouting, refactoring, searching, diffing ...).

follow me on twitter, I need some friends :-)