Ok, I’ll come clean… this is my pet topic in software development, and one day my tag cloud will agree with me ;^)
To me the “Holy Grail” of software development is to have languages, tools and techniques that allow us to easily discover (and transcribe) the intent of software i.e. what the software is actually trying to *achieve*, not how it does it.
Why is this so important? Well, without understanding the intent of a piece of code, a developer has very little chance of successfully adding new features or fixing bugs. Changing code without understanding the intent (usually by just copying and pasting existing code) is just MSMD (“Monkey See, Monkey Do”) programming and even when combined with a process such as TDD, such an approach is going to be painfully slow and error-prone. You might well get your next test to pass, but the chances are it will be the wrong test!
Now I know that these ideas are not new, the JetBrains paper on Language Oriented Programming was written back in 2004, and Charles Simonyi’s company Intentional Software was started even earlier in 2002, but it seems to me that the mainstream world of software development is also heading this way with the rise of more declarative languages , APIs and DSLs (which I would say are excruciatingly trendy right now ;^)
As the popularity of things like Domain Driven Design, DSLs, Language Workbenches etc., increases, I am reminded of what one of my professors said some 15 or so years ago – that all programming is actually language extension.
We start at the lowest-level with our implementation language and we gradually build in concepts that extend it towards the highest-level which is the application domain. Now, obviously, in a non-language workbench world, we are bound by the syntax of the implementation language (and hence our extensions are in the form of types, functions etc) but we can still see the extensions as building layers of new “languages” until the final language that can (in theory) be shown (and explained) to domain expert.
For me, the key is that, DSL or no-DSL, the outermost layer should preferably be readable by the domain-expert, and even better, be writeable too ;^)
The API vs DSL debate is an interesting one. IMHO, all good software is made up of layers, with each layer representing a domain. The outermost (application) domain might model a shipping company and hence any API/DSL at this layer should easily describe how to manipulate ships, cargos, routes etc. The innermost layer(s) might model web services or relational databases etc and so any API/DSL at these layers must model the low-level concepts that a developer would expect of these technologies. The key point is that at each layer the API/DSL should allow the user (be it, an application domain expert, or a hacker-geek dude!) to express the intent of what they want to do as easily as possible.
Now, my first guess is that if I was going to provide a DSL, I would probably only provide it at the application layer, and I would just create (hopefully) nice, clean APIs for the other layers (and of course, the DSL would really just be a thin wrapper around a nice clean application level API!)…
Of course, you could go DSL crazy and create one for each layer, although this seems like overkill to me, and brings up some pragmatic issues (which, actually apply even to a single DSL architecture):-
1) The ability of the user to learn a new language (even one targeted to the domain that they are in). This would obviously require the usual array of tutorials and reference manuals etc.
Currently when I code (I use TDD) I write tests in the same language as the shippable code. Obviously, every DSL can’t have extensions for testing, so would we use a low-level language or have a DSL for testing?