Getting Groovy (and Grails)

Last week I attended the excellent inaugural gr8 in the US in Minneapolis (look for a review in the May edition of GroovyMag). This week I’m back in MN presenting at a ColdFusion conference on agile development and productive work practices.

in May, my Grails world tour begins! First stop, a presentation for the Sydney Groovy group on Tuesday May 4th while I’m down to present at WebDU. The week after I’ll be presenting in New York on Tuesday May 11th, and the week after I’ll hopefully be either attending or presenting at the London Groovy/Grails Usergroup – probably on Monday 17th.  I’ll be presenting on DSL design and the Eclipse Modeling Framework that week at the British Computer Society Software Practices Advancement conference, before flying off for a day to the awesome Gr8 conference in Copenhagen (I’d love to attend both days, but am presenting at SPA on the Wednesday in London). Then back to London for another ColdFusion conference presentation (I love *all* dynamic languages on the JVM!) before returning briefly to chillax in New York (my home base) for a while. Back to the UK in June to present on Spring Roo at Code Generation 2010.

I’ve been doing work on domain specific modeling, DSL’s and software product lines for a while, but I’m really excited to finally be porting a lot of my SPL functionality on to the Grails framework. It’s an excellent stack with a really exciting community and I can’t wait to meet more Groovy and Grails developers around the world in the next few weeks. So, if you’re in Sydney, New York or London (or are going to gr8.eu), please say “hi” to the slightly jetlagged guy with a strange English/US accent – I’d love to learn from you all!

As to the preso’s, I’m focusing on practical DSL design. Guillaume Laforge does a great job of presenting on the syntax of DSLs, but there are a bunch of other issues to consider when designing, developing, testing, maintaining and evolving DSLs and I’m looking forward to presenting more information on that as it relates to Groovy and Grails as I continue to get more familiar with the stack.

(Hopefully) see you somewhere!

I’m really looking forward to attending the gr8 conference in Minneapolis this Friday. I have been doing more and more Groovy and Grails projects recently and am looking forward to get to know the broader US community (the Copenhagen conference last year was great – but there weren’t that many US attendees).

I’ll be writing an article for Groovy Mag on the conference, so if you are presenting and would like to be interviewed briefly, I’ll be in the conference hotel Thursday evening and all day Friday (I’m flying back to NYC first thing Saturday) so make yourself known (a tweet to @peterbell is probably the easiest option) and I look forward to meeting you!

This session by Guillaume LaForge was probably the one I was looking forwards to the most. I do a lot of work with Software Product Lines, Domain Specific Modeling and Domain Specific Languages. I’d seen Neil Ford the other month at No Fluff Just Stuff Boston presenting on using Java, Groovy and Ruby for internal DSLs and I was interested to see what Guillaume would cover. Historically Groovy has been somewhere in the middle of Java and Ruby in terms of support for the kind of syntactic sugar that allows for the creation of business user readable DSLs and I was interested to see both how Guillaume presented the topic and what specific information  he’d include.

He started off by introducing the problems that vertical (business – as opposed to horizontal technology DSLs like SQL or RegEx’s) DSLs are designed to solve. They allow for clean separation of business logic from application code, allow Subject Matter Experts/business analysts to read (not necessarily write) the DSLs and allow for business rules to have their own lifecycle independent of the usual re-build, re-deploy lifecycle required when the business logic is encoded in Java.

He mentioned that there are a range of possible notations for DSLs – from the “e4 – e5″ chess notations to the notations used to describe solutions for rubix cubes and sheet music notation. He then also introduced some real world examples of business DSLs such as anti malaria drug resistance simulation, hr employee skill representation, insurance policies risk calculation engine, loan acceptance roles engine for a financial platform, mathematica like lingua for nuclear safety and market data feeds evolution scenarios.

Groovy and DSLs
Groovy has a number of capabilities that make it nicer than Java for implementing business user readable internal DSLs.
- There is no need to write full blown classes – you can use scripts
- Optional typing (def) – cut down on visual noise – in scripts can even omit def
- Native syntax constructs (lists, maps and ranges – can create own custom ranges e.g. monday..friday)
- Optional params (for top level statements) and semicolons so “move(left);” becomes “move left”
- Named arguments: Can mix named and uunnamed – named params are put in a map parameter.
E.g. take 1.pill, of: Chloro after: 6.hours
==     def take(Map m , MedicideQuantity mq)
- Custom control structures: when closures are last they can be put “out” of the parenthesus surrounding parameters
unless (account.balance > 100.euros , {account.debit 100 euros})
== unless (account.balance > 100.euros) {account.debit 100 euros}
Signature “def unless (boolean b , Closure c)”
- Operator overloading: -15.euros + 10.dollars, -10.kilomaters – 10.meters, taskA | taskB & taskC
e.g. a + b – just put a.plus(b) method
e.g. a – b – just put a.minus(b) method
e.g. a * b – just put a.multiply(b) method
etc.

Options for crediting an  account:
account << 10.dollars
account += 10.dollars
account.credit.10.dollars
depends on users which syntax they prefer . . .

Groovy’s dynamic heart: the MOP – The MetaObject Protocol
All accesses to methods, properties, constructuors, operators, etc can be intercepted thanks to the MOP. While Javas behavior is hard wired at compile time in the class, with groovy, behavior is adaptable at runtime through the metaclass.

Different hooks for changing the runtime behavior:

GroovyObject
getProperty() , setProperty , invokeMethod() , getMetaClass() , setMetaClass()
A GO can have “pretend” methods and properties

MetaClass
Core of the Groovy MOP system
- invokeConstructor
- invokeMethpd
- invokeStaticMethod
- invokeMissingMethod
- getProperty
- setProperty
- getAttribute
- setAttribute
- respondsTo
- hasProperty
- Metaclasses can change the behavior of existing third party classes – even java one.

ExpandoMetaClass
- A DSL for metaclasses!
MoneyAbount.metaclass.constructor = { . . }

Note: 10.euros == 10.getEuros()

Class CurrencyAmount {
Number value
String currency
String toString() { “$value $currency”}
}

Number.metaClass.getEuros = { ->
new CurrencyAmount(value: delegate, currency: “EUR”)
}

10.euros
The delegate variable in closure represents the current instance, and “it” the default parameter

The Builder pattern
This is a great pattern for certain classes of DSLs. You’re probably familiar with the markup builder for XML/HTML, but how about having a builder for HR?
softskills {
ideas {
capture 2
formulate 3
}
. . .
}
knowhow {
languages {
java 4
groovy 5
}
. . .
}

Builders are a mechanism, for creating any tree structured graph – a realization of the GoF builder pattern. It uses hooks like GroovyObject#invokeMethod() to catch all non-existing method calls

Adding properties to  Numbers
One of the common approaches when creating a fluent API/DSL in a dynamic language is to add properties to numbers. There are three ways of implementing this in Groovy: create a category, create a custom metaclass or use the ExpandoMetaClass.
- Create a category: A kind of decorator for default MCs. Interesting scope – thread bound and lexical, but doesn’t work across hierarchy of classes (doesn’t work for subclasses).
- Create a custom MetaClass: A full blown MC class to implement and to set on the POGO instance.
- With an expandometaclass: Works for class hierarchy for POJOs and a flag exists to make it work for POGOs too, but the catch is it’s really a global change so beware collisions with other libraries.

Compile time metaprogramming:
Groovy 1.6 now support compile-time metaprogramming via AST transformations. They are compile time, so there is no runtime performance penalty.

There are two kinds of AST transformations available:
- Global – applicable to all compilation units
- Local – applicable to marked (annotated) program elements, using specific market annotations
E.g. added @Singleton and @Delegate

To create a global transformation:
Implement ASTTransformation
Annotate the transformation specifying a compilation phrase

For discovery, create the file META-INF/services/org.codehaus.groovy.transform.ASTTransformation
Add the fully qualified name of the class to that file

Local is same as global, but don’t need to add to meta-inf – just create annotation to specify on which element the transformation should apply.

For example, the Spock framework does this using syntax highjacking with AST transformations to create a BDD framework.

You can pre-compile pogos like pojos in a spring app, but can also load on the fly to decouple the lifecycle of the business rules form that of the application. Also have eval, GroovyShell, or GroovyClassLoaader – most powerful mechanism. Could also visit or change the AST. Scripts and classes can be loaded elsewhere
Groovy DSLs CAN be embedded in groovy classes, but should store elsewhere – database, xml file, etc

Considerations to remember when designing DSL
- Start small with key concepts – beware of over engineering!
- Grow the language progressively, get hands dirty, play with end users
- Let DSL fly – theirs, not yours!
- Tight feedback loop – iterative process
- Stay humble – can’t get it right first time, don’t design alone at your desk – involve the end users from the start!

And if I might add something, check out Eric Evans book on Domain Driven Design. I did an presentation and an interview at ddd.org on using DSM for DDD’ers. I’d also argue that the DSM/DSL community could learn a lot from Erics work on eliciting and developing Ubiquitous Languages – many of which can be executable (making them DSLs).

Maybe play it safe in a sandbox
- Groovy supports usual java security model
- Use metaprogramming tricks to prevent calling or instantiating certain classes
- Create a special GroovyClasLoader AST code visitor to filter only the nodes of the AST you want to keep – e.g. the ArithmeticShell in Groovy’s sample.

- Don’t just test for nominal cases – explicitly test for errors!
- Ensure end users get meaningful error messages

Groovy is often used for mission critical DSLs . . .

That’s the notes I managed to capture from the session. I’d thoroughly recommend catching any of Guillaume’s presentations as in addition to his deep knowledge about Groovy he clearly has a robust understanding of DSM and the practical uses of DSLs. It was a great session both as an intro to DSLs and with lots of detailed information about using Groovy for implementing those languages. I certainly can’t wait to play with Groovy DSLs!

My latest article – this time comparing Grails and Rails – has been published in Groovy Mag. It provides an overview of the Groovy/Grails and Ruby/Rails stacks and looks at some of the reasons you might choose one over another.

It’s been almost three years since I set up my main blog at pbell.com which has covered everything from software product lines and domain specific languages/modeling through domain driven design, agile/lean processes and tooling to language specific posts on ColdFusion, Javascript and Flex. In three years it has received over 1.7 million page views (which isn’t bad for a blog that started off focused on code generation techniques in CFML – not exactly a huge target audience) and it’s definitely where you should go if you want to know where I’m presenting/traveling, what I’m publishing and what I’m currently thinking about DSM, code gen, agile/lean, tooling and any languages except for Groovy and Grails.

However, I decided recently to get serious about learning Groovy and Grails. I still remember when  I first mentioned Groovy to a friend who teaches CS in the UK. He rather huffily pointed out that there were fundamental flaws in the original compiler and that if the team had taken his compiler class he’d have flunked them! Because of that I (unfortunately) didn’t take another look until last fall, but recently I’ve been getting increasingly interested in having more powerful metaprogramming capabilities (I’m down with the metaclass stuff, haven’t yet figured out the core use cases for the AST transformations in 1.6+ but I’m looking forwards to figuring it out) and ideally in leveraging all the great work that the Grails team are doing so I can focus on my product line (metadata re-use and configuration, automated generation of apps and docs from end user manageable specifications, etc.) rather than the details of writing a framework to implement the DSL statements and make them run on a given language.

Anyway, this blog will catalog my journey, and hopefully also be useful for anyone else looking to figure out how to build great Groovy an Grails applications using software engineering best practices. And with that, let’s get groovy