John of All Trades

You could probably say I’m getting a dose of my own medicine.

After all, I’m one of the people responsible for Apache Maven 2.x, including its plugin framework which seems to confuse the hell out of Maven newbies. I used to chalk most of that up to a lack of documentation, and I’m still sure that is part of the problem. But now, I’ve got a somewhat different perspective on plugin development in a dependency-injected context, and I think there’s probably more making life difficult for third-party plugin developers than simply being too lazy to read a book.

Recently, I’ve been working on a few custom plugins to work with Sonatype Nexus for Red Hat (whom I joined back in January). It may sound surprising that I’m having some trouble with this since I used to work at Sonatype, after all. But alas, the closest I ever got to working on Nexus was writing a client API that I could wrap in a Maven plugin, and a few tools and configurations for building Nexus itself. Anyway, while the Nexus documentation for plugin developers is a little scant and shallow, I don’t really think it’s the main problem. Rather, it seems like developers have a tendency to include a plugin framework for their application without thinking much about the fact that people – people who don’t have a deep understanding of the application’s architecture – will actually try to use that framework. I haven’t understood this problem as well as I should with Maven, since I do have such a deep architectural understanding of it. Now I see that by providing a plugin framework for your application, you’re combining the hardest parts of standalone application development with the hardest parts of library development. In building a customer-facing application, you’re concerned with providing clean UI and an unbreakable user experience. These things require quite a bit of focused testing, and hiding a lot of the debug-level information from the user, so the output (console and UI) is clean and understandable. On the other hand, when building a good library, you need to worry about maintaining a strict respect of the library’s API contract over time, good developer documentation, the accessibility (and tunability) of debug-level information, and an absolute minimum of unexpected or magical behavior. Again, this requires a ton of focused testing, and some deep thinking about how to provide useful feedback to the developer in the event of an error. When the application is deployed, the developer should be able to hide this troubleshooting information (see application concerns, above).

Where many applications with plugin frameworks seem to really fall down is in supporting the plugin developer. The plugin framework itself seems to be a concession to make the lives of internal developers simpler, and this “simplicity” is usually passed on to third-party developers as well – as an afterthought, without considering that third-party developers don’t work with this stuff all day, every day. The result is an application that taunts the outside developer with the promise of extensibility, only to make life hell for those who succumb to that particular siren song. Finding themselves knee-deep without a shovel, such developers often have no other choice but to hook up a debugger, set some optimistic breakpoints, and continue down the rabbit hole into an endless iteration of debug session after debug session. “OK, I think I’ve finally isolated the code I need…Whoops! Just missed it.” CTL-C, set another breakpoint (or refine an existing one because unqualified, it seems to be the usage equivalent of Object.hashCode()) and run it again…and again….and again. Welcome back to the world of trial-and-error programming. If you’re a decent programmer, you haven’t been this helpless to fix a coding error since you learned to write code. Often, after eight hours of debugging or more, our poor developer finds out that he has an extra %s in his String.format() call, and the resulting RuntimeException is being swallowed up by the plugin framework. Or, he realizes that he’s been trying to retrieve an attribute named ‘repository-id’ from the Context attributes, when he should have been retrieving it from the Request attributes. Isn’t it obvious that the context has a different lifecycle than the request? Well, they’re both passed in next to one another in the same method signature, so, well…No, it’s not obvious.

And maybe that’s the real problem with plugin development: the pitfalls, the potential for errors, even the philosophy that went into the application’s architecture is not obvious. Since most third-party developers aren’t psychic, confusion and torture ensues. And, while most developers these days – myself included – have a big ole crush on dependency-injection frameworks like Guice, in many ways they only exacerbate the confusion by adding a new dimension of spooky action at a distance. Plugins have always been collections of hook implementations, event listeners, and callbacks that tie into an existing application…that’s sort of the definition. But with DI frameworks, these listeners and callbacks have never looked more disembodied. Making matters worse, DI frameworks themselves often over-promise and under-deliver. Like the rest, they are pieces of software, and may contain their own quirks or bugs. At times, the application developers may even place strange and seemingly arbitrary restrictions on how the underlying DI system is used (like forcing a component interface and its implementation to come from the same jar) that undermine the promise of a componentized architecture. In the end, the unflinching observer is all too often challenged to say that using a DI container to support a plugin framework saves anyone any pain at all.

To some extent, the fragmentation issues are unavoidable in plugin development. Fundamentally, you’re adding a small slice of functionality onto carefully selected extension points in an existing application; when you don’t understand how that application functions internally, finding the right leverage points for your new feature will always be hard. But how much harder is your application actually making the life of your third-party developers? Providing and advertising a plugin framework for your application implies the desire to create an ecosystem around that application. But providing a plugin framework and API that doesn’t behave as expected, doesn’t provide much help to third-party developers, isn’t well documented, and doesn’t have an comprehensive suite of tests to continually verify the plugin contract…all of this is counter-productive at best. At worst, it can alienate some of your application’s biggest fans.