20 November 2015

Devoxx 2015 Impressions

Devoxx Belgium was last week and it was a great experience to be there. It’s been a great conference for developers since 2002 and has grown into a global brand now, with Devoxx conferences all over Europe. The conference in Belgium will always be the original though.

Compared to the first time I was there in 2009, it’s intriguing to see how much the conference has grown. Back then, there were only a few tracks at a time; now there were about eight sessions running concurrently!
The most popular news from Devoxx this year was the fact that all conference sessions were uploaded to YouTube. This is great news for all - it means that people who didn’t get to attend the conference can now watch the sessions for free; the attendees can catch up on the sessions that they missed; and finally poor old Stephan and his crew didn’t have to manually upload videos to Parleys afterwards…

Since the sessions are all available for free on YouTube, I would definitely recommend watching the opening keynote by Mark Reinhold. For those who don’t know, Mark is the chief architect for the Java Platform Group at Oracle and has been working hard on Java 9. Yes, for a lot of us, Java 8 is new enough anyway; but it’s great to see that there are big improvements coming. In separate sessions, Mark also dived further into project Jigsaw, which is what will introduce modules into Java come version 9.

Microservices was still a big theme this year, with Docker and Kubernetes complementing it. IoT talks and demos are a personal favourite of mine, so I had great fun watching Bert Jan Schrijver talk about how he tracked aircarft and ships near his home near Schiphol. The way he managed to set up a demo in the room to eavesdrop on Dan Allen while he was speaking in the room next door concurrently was pure genius!

If you're into Microservices you might also want to check out my presentation on Payara Micro here. In this short Quickie Session I demonstrate the ease of deployment of applications and services to Payara Micro, along with showing how simple it is to build and scale a cluster of instances running these services.

For anyone who interested in astrophysics, you might be interested in the other opening keynote by Prof. Lawrence Krauss; who spoke about how we can attempt to visualise what had happened when the Big Bang occurred.

Summing up: Devoxx was a great conference, it was great to meet some very interesting people… Oh and the beer was awesome!

Navin Surtani
Senior Consultant

10 November 2015

SOA vs Microservices

There is much discussion recently regarding the concept of microservices, and much discussion of how this fits in with SOA – whether it is the final nail in the coffin of SOA, or the latest cure-all panacea that will save software engineering. In this article I am going to look at the concepts behind both SOA and microservices, and how there is really very little difference between them in terms of principles, but that the products that claim to be in one camp or another do have some significant differences, making them suitable for different use cases.

The concept of microservices has been around for some time now, and relates to the approach of breaking your application or system down into a number of components, and implementing each component separately. Creating a well-defined interface for each service allows you to integrate the components together in order to solve your business requirement. 


The advantages of microservices are fairly well understood at this stage. Separating out components allows them to have separate lifecycles, and to scale as required, it also breaks many of the technology dependencies between components, allowing each service to be implemented in the most appropriate technology. Breaking down the larger problem into several smaller ones makes each individual problem easier to analyse, making it easier for developers to come up with the most suitable solution. The disadvantages of microservices are also well understood, although less talked about. Breaking down the problem like this increases the complexity of the overall solution, especially if different technologies or approaches are used to build different services. By pushing the integration points to be at interfaces between services, it is important that these interfaces are well defined, and have agreed service levels and other non-functional requirements defined. Tooling for microservices is also in its infancy, with much scope for improvements that would make developers and architects lives easier. For me however, the key disadvantage of a microservice architecture is the problem of data management and ownership. When something that would previously have been a monolithic application is broken down into a number of smaller services, it often becomes the case that data that would be stored in one place in the monolithic application is stored in multiple places for the microservice applications. This presents challenges in maintaining data consistency.

Products that enable microservices primarily work to manage the lifecycle of the service components, and providing a set of common services such as connection management, deployment, and abstraction of underlying OS resources or external systems. This leaves a lot of room for the developers to choose how they implement services, what technologies and protocols are used, etc. Common technologies used to enable microservice architectures range from containerisation technologies like docker, through to inverted application containers like spring boot. While the products themselves are fairly independent of protocol or message format, there is a general assumption that RESTful implementations are the best approach when building microservices. While RESTful services are generally best focussed on providing CRUD operations for a data model, it is often the case that the services in a microservice architecture can easily be broken down into these CRUD type services where RESTful is a good fit. For other services, it is often still appropriate to look at RESTful-like services, where HTTP is the transport protocol, but the service doesn’t necessarily stick 100% to the RESTful principles. 


It is common to talk about the disadvantages of SOA these days, but when you really look into it, most of the disadvantages of SOA are the same as those of microservices, but with more concrete examples behind them. The same can be said for the advantages, because fundamentally what we are doing is the same thing – taking a big problem and breaking it down into smaller problems. The companies that are held up as being the leaders of microservice architectures (Netflix, Amazon, Ebay etc) are all quite happy to describe their architecture as service oriented. What they don’t do is use traditional SOA products. The advantages of SOA are also the same as those of microservices, but applied at a different scale. An enterprise wide Service Oriented Architecture is intended to represent company-wide business as a series of transactions between services.

The products that are sold badged as Service Oriented Architecture are very different though. Most of these products will be sold as an Enterprise Service Bus – a framework intended to allow the orchestration of services on an enterprise scale. As such the features are focussed primarily on this enterprise use-case, with ways of tracking business-unit level SLAs. Most SOA products will mandate that services communicate using one, or a small number of protocols and message formats, such as HTTP, FTP, SOAP, JMS etc, and provide a library of connectors that enable this.

SOA products have received a bad press recently, and much of this is due to their miss-use (or miss-selling) on projects where the goal was to build an application, rather implement an enterprise-wide architecture.

More in common than different

So both SOA and microservices are the same set of principles, applied at different layers of the organisation. With SOA we are looking to orchestrate large services, but those large services could in turn be composed of a number of microservices. The very existence of microservices comes down to the success of SOA principles. The fact that the products advertised as SOA were not well suited for use in developing mid-size applications has led directly to the suite of products that we now call 

Pick the right solution

This gives us a range of products that can be applied to a range of use-cases, without any direct competition between them. If you are developing an application, then a microservice framework is going to be more agile and give you greater control as a developer. If what you are trying to do is orchestrate a number of business processes across your whole enterprise, then a SOA product probably provides a better set of tools.

The real answer to SOA vs Microservices is that both are appropriate for different solutions, but as there are a lot more applications being produced than enterprise architectures, the chances are that a microservice framework is a better fit for your project than an ESB.

1 November 2015

Introduction to JBoss Fuse – Part 2 - deploying, monitoring, debugging & error handling


This is the second part in a blog series on JBoss Fuse. The first part looked at building a basic application from scratch and is available here:


This follow up uses the application built in part one and therefore you will need to do that first in order that you can follow through this blog.

Deploying in Fuse

In Part One we did all the testing of our application from within JBoss Developer Studio. Now we're going to look at deploying our application to Fuse.

First of all we need to make some simple changes to our project. In our File components the directories are currently relative to our code. Once we are running in Fuse they will be relative to the Fuse home directory. Rather than store them there let’s create our own directories.

To keep things simple I'm going to place all the file directories in a folder called cartest-files in my home directory. So from my home directory I run the following commands:

mkdir cartest-files
mkdir cartest-files/input
mkdir cartest-files/sales
mkdir cartest-files/purchases
mkdir cartest-files/invalid

Then I copy the test files over: 

cp workspace/CarTest/src/test/resources/*.xml cartest-files/

Now we have the relevant directories and files in the right place we need to alter each of our File components and give them the full path to the relevant directories. So for the input directory, instead of:


This now needs to be:


You will need to change all 4 file components to point to the new directories. 

Next we're going to change the ActiveMQ broker to use the default broker rather than the embedded one. To set the default broker username and password go to the admin console at http://localhost:8181

Click on admin (top right) and select preferences. Select ActiveMQ and set the username and password to admin/admin

Next we need to alter the ActiveMQ broker settings in our blueprint.xml file to use the default broker rather than the embedded broker.

So, where previously we had:
<property name="brokerURL" value="vm://localhost"/>

This needs changing to:
<property name="brokerURL" value="tcp://localhost:61616" />
<property name="userName" value="admin" />
<property name="password" value="admin" />

Make sure you've saved all of your changes.

Now we need to rebuild our project. From a terminal window run the following:
cd ~/workspace/CarTest
mvn clean install

Now we’ve made the changes needed we can deploy our application to Fuse. First of all we need to start Fuse. To do so run the following (changing fuse_install_dir to wherever you’ve installed it).
cd <fuse_install_dir>/bin

Once Fuse has started run the following command from the fuse terminal:

JBossFuse:karaf@root> osgi:install -s mvn:uk.co.c2b2/camel-blueprint/1.0.0-SNAPSHOT

You can get the details of the name to enter here when you do your maven build. Just before the build success message you should see a message similar to the following:
Installing uk/co/c2b2/camel-blueprint/1.0.0-SNAPSHOT/camel-blueprint-1.0.0-SNAPSHOT.jar

When you do the osgi:install it will return the ID of the installed bundle. If you want to uninstall it for some reason you can run the following command:
osgi:uninstall <bundle_id>

The command osgi:list will list all the OSGi bundles currently deployed.

Check the log to make sure your route is running and awaiting files in the input directory. You can check the log from the Fuse terminal with:

Or the log itself is available at <FUSE_INSTALL_DIR>/data/log/fuse.log

You should see similar to the following:
2015-10-16 12:03:00,407 | INFO  | rint Extender: 2 | BlueprintCamelContext            | 198 - org.apache.camel.camel-core - 2.15.1.redhat-620133 | Route: PurchaseAndSaleHandling started and consuming from: Endpoint[file:///home/andy/cartest-files/input?delete=true]
2015-10-16 12:03:00,407 | INFO  | rint Extender: 2 | BlueprintCamelContext            | 198 - org.apache.camel.camel-core - 2.15.1.redhat-620133 | Total 1 routes, of which 1 is started.
2015-10-16 12:03:00,408 | INFO  | rint Extender: 2 | BlueprintCamelContext            | 198 - org.apache.camel.camel-core - 2.15.1.redhat-620133 | Apache Camel 2.15.1.redhat-620133 (CamelContext: camel-1) started in 0.702 seconds

Test your app by copying files from the cartest-files directory into the input directory. They should all be handled correctly and moved into the relevant purchases/sales/invalid folders. 


Next up we’re going to take a brief look at monitoring our application. First of all point your browser at http://localhost:8181 to open the Fuse Management Console.  

Viewing bundle details

Click on the OSGi tab. Select table view - top left

Scroll down and you should see your newly installed bundle at the bottom.

Click on the bundle name to view details on the bundle.

Viewing logs

Click on the Logs tab to bring up the latest log messages. From here you can filter the log messages to search for certain strings or on the log level.

Click on a message to see complete details.

Viewing Active MQ details

Select the ActiveMQ tab. Click on Queue on the left to show details of all queues. You should see our purchaseandsale queue. Click on the queue itself to show details of the individual queue.

Once you've selected your queue, click on Browse to show details of all messages on the queue. Clicking on the individual messages will show you details of the message contents.

Send allows you to send a message which can be useful for debugging purposes.

Operations gives you a list of all the operations/methods that can be called on the queue. Clicking on the operation name allows you to call it passing in any relevant parameters. This again is useful for debugging meaning you don't have to write code in test operations.

Chart allows you to monitor all available metrics regarding your queue in real time. Try copying over some messages and you will see the chart change.

Viewing camel route details

Select the Camel tab.

Select Routes on the left hand side. This will show you in depth details of all your routes including number completed, number failed, amount of time to run, last message time etc.

Selecting a route brings up details on an individual route.

Source allows you to view the route XML. Note - you can make changes to your route from the console, click update and your route will be changed. This can be very useful for testing out something quickly before adding it to your project in JBDS.

Route Diagram shows you a similar view to the design view in JBoss Developer Studio

Clicking on the dropdown next to your route in the left hand panel allows you to drill down into each part of your route. You can also click on the different elements from the design view.

Selecting Endpoints from the left hand panel allows you to view all the different endpoints in your route. From here you can click on the individual endpoints in order to view properties, browse messages, send test messages and call operations on the endpoints.

Debugging and Tracing

Next up we’re going to take a brief look at debugging and tracing to find where problems lie in your route.

Tracing allows you to send messages to a route and then step through and see the messages flow through a route to aid debugging and to help diagnose issues.

Debugging is very similar except you can also set breakpoints.


Go to the Camel tab and select your route. Click on Trace

Click on Start tracing and it will bring up your route along with a list of processed messages at the bottom.

Now, drop a test message in the input folder.You will see a list of processed messages underneath the route.

Clicking on any of these will highlight the node that dealt with that message in the route diagram and display the message that was processed.

Using the control buttons you can navigate forward and backward through the route and see the messages at each stage.

Note - in our example we're not making changes to the message so the only real change you will see is in the header when we add in the timestamp to the file name.


Go to the Camel tab and select your route. Click on Debug. You will see the same view as when tracing.

To set a breakpoint in the route you can either double click on a node or select a node and click the + button. You can see where your breakpoints are by the yellow dot in the node. When a message arrives at the node, the dot will turn from yellow to blue.

Once your message hits a breakpoint you can examine the message before continuing. To continue you can either step through or click the Resume button which will cause the message to continue routing until it arrives at the next breakpoint.

If you have multiple messages only the first message will be tracked through the debugger. Any other messages will continue routing without being suspended at breakpoints until the message in the debugger has completed its routing.

Exception Handling

Finally, we're going to take a look at how to handle exceptions within your route.

There are 3 ways to handle errors:

· default

· try-catch-finally

· onException

The default is to propagate the exception back to the caller and end the route.

Try-catch-finally is basically the same as in Java code.

onException allows you to specify the error handling you require on a per exception type basis.

For our example we'll use the onException method.

In our current route we handle 3 types of message - Sale, Purchase and Invalid. However, the Invalid message handling only deals with XML messages where the messageType is unknown. To show the effect this has create a new file called garbage.xml and fill it with some random characters. Now, try dropping that file in the Input directory.

If you look at the log you will see that this file causes a big problem. You should see an error like the following:

Consumed message from purchase/sale queue : adgfdfdgsdfscvgadsyjgh
2015-10-21 10:00:38,484 | ERROR | test-files/input | DefaultErrorHandler              | 198 - org.apache.camel.camel-core - 2.15.1.redhat-620133 | Failed delivery for (MessageId: ID-ubuntu-39576-1445348432584-5-137 on ExchangeId: ID-ubuntu-39576-1445348432584-5-138). Exhausted after delivery attempt: 1 caught: org.apache.camel.TypeConversionException: Error during type conversion from type: java.lang.String to the required type: org.w3c.dom.Document with value [Body is instance of java.io.InputStream] due org.xml.sax.SAXParseException: Content is not allowed in prolog.

If you view the message history below the error you can see where the issue arises:

Message History
RouteId              ProcessorId          Processor                                                                        Elapsed (ms)
[PurchaseAndSaleHan] [PurchaseAndSaleHan] [file:///home/andy/cartest-files/input?delete=true                             ] [        16]
[PurchaseAndSaleHan] [log37             ] [log                                                                           ] [         0]
[PurchaseAndSaleHan] [setHeader7        ] [setHeader[CamelFileName]                                                      ] [         1]
[PurchaseAndSaleHan] [log38             ] [log                                                                           ] [         0]
[PurchaseAndSaleHan] [PurchaseAndSaleQue] [activemq:queue:purchaseandsale                                                ] [         6]
[PurchaseAndSaleHan] [log39             ] [log                                                                           ] [         1]
[PurchaseAndSaleHan] [choice7           ] [when[xpath{XPath: /CarDeal[@messagetype="Purchase"]}]choice[when[xpath{XPath: ] [         8]

When the choice node tries to check the messagetype attribute it fails as our file doesn't contain XML as expected. Another issue here is that it keeps retrying the message so we have a situation where it tries to parse the message, fails, tries to parse it again and then repeatedly fails causing an infinite loop.

Now, the correct way to deal with this would be to ensure that the XML could be validated against a schema. However, to keep things simple and to show exception handling we will handle the exception thrown instead.

Open the route in Developer Studio.

From the palette, select an OnException node (this can be found under Control Flow), drag it under the Choice node and join the two.

Open up the properties view. Here you can define what Exceptions to handle and how you want them to be dealt with.

There are a number of options:

· Continued - if this is set to true then the Exception is ignored and the route continues – generally not a good idea.

· Retry While - allows you to set a predicate so it will keep retrying until the predicate is false.

· Handled - tells Camel not to propagate the error back to the caller.

In our case we want to handle the SaxParseException and we will put files that could not be parsed in a new Error folder.

First of all add the org.xml.sax.SaxParseException and set handled to true.

Next, add a log component with the message “Could not parse file. Contents : ${body}”

Finally, add a file component with the URI file:/home/andy/cartest-files/error

Now, build and redeploy your route and test with the garbage.xml file again.

At this point you may run into an issue with JBoss Developer Studio where editing the onException node by changing the properties causes it to create an invalid route as it adds in both the handled and continued elements.

If you see an error like the following:

java.lang.IllegalArgumentException: Only one of handled or continued is allowed to be configured on: OnException[[class org.xml.sax.SAXParseException] -> [Log[Could not parse file. Contents : ${body}], To[file:/home/andy/cartest-files/error]]]

You will need to edit the source file and remove the following lines:


Your source should look similar to the following:

    <log message="Could not parse file. Contents : ${body}"/>
    <to uri="file:/home/andy/cartest-files/error"/>

Once you have built and redeployed and tested with the garbage.xml file you should see the message consumed and dealt with by the exception handler and placed in the error directory:

2015-10-21 14:20:22,997 | INFO  | test-files/input | PurchaseAndSaleHandling          | 198 - org.apache.camel.camel-core - 2.15.1.redhat-620133 | Consumed message from purchase/sale queue : adgfdfdgsdfscvgadsyjgh
2015-10-21 14:20:23,007 | INFO  | test-files/input | PurchaseAndSaleHandling          | 198 - org.apache.camel.camel-core - 2.15.1.redhat-620133 | Could not parse file. Contents : adgfdfdgsdfscvgadsyjgh

That concludes Part 2 of this blog series. Hopefully, it’s given you a good starting point with regards to how to go about monitoring and debugging your application and a brief overview of building in exception handling. Happy Fusing!