Table of Contents
- Apache Camel
- Overview
- The Fuse Implementation
- Summary
Apache Camel
Apache Camel is an open source project to allow developers to implement integration services quickly and easily. It comes with many readily available components for integration and together with the Enterprise Integration Patterns, provides the portability to deploy the integration services on many runtimes makes it the ultimate organizations’ choice.
From the business perspective, organizations not only wish to be able to support their business with flexible, agile and cost effective integration solution but also wish to be supported whenever there is problem with the integration solution. This is where we are looking into Red Hat Fuse which it is meant for enterprise and businesses. Fuse is Red Hat commercial version base on Apache Camel.
In this article, we are going to look at how can we implement integration service with Red Hat Fuse. As usual for open source solution, it is practically similar steps for Fuse and Camel in terms of the implementation however we are not going into basic tutorial in here but instead this article is part of a series on bigger use case implementation that I had published earlier — Implementing Event-Based Microservices Applications on Red Hat OpenShift.
Overview
The following diagram shows the updated implementation since my last article. You will notice the use case has been expanded with more components. I will be specifically focus on the Customer Service in this article. Please subscribe to this blog if you wish to be informed of the future articles covering the other components. I will continue to expand and enhance the implementation with more use cases and components. To benefit the readers, I had shared the complete deployable asset on the GitHub. Feel free to try it out and feedback.
Oh yeah, I call the project as Payment Gateway now.
Let’s have quick understanding of the context of the implementation before we dive into the detail.
The Customer Service component basically provides a single microservices API with easily consumable JSON data for the Customer UI component. It communicates with the Account Service and Account Profile Service components to retrieve the customer account balance data and customer profile data, which each of them has their own DB service and data format.
The Fuse Implementation
Overview of The Camel Context
The following Camel context diagram shows the overall Customer Service implementation. As you can see the routes are designed in such a way that they are made as common as possible so that I can reuse them and cut down the implementation effort. The Direct component is used to send the message exchange to the respective route based on the request nature.
The Camel implementation provides the endpoints to retrieve all customer information or a specific customer information via the following interfaces:
GET /ws/pg/customer/{accountId}
GET /ws/pg/customer/all
You can take a look at the sample codes for this implementation at the same GitHub repo at sc/CustomerService.
The Producer Camel Route
The first Camel route implements the REST producer using the Camel CXF component. This is the endpoint that will be consumed by the Customer UI.
The Choice Camel component is used to verify the CamelHttpPath header whether the request context path is ended with /all or /{accountId}, and then direct the request to the respective route. If the request context path ends with /all, the message will be directed to RequestAllProfiles route, otherwise direct the message to RequestAccountProfile and RequestAccountBalance routes. We will cover more detail of these routes in later part of this article.
There is a Camel Processor named dataProcessor at the end of the Otherwise Camel component implementation which takes the customer profile data and the account balance data, and merges them into a single list of data entity. Notice that this dataProcessor is also being reused in the RequestAllProfiles route.
Take note that this producer endpoint is listening at all address and a configurable port as per the following Camel context codes. This is crucial when we are deploying the service on OpenShift later or on any other environments.
<cxf:rsServer address="http://0.0.0.0:{{service.port}}/" id="customerEndpoint" loggingFeatureEnabled="true" serviceClass="com.gck.demo.paymentgateway.customer.services.CustomerService">
<cxf:providers>
<bean class="com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider"/>
</cxf:providers>
</cxf:rsServer>
The following shows the code snippet for the CustomerService serviceClass. This class implementation tells the Camel CXF server component how to expose our REST API.
@Path("/ws/pg")
public class CustomerService {
public CustomerService(){}
// ... more codes are omitted ...
@GET
@Path("/customer/{cust_id}")
@Consumes(MediaType.APPLICATION_JSON)
@Produces (MediaType.APPLICATION_JSON)
public String get(@PathParam("cust_id") String custId){
return "{\"status\":\"OK\"}";
}
// ... more codes are omitted ...
}
The Consumer Routes
There are 2 consumers in these routes: the consumer endpoint for Account Service and the consumer endpoint for Account Profile Service. These are implemented with Apache CXF client component.
Camel Route implementation for consuming and processing data from account profile service
The RequestAccountProfile route provides the implementation to make REST API call to the Account Profile service and make a decision (Camel Choice component) whether to return the result from the consumer endpoint as List of POJO objects or as an POJO object. A List of POJO objects is returned if the request context path from the producer endpoint earlier ends with /all. The route also perform a simple enough exception handling using the OnException component.
The Camel CXF client is implemented with the following codes. Notice the {{cost.profile.host}} and {{just.profile.port}} allows us to configure these as parameters during deployment. The serviceClass is shown as per the next code snippet.
<cxf:rsClient address="http://{{cust.profile.host}}:{{cust.profile.port}}" id="custProfileEndpoint" loggingFeatureEnabled="true" serviceClass="com.gck.demo.paymentgateway.customer.services.ProfileService">
<cxf:providers>
<bean class="com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider"/>
</cxf:providers>
</cxf:rsClient>
The following shows the Java Interface used to tell the custProfileEndpoint CXF client how to invoke the Account Profile Service. Note that for the Camel CXF client, a Java Interface is used instead of Java Class. This Interface basically just define how the Camel CXF client should consume the remote REST APIs.
@Path("/ws/pg")
public interface ProfileService {
@GET
@Path("/account/{accountid}")
@Consumes(MediaType.APPLICATION_JSON)
@Produces (MediaType.APPLICATION_JSON)
public String getCustomerProfile(@PathParam("accountid") String accountId);
}
The Account Profile Service REST API requires an accountId being sent as path variable. We need to set this information and some additional headers before invoking the consumer endpoint. The following shows how we can use Camel Set Header component to set the operationName header. This header tells the Camel CXF client which method in the ProfileService should be called later. More details can be found from Camel CXF documentation.
For the rest of the header information and path variable, I decided to do it with the Camel Processor as per the following, which provide a more cleaner way to do instead of multiple Camel Set Header components. You may refer to the GitHub source code for the Java Class implementation.
<bean class="com.gck.demo.paymentgateway.customer.processors.RequestProcessor" id="requestProcessor"/>
The Camel Choice component is used to decide whether to return the result as List of POJO objects or a single POJO object. This is very much simple to implement in Fuse/Camel. Take a look at the following screen capture, the Camel Choice component is used to check the value of ${exchangeProperty.SINGLE_RECORD} that is set earlier in the QueryCustInfo route. If the request is not for a single record (i.e. context path ends with /all), we use the Camel Unmarshal component in the Camel Otherwise component to specify the Unmarshal Type Name and also check the Use List to indicate we would like the result to be unmarshal into java.util.List<AccountProfile>.
Camel route implementation for consuming and processing data from account service
The RequestAccountBalance Camel route is simpler compared to the RequestAccountProfile Camel route. The approach of implementation is very similar to RequestAccountProfile except it is always returning single POJO object.
The following shows the code snippet in the Camel context file and the code snippet for serviceClass BalanceService at the next.
<cxf:rsClient address="http://{{acc.service.host}}:{{acc.service.port}}" id="balanceEndpoint" loggingFeatureEnabled="true" serviceClass="com.gck.demo.paymentgateway.customer.services.BalanceService">
<cxf:providers>
<bean class="com.fasterxml.jackson.jaxrs.json.JacksonJsonProvider"/>
</cxf:providers>
</cxf:rsClient>
@Path("/ws/pg")
public interface BalanceService {
@GET
@Path("/balance/{accountid}")
@Consumes(MediaType.APPLICATION_JSON)
@Produces (MediaType.APPLICATION_JSON)
public String getBalance(@PathParam("accountid") String accountId);
}
The Camel Route That Does the “Request All”
This RequestAllProfiles Camel route handles the /all request to invoke the Account Profile Service and return multiple account profiles. It then Loop through each of the profile to invoke the Account Balance service to retrieve the account balance data. As you can see here, the implementation become so much simple because we are re-using all the Camel routes that we have covered in the previous sections.
The Environmental Variables
In order to make our Camel service to be flexible and agile so that we can deploy them on any environment, we need to ensure we parametrised all the environmental variables. There are a few way of doing these for Camel implementation:
For my case, I am using the resource fragments approach. I have 2 yaml files in this case. One is the deployment.yml to tell the maven how to deploy the project as container to OpenShift, and the service.yml to give instruction to create multiple OpenShift services. Both of these .yml files are placed in src/main/fabric8 in the project folder.
The following shows the deployment.yml for all the endpoints with the environmental variables. These variables will be mapped into the variables in the application.properties. In case you would like to adapt to different deployment requirement, you just need to pass these variable values as command parameters (e.g. -DSERVICE_PORT=8081) when the mvn clean install fabric8:deploy command is called. The command line parameter values will always overwrite the values in the .yml and the application.properties.
spec:
template:
spec:
containers:
-
resources:
requests:
cpu: "0.2"
limits:
cpu: "1.0"
env:
- name: SPRING_APPLICATION_JSON
value: '{"server":{"tomcat":{"max-threads":1}}}'
- name: SERVICE_PORT
value: "8080"
- name: CUST_PROFILE_HOST
value: "accountprofile"
- name: CUST_PROFILE_PORT
value: "8080"
- name: ACC_SERVICE_HOST
value: "accountservice"
- name: ACC_SERVICE_PORT
value: "8080"
The following shows the application.properties. Notice that we are not hard coding the environmental variables in here. The ${} variables are matching those from the deployment.yml or from the command parameters.
logging.config=classpath:logback.xml
# the options from org.apache.camel.spring.boot.CamelConfigurationProperties can be configured here
camel.springboot.name=MyCamel
# lets listen on all ports to ensure we can be invoked from the pod IP
server.address=0.0.0.0
management.address=0.0.0.0
# lets use a different management port in case you need to listen to HTTP requests on 8080
management.port=8091
server.port=8090
# disable all management enpoints except health
endpoints.enabled = false
endpoints.health.enabled = true
spring.main.web-application-type=none
camel.springboot.main-run-controller=true
service.port=${SERVICE_PORT}
webPort=${SERVICE_PORT}
# Account Profile Hostname and Port
cust.profile.host=${CUST_PROFILE_HOST}
cust.profile.port=${CUST_PROFILE_PORT}
# Account Service Hostname and Port
acc.service.host=${ACC_SERVICE_HOST}
acc.service.port=${ACC_SERVICE_PORT}
Notice the server is listening on the 0.0.0.0 (server.address) with port 8090 (server.port). This is the port that will be called by build process when we deploy the project onto OpenShift using maven fabric8 plugin. This will be the default service port created by OpenShift when it is deployed if no other specific configuration is made.
How about the service.port that the Camel producer endpoint that it is listening to?
We need to explicitly define any additional OpenShift services that we wish to create when the Fuse is deployed onto OpenShift. This is where can use the following service.yml file to do that. Make sure the server-port is at the first list of the following configuration as the first service will always be picked by the build process when we are deploying the project.
spec: ports: - name: server-port port: 8090 protocol: TCP targetPort: 8090 - name: http port: 8080 protocol: TCP targetPort: 8080
Finally you can execute the following command at the project folder to deploy the Fuse service onto OpenShift. The deployment of Fuse onto OpenShift is using the S2I deployment strategy, paired with fabric8 maven plugin, which makes thing so much simpler.
mvn clean install fabric8:deploy -Dfabric8.deploy.createExternalUrls=false -Dopenshift.namespace=$APPS_NAMESPACE
Please note that -Dopenshift.namespace does not fully working here. I encountered situation where the build works fine at initial build stage, but at later build stage, the build process intend to use the current project set by the user. This can be workaround by explicitly call oc project $APP_NAMESPACE to set the correct OpenShift project before the maven build is started.
Summary
This article and the demo asset demonstrate one possible way of how to implement the integration service using Red Hat Fuse. With the enriched list of technology and components brought by Camel, you may find a different way to implement the same solution. This is where the EIP (Enterprise Integration Pattern) is in placed to ensure however the developer is implementing the integration, it will be ensured the developers are following the best practices outlined by the EIP.
With the nature and flexibility of Camel integration approaches, we also see how easy it is for us to implement endpoints integration, message marshalling and unmarshalling, and most importantly how we can embrace reusability allowing us to bring our solution to market faster.