14 July 2016

Monitoring Tomcat with JavaMelody

In this post, troubleshooting specialist, Andy Overton describes an on-premise monitoring solution he deployed for a customer using a small number of Tomcat instances for a business transformation project. Using a step-by-step approach, he walks through his JavaMelody configuration and how he implements alerts in tandem with Jenkins.




Introduction

Whilst working with a customer recently I was looking for a simple, lightweight monitoring solution for monitoring a couple of Tomcat instances when I came across JavaMelody.

https://github.com/javamelody/javamelody/wiki

After initial setup - which is as simple as adding a couple of jar files to your application - you immediately get a whole host of information readily available with no configuration whatsoever.

Playing about with it for a while and being impressed, I decided to write this blog because I thought I might be able to use my experiences to throw some light on a few of the more complex configurations (e-mail notifications, alerts etc.).

Technology Landscape

I’m going to start from scratch so you can follow along. To begin with, all of this was done on a VM with the following software versions:
  • OS – Ubuntu 16.04
  • Tomcat – 8.0.35
  • JDK - 1.8.0_77
  • JavaMelody – 1.59.0
  • Jenkins - 1.651.2

Tomcat Setup

Download from http://tomcat.apache.org/download-80.cgi

Add an admin user:
Add the following line to tomcat-users.xml:


<role rolename="manager-gui"/>
<user username="admin" password="admin" roles="manager-gui"/>


Start Tomcat by running <TOMCAT_DIR>/bin/startup.sh

The management interface should now be available at: http://localhost:8080/manager


JavaMelody Setup

Download from https://github.com/javamelody/javamelody/releases and unzip.

Add the files javamelody.jar and jrobin-x.jar to to the WEB-INF/lib directory of the war file you want to monitor.

I used a simple testapp used for testing clustering. Obviously we’re not testing clustering here but it doesn’t actually matter what the application does for our purposes.

Download the clusterjsp.war from here (or use your own application):
http://blogs.nologin.es/rickyepoderi/uploads/SimplebutFullGlassfishHAUsingDebian/clusterjsp.war

Drop the war file in the <TOMCAT_DIR>/webapps directory and it should auto-deploy.

Point a browser to http://localhost:8080/clusterjsp/monitoring and you should see a screen similar to this screen grab from github:





First Look

For new users, I'll just offer a quick run-down of my out-of-the-box experience. First thing you see are the graphs you have immediately available:

  • Used memory
  • CPU
  • HTTP Sessions
  • Active Threads
  • Active JDBC connections
  • Used JDBC connections
  • HTTP hits per minute
  • HTTP mean times (ms)
  • % of HTTP errors
  • SQL hits per minute
  • SQL mean times (ms)
  • % of SQL errors
You can access additional graphs for such things as garbage collection, threads, memory transfers and disk space via the 'Other Charts' link, and helpfully these can be easily expanded with a mouse click. Less helpfully, there's no auto-refresh so you do need to update the charts manually.


If you scroll down, you'll find that 'System Data' will make additional data available and here you can perform the following tasks:
  • Execute the garbage collector
  • Generate a heap dump
  • View a memory histogram
  • Invalidate http sessions
  • View http sessions
  • View the application deployment descriptor
  • View MBean data
  • View OS processes
  • View the JNDI tree

You can also view the debugging logs from this page - offering useful information on how JavaMelody is operating.


Reporting Configuration Guide

JavaMelody features a reporting mechanism that will produce a PDF report of the monitored application which can be generated on an ad-hoc basis or be scheduled for daily, weekly or monthly delivery.

To add this capability simply copy the file itext-2.1.7.jar, located in the directory src/test/test-webapp/WEB-INF/lib/ of the supplied javamelody.zip file to <TOMCAT_DIR>/lib and restart Tomcat.

This will add 'PDF' as a new option at the top of the monitoring screen.

Setting up an SMTP Server
In order to set up a schedule for those reports to be generated and sent via email, you first need to set up a Send Only SMTP server.

Install the software: sudo apt-get install mailutils

This will bring up a basic installation GUI and here you can select 'Internet Site' as the mail server configuration type. Then simply set the system mail name to the hostname of the server.

You'll then need to edit the configuration file /etc/postfix/main.cf and alter the following line from inet_interfaces = all to inet_interfaces = localhost

Restart postfix withsudo service postfix restart

You can test it with the following command (replacing the e-mail address):
echo "This is a test email" | mail -s "TEST" your_email_address


Scheduling the Report
With the email done, the next step is to schedule JavaMelody to send out daily e-mails of the PDF report. Firstly we need to download a couple of additional libraries.



When you have these, copy both files to <TOMCAT_DIR>/lib and add the following code to <TOMCAT_DIR>/conf/context.xml (replacing the e-mail address):


<Resource name="mail/MySession" auth="Container" type="javax.mail.Session"
                                 mail.smtp.host="localhost"
                                 mail.from="no-reply@test.com"
/>
<Parameter name="javamelody.admin-emails" value="your_email_address" override="false"/>
<Parameter name="javamelody.mail-session" value="mail/MySession" override="false"/>
<Parameter name="javamelody.mail-periods" value="day" override="false"/>


Once the server is started, you can send a test mail by calling this action:

http://<host>/<context>/monitoring?action=mail_test

Alerts (Using Jenkins)

Alerting takes a little more setting up and isn’t provided by JavaMelody itself. Instead, it's provided by Jenkins with a Monitoring add-on, so first of all, you'll need to download Jenkins from:


Use the following command to run Jenkins (we need to run on a different port as we have Tomcat running on the default 8080):  java -jar jenkins.war --httpPort=9090

Jenkins is now available at: http://localhost:9090

The nest step is to install the following plug-ins for Jenkins:
  • Monitoring – Needed for linking in with JavaMelody
  • Groovy – Needed to run Groovy code. This is required for setting up the alerts.
  • Email Extension – Needed to customise the e-mails Jenkins sends out

To install the monitoring plugin:
  1. Click 'Manage Jenkins'
  2. Select 'Manage Plugins'
  3. Select 'Available'
  4. Find and select the 'Monitoring Plugin'
  5. Click 'Install without restart'

Then follow the same procedure for Groovy and Email Extension. 


Groovy Configuration

Now, let's make sure the Groovy runtime is installed and configured by using sudo apt-get install groovy to install it to /usr/share/groovy

In order to run our Groovy scripts and call JavaMelody methods we'll need log4j and JavaMelody on the Groovy classpath. JavaMelody uses an old version for log4j (1.2.9) which can be downloaded from:


To configure Groovy:
  1. Go to Manage Jenkins, select 'Configure System'
  2. Under the Groovy section, select 'Groovy Installations'
  3. Add a name for your installation.
  4. Set GROOVY_HOME to /usr/share/groovy


Email Extension Plugin Configuration
  1. Go to Manage Jenkins, select 'Configure System'
  2. Under Jenkins location, set the URL to: http://hostname:9090 (replacing hostname with your hostname)
  3. Set the System Admin e-mail address to: donotreply@jenkins.com (or something similar – this is the address that alert e-mails will be sent from.
  4. Under the Extended E-mail Notification section, set SMTP server to localhost

Creating Alerts
Next up we'll set up a test alert, which triggers when there are more than 0 HTTP sessions - obviously not realistic, but good for demo and testing purposes.

From the main Jenkins menu:
  1. Select 'New Item'
  2. Select 'Freestyle' project
  3. Add the following details:
    • Name - High Session Count Alert
    • Description - Test alert triggered when there are more than 0 HTTP sessions
  4. Under 'Build Triggers', select 'Build' and 'Periodically'

    Now you can schedule how often to run your alert check. The syntax is exactly like a cronjob. Here we will set it to run our check every 10 minutes using the following: */10 * * * *
  5. Under 'Build', click 'Add build step'
  6. Select 'Execute Groovy' script
  7. Set the 'Groovy Version' to whatever you called it previously
  8. Add the following Groovy code:

import net.bull.javamelody.*;

url = "http://localhost:8080/clusterTest/monitoring";

sessions = new RemoteCall(url).collectSessionInformations(null);

if (sessions.size() > 0) throw new Exception("Oh No - More than zero sessions!!!");

This simple piece of code calls the URL of JavaMelody, retrieves the sessions information and then if that size is greater than zero, throws an Exception. Add javamelody.jar and log4j jar to the classpath (under Advanced) e.g.:


/home/andy/javamelody/javamelody.jar:/home/andy/logging-log4j-1.2.9/dist/lib/log4j-1.2.9.jar

Under 'Post-Build Actions', select 'Add post build action', then select 'Email Notification', add the email address to send the alert to and finally, Save.

Testing

In order to test the alert triggers as required simply call your application e.g.


You should receive an e-mail with the subject 'Build failed in Jenkins', which looks something like this:


------------------------------------------
Started by user anonymous
Building in workspace <http://192.168.43.129:9090/job/High%20Session%20Count%20Alert%202/ws/>
[workspace] $ /usr/share/groovy/bin/groovy -cp /home/andy/javamelody/javamelody.jar:/home/andy/logging-log4j-1.2.9/dist/lib/log4j-1.2.9.jar "<http://192.168.43.129:9090/job/High%20Session%20Count%20Alert%202/ws/hudson4959397560302939243.groovy">
Caught: java.lang.Exception: Alert-Start
Oh No - More than zero sessions!!! Number of sessions: [SessionInformations[id=9BBFCF23C5126EDDBD44B371F1B11FD0, remoteAddr=127.0.0.1, serializedSize=229]]
 Alert-End
java.lang.Exception: Alert-Start
Oh No - More than zero sessions!!! Number of sessions: [SessionInformations[id=9BBFCF23C5126EDDBD44B371F1B11FD0, remoteAddr=127.0.0.1, serializedSize=229]]
 Alert-End
        at hudson4959397560302939243.run(hudson4959397560302939243.groovy:7)
Build step 'Execute Groovy script' marked build as failure


As Jenkins is generally used as a build tool, the outgoing e-mail isn’t the most user friendly when we’re looking to use it for alerting purposes. So, the final thing we will look at is altering the outgoing e-mail into something more legible.

Editing the Outgoing Email


First of all we will alter the Groovy script so that we can strip out the stack trace and additional information that we don’t need as we’re alerting on a specific condition of our app, not the underlying JavaMelody code.


In order to do so we will use Alert-Start and Alert-End to indicate the start and end of the alert message we want to put in the e-mail we will send out. Later we will use a regular expression to extract this from the whole Exception.

Go to the High Session Count Alert project and alter the last line of the Groovy script, changing it from:


if (sessions.size() > 0) throw new Exception("Oh No - More than zero sessions!!!");

to 

if (sessions.size() > 0) throw new Exception("Alert-Start\nOh No - More than zero sessions!!! Number of sessions: " + sessions.size() + “\nAlert-End);

  1. Click Configure
  2. Delete the e-mail notification post-build action
  3. Add a new one - Editable Email Notification
  4. Set Project Recipient List, add your e-mail address
  5. Set the Default Subject to - JavaMelody - High Session Count ALERT
  6. Set the Default Content to the following:

Build URL : ${BUILD_URL}

Alert : ${PROJECT_NAME}

Description: ${JOB_DESCRIPTION}

${BUILD_LOG_EXCERPT, start="^.*Alert-Start.*$", end="^.*Alert-End.*$"}

This will result in an e-mail containing the following:

Build URL : http://192.168.43.129:9090/job/High%20Session%20Count%20Alert/252/

Alert : High Session Count Alert

Description: Test alert triggered when there are more than 0 HTTP sessions

Oh No - More than zero sessions!!! Number of sessions: 1

The key thing here is the BUILD_EXCERPT. This takes in 2 regular expressions to indicate the start and end lines within the build log. This is where we strip out all of the extraneous stack trace info and just get the message between the Alert-Start and Alert-End tags.


To see a list of all available email tokens and what they display, you can click the "?" (question mark) next to the Default Content section.

Conclusion

Hopefully, this blog has given you a good starting point for using JavaMelody and Jenkins to monitor your Tomcat instances. There is a lot more that I haven’t covered but I’ll leave that as an exercise for the reader to dig a little deeper.

I’ve been impressed by it as a simple to set up, free monitoring tool. Configuring the alerts is a bit more of an effort but it’s nothing too difficult and it’s a tool I’d certainly recommend.

6 July 2016

Planning for a JBoss EAP 7 Migration


by Brian Randell




In my previous post, I had a ‘First Look’ at Red Hat JBoss EAP 7 and highlighted a few fundamental changes from EAP 6. This post has been written to dive deeper under the covers, and aims to examine the key differences between the two versions, looking primarily at the impact of migrating to this version from EAP 6.

I want to consider whether there are any operational considerations regarding migration, and further expand on some of the points raised when I first opened the EAP 7 box.


Support Timeline


Full Support for JBoss EAP 6 finishes at the end of June 2016 - which means there will be no minor releases or software enhancement to the EAP 6 code base from then on. Maintenance support ends June 2019 and Extended Life Support ends in June 2022.

So, if you're happy with the features you have and the system's stability, then bug fixes will still be provided for a while to come. However, if you're looking to use newer features and take advantage of those provided by Java EE 7, for example, then it's worth starting the evaluation cycle for JBoss EAP 7 now, so that when the first point release arrives (which is historically the more stable release) you are ready to implement into production.

Differences

As is the nature of new releases, some of the older technologies are not supported or are untested - and hence unverified whether they work. For JBoss EAP 7, it is only supported for Java 1.8+ and has not been tested on RHEL 5 or Microsoft 2008 server (Note: it has been tested on Microsoft 2008 R2 server).  

Some of the notable untested database integrations include Oracle 11g R1, Microsoft SQL Server 2008 R2 SP2 and PostgreSQL 9.2 – though I would expect these to be added to over time if there is demand. One addition to the database integration testing has been for MariaDB. Fundamentally, though, the support and testing is in line with previous versions of EAP and what you would expect.

Looking at the Java EE standards supported in EAP 7, JAX-RPC is now not available with preference to use JAX-WS. The standards that have been updated are:

  • Java EE
  • Java Servlet
  • JSF (JavaServer Faces)
  • JSP (JavaServer Pages)
  • JTA (Java Transaction API)
  • EJB (Enterprise Java Beans)
  • Java EE Connector Architecture
  • JavaMail
  • JMS (Java Message Service)
  • JPA (Java Persistence)
  • Common annotations for the Java Platform
  • JAX-RS (Java API for RESTful Web Services)
  • CDI (Contexts and Dependency Injection)
  • Bean validation


The major updates here are primarily for Java EE, JMS and JAX-RS that all have major version changes.

Corresponding to the standards updates, notable component changes from EAP 6 are :

  • JBoss AS has been replaced with the Wildfly Core
  • JBoss Web has been replaced with Undertow
  • HornetQ has been replaced with Active MQ Artemis (though the HornetQ module is retained for backwards compatibility)
  • Apache Web Server has been removed
  • jBeret has been added
  • JBoss Modules has been removed
  • JBoss WS-Native has been removed
  • JSF has been removed
  • The Jacorb subsystem has been removed and switched to OpenJDK ORB
  • The JBoss OSGi framework has been removed


With the standards, components and module changes, you can see that there are a lot of areas that will need to be checked, reconfigured and tested before using EAP 7 with existing code.

Migration

There should always be care consideration given to migrating major versions of an application. In all cases, full evaluation and testing should be undertaken to reduce the risk when deploying to the new environment. 

There are a significant number of changes between EAP 6 and EAP 7 with the updated standards used - deprecated APIs, modules and components and modified configuration structure. However there are also a number of compatibilities and interoperabilities provided in EAP 7 that should make the migration easier with proper planning and testing.

Migration tasks should be thought of from various points of view. The main ones I think about when migrating are:

1. Environment
  • Do I need to modify CPU, Memory, Storage, Network, Architecture for the new solution?
  • Can I upgrade inline or side by side, all at once or some servers at a time?


2. Code
  • Are there deprecated APIs that are used that need to be updated?
  • Do current API calls behave in the same way?


3. Server Configuration
  • Are there server configuration settings that need to be changed?
  • Are the CLI commands the same or are there new ones?


4. Monitoring
  • Is the monitoring you have in place compatible with the new solution?
  • Are there new configurations to add or amend for the updated components and modules?
  • Does the logging behave in the same way?


5. Process / Procedure
  • Are your procedures for operational tasks the same or do they need amending?
  • Are your operational scripts still fit for purpose?


6. Testing
  • Functional, Integration and Performance testing is required to ensure the application behaves within agreed thresholds.

Code

From a code perspective, as mentioned, there are a number of deprecated features and updated standards, so the code will need to be checked and verified to understand whether any code will need changing to ensure compatibility with the new and updated modules.

For this there is a tool called Windup which is part of the Red Hat JBoss Migration Toolkit that provides an analysis of your code and what will need to be changed.

Some areas that the developers need to be aware of that haven’t already been mentioned are:

  • RESTEasy has a number of deprecated classes
  • Hibernate Search changes
  • JBoss logging annotations change

There are a lot of areas to check in the code, so as a first pass it is sensible to use the Windup tool.

Server Configuration

For the server configuration there are several approaches:

The recommended approach is to use the JBoss server migration tool - but this is currently in alpha (and hence unsupported) and only currently works against EAP 6.4 standalone servers. This is currently being developed though, and I expect it to be expanded to work across more versions and be a full release.

An alternative is to use the EAP 6 configuration as the configuration for EAP 7 and use the inbuilt CLI commands for migrating the subsystems of messaging, web and jacorb over to the new subsystems. This though does not update all the required server configuration so you are caught with still potentially having to make changes in order to achieve a finalised configuration.

I personally will always keep CLI scripts to configure the server so if a new server is required I can easily run these scripts and the server will be configured. These can be run on a newly installed version of EAP 7 and amended as required to use the new subsystems and configuration structure.

None of the ways described are clean and simple solutions, so there will need to be close attention paid to ensuring the configuration is correct.

Some of the areas that you need to be aware of are:

  • The ‘web’ subsystem is now the ‘undertow’ subsystem
  • Valves are not supported and need to be migrated to Undertow handlers
  • Rewrite sub-filters need to be converted to expression-filters
  • The ‘messaging’ subsystem is now the ‘messaging-activemq’ subsystem and rather having a ‘hornetq-server’ it is now simply ‘server’
  • To allow EAP6 JMS connections ‘add-legacy-entries’ needs to set to true when migrating via CLI
  • The threads subsystem has been removed and now each subsystem uses its own thread management

It should be noted that the list of other changes that will affect the server configuration is large and too numerous to be listed here. This highlights how much care will need to be taken to get the server configuration right. When the JBoss Server Migration Tool is fully available then this will be a good option.


Architecture

There are also architecture concerns you need to be aware of when planning your migration. Some notable ones are :

  • Clusters must be the same version of EAP (So you will need to upgrade an entire cluster at a time)
  • JGroups now use a private interface (as opposed to public) as best practice means they should use a separate network interface.
  • Infinispan now uses distributed ASYNC caches for its default clustered caches rather than replicated
  • Messaging directory and file structure has changed. They are now named with reference to ActiveMQ rather than HornetQ.
  • Log messages are newly prefixed with the WFLY project codes

There are significant enough differences here to revisit the architecture design of your environment and verify it still fits for EAP 7.

Summary

There are a significant number of changes between JBoss EAP 6 and JBoss EAP 7, with a number of modules and components being updated to cater for the updated standards - resulting in API deprecation and configuration changes. 

This means the migration path may not be simple, the architecture may need to be reconsidered and the operational procedures may need to be modified. This can be eased by the use of Windup for code analysis, and the JBoss Migration Toolkit.

However there is still a lot to verify, reconfigure and test both from a code perspective and a server configuration/architecture perspective.



Resources


EAP 7 Supported Configurations -> https://access.redhat.com/articles/2026253
EAP 7 Included modules (requires a RedHat subscription account) -> https://access.redhat.com/articles/2158031
EAP 7 Component Details -> https://access.redhat.com/articles/112673

3 June 2016

How to Configure Oracle Wallet with Tomcat

by Claudio Salinitro

In my last post, I explained how I used Oracle Wallet with PHP as a way of offering my client a way of preventing the PHP developers gaining password access to the Oracle database. In this, a follow-on post, I'll be showing you how to configure Oracle Wallet with Tomcat.






Note: The wallet is created in exactly the same way as I showed in the first post. If you refer back to those instructions, we can then move quickly on to the software you'll need:

  • Tomcat
  • Oracle database 10g Release 2+
  • Oracle jar files:
    • oraclepki.jar
    • osdt_core.jar
    • osdt_cert.jar
    • ojdbc6.jar if you are using jdk6
    • ojdbc7.jar if you are using jkd7 or jdk8
Except for the ojdbcX.jar file, it seems you can find the other jars only inside the Oracle full client directories (not in Oracle instantclient) or in the Oracle database server directories.


Oracle Wallet configuration

Transfer the wallet on the Tomcat machine and create a file named tnsnames.ora inside the wallet directory with the following content:


connection_string =  
  (DESCRIPTION =  
    (ADDRESS = (PROTOCOL = TCP)(HOST = database_host)(PORT = database_port))    
    (CONNECT_DATA = (SID = database_sid))    
  )   

Where...
  • connection_string = the connection string related to the credentials stored in the wallet
  • database_host = the database hostname
  • database_port = the database listen port
  • database_sid = the sid of the database

An entry for each credential that is configured in the wallet and wants to be used by the application should be created.

Create a file named sqlnet.ora inside the wallet directory with the following content:


WALLET_LOCATION =    
   (SOURCE =     
     (METHOD = FILE)     
     (METHOD_DATA =     
       (DIRECTORY = [wallet path])     
     )      
    )     

SQLNET.WALLET_OVERRIDE = TRUE 

Note. The user running Tomcat needs to have read permission on the wallet files.

Tomcat configuration

Add the jar files (ojdbc drivers + oraclepki + osdt_core and osdt_cert) in the $CATALINA_BASE/lib directory.

Set the JAVA option -Doracle.net.tns_admin=wallet_path


export CATALINA_OPTS=-Doracle.net.tns_admin=/apps/tomcat/conf/wallet

Finally restart Tomcat.

Test JSP page

If everything is OK, the configuration should already be working - but we need a test web application to to confirm it.

For testing purposes, we can use the ROOT web application that is already in place, and which is shipped with any Tomcat distribution. Create a walletTest.jsp file in your $CATALINA_BASE/webapps/ROOT folder, with the following content:


<%@page import="java.sql.*, javax.sql.*, javax.naming.*, java.io.*"%>

<html>
  <head>
    <title>DB Test</title>
  </head>
  <body>

<%
    Context context = new InitialContext();
    Context envCtx = (Context) context.lookup("java:comp/env");
    DataSource ds = (DataSource)envCtx.lookup("jdbc/myoracle");
    PrintWriter res = response.getWriter();
    Connection conn = ds.getConnection();
    Statement stmt = conn.createStatement();
    ResultSet result = stmt.executeQuery("SELECT 1 FROM DUAL");
    while (result.next()) {
      String code = result.getString(1);
      res.print(code);
    }
 %>

  </body>
</html>

And add the following configuration to the $CATALINA_BASE/conf/context.xml file, between the <context></context> tags:


<Resource name="jndi_name" auth="Container" type="javax.sql.DataSource" driverClassName="oracle.jdbc.OracleDriver" url="jdbc:oracle:thin:/@connection_string" connectionProperties="oracle.net.wallet_location=wallet_path"/>

Where...


  • wallet_path is the path to the directory containing the wallet - in my case    /apps/tomcat/conf/wallet
  • connection_string is a text string that will be used by our application to connect to the database with the related credentials - in my case db_credentials1
  • jndi_name is the jndi name used by the applications to use the datasource - in my case jdbc/myoracle

Security considerations

As it was for Apache, also for Tomcat or any other application server, the wallet security relies on the OS file permissions. Any users who have access to the wallet files will have access to the database data.

Also keep in mind that you cannot configure more than one wallet on the same Tomcat instance. This means that any applications running on the same Tomcat instance can potentially access the data of other applications once it discovers the connection string.




How to Configure PHP to use the Oracle Wallet

by Claudio Salinitro


In today's post, I'll be looking at a problem one of our clients encountered when trying to prevent the PHP application developers from gaining access to their Oracle database passwords. Whilst the obvious answer might be to move the connection parameters to an environment-specific file, the best solution I found for our client was to configure PHP to use the Oracle Wallet. Here, I'll go through the configuration process I undertook step-by-step.







The problem

As I suggested in my introduction, the first obvious solution to the problem might be to move the connection parameters to an environment-specific file, but this will have 2 consequences: 

1. It will demand a change to their deployment procedure

2. The password would be stored in clear text on the filesystem

The solution I wanted would have to overcome these issues, and what I chose to do was configure PHP to use the Oracle wallet.


What is Oracle Wallet?

Well, from the Oracle documentation...

Oracle Wallet provides a simple and easy method to manage database credentials across multiple domains. It allows you to update database credentials by updating the Wallet instead of having to change individual datasource definitions. This is accomplished by using a database connection string in the datasource definition that is resolved by an entry in the wallet.

Exactly what we need!


Software needed

  • Apache configured with mod_php or php-fpm (compiled with oci8)
  • Oracle instant client (basic + sdk)
  • Oracle database 10g Release 2+


Create the Oracle Wallet

Create a wallet with the following command:


mkstore -wrl "wallet_path" -create
Enter password:

Enter password again:

This will create an empty password protected container (the wallet) to store your database credentials. The wallet is composed of two files, cwallet.sso and ewallet.p12, and these will be created in the [wallet path] location.

Let’s start adding some credentials to the newly created wallet, execute:


mkstore -wrl "wallet_path" -createCredential connection_string username password

Where:

  • wallet_path is the path to the directory containing the wallet
  • connection_string is a text string that will be used by our application to connect to the database with the related credentials
  • username and password are the database credentials that will be used for the connection

You can add all the credentials you want in the same wallet, as soon as they have different connection string, and you can list the credentials stored in the wallet with the following command:


mkstore -wrl "wallet_path" -listCredential


Oracle instantclient

On the machine running php, we need an Oracle client to connect to the database. Usually on the web server there is no need to install the full Oracle client, and for this reason I prefer to stay light and use the Oracle instantclient. It's a lightweight version, with no installation needed - and  is more than enough for our needs.

Download the following Oracle instantclient files from the Oracle website:

  • Instant Client Package - Basic: All files required to run OCI, OCCI, and JDBC-OCI applications

  • Instant Client Package - SDK: Additional header files and an example makefile for developing Oracle applications with Instant Client

Then unzip both packages so that you have the sdk folder inside the basic instant client folder:


/opt/oracle/instantclient_12_1
/opt/oracle/instantclient_12_1/sdk

Create the following symbolic links:


cd /opt/oracle/instantclient_12_1
ln -s libclntsh.so.12.1 libclntsh.so
ln -s libocci.so.12.1 libocci.so



Set the environment variable LD_LIBRARY_PATH:



export LD_LIBRARY_PATH=/opt/oracle/instantclient_12_1:$LD_LIBRARY_PATH

Compile PHP with the option:
--with-oci8=instantclient,/opt/oracle/instantclient_12_1

Apache/PHP configuration


Transfer the wallet on the Apache web server machine, and create a file named tnsnames.ora inside the wallet directory with the following content:


connection_string =  
  (DESCRIPTION =  
    (ADDRESS = (PROTOCOL = TCP)(HOST = database_host)(PORT = database_port))    
    (CONNECT_DATA = (SID = database_sid))    
  )   

Where:

  • connection_string = the connection string related to the credentials stored in the wallet
  • database_host = the database hostname
  • database_port = the database listen port
  • database_sid = the sid of the database
An entry for each credential that is configured in the wallet and wants to be used by the application should be created.
Create a file named sqlnet.ora inside the wallet directory with the following content:


WALLET_LOCATION =    
   (SOURCE =     
     (METHOD = FILE)     
     (METHOD_DATA =     
       (DIRECTORY = [wallet path])     
     )      
    )     

SQLNET.WALLET_OVERRIDE = TRUE 

Note: The user running Apache needs to have read permission on the wallet files.


Configuration for mod_php:

Add the following environment variables to the apache startup:


export ORACLE_HOME=/opt/instantclient   
export LD_LIBRARY_PATH=/opt/instantclient   
export TNS_ADMIN=/opt/wallet  

I usually add these variables at the beginning of the apachectl startup script, but any other way is fine.

Stop Apache, and start it again. The apachectl restart option doesn’t work in this case because the master process will not be restarted, and will not see the new variables.

Configuration for php-fpm


If you are using php-fpm, you have to add the following settings in your pool configuration section:


env[ORACLE_HOME] = /apps/instantclient
env[LD_LIBRARY_PATH] = /apps/instantclient
env[TNS_ADMIN] = /apps/httpd/conf/wallet
 

Restart php-fpm processes.

Check if everything works

Create a PHP file in the DocumentRoot of your web server to test the connection:


<?php
$conn = oci_connect("/", "", "[connection string]", null, OCI_CRED_EXT);

$statement = oci_parse($conn, 'select 1 from dual');
oci_execute($statement);
$row = oci_fetch_array($statement, OCI_ASSOC+OCI_RETURN_NULLS);

print_r($row);
?>

And from the browser, call that page. Everything is working if the output is something like this:


Array ( [1] => 1 )

Security considerations

Keep in mind that the wallet security relies on the OS file permissions. Any users who have access to the wallet files will have access to the database data.

Also keep in mind that with mod_php you can have one wallet per Apache installation. This means that, in a shared environment, any applications running under the same Apache can potentially access to the data of other applications once it discovers the connection string.

With php-fpm you can have a different wallet and different configuration for each fpm pool.


Next Time...

Hopefully, the solution I found for this client will work for you, and in my next post, I'm going to continue on the same theme, but this time look at using the Oracle Wallet with Tomcat.









24 May 2016

JBoss EAP 7 - A First Look

by Brian Randell


In this post, Brian Randell takes a peek at the new JBoss EAP 7.0.0 release and gives his impression as a JBoss consultant and administrator. He'll dig deeper under the cover and a throw little light on some of the new features and enhancements in future posts, but for his first look he'll reveal what he sees when getting it up-and-running for the first time.





The first version of JBoss EAP 7 was released on 10th May 2016 (Red Hat JBoss EAP 7.0.0). It's based on WildFly 10 (http://wildlfy.org/and uses Java SE 1.8 implementing Java EE 7 Full Platform and Web Profile Standards. The full list of supported configurations are listed here (please note that access requires a Red Hat subscription)https://access.redhat.com/articles/2026253 


...I could see a number of areas that were of immediate interest to me:

  • The replacement of HornetQ with Active MQ Artemis(https://activemq.apache.org/artemis/index.html)
  • The replacement of JBoss Web with Undertow (http://undertow.io/)
  • Ability to use JBoss as a Load Balancer
  • Server Suspend Mode
  • Offline Management CLI
  • Profile Hierarchies
  • Datasource Capacity policies
  • Port Reduction
  • Backwards compatibility with EAP 6 and some interoperability with EAP 5

There are many more enhancements and features listed in the release notes and I am sure you will have others spring out at you as items you want to investigate more. Putting these aside for now let’s get it installed.

When I'm looking at a new system, I like to dive in and get it running, then investigate it from a first look (where I concentrate on normal operation), through to a more detailed investigation on those areas that are of interest to me.

For my first look at JBoss EAP 7, I used an Amazon EC2 t2.medium tier shared host running Red Hat Linux 7.2 (Maipo) with 4GB Ram and 2vCPUs.  I downloaded the Oracle Java JDK 8u92 (http://www.oracle.com/technetwork/java/javase/downloads/index.html ) and JBoss EAP 7.0.0 (http://developers.redhat.com/products/eap/download/) (requires Red Hat subscription) zip files and extracted them into /opt/java and /opt/jboss directories respectively. I then created users java and jboss and chown’d the respective files. I set up JAVA_HOME environment variable and I was good to go.

Fundamentally – running JBoss EAP 7 is the same as running EAP 6. You install it in the same way and run it in a similar way. The only difference for me was running on RHEL 7.2 where if you set up JBoss to run as a service you run the command without the ‘.sh’ for the script. Placing the script jboss-eap7.sh in /etc/init.d/ and then registering it through chkconfig the service is run by the command:


service jboss-eap7 start

whereas on RHEL 6 you run it as:


service jboss-eap7.sh start

The first difference I noticed when running JBoss EAP 7 for the first time is that as the libraries are based on WildFly rather than jboss-as, the logs show WFLY references rather than JBAS references. For any of us that look for the references to search for certain log entries and have monitoring set up against them this will be a big change. For example the JBoss start message is now under reference WFLYSRV0025 (whereas it used to be JBAS015874)


INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: JBoss EAP 7.0.0.GA (WildFly Core 2.1.2.Final-redhat-1) started in 4142ms - Started 306 of 591 services (386 services are lazy, passive or on-demand)
INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss EAP 6.4.0.GA (AS 7.5.0.Final-redhat-21) started in 3441ms - Started 192 of 229 services (68 services are lazy, passive or on-demand)

You will also notice here that even though I am running the same configuration file (standalone-full.xml) the new EAP 7 server starts a lot more services which making it start slower that EAP 6. On average (over 10 starts) EAP 7 took 4180ms, whereas EAP 6 took 3573ms

We can also compare with using the standalone.xml and you can see that also starts a lot more services for EAP 7 which means it starts slower than EAP 6. An average of 3289ms for EAP 7 and 2667ms for EAP 6.


INFO  [org.jboss.as] (Controller Boot Thread) WFLYSRV0025: JBoss EAP 7.0.0.GA (WildFly Core 2.1.2.Final-redhat-1) started in 3227ms - Started 267 of 553 services (371 services are lazy, passive or on-demand)
INFO  [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss EAP 6.4.0.GA (AS 7.5.0.Final-redhat-21) started in 2652ms - Started 153 of 191 services (57 services are lazy, passive or on-demand)

The next difference I noticed was on the admin console where the layout has been changed. There are the same high level options as from the 6.4 console but when navigating into the sections the layout changes become more noticeable.




Figure 1 - JBoss EAP 7 Admin Console



Figure 2 - JBoss EAP 7 Subsystem Navigation



Figure 3 - JBoss EAP 7 Subsystem Settings

This unfortunately means more clicks to navigate to the same point you would have got to with 6.4, and as the settings encompass a whole screen you have to click back before you can navigate elsewhere. On first look this could become a frustration of using the console.

When using the CLI there are some other differences to be seen. The default port for connection to the CLI has changed from 9999 to 9990. Looking at the port configuration you can see a limited range of ports configured in EAP 7. This is because the http and management ports are used for a variety of protocols.



You can see there are no management-native or messaging ports.
It is also worth noting that the default management-https port is now 9993 rather than 9443 as it was before.

There are also some new CLI commands that can be used, such as set and unset to assign variables, unalias so you can turn off a defined alias and connection-info to show details of the connection.

There are also some new CLI operations that can be used such as list-add, list-get, list-clear, list-remove, map-get, map-clear, map-put, map-remove and query which can add and set attributes to an entity. These aren’t very well documented and will need further investigation.

There is also suspend and resume which suspends the server enabling it to complete its tasks gracefully without accepting new requests at which point you can resume it again.

Future blog posts will delve into the technology changes, features and enhancements, but from an initial first look at JBoss EAP 7 and before we do any deep investigation there are some immediate differences that will need to be thought of and evaluated when converting production systems from EAP 6.

References


JBoss EAP 7 download (requires Red Hat subscription)
http://developers.redhat.com/products/eap/download/


Red Hat EAP 7 – supported configurations (requires Red Hat subscription)
https://access.redhat.com/articles/2026253

Undertow - http://undertow.io/