Introduction and Background Information
Apache Tomcat is a Java based web container that is used to host a variety of applications. In 1999, Tomcat was initially introduced to the open source group. Since then, it has been supported by the open source community and widely accepted in the IT industry. This project was actually developed by James Duncan Davidson (software architect at Sun Microsystems). James made his project open source and played an important role in donating the project to Apache Software Foundation. Since its donation, Tomcat has undergone rapid development in many areas from the developer community. Tomcat has redefined the meaning of Java server and it is one of the most widely used open source software packages.
The Tomcat web server architecture is depicted below.
In current scenario, Tomcat is running in production environments and it is also being used for mission critical projects in many enterprises.
This white paper focuses on the performance improvement strategies for Tomcat server.
The Tuning Approach
Performance tuning plays an important role in running a web application without downtime. Tuning the server performance may vary from one application to the other. As each application has its own requirements, it is a tricky task to tune Tomcat.
By default, Tomcat packages are customized for the production run.
Why is it necessary to tune Tomcat web server?
The answer to this question is very subjective. Each web application has its own requirements. Some applications may need high memory while others need less memory. The administrator has to tune Tomcat as per the requirement of an application. Performance tuning is still incomplete with JVM tuning. There are certain things that play a major role in the performance of an application such as database configuration, OS level setting and the hardware.
The following figure illustrates various types of performance tuning:
Different aspects that can affect the performance of Tomcat server:
Application Code
If the code is not developed properly, it may cause performance issues. For instance, if the database connection from the application code is not closed properly then a stale connection gets created in the database and the application runs slowly.
Middleware Services
Middleware services may lead to application connectivity issues with the external interface. Now-a-days, many mobile applications use web services to connect to the server and fetch the application data. In this way, we are only exposing web services to the Internet and not the entire application server.
Database Tuning
The database can cause major issues in the performance of an application hosted in the Tomcat server. If the query processing is slow then it sends a delayed response to all queries.
JVM Tuning
Every application has its own memory requirement. If an application has a very huge memory requirement and you have allocated less memory then you may face 'out of memory' issues.
Infrastructure
Infrastructure issues may also affect the performance. If there is an Internet connectivity issue with the network or a packet drop in the network, it can degrade the performance.
How to Start Performance Tuning?
When you are prepared to run Tomcat server, you may need to do some performance tuning so that it serves all requests efficiently. The art of tuning the server is a complex one. It consists of measuring, proper understanding, changing and measuring again.
At the time of developing an application, we may define the application architecture, how the application is going to perform and the resources required for an application. Administrators should follow few thumb rules for increasing the performance.
The process flow for performance tuning is shown in the figure below:
Being Proactive
This process deals with the estimation of most common application failures that may arise. To avoid application failures, you need to prepare appropriate solutions. You can use latest profiling tools and keep track of such issues.
Note: Around 70% of issues could be resolved in the production environment if you can proactively check and eliminate all the bottlenecks and this method gives your customers great satisfaction.
Proper Logging and Monitoring
You must enable proper logging for Tomcat and this may help you in tracing the major issues that are about to occur in production environment. It is advised to monitor the system accordingly.
Knowledge base for the application and issue trends
A knowledge base is recommended for applications running 24x7. A good documentation for the application's support is also recommended so that the team has same information regarding all the issues.
Following the problem management approach
It is a good approach to perform root cause analysis (RCA) for all the issues so that recurrence of such issues could be avoided in future.
Thread Tuning
Thread tuning plays a major role in improving Tomcat's performance. Improper tuning of the thread may degrade the performance. The thread pool is defined as the ability of the web server to accept number of requests.
You can configure two types of thread pools such as shared pool and dedicated pool.
These configurations need to be performed in TOMCAT_HOME/conf/server.xml.
Increasing the Heap Size in Tomcat
To increase the heap size, we must add JAVA_OPTS parameter in catalina.sh, which could be found in TOMCAT_HOME/bin.
For instance, to increase max heap size to 512 MB and to set Perm Gen = 256 MB, the JAVA_OPTS parameter is:
JAVA_OPTS="-Xms128m -Xmx512m -XX:MaxPermSize=256m"
Note: All the changes in configuration for the JVM parameter will take effect when you restart the Tomcat server. You can verify the changes done in the JVM parameter by running the jmap command.
OS Tuning
Every operating system has its own prerequisites to run Tomcat server and has to be tuned according to the requirements.
Some useful tips:
64 bit versus 32 bit VM
We can address larger amounts of memory by using 64 bit VMs as compared to 32 bit VMs. In case of memory intensive applications, we can allocate more than 4 GB JVM for certain applications.
File size
As per the requirement of application, the file size is set in the operating system. If the application is using large number of transactions then we must increase the file limit.
Ulimits
Depending on the session, users can increase limits.
Huge page size
In many instances, applications usually send a large page size that may slow down your system. So the only option is to increase the page size as per the needs of an application.
Tomcat Clustering
A cluster is a group of servers connected together that can perform similar functionality. They are generally connected to each other through high speed Ethernet. Clusters are highly used in security areas, financial & banking sectors and so on.
Benefits of Clustering
There are many benefits of clustering especially in middleware environment. The benefits include:
Scalability: Scalability helps a system architect to accommodate future enhancements. Assume that an application has 150 concurrent users. At the time of an event, you may expect around 600 concurrent users. So clustering is considered as one of the best options to ensure that the system runs as expected.
High availability: High availability systems are implemented in banking and financial sectors where all the transactions need to be recorded and the environment should be up 99.99%. These businesses can't afford to have their websites down even for one minute.
High performance: A major benefit of clustering is that it boosts up the performance by n times, where n is the number of systems. For instance, if you are running the system with a single server and the system supports 120 concurrent users, then by adding another system you can support more than 200 concurrent users. If you wish to minimize the response time, you can just use JVM performance tuning.
Cloud computing: Clustering is also very useful in cloud computing platforms. It is used to improve the performance at the time of setting the grid computing architecture for cloud computing models.
The two types of clustering architectures are Vertical clustering and Horizontal clustering.
Vertical Clustering
It consists of a single hardware with multiple instances running and using the shared resources from the system. This is very useful in the development and quality systems for the team to test application's functionality. Vertical clustering could also be utilized in some cases where there is a resource crunch for the hardware. It normally uses the concept of shared resources such as RAM, CPU and so on.
Horizontal Clustering
In horizontal clustering, instances are actually configured separately on individual machines and they are connected through high speed Ethernet. This is a popular implementation method in production environments. Resources of one machine are not allowed to share with the other machine and failover could be achieved in case of failures in the hardware.
Now let us consider a cluster of Apache httpd and Tomcat instances. Here we can implement all the necessary features in multiple spots.
The following figure shows the request sequence where a website is served on a cluster of Apache httpd and Tomcat instances.
Few Clustering Technologies Include:
DNS request distribution
Usually DNS server is configured to give one IP address for one Apache httpd server instance. With DNS request distribution, we can configure to give three IP addresses. Each address is allocated to a separate httpd or tomcat instance.
NAT (TCP Network Address Translation) request distribution
DNS may give many IP addresses to various browsers. But the initial IP address could be answered by NAT request distributor. NAT distributor can act as a gateway to many servers behind it. We can utilize this type of distributor to resolve failover and load balancing issues.
mod_proxy_balancer load balancing
If you would like to run many instances of Tomcat behind httpd instances then you need to use mod_proxy_balancer to distribute all the requests. It is also useful to keep various requests from being distributed to failed instances of Tomcat.
Troubleshooting
Troubleshooting is an art of resolving crucial issues. It is not possible to troubleshoot any issue by just referring to the problem statements. You have to narrow down the problem to its root level and solve it. Slowness of application is a major issue that makes it very difficult for administrators to solve. In many cases, the components of a web application such as the operating system, database and the web server are responsible for slowness.
End-User Troubleshooting
Here you need to perform the following steps:
- Application could be accessed from the browser and you can check how much time it takes to load the page.
- Check the ping response of the server from the user side. An appropriate response denotes that the connectivity for server and user machine is good.
Web Server Troubleshooting
Here we have to dig down in the server to check whether there are any issues.
- If the web server process is running, check how many processes are running by using the command: ps –aef|grep httpd. This command shows all the processes running for the Apache httpd server. If they are greater than 50, it indicates that there is an issue such as high user traffic, high CPU utilization or high disk I/O.
- You can also check memory status and CPU utilization to see if any Apache processes are consuming a high CPU usage by using appropriate commands.
- You must verify Apache logs and look for errors in the error and access logs.
Java Based Applications
In case of Java based applications, slowness is caused due to the issues such as improper application deployment, JVM memory and incorrect DB configuration.
- Check Java processes and the load average for the instance machine by using the command ps –ef|grep java. The load average can give you substantial clues.
- Check the Tomcat logs that could be found in TOMCAT_HOME/logs and search for the exceptions.
Capacity Planning
In production, capacity planning is the most important thing to keep in mind while tuning Tomcat server. Here bandwidth and hardware play a vital role in serving high volume of traffic.
Capacity planning is the method of estimating the operating system, hardware and the bandwidth required for a website by understanding the total network traffic a website is supposed to handle. Here you need to finalize the characteristics of acceptable services and identify appropriate operating system and hardware to meet all the service requirements.
Procedure for Tomcat Capacity Planning
1. Characterizing the workload
If your website is up and running then you can measure the number of requests per second, summarize various kinds of other requests and also measure the utilization of resources. If your website is not running yet then you need to make some calculated guesses at the requests. To determine the resource requirements, you can just run staging tests.
2. Analyzing the performance trends
You must know what requests can generate maximum load and how these requests are different when compared to other requests. This is very much useful in knowing what to optimize so that you may have a good impact on all your servers. For instance, a servlet that queries a database may take too long to send a response. In this case, the response time could be improved by caching some data in RAM.
3. Minimum acceptable service requirements
In some cases, you may want the end user to get a web page response within 10 to 15 seconds. Such minimum acceptable service requirements may vary from one enterprise to the other. Other types of requirements include minimum number of service requests per second, number of users, number of sessions and so on.
4. Infrastructure resources
Infrastructure resources include bandwidth circuits, computer hardware, operating system and so on. You need to deploy and test at least one server that mirrors what you may have for production to make sure that it meets all your requirements. While testing Tomcat, it is recommended to test with more than one JVM, different memory settings and also request thread pool sizes.
If the 4th step meets your requirements then you may deploy and use Tomcat as your production server. Else you have to redo the above step until all the needs and service requirements are met.
Conclusion
As performance testing is an iterative process, it is very essential to document the test results for all iterations as well as configuration settings. After the capacity planning, your website would be much better tuned for performance. This is mainly due to rigorous testing of a number of options. You will gain a considerable amount of performance by just deploying appropriate hardware, operating system and JVM combination for the use of Tomcat server.