Asynchronous EJBs on TomEE

After configuring Asynchronous EJB invocations on WebSphere for the last few weeks at work, I had some ideas on how Asynchronous EJBs would work on TomEE and how they can be configured. EJB 3.1 specification does not say anything about how to implement asynchronous invocations and the semantics can differ a bit between application servers.

This article:

• Gives a short introduction into Asynchronous EJB
• Explains the configuration options of TomEE with respect to Asynchronous EJBs
• Provides some ideas on how to configure your own system
• Includes a project demonstrating configuration options with TomEE

A Quick Introduction into Asynchronous EJBs

The EJB 3.1 specification introduced the concept of asynchronous method invocations of session beans. An asynchronous session bean invocation is not a choice of the client; the session bean marks a business method as @Asynchronous as shown in the example below, and thereby, this method is always executed asynchronously.

import javax.ejb.AsyncResult;
import javax.ejb.Asynchronous;
import javax.ejb.Stateless;
import java.util.concurrent.Future;

@Stateless
public class AsyncEJB {

    @Asynchronous 
    public Future sayHello(String name) { 
        return new AsyncResult<>("Hello, " + name + "!");
    } 
}

The client can simply invoke the method sayHello, which causes the container to make the method invoke asynchronously (most likely on another thread) but it will not get the result immediately.  Instead, the client can use java.util.concurrent.Future to get the result. It has the choice to block or simply to check if a result is available as the example below shows:

@EJB
private AsyncEJB asyncEjb;
...
public void f() {
    Future asyncResult = asyncEjb.sayHello("World");
    if (asyncResult.isDone()) { <1>
        String result = asyncResult.get(); <2>
    }
    String result2 = asyncResult.get(1, TimeUnit.SECONDS); <3>
}

1. Check if the asynchronous method has already finished execution.
2. Get the result Hello, World! This will block until the method execution has finished, if it has not before.
3. Returns the result if the asynchronous method execution terminated within 1 second. If the method did not finish within this time the method will throw a TimeoutException.

The specification says nothing about how an application server has to implement asynchronous method invocations and how it should be configured. The EJB 3.1 specification states:

“A session bean can expose methods with asynchronous client invocation semantics. For asynchronous invocations, control returns to the client before the container dispatches the invocation to a bean instance. An asynchronous method is a business method exposed through one or more of the Remote business, Local business, or no-interface session bean views.”

— JSR 318: Enterprise JavaBeans^TM Version 3.1, Section 4.5

It only says that the container must return to the client while the asynchronous method is invoked. Although though not mentioned, it is probable that most implementations will use a java.util.concurrent.ThreadPoolExecutor to implement this process. This class is highly configurable and it is up to the container as to how much of this configuration is exposed to the client.

The next section will show what configuration options TomEE offers to configure and tune asynchronous method executions.

Configuration of TomEE for Asynchronous EJBs

The following configuration options allow TomEE to configure the internal ThreadPoolExecutor that processes asynchronous EJB calls:

AsynchronousPool.Size

This parameter serves as a default for AsynchronousPool.CorePoolSize and AsynchronousPool.MaximumPoolSize. If not defined, the default is 5.

AsynchronousPool.CorePoolSize

The number of core threads the ThreadPoolExecutor uses to execute the asynchronous EJB calls. If the thread pool has less core threads, at the time of a call, than configured it will create a new core thread even if other worker threads are idle. The default for this value is the value of AsynchronousPool.Size.

AsynchronousPool.MaximumPoolSize

The maximum number of threads the thread pool will use at a time, including the core threads. The ThreadPoolExecutor may allocate more threads when new tasks are submitted and all other threads are already busy. The default is AsynchronousPool.CorePoolSize so that no threads other than the core threads are created.

AsynchronousPool.QueueSize

The maximum number of invocations that can be queued when all worker threads are busy. It depends on the option AsynchronousPool.QueueType, if this parameter is used at all. Configuring this value requires some knowledge about the concrete asynchronous tasks; how long they take, what quality of service is expected, etc. It cannot be simply derived from the CPU and the memory that TomEE has access to. Its default is the same value as AsynchronousPool.CorePoolSize; that is by default 5 worker threads process asynchronous methods while 5 more invocations can be queued.

AsynchronousPool.KeepAliveTime

Configures the time after which idle worker threads are stopped. The default is 60 Seconds.

AsynchronousPool.QueueType

This configures the work queue of the ThreadPoolExecutor. Depending on the type of work queue the semantics of invoking an asynchronous method can change.

LINKED

Tasks are queued in a java.util.concurrent.LinkedBlockingQueue of size AsynchronousPool.QueueSize. This is the default when QueueSize is > 1. LinkedBlockingQueues can be unbounded but TomEE will always set a queue size of at least 1.

ARRAY

Tasks are stored in a java.util.concurrent.ArrayBlockingQueue of size AsynchronousPool.QueueSize. This is very similar to using the LINKED queue type.

SYNCHRONOUS

Tasks are queued in a java.util.concurrent.SynchronousQueue. This queue type has no size. It will only accept an asynchronous method invocation if there is a free thread waiting. Thereby, the semantics of invoking an asynchronous method changes from scheduling a method invocation to starting it. For example, the calling thread blocks until the asynchronous method starts. The fair policy of the SynchronousQueue can be configured via AsynchronousPool.QueueFair, which defaults to false. It is the default when the QueueSize is 1.

AsynchronousPool.ShutdownWaitDuration

This defines how long TomEE will wait, at most, for all currently executing and scheduled invocations when shutting down. If this timeout elapses, and there are still asynchronous method executions running, TomEE will continue shutting down.  The default is 1 minute.

AsynchronousPool.AllowCoreThreadTimeOut

This option defines if core threads will be stopped after idling for longer than Asynchronous.KeepAliveTime. The default is true.

AsynchronousPool.OfferTimeout

This is the time a thread waits at most to put an asynchronous method invocation into the work queue. If the invocation cannot be put into the queue after this timeout has elapsed, the client will receive an exception.

This parameter is only used when no AsynchronousPool.RejectedExecutionHandlerClass is configured. (The reason for this is that the default implementation, the OfferRejectedExecutionHandler, tries to submit the task with this timeout if the initial try failed.)  Possible values are 1 second, 1 minute, 10 seconds, etc. The default timeout is 30 seconds.

AsynchronousPool.RejectedExecutionHandlerClass

This is when a configured instance of this class is called and if submitting the task to the Executor fails. Note that if all workers are currently busy, and the work queue is full, the invocation will fail immediately. TomEE will create an instance of this class calling the default constructor and pass it to the ThreadPoolExecutor.

Finding the Right Pool, Queue and OfferTimeout Settings

This is one of the hardest parts as it is nearly impossible to provide simple rules on how to configure these options; especially without knowing the actual tasks being executed asynchronously.

Pool Sizes

Depending on the type of asynchronous code that is executed, the thread pool size value may vary from small to large. Methods that have a lot of I/O, (such as calling other remote systems or databases) will benefit from larger pool sizes as the CPU then does not need to wait for a response from a database. If the asynchronous methods are basic Java code, without any further remote calls, pool sizes that are about the number of CPU cores are reasonable.

Queue Sizes

If no synchronous queue is configured the queue size plays an important role.

Higher queue sizes can make the application consume too much memory. Too big of a value will also hide that the system is currently choking because the asynchronous methods executions hang. This will not become apparent before the queue is saturated.

On the other hand, larger queues make the system more responsive with respect to the synchronous calls. The system will less often  use the OfferTimeout to schedule an invocation.

OfferTimeout

Looking back at WebSphere, it does not support an OfferTimeout, it only has the two extremes; block until the call can be scheduled, or to fail if the queue is saturated. This makes the application development a little easier;  you either care about the work queue being saturated (and accept the invocation to fail) or expect the invocation to be successful at some point in time. Unfortunately, the anti-pattern to make an asynchronous method execution invoke another asynchronous method can jam the whole server (and that topic is cause for a whole new article).

TomEE allows for a finer configuration with the OfferTimeout. If your application is able to handle failed scheduling of asynchronous method invocations, then you can configure lower values.

On the other hand if your application is not able to handle failed invocations you can configure higher values but still ensure that the system stays running smoothly (assuming the fact that an error might fall back to the client). If it has its own RejectedExecutionHandler, you can handle scheduling the invocation yourself in any way you want without modifying the application.

Try it out yourself

This project contains an Arquillian based test that starts a remote TomEE instance and schedules some asynchronous methods. You can simply start the test like this:

cd AsyncEJBTest
 ./gradlew test

The project contains only a stateless EJB with one asynchronous method that sleeps for 5 seconds:

@Stateless
public class AsyncEJB1 {

    private final Logger LOG = Logger.getLogger(getClass().getSimpleName());

    @Asynchronous
    public Future simpleCall(int i) {
        LOG.info(">> simpleCall " + i + " Thread name: " + Thread.currentThread().getName());
        try {
            Thread.sleep(5000);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        LOG.info("<< simpleCall " + i);
        return new AsyncResult<>(i);
    }
}

The test invokes the asynchronous method simpleCall 11 times:

@RunWith(Arquillian.class) 
public class AsyncTest {

    private static final Logger LOG = Logger.getLogger(AsyncTest.class.getSimpleName());

    @Deployment
    public static WebArchive deploy() throws Exception {...}

    @EJB
    private AsyncEJB1 asyncEJB1;

    @Test
    public void test() throws Exception {
        for (int i = 1; i <= 11; i++) {
            try {
                Future asyncResult = asyncEJB1.simpleCall(i);
            } catch (Exception e) {
                LOG.log(Level.SEVERE, "Error while scheduling task " + i, e);
                throw e;
            }
            LOG.info("Submitted task " + i);
        }
    }
}

The configuration of the AsynchronousPool is already in the arquillian.xml predefined with its default values. To play around and see what happens with different values simply change the properties in the arquillian.xml and restart the test:

AsynchronousPool.CorePoolSize=5
AsynchronousPool.MaximumPoolSize=5
AsynchronousPool.QueueSize=5
AsynchronousPool.OfferTimeout=30 seconds
AsynchronousPool.QueueType=LINKED
AsynchronousPool.OfferTimeout=30 seconds

For example, setting the AsynchronousPool.OfferTimeout to 1 second will make the test fail. The reason is because the first 5 invocations will be started immediately, the next 5 invocations will be put into the queue, but the eleventh invocation will be refused.

I have created a starter project that allows to play with asynchronous EJBs and TomEE. You can get that project by simply cloning https://github.com/robertpanzer/AsyncEJBTest

git clone https://github.com/robertpanzer/AsyncEJBTest