Driving Performance Gains in Java Virtual Threads

The Java platform is nearly 30 years old, yet it consistently maintains its position in the top three most popular programming languages. One of the key reasons for this is the Java Virtual Machine (JVM). By abstracting away concerns like memory management and compiling code as it is running, the JVM can deliver internet-level scalability beyond the reach of other runtimes.

What also helps to keep Java so popular is the rate of evolution of the language, the libraries and the JVM. In 2019, OpenJDK, the open source project for Java development, switched to a time-based rather than feature-based release schedule. We now have two new versions of Java each year, rather than having to wait between two and four years.

With so many releases, it is not practical for distributions to offer extended maintenance and support for all versions. Only specific ones classified as long-term support (LTS) include this for a standard eight years from release. (JDK 8 is supported until December 2030, JDK 11 is supported until January 2032, and JDK 6 and 7 are only supported by Azul until December 2027). All releases are suitable for use in production, but most enterprise users will opt only to deploy applications using an LTS JDK.

How Threads Work in Java

The current LTS release is JDK 21, released in September 2023, included some interesting new features that will make it attractive for applications that support large numbers of simultaneous users.

From the beginning, Java was a language that supported the idea of concurrent execution of tasks. Unlike languages like C and C++, which rely on external libraries for this support, Java has the concept of threads built into the language.

Suppose you’re developing a web-based application that will support many simultaneous users. In that case, each user’s connection can be handled concurrently by allocating their own thread or threads to process the necessary transactions. All this is done independently, keeping each user’s data isolated from others. This is referred to as a thread-per-request (TPR) programming model.

While this is excellent for the developers of these types of applications, it does come with some limitations. Linux, for example, is a hugely popular operating system (OS) that can handle all of the low-level aspects of threads and assign them to the available CPUs and cores of the hardware in use. Prior to JDK 21, all Java threads were mapped directly to OS threads, so the JVM did not need to handle the low-level aspects.

The drawback is scalability when handling hundreds of thousands (or more) of simultaneous connections. The memory requirements when using so many threads make it impractical to provision cost-effective server hardware, either on premises or in the cloud.

If you look at how most TPR applications work, you will find that much of the processing involves calls to other parts of the system, whether it be databases, files or network connections to other services. These connections require the thread to wait (or block) until the database, for example, responds. The reality is that the thread spends most of its time waiting, so it is not actively using the OS thread it has been allocated.

Introducing Virtual Threads to Increase Scalability

JDK 21 introduced the Virtual Thread feature. Rather than using a one-to-one mapping between Java and OS threads, we can now have a many-to-one mapping. Multiple Java threads share a single OS thread.

For developers, migrating to virtual threads is very straightforward, only requiring a change to the way the thread is created, not how it is used. When the application runs, the JVM takes responsibility for switching between the virtual threads sharing an OS (now called platform) thread.

When a Java thread makes a call that will block it, the JVM will record all the details of the thread’s state and switch the platform thread to a different Java thread that has work to do. The memory requirements of a virtual thread are roughly a thousand times less than a platform thread by default, so scalability can be increased massively without adding hardware.

When considering virtual threads, it is vital to understand the difference between scalability and performance. For example, the scalability of an e-commerce application determines how many users can access it at the same time. The performance of that application determines how quickly the system responds to each request. Virtual threads will potentially increase the scalability of an application, allowing more connections to be handled simultaneously.

What it will not do is improve performance to deliver the results to those connections more quickly. Care should also be taken about how an application processes each connection. If that connection requires CPU-intensive tasks, rather than ones that spend most of the time waiting, you could easily end up with worse scalability and connections timing out because they cannot access the shared platform thread.

A Solution for Better Java Performance

To deliver better scalability and performance for your applications, you should consider a performance-optimized JVM. By replacing certain internal components, like memory management, garbage collection and just-in-time compilation, both reduced latency and improved throughput can be delivered. Azul’s Zing JVM delivers the C4 garbage collector, Falcon JIT compiler and ReadyNow warm-up elimination technology.

According to Gartner, you have options to choose from. Make sure you choose a JVM that fully conforms to the Java SE specification and passes all required Technology Compatibility Kit (TCK). Azul offers an option that doesn’t require you to rewrite or even recompile any code.

With JDK 21, you have options for how to take advantage of the virtual threads to increase scalability, but it’s best if you combine that with a JVM that delivers performance improvements for your Java applications.