Concurrency that actually helps

Virtual threads made thread-per-request cheap again. They did not make anything faster.

What they change

A platform thread is an OS thread: roughly a megabyte of stack, scheduled by the kernel, expensive enough that you pool them. A virtual thread is managed by the JVM, costs a few hundred bytes, and when it blocks it is unmounted from its carrier thread so the carrier can run something else.

So the workload they help is blocking I/O: a request that spends its life waiting on a database, an HTTP call, a queue. Ten thousand of those can be ten thousand virtual threads on a handful of carriers.

The workload they do not help is CPU-bound work. Sixteen threads computing on sixteen cores are already using the machine; making them virtual changes nothing except the bookkeeping. If a service is at 100% CPU, virtual threads are not the answer — there is no waiting to reclaim.

Why throughput sometimes does not move

You switch a thread-per-request service to virtual threads and the numbers barely budge. Three usual causes, in the order worth checking:

The bottleneck was never the threads. A connection pool of ten means ten concurrent queries whatever the thread count. Ten thousand virtual threads all waiting on ten connections is the same throughput as before, plus a longer queue. This is the common one, and it is why enlarging the thread pool without enlarging what it waits on achieves nothing.

Pinning. A virtual thread blocking inside a synchronized block cannot unmount — it holds its carrier hostage for the duration. Recent JDKs have removed most of this, but a synchronized method around an I/O call is still the shape to look for. ReentrantLock does not pin.

Native calls. Blocking in native code cannot unmount either.

StructuredTaskScope for fan-out

The problem with launching several concurrent calls by hand is not the launching, it is the cleanup: if one fails, who cancels the others, and who waits for what.

try (var scope = new StructuredTaskScope.ShutdownOnFailure()) {
    Subtask<Profile> profile = scope.fork(() -> fetchProfile(id));
    Subtask<Orders>  orders  = scope.fork(() -> fetchOrders(id));

    scope.join().throwIfFailed();          // both done, or the first failure won
    return new Dashboard(profile.get(), orders.get());
}

The scope owns the subtasks. If either fails, the other is cancelled; if the enclosing thread is interrupted, both are. The lifetime of the concurrency is the lifetime of the block, which is the whole point of the name — and the reason a leaked task is not possible here in the way it is with a bare executor.

Where CompletableFuture still wins

Virtual threads make blocking code cheap, so most fan-out becomes simpler as plain blocking calls in a scope. CompletableFuture is still the better tool when the composition itself is the logic:

• a pipeline where each stage transforms the previous result, and you want that written as a chain rather than a sequence of assignments
• combining results with timeouts and fallbacks per stage
• an API you do not control that already returns futures

Reaching for CompletableFuture to avoid blocking a thread is the reason that no longer applies.

Parallel streams: the sharp edge

list.parallelStream().map(this::callService).toList();

By default this runs on the common ForkJoinPool, shared by the whole JVM. Put blocking I/O in there and you starve everything else that uses it — including other parallel streams and some library internals. Parallel streams are for CPU-bound work over a large, cheaply-splittable source. For I/O fan-out, use a scope or an executor you own.

And parallel() does not make a small stream faster. Splitting, scheduling and merging cost more than the work when the source is a few hundred elements.

Two small facts that catch people

A stream is single-use: consume it twice and you get IllegalStateException: stream has already been operated upon or closed. And a stream does nothing until a terminal operation runs — a map with a side effect and no collect is a no-op that looks like code.

The production side of this, with the pool sizing that actually matters: Surviving Production Java.