ReadWriteLock Upgrade Trap

ReadWriteLock Upgrade Trap: practice a Java concurrency bug with symptoms like Request hangs, Thread waits unexpectedly, Confusing lock behavior. Inspect...

Read/write locking
ReadWriteLock
Locking Protocol
Java
Intermediate

Production symptoms

Request hangs
Thread waits unexpectedly
Confusing lock behavior

Failure scenario

Code

Java example

rw.readLock().lock();
try {
    Value value = cache.get(key);
    if (value == null) {
        rw.writeLock().lock();
        try {
            cache.put(key, load(key));
        } finally {
            rw.writeLock().unlock();
        }
    }
} finally {
    rw.readLock().unlock();
}

Prod Symptoms

A cache or routing-table lookup holds a read lock for normal reads, then tries to acquire the write lock on a miss or refresh path.

Key signal: ReentrantReadWriteLock supports downgrading from write to read, but upgrading from read to write is a trap.

Hot-cache reads may keep working while cold-key or refresh requests hang
CPU is usually low because the stuck thread is parked, not spinning
The hang appears near cache miss, reload, or invalidation code
Thread dumps show a waiter parked in ReentrantReadWriteLock write-lock acquisition
The loader may look suspicious, but the root cause is the lock upgrade protocol

Run Locally

The thread reports a cache miss while holding the read lock
It then waits trying to acquire the write lock
The process stays alive because the upgrade path cannot complete
No second application thread is required to demonstrate the protocol problem

What to look for

upgrade-thread parked in ReentrantReadWriteLock write lock acquisition
Code path that still owns the read lock when it tries to acquire the write lock

Run

javac ReadWriteLockUpgradeDemo.java
java ReadWriteLockUpgradeDemo

Inspect while stuck

jps
jcmd <pid> Thread.print

ReadWriteLockUpgradeDemo.java

import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.locks.ReentrantReadWriteLock;

public class ReadWriteLockUpgradeDemo {
    private static final ReentrantReadWriteLock rw = new ReentrantReadWriteLock();
    private static final Map<String, String> cache = new HashMap<>();

    public static void main(String[] args) throws Exception {
        Thread thread = new Thread(() -> getOrLoad("a"), "upgrade-thread");
        thread.start();

        Thread.sleep(1_000);
        System.out.println("thread state = " + thread.getState());
    }

    private static String getOrLoad(String key) {
        rw.readLock().lock();
        try {
            String value = cache.get(key);
            if (value == null) {
                System.out.println("cache miss while holding read lock");
                System.out.println("trying to acquire write lock");
                rw.writeLock().lock();
                try {
                    cache.put(key, "loaded");
                    return "loaded";
                } finally {
                    rw.writeLock().unlock();
                }
            }
            return value;
        } finally {
            rw.readLock().unlock();
        }
    }
}

Note: A single thread can block itself by trying to acquire the write lock while still holding a read lock.

Diagnosis and fix

Explanation

ReentrantReadWriteLock allows downgrading from write lock to read lock, but it does not safely support upgrading from read lock to write lock.

Key signal: Treat read-to-write upgrade as a protocol redesign, not as another nested lock.

A write lock requires exclusive access
The current thread is still counted as a reader while it holds the read lock
The write lock cannot be acquired while readers exist
The thread cannot release its read lock because it is waiting inside writeLock().lock()
This can block progress even with one application thread
With multiple readers, the protocol becomes even harder to reason about

How to Diagnose

Use a thread dump to find the waiter, then inspect the lock acquisition order in code.

Look for a thread parked under ReentrantReadWriteLock write-lock acquisition
Check whether the same call path already acquired readLock()
Inspect cache miss, refresh, reload, and invalidation paths
Check whether the code re-checks the cache condition after acquiring the write lock
jstack can show the parked writer, but it may not print a Java-level deadlock section because ReentrantReadWriteLock waits through AQS/LockSupport, not intrinsic monitor ownership

Commands

jps
jstack <pid>
jcmd <pid> Thread.print

Expected dump shape

"upgrade-thread" #... WAITING (parking)
  at jdk.internal.misc.Unsafe.park(Native Method)
  at java.util.concurrent.locks.LockSupport.park(LockSupport.java:...)
  at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:...)
  at java.util.concurrent.locks.ReentrantReadWriteLock$WriteLock.lock(ReentrantReadWriteLock.java:...)

How to Fix

Read under the read lock first
Release the read lock before acquiring the write lock
Re-check the condition under the write lock because another writer may have filled the value
Keep the write-locked update small
Avoid slow remote loading while holding the write lock unless blocking all readers is acceptable
Do not expect fairness mode, bigger pools, or timeouts to fix a broken upgrade protocol
Consider a higher-level cache abstraction when get-or-load semantics are the real requirement

ReadWriteLockUpgradeFixed.java

import java.util.HashMap;
import java.util.Map;
import java.util.concurrent.locks.ReentrantReadWriteLock;

public class ReadWriteLockUpgradeFixed {
    private static final ReentrantReadWriteLock rw = new ReentrantReadWriteLock();
    private static final Map<String, String> cache = new HashMap<>();

    public static void main(String[] args) throws Exception {
        Thread first = new Thread(() -> System.out.println(getOrLoad("a")), "reader-1");
        Thread second = new Thread(() -> System.out.println(getOrLoad("a")), "reader-2");

        first.start();
        second.start();
        first.join();
        second.join();
    }

    private static String getOrLoad(String key) {
        rw.readLock().lock();
        try {
            String value = cache.get(key);
            if (value != null) {
                return value;
            }
        } finally {
            rw.readLock().unlock();
        }

        String loaded = load(key);

        rw.writeLock().lock();
        try {
            String value = cache.get(key);
            if (value == null) {
                value = loaded;
                cache.put(key, value);
            }
            return value;
        } finally {
            rw.writeLock().unlock();
        }
    }

    private static String load(String key) {
        System.out.println(Thread.currentThread().getName()
                + " loading " + key + " outside the write lock");
        try {
            Thread.sleep(200);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
            throw new RuntimeException("load interrupted", e);
        }
        return "loaded-" + key;
    }
}

Note: Loading outside the write lock keeps readers moving, but concurrent misses may duplicate the load. The second check prevents duplicate cache updates; a production cache may also coalesce in-flight loads.