Cache Layers

In this topic talking about 

Requet Level Cache - HashMap

Application Level Cache - Caffeine Cache

Cluster Level Cache - Redis

DoorDash standardized caching across its microservices to address fragmentation and performance issues. Their new multi-layered system boosts scalability while simplifying adoption for engineering teams.​

Problems Faced

Teams used varied tools like Caffeine, Redis Lettuce, and HashMaps, leading to repeated issues such as cache staleness, Redis overload, and inconsistent key schemas. This fragmented approach complicated observability and debugging, especially under high traffic in services like DashPass.

Core Solution

Engineers created a shared Kotlin-based library with two key interfaces: CacheManager for cache creation and fallbacks, and CacheKey for abstracting keys. This enables uniform API calls via dependency injection and polymorphism, hiding backend details from business logic.​

Cache Layers

• Request Local Cache: HashMap-bound to a single request's lifecycle for ultra-fast access.

• Local Cache: Caffeine-powered, shared across workers in one JVM.

• Redis Cache: Distributed via Lettuce, accessible across pods in a Redis cluster.

Data flows from fastest (local) to slowest (Redis), populating upper layers on misses.

Key Features

Runtime controls let operators toggle layers, adjust TTLs, or enable shadow mode (sampling cache vs. source-of-truth for validation). Built-in metrics track hits/misses, latency, and staleness, with logging for observability

Client Request

       |

       v

+--------------------+

| 1. Request Local   |  (HashMap, request-lifetime)

|    Cache (Fastest) |

+--------------------+

       | Miss?

       v Yes

+--------------------+

| 2. Local Cache     |  (Caffeine, JVM-wide)

+--------------------+

       | Miss?

       v Yes

+--------------------+

| 3. Redis Cache     |  (Lettuce, Cluster-wide)

+--------------------+

       | Miss?

       v Yes

+--------------------+

| Source of Truth    |  (DB/Service)

+--------------------+

       ^

       | Populate all layers on hit

Resilience4J - SlidingWindow Protocal with CircuitBreaker

 In previous articel discussed about the BulkHead Pattern, Now we are discussing on Sliding Window.

Good 👍 this is core internal logic of CircuitBreaker in Resilience4j.

Most developers use @CircuitBreaker but don’t understand how sliding window actually calculates failure rate.

Let’s break it clearly.

🔥 What is Sliding Window in Resilience4j?

Sliding window is the statistical window used by CircuitBreaker to decide:

Should we OPEN the circuit or keep it CLOSED?

It calculates:

• Failure rate %

• Slow call rate %

• Total calls count

Based on last N calls or last N seconds.

---

📌 Two Types of Sliding Windows

1️⃣ COUNT_BASED Sliding Window

Based on number of calls.

Example:

CircuitBreakerConfig config = CircuitBreakerConfig.custom()
    .slidingWindowType(SlidingWindowType.COUNT_BASED)
    .slidingWindowSize(10)
    .failureRateThreshold(50)
    .build();

Meaning:

• Observe last 10 calls

• If more than 50% fail

• Circuit goes OPEN

---

Example Scenario:

Last 10 calls:

S F S F F S F F S F

Failures = 6
Failure rate = 60%

If threshold = 50% → Circuit OPEN

---

2️⃣ TIME_BASED Sliding Window

Based on time duration.

Example:

.slidingWindowType(SlidingWindowType.TIME_BASED)
.slidingWindowSize(10)

Meaning:

• Observe calls in last 10 seconds

• Calculate failure rate

• If threshold crossed → OPEN

---

🧠 How Sliding Window Internally Works

Internally it maintains:

• Circular array (ring buffer)

• Buckets for time-based

• Atomic counters

Every new call:

1. Old data expires

2. New result added

3. Failure rate recalculated

4. Decision made

This is O(1) time complexity per update.

Very efficient.

---

🎯 Important Configurations (Architect Level)

🔹 Minimum Number of Calls

.minimumNumberOfCalls(5)

Circuit will not evaluate failure rate unless at least 5 calls happen.

This avoids false positives in low traffic systems.

---

🔹 Failure Rate Threshold

.failureRateThreshold(50)

If failure % > threshold → OPEN

---

🔹 Slow Call Rate Threshold

.slowCallRateThreshold(60)
.slowCallDurationThreshold(Duration.ofSeconds(2))

If 60% calls take > 2 seconds → OPEN

This protects against latency spikes.

---

🏦 Real Banking Example (APS Context)

Let’s say:

Loan SOR:

• Sliding window size = 20 calls

• Failure threshold = 40%

• Minimum calls = 10

If last 20 calls:

• 8 failures

• Failure rate = 40%

Circuit remains CLOSED.

But if 9 failures:

• 45%

• Circuit OPEN

---

🔄 Difference Between Count vs Time Based

Feature	COUNT_BASED	TIME_BASED
Best For	Stable traffic	Variable traffic
Banking Core APIs	✅ Good	⚠️ Depends
High burst systems	❌ Risky	✅ Better
Predictability	High	Medium

Bulkhead Pattern – Every SOR Should Have a Separate Thread Pool/Executor Service

Resilence4J Patterns - These patterns will focus only on threads.

Bulkhead Pattern – Every SOR Should Have a Separate Thread Pool/Executor Service

In enterprise banking systems, a single application often communicates with multiple Systems of Record (SORs) — such as Customer SOR, Loan SOR, Payment SOR, or Core Banking.

If one SOR becomes slow or unavailable, it should not impact other SOR integrations.

This is where the Bulkhead Pattern becomes critical.

Ex:

APS Service

   ├── Customer SOR

   ├── Loan SOR

   ├── Payment SOR

   └── Notification SOR

All SOR calls share the same thread pool.

❌ What happens if Loan SOR becomes slow?

• Threads get blocked

• Thread pool gets exhausted

• Customer SOR calls start waiting

• Entire APS system becomes unresponsive

• Production incident

This is called resource starvation.

✅ Solution: Separate Thread Pool Per SOR

Each SOR should have:

• Dedicated thread pool

• Dedicated timeout

• Dedicated circuit breaker

• Dedicated monitoring metrics

Architecture becomes:

Customer SOR → ThreadPool-A
Loan SOR     → ThreadPool-B
Payment SOR  → ThreadPool-C
Notification → ThreadPool-D

Now:

• Loan SOR failure affects only ThreadPool-B

• Other SORs continue working normally

• System stability increases dramatically

JAVA 8 - Lambads,Streams - Very Simple Manner

Lambda - Nameless methods we are calling as lambda,

We heard anonymous class as nameless class, What is nameless function ?

Lambda are advanced short cut version of anonymous class.

Usecase: Convertion of Method as Nameless Method (Lambda)

public void fan() {

      System.out.println("Ramesh is megastar fan");

}

As I told lambda means nameless function/method, Now I am going to remove the method name "public void fan"

()  -> {

             System.out.println("Ramesh is megastar fan");

         }

For the single statement we don't require curly brackets {,} and then

() -> System.out.println("Ramesh is megastar fan");

2nd Ex: methods with arguments .

public void add(int a,int b) {

        System.out.println(a+b);

}

As I told lambda means nameless function/method, Now I am going to remove the method name "public void add"

(int a,int b) -> {

                          System.out.println(a+b);

                      }

For the single statement we don't require curly brackets {,} and then

(int a,int b) ->  System.out.println(a+b);

                      

 If the type of the parameter can be decided by compiler automatically based on the context then we can remove types also. 

(a,b) ->  System.out.println(a+b);

3rd Ex: method with arguments and return type

public String str(String str) { 

                return str;

 }

As I told lambda means nameless function/method, Now I am going to remove the method name "public String str"

(String str) -> { return str };

 If the type of the parameter can be decided by compiler automatically based on the context then we can remove types also. 

(str) -> { str };

(str) ->  str ;

Hope you understand lambda secret, As I told lambda is advanced version of anonymous class, Now we will see the scenario.

Concept: Functional Interface.

If an interface contain only one abstract method, such type of interfaces are called functional interfaces and the method is called functional method or single abstract method (SAM). 

Ex: 

1) Runnable -> It contains only run() method 

2) Comparable ->  It contains only compareTo() method 

3) ActionListener -> It contains only actionPerformed() 

4) Callable ->  It contains only call() method 

Inside functional interface in addition to single Abstract method (SAM) we write any number of default and static methods.

Usecase: Convertion of Anonymous Class as Lambda (Anonymous method)

class Test { 

    public static void main(String[] args) { 

        Thread t = new Thread(new Runnable() {

                                                                            public void run() { 

                                                                                 for(int i=0; i<10; i++) { 

                                                                                    System.out.println("Child Thread"); 

                                                                                } 

                                                                           }  

                                              }); 

          t.start(); 

         for(int i=0; i<10; i++) 

                System.out.println("Main thread");  

        }  

}  

In the above example anonymous (Nameless class) -> 

new Runnable() {

           public void run() { 

                  for(int i=0; i<10; i++) { 

                          System.out.println("Child Thread"); 

                  } 

           }  

 }

Converting above one into Lambda (Nameless Function/Method)

For the nameless Function - we only focus on function or method, here function/method is run() method.

 public void run() { 

           for(int i=0; i<10; i++) { 

                     System.out.println("Child Thread"); 

           } 

 }

Remove the function name -> 

 ()  ->  { 

           for(int i=0; i<10; i++) { 

                     System.out.println("Child Thread"); 

           } 

 }

We successfully converted the Anonymous class into Lambda.

Few more scenarios:

interface Calculator { 

        public void sum(int a,int b); 

}

 class Demo implements Calculator { 

        public void sum(int a,int b) { 

             System.out.println("The sum:"+(a+b)); 

        } 

} 

public class Test { 

    public static void main(String[] args) { 

       Calculator  cal = new Demo(); 

        cal .sum(20,5); 

    } 

}

Convert into anonymous class approach

public class Test { 

    public static void main(String[] args) { 

       //Calculator cal = new Demo();

 Calculator cal = new Calculator() {

            public void sum(int a,int b) { 

                                  System.out.println((a+b)); 

                            }

                 };

        cal .sum(20,5); 

    } 

}

Now Converting anonymous class into anonymous function/method (Lambda approach)

public class Test { 

    public static void main(String[] args) { 

       //Calculator cal = new Demo();

  Calculator cal = (a,b) -> { 

                                                 System.out.println(a+b); 

                                               };

        cal .sum(20,5); 

    } 

}

Hope you understand lambda alias nameless function/method concept.   

Microservices/Distributed Design Patterns

 I had started posting of the microservices/Distributed design patterns.

Circuit Breaker Section:

Importance of urandom egd in containers -> java -Djava.security.egd=file:/dev/urandom Importance

 In high distributed, scaled applications , One System/Server needs to connect another Downstream System/Server to establish session, certificates,

Here session/certificates are having random numbers,

Our java security pickup the random number from Linux/Window entropy pool.

What is entropy pool ?

         In any server, user hits the server through keyboard some other noises, Linux server capture those noises put them into entropy pool.

EGD - Entropy Gathered Device

our java security device -> java.security.egd=file:/dev/random

java.security.egd=file:/dev/random

      By default java8 and java11 this property is there in the java.security file, 

If entropy pool having full entries, file:/dev/random works fine,

If entropy pool having no entries, file:/dev/random will waits for long time, it is blocking the thread, it is hitting the performance.

Solution: Always use the java.security.egd=file:/dev/urandom in PCF/Kubernets/Virtual Server.

java.security.egd=file:/dev/urandom

      need to pass this argument in java argument.

If entropy pool having full entries, file:/dev/urandom works fine,

If entropy pool having no entries, file:/dev/urandom will not wait, it will genereate the random number hence it will improve the performance, no blocking of the thread.

Book Review

React: For React js I had seen videos by experts, but not get clarity. But I had read very good book
"Learning React A Hands-On Guide to Building Web Applications Using React and Redux by Kirupa Chinnathambi" It's very good book.It's covered react fundamental very good way.

for redux storage below link given very good
https://chriscourses.com/blog/redux - react-redux

Spring Boot Auto-Configuration:

https://dzone.com/articles/what-is-spring-boot-auto-configuration

Kubernetes Administration certification:

Basically Kubernetes is wrapper around the docker container. 

Try to understand kubernetes components. Intention of each kubernets component.

While doing monitoring (logs & metrics) I had taken helm install those packages. Here important point

Metrics :- ( Kubernetes Components ) Level and Application ( Deployed application) Level, Don't confuse here.

First complete extra ordinary course kubernetes administration certification course.

https://www.udemy.com/course/certified-kubernetes-administrator-with-practice-tests/

then follow below book some more understanding

"Kubernetes Up and Running Dive Into the Future of Infrastructure by Brendan Burns Joe Beda Kelsey Hightower" best book for kubernetes administrators