Unknown fields should be preserved, not destroyed - encoding json,avro,protobuff etc

 Excellent question. This is one of the most frequently asked Architect interview questions from DDIA.

The statement:

"Unknown fields should be preserved, not destroyed."

is true for Protocol Buffers (modern implementations) but not generally true for Avro. This is a key difference that interviewers often test.

---

First understand the problem

Suppose we have three services:

Order Service (V2)
        |
        |  orderId, amount, couponCode
        |
        V
Kafka
        |
        V
Inventory Service (V1)

V2 sends

{
  "orderId":101,
  "amount":1000,
  "couponCode":"NEW100"
}

Inventory Service V1 knows only

orderId
amount

It doesn't know couponCode.

---

What happens in Avro?

Assume the schemas are:

Producer Schema (V2)

{
  "type":"record",
  "name":"Order",
  "fields":[
    {"name":"orderId","type":"int"},
    {"name":"amount","type":"double"},
    {"name":"couponCode","type":["null","string"],"default":null}
  ]
}

---

Consumer Schema (V1)

{
  "type":"record",
  "name":"Order",
  "fields":[
    {"name":"orderId","type":"int"},
    {"name":"amount","type":"double"}
  ]
}

---

During deserialization,

Avro matches fields by name.

It reads

orderId ✔

amount ✔

couponCode ❌ Unknown

Unknown field is simply ignored.

The consumer object becomes

Order

orderId = 101

amount = 1000

couponCode never exists inside the Java object.

---

If Inventory republishes the message

Suppose

Producer

↓

Kafka

↓

Inventory

↓

Kafka Again

Inventory reads

orderId

amount

and writes again.

The new message becomes

{
  "orderId":101,
  "amount":1000
}

The field

couponCode

is lost.

This is because Avro does not automatically preserve unknown fields during normal deserialize → serialize cycles.

---

Why?

Avro is schema resolution based.

It creates an object using only fields present in the reader schema.

Unknown fields are discarded during deserialization.

So they cannot be written back later unless the application explicitly carries them along.

---

Protocol Buffers behave differently

Protocol Buffers internally stores unknown fields.

Imagine

Producer

↓

Protobuf Binary

↓

Consumer

Consumer understands

orderId

amount

Unknown

couponCode

is kept inside an internal UnknownFieldSet.

The object looks conceptually like:

Order

orderId

amount

UnknownFieldSet

     |

couponCode

If the consumer forwards the message,

serialize()

includes

couponCode

again.

Nothing is lost.

---

Visual Comparison

Avro

Producer V2

↓

orderId
amount
couponCode

↓

Consumer V1

↓

Reads

orderId

amount

↓

couponCode discarded ❌

↓

Writes again

↓

orderId

amount

---

Protocol Buffers

Producer V2

↓

orderId

amount

couponCode

↓

Consumer V1

↓

Reads

orderId

amount

↓

UnknownFieldSet

↓

couponCode preserved

↓

Serialize again

↓

orderId

amount

couponCode

---

How does Avro achieve compatibility then?

Avro's strength is schema evolution, not unknown field preservation.

It achieves compatibility using writer schema + reader schema resolution:

1. The writer's schema (or its ID via a Schema Registry) is available.

2. The reader uses its own schema.

3. Avro resolves differences:

   • Matching fields are read.

   • New writer fields missing from the reader are ignored.

   • Reader fields missing from the writer use default values (if defined).

4. The application works without errors, but unknown fields are not retained after deserialization.

---

Real-world Kafka Example

Order Service V2

↓

Kafka

↓

Payment Service V1

↓

Notification Service V2

If Payment Service only validates payment and republishes the event:

• With Avro: If it deserializes into a V1 object and reserializes, couponCode is lost unless the application preserves it explicitly.

• With Protocol Buffers: The unknown field is retained automatically and forwarded.

---

Architect Interview Answer ⭐

Question: How does Avro preserve unknown fields?

Answer:

Avro generally does not preserve unknown fields during a deserialize–serialize cycle. It resolves schema differences using the writer and reader schemas, ignoring fields that the reader doesn't know. This enables schema compatibility, but unknown fields are discarded unless the application explicitly carries them forward. In contrast, Protocol Buffers stores unknown fields internally (using an UnknownFieldSet in many implementations) so they can be reserialized without loss.

This distinction between schema compatibility (Avro) and unknown field preservation (Protocol Buffers) is an important concept for distributed systems and frequently comes up in Senior Java Architect interviews.

SOLID principles with Design Patterns

 

SOLID + Design Patterns (Java Examples)

One of the most common Java interview questions is:

Which Design Patterns follow which SOLID Principle?

The answer is that design patterns are practical implementations of SOLID principles.

---

SOLID vs Design Patterns

SOLID Principle	Design Patterns	Why?
SRP	Facade, DAO, Repository, Service Layer	One class = One responsibility
OCP	Strategy, Decorator, Template Method	Add new behavior without changing existing code
LSP	Factory Method, Template Method	Child classes can replace parent classes
ISP	Adapter, Bridge	Small focused interfaces
DIP	Factory, Abstract Factory, Dependency Injection, Builder	Depend on interfaces, not implementations

---

1. SRP + DAO Pattern

❌ Bad Design

class Employee {

    void calculateSalary(){}

    void saveEmployee(){}

    void sendEmail(){}
}

Three responsibilities.

---

✅ Good Design

class EmployeeService {

    void calculateSalary(){}
}

class EmployeeRepository {

    void save(Employee e){}
}

class EmailService {

    void sendMail(){}
}

Pattern Used

Controller
      |
      V
Service
      |
      V
Repository (DAO)

Every class has one responsibility.

---

2. OCP + Strategy Pattern

Suppose Flipkart offers different payment methods.

Instead of:

if(payment.equals("UPI"))

if(payment.equals("CARD"))

if(payment.equals("NETBANKING"))

Use Strategy.

---

Strategy Interface

interface PaymentStrategy {

    void pay(double amount);

}

---

UPI Strategy

class UpiPayment implements PaymentStrategy {

    public void pay(double amount){

        System.out.println("UPI Payment");

    }

}

---

Card Strategy

class CardPayment implements PaymentStrategy {

    public void pay(double amount){

        System.out.println("Card Payment");

    }

}

---

Client

class PaymentService {

    private PaymentStrategy payment;

    PaymentService(PaymentStrategy payment){

        this.payment = payment;

    }

    void checkout(){

        payment.pay(1000);

    }

}

Adding Wallet Payment?

Just create

class WalletPayment implements PaymentStrategy{}

No existing code changes.

OCP achieved.

---

3. OCP + Decorator Pattern

Imagine Coffee Shop.

Base Coffee

interface Coffee {

    String getDescription();

}

Simple Coffee

class SimpleCoffee implements Coffee {

    public String getDescription(){

        return "Coffee";

    }

}

Decorator

class MilkDecorator implements Coffee{

    private Coffee coffee;

    MilkDecorator(Coffee coffee){

        this.coffee=coffee;

    }

    public String getDescription(){

        return coffee.getDescription()+" + Milk";

    }

}

Usage

Coffee coffee = new MilkDecorator(
                    new SimpleCoffee());

System.out.println(coffee.getDescription());

Output

Coffee + Milk

Want Sugar?

Create

SugarDecorator

No modification.

Only extension.

---

4. LSP + Factory Method

Vehicle example

abstract class Vehicle{

    abstract void start();

}

class Car extends Vehicle{

    void start(){

        System.out.println("Car Started");

    }

}

class Bike extends Vehicle{

    void start(){

        System.out.println("Bike Started");

    }

}

Client

Vehicle vehicle = VehicleFactory.create("CAR");

vehicle.start();

Client never worries whether it is Car or Bike.

Any subclass works.

LSP satisfied.

---

5. ISP + Adapter Pattern

Suppose Printer

Bad Interface

interface Printer{

    print();

    scan();

    fax();

}

Old Printer

Needs only print.

Forced to implement scan().

Violation.

---

Split Interfaces

interface Printable{

    void print();

}

interface Scannable{

    void scan();

}

Adapter

class OldPrinterAdapter implements Printable{

    OldPrinter printer;

    public void print(){

        printer.print();

    }

}

Only required methods implemented.

---

6. DIP + Factory Pattern

Bad

class NotificationService{

    EmailSender sender=new EmailSender();

}

Tightly coupled.

---

Better

interface MessageSender{

    void send();

}

class EmailSender implements MessageSender{}

class SmsSender implements MessageSender{}

Factory

class SenderFactory{

    static MessageSender getSender(String type){

        if(type.equals("EMAIL"))
            return new EmailSender();

        return new SmsSender();

    }

}

Client

MessageSender sender =
        SenderFactory.getSender("EMAIL");

sender.send();

NotificationService depends only on

MessageSender

DIP achieved.

---

7. DIP + Spring Dependency Injection ⭐

Without Spring

class OrderService{

    MySqlRepository repo =
            new MySqlRepository();

}

Coupled.

---

With Spring

interface OrderRepository{

    void save();

}

@Repository
class MySqlRepository
implements OrderRepository{

}

@Service
class OrderService{

    private final OrderRepository repo;

    OrderService(OrderRepository repo){

        this.repo=repo;

    }

}

Spring injects implementation automatically.

This is Dependency Inversion in action.

---

8. SRP + Builder Pattern

Instead of

Employee e = new Employee(
    "Ramesh",
    30,
    "Architect",
    "Bangalore",
    true,
    150000,
    "Java");

Use Builder

Employee employee = Employee.builder()
        .name("Ramesh")
        .age(30)
        .city("Bangalore")
        .designation("Architect")
        .salary(150000)
        .build();

Builder focuses only on object creation.

Employee focuses only on business data.

SRP improved.

---

SOLID + Spring Boot Mapping

Spring Boot Feature	SOLID Principle	Pattern
@Service	SRP	Service Layer
@Repository	SRP	Repository
Dependency Injection	DIP	IoC
@Autowired Constructor	DIP	Dependency Injection
Strategy Bean Selection	OCP	Strategy
JpaRepository	ISP	Repository Interface
Bean Polymorphism	LSP	Factory
RestTemplateBuilder	SRP	Builder
ResponseEntity	OCP	Builder

---

Complete Mapping

Design Pattern	SOLID Principle
Strategy	OCP
Decorator	OCP
Factory Method	DIP + LSP
Abstract Factory	DIP
Builder	SRP
Adapter	ISP
Bridge	ISP + DIP
Observer	OCP
Command	OCP + DIP
Template Method	OCP + LSP
Proxy	OCP
Facade	SRP
Repository	SRP
Singleton	(Not directly tied to SOLID; often overused and can violate SRP/DIP if misapplied)

---

Interview Cheat Sheet

Interview Question	Best Answer
Which pattern best demonstrates OCP?	Strategy and Decorator
Which pattern demonstrates DIP?	Factory, Abstract Factory, Dependency Injection
Which pattern uses LSP?	Factory Method, Template Method
Which pattern follows ISP?	Adapter, Bridge
Which pattern improves SRP?	Repository, Facade, Builder, Service Layer

Tip for Senior Java Architect Interviews

Rather than memorizing mappings, explain the reasoning:

• Strategy supports OCP because you add new algorithms without modifying existing client code.

• Decorator supports OCP by extending behavior dynamically instead of changing the original class.

• Factory and Dependency Injection support DIP by ensuring clients depend on interfaces rather than concrete implementations.

• Repository and Service Layer encourage SRP by separating persistence, business logic, and presentation concerns.

• Adapter supports ISP by exposing only the operations a client actually needs.

• Template Method supports LSP because subclasses provide specialized behavior while preserving the base class contract.

This explanation demonstrates a deeper understanding than simply listing patterns against SOLID principles, which is often what interviewers for Senior Architect roles look for.