This Distributed System Design, I will update based on my reading.
Reference: Designing Distributed Systems By Brendan Burns ( Very Good Book which will cover main concepts)
Replica Service: Stateless Service, Assume Request A can answer by any of #1,#2,#3
Sharded Service: Stateful Service, Assume Request A can answer by #1
Now I am started to work on Kubernetes - Below is the best material to learn the Kubernetes
https://kodekloud.com/courses/675122/lectures/12039431
This author covered all the kubernetes topics very well.
To setup the minikube on window 10 home edition: https://www.assistanz.com/installing-minikube-on-windows-10-home-edition-using-virtualbox/
I had tried minikube is working with virtualbox 5.2.x only, for virtualbox 6.x I had got error. While installing the minikube in your local please refer the virutalbox.
Docker daemon is a process, it will come along with minkube, Hence in my local i am utilizing the minikube docker daemon.
Below is the command to check:
minikube docker-env
Result:
SET DOCKER_TLS_VERIFY=1
SET DOCKER_HOST=tcp://192.168.99.100:2376
SET DOCKER_CERT_PATH=C:\Users\rames\.minikube\certs
SET MINIKUBE_ACTIVE_DOCKERD=minikube
What is sticky session ?
2 Webservers - server1 and server2
Application Load Balancer - default time - 5 min
10:00 Client1 > ALB > sticky session > server1
10:03 Client1 > ALB > sticky session > server1
10:08 Client1 > ALB > sticky session > server2 ( why ? because sticky session timeout 5 min means same client --> attach to server1 until 5 min, if any client1 request coming with in a 5 min, it will map to server1 means sticky session attached for particalr server).
What is Anti Corruption Layer ?
https://dev.to/asarnaout/the-anti-corruption-layer-pattern-pcd, It is going to protect existing legacy system through combination of patterns those are facade, adapters and translators.
https://docs.microsoft.com/en-us/azure/architecture/patterns/anti-corruption-layer
Today I had went through the twitter system design.
Twitter System Design: https://imgur.com/6TFkVL2
Twitter Technical Stack:
1) Redis Cluster
2) Apache Storm/Kafka Streams
3) Search - EarlyBird
4) http push websocket - million of mobiles
5) Apache Zookeper - maintains nodes
Refer: https://www.youtube.com/watch?v=wYk0xPP_P_8
Whatsup System Design
Whasup Techical Stack:
1) Erlang, the functional programming language
2) Webserver for load balancer - Yaws webserver
3) Message Server - XMPP Server (XMPP protocol)
4) Database - Mnesia / CouchDB Cluster
5) Apache HttpServer for Media Handling
Refer: https://medium.com/codingurukul/whatsapp-engineering-inside-2-bdd1ec354748
https://www.youtube.com/watch?v=L7LtmfFYjc4
Netflix System Design
Front End : React JS
Server: Netty Server (TCP Server), Apache EC2 Cluster(My sql), AWS ELB
Cache : EVCache (Memcached)
Streaming: Apache Kafka
Compute : Spark cluster, Apache Samza
Storage: AWS S3,Cassandra Cluster
Services: Microservices ( ZUUL, Hystrix etc), Amazon EMR ( Managed Hadoop Framework), Archer - MapReduce Style Platform, Elastic Search, Apache Chukwa (Apache Chukwa is an open source data collection system for monitoring large distributed systems)
Service Orchestra: TITUS ( Container Managed Platform, like kubernets) AWS Application Auto Scaling feature ((Titus is a container management platform that provides scalable and reliable container execution and cloud-native integration with Amazon AWS))
Refer: https://medium.com/@narengowda/netflix-system-design-dbec30fede8d
Whatsup System Design
Whasup Techical Stack:
1) Erlang, the functional programming language
2) Webserver for load balancer - Yaws webserver
3) Message Server - XMPP Server (XMPP protocol)
4) Database - Mnesia / CouchDB Cluster
5) Apache HttpServer for Media Handling
Refer: https://medium.com/codingurukul/whatsapp-engineering-inside-2-bdd1ec354748
https://www.youtube.com/watch?v=L7LtmfFYjc4
Netflix System Design
Front End : React JS
Server: Netty Server (TCP Server), Apache EC2 Cluster(My sql), AWS ELB
Cache : EVCache (Memcached)
Streaming: Apache Kafka
Compute : Spark cluster, Apache Samza
Storage: AWS S3,Cassandra Cluster
Services: Microservices ( ZUUL, Hystrix etc), Amazon EMR ( Managed Hadoop Framework), Archer - MapReduce Style Platform, Elastic Search, Apache Chukwa (Apache Chukwa is an open source data collection system for monitoring large distributed systems)
Service Orchestra: TITUS ( Container Managed Platform, like kubernets) AWS Application Auto Scaling feature ((Titus is a container management platform that provides scalable and reliable container execution and cloud-native integration with Amazon AWS))
Refer: https://medium.com/@narengowda/netflix-system-design-dbec30fede8d
Varnish - Open source web cache
Below diagram talks about Replica Vs Sharding
Below diagram talks about Replica Vs Sharding
Replica Service: Stateless Service, Assume Request A can answer by any of #1,#2,#3
Sharded Service: Stateful Service, Assume Request A can answer by #1
Now I am started to work on Kubernetes - Below is the best material to learn the Kubernetes
https://kodekloud.com/courses/675122/lectures/12039431
This author covered all the kubernetes topics very well.
To setup the minikube on window 10 home edition: https://www.assistanz.com/installing-minikube-on-windows-10-home-edition-using-virtualbox/
I had tried minikube is working with virtualbox 5.2.x only, for virtualbox 6.x I had got error. While installing the minikube in your local please refer the virutalbox.
Docker daemon is a process, it will come along with minkube, Hence in my local i am utilizing the minikube docker daemon.
Below is the command to check:
minikube docker-env
Result:
SET DOCKER_TLS_VERIFY=1
SET DOCKER_HOST=tcp://192.168.99.100:2376
SET DOCKER_CERT_PATH=C:\Users\rames\.minikube\certs
SET MINIKUBE_ACTIVE_DOCKERD=minikube
What is sticky session ?
2 Webservers - server1 and server2
Application Load Balancer - default time - 5 min
10:00 Client1 > ALB > sticky session > server1
10:03 Client1 > ALB > sticky session > server1
10:08 Client1 > ALB > sticky session > server2 ( why ? because sticky session timeout 5 min means same client --> attach to server1 until 5 min, if any client1 request coming with in a 5 min, it will map to server1 means sticky session attached for particalr server).
What is Anti Corruption Layer ?
https://dev.to/asarnaout/the-anti-corruption-layer-pattern-pcd, It is going to protect existing legacy system through combination of patterns those are facade, adapters and translators.
https://docs.microsoft.com/en-us/azure/architecture/patterns/anti-corruption-layer
No comments:
Post a Comment