Components of Microservices Architecture for beginners
As the name tells, running your application as small independent separate loosely coupled services is called a Microservice architecture (MSA). Each service has its own functional or logical boundary, so that it can be reused. And, in case of upgrading a logic or fixing bugs in a logic, its not necessary to restart the entire application. Only the service which is modified can be restarted. And particlularly, a monolithic application is developed based on a single programming language. In MSA, its possible to use the best programming language for specific functionalities. For example, if you are making a machine learning application using python, you can make server side APIs using nodejs since nodejs is good in dealing with APIs than python, which is good for machine learning.
For a beginner to deploy a microservices architectute, here is a quick gist of the different opensource components that can be used.
MONGODB:
Mongodb is a document-based opensource storage distributed database which is one of the highly used db by fullstack developers. Data is stored in JSON format. The documents are schema less, meaning, the JSON structure can vary from document to document.
Mongodb is highly scalable too and it supports a lot of programming languages like C, C++, C#, Java, Node.js, Perl, PHP, Python, Ruby, Scala, Go and Erlang.
Install and use mongodb from : https://docs.mongodb.com/manual/installation/
For documentation: https://docs.mongodb.com/?_ga=2.192784556.740909059.1553338448-56655559.1552390234
ELASTICSEARCH:
Elasticsearch is like a standalone database which makes ‘SEARCH’ easy. It is a full-text search engine based on Lucene developed in Java. Elasticsearch can be clustered in different nodes which acts as a failover mechanism. There is automatic backup in case of failover using the concept of replicas. Elastic also supports a good number of programming languages like JAVA, Python, .NET, C#, PHP, SQL etc.
Kibana is a tool used to visualize what is inside elasticsearch using different visualization techniques like graphs, piecharts, histograms, sunburst chart etc. You can make your custom visualization components using Vega grammar.
Logstash is a tool used to ingest data from different sources into elasticsearch. Logstash supports a variety of input plugins like http, graphite, heartbeat, imap, kafka, kinesis, redis, s3, sql etc. and a lot more. Can easily ingest data from our logs, metrics, datasources, services etc. continuously and simultaneously.
To play with elasticsearch, refer, https://www.elastic.co/
KAFKA:
Kafka is an event queue system which works based on pub-sub technique. There can be multiple producers publishing events to a topic on kafka and there can be multiple consumers subscribing to a topic.
All transactions happen via the event queue, hence avoiding the web-like random interactions between services. Using kafka makes the architecture more clean and robust.
To learn more about Kafka, refer https://kafka.apache.org/intro
JENKINS:
Jenkins is an automation tool that enables continuous integration and continuous deployment (CI/CD). You can have multiple stages in a Jenkins pipeline to build, test, approve, deliver or deploy a software or project.
Jenkins supports hundreds of plugins to easily integrate with almost every tools used for integration. Refer Jenkins at https://jenkins.io/
DOCKER:
All the applications you develop are to be deployed as containers. There are many containerizing tools like supergiant, portainer, sysdig etc., but docker is leading the market. When you develop an application, you will have an environment. When you deploy, the environment varies. Docker containers act as virtual machines ( there are many differences though) and provide a static environment defined in the .env file, so that your application runs in all the servers in the same environment avoiding deployment issues. Moreover it reduces the number of steps involved in deploying the application and replaces with just a single step.
You will write all the steps, installation, and procedure to be followed in a Dockerfile. When you have multiple containers, you use a docker-compose.yml file to start all containers simultaneously.
You can find many public images in dockerhub. You can use those images from https://hub.docker.com/
Install and use docker from https://www.docker.com/
KUBERNETES:
When there are thousands of services running as containers, there needs to be an engine to orchestrate the process. Google open-sourced the Kubernetes platform in 2014 that enables autoscaling of containers, automatic deployment and managing containers. You can read about kuberenetes in https://kubernetes.io/
JAEGER:
Jaeger is an open-source end to end distributed tracing tool. Debugging a monolith application is more easy than a distributed architecture. In the latter, you have to figure out dependencies between the services.
Jaeger is an excellent tool to do that. Jaeger can monitor distributed transactions, finds latency of each module and helps in performance optimization, gives a root cause analysis, gives you the dependencies between services, and etc. Learn tracing from https://www.jaegertracing.io/
FLUENTD:
Logging is very important in any application. But in a microservice architecture, where you have multiple services running independently, the logs are incomplete and discontinuous unless they are extracted into a single layer. Fluentd is an open-source data collector and provides a single logging layer.
This allows you to understand what is happening in the application, better. You can collect logs from anything like system logs, web logs, nginx logs, sensors data, etc., and aggregate them into any datasource like elasticsearch, mongodb etc. You can learn about fluentd in https://www.fluentd.org/
PROMETHEUS:
Monitoring is very important when an application is deployed, to check if everything is working fine. Prometheus is a time-series data store which scrapes metrics from your application and displays it as a graph.
PromQL is the querying language for prometheus. There are already exporters for many applications like elasticsearch, HAProxy, statsD etc., and prometheus supports inbuilt integrations to get these third party data. There are also alerting mechanism. Get prometheus at https://prometheus.io/
GRAFANA:
How interesting it would be to have a lot of colorful visualizations for our monitoring application? Yes. Grafana is a open platform for beautiful analytics and monitoring, which provides many visualization techniques like graphs, counters, histograms, tables etc., for the metrics extracted by prometheus.
You can alert the user if a metric crossed a threshold. There are many dashboards available already and you can import the dashboard json and use it. Get grafana at https://grafana.com/
NGINX:
Nginx acts as a reverse proxy and acts as a single entry point where all the API calls are made via nginx. It helps in load balancing, caching, webserving etc.
In addition to its HTTP server capabilities, NGINX can also function as a proxy server for email (IMAP, POP3, and SMTP) and a reverse proxy and load balancer for HTTP, TCP, and UDP servers.
— — — — — — — — — -
I have almost covered all components to build a single end to end microservice architecture.(left out frontend though). For a beginner, this would be helpful for sure. And there are a lot more to talk about MSA and lot more components are required to support a pucca microservice architecture which you can learn as you develop.
