Some Ideas About Microservices. Part 2

How is an Architecture Ready for Micro Services ?

Ready for adopting MSA?
An apparently easy question but it's not so easy...

Is it monolithic? Yes, but in general terms the app is working fine.
OK, do not change.
Do you expect grow in your concurrent users? Yes, but I have metal available over there.
OK, do not change.

Are your apps going badly? Do you expect more and more concurrent users and your budget is getting lower?
Your processes are more often in bottlenecks.
You have a big problem with each release version.
You need a specialized team for deploy changes.
You are involving more and more complex processes.

Maybe you need a change towards Microservices, towards Non blocking IO, towards an automated work distribution. It is worthwhile.

What should be the main features in a MS Architecture?

Some times a diagram is better than words..

But I´d like to consider before some good pieces of advice. For example..

Topics we should consider

Above diagram suggests several main features in MSAs:

• For Gods's shake! The size! If you need a MACRO service, OK, go ahead. We are talking here about Microservices. One, two or three small classes. Single very well defined responsability, full focus on the functionality.
• Non blocking API!!! Use asynchronous APIs. User Reactive programming if you can.
• Consider a multi project structure. Not always Maven modules are the solution although Maven is great for these tasks. Gradle leverages project management in this way. SBT have similar features. This type of projects allow multi formats, multi tasks, different better adapted lifecycles and much more. It´s increased flexibility what we need.

• Auto deployment. The build pipeline ends with the deployment and promotion. OK, this is Continuous Delivery!!
• We should be agnostic. Commitment with a given programming language (to be dependant on a given framework is something I cannot imagine) is not good in the mid term. Dependencies on JVM version, libraries, etc are factories of problems. Consider a poliglot system, with the most appropriate language for each specific service. 
• Be resilient. Your services should be prepared for failure. There are several ways to get be relient. Use supervisors, manage the failures and adopt a specific pattern.
• Stateless. Each microservice is adjusted under the principle of single responsability, totally focused on a specific absolutely well defined task under heavy concurrency conditions. Stateful services are slow, are complex and are out of scope in early stages of a Microservices architecture.
• Architectural Patterns for microservices. It can seem is obvious but it´s not! Most of MS based systems have not an architectural pattern. Services are deployed without considering special roles and functions in the system as a whole.
• This drives us to containerization. From a nerd idea to be almost a standard. Containerization is one of the best solutions you can find to deploy your microservices and put them on working. It is alone in its container, it can be cllustered, balanced, isolated. I´ve used Docker, Docker files for the pipeline and other tools for clustering and load balance and works perfectly. 
• Scalable services are clustered. Containerization allow us in the easiest way.
• Yes, we need a Source Code Management workflow!! 
• Performance: metrics & self-monitorization. Each MS has a sub component with this function.

• Error early warning. Self supervision is based on self-monitorization. It should be provided by the selected framework.
• Integrated log. Try to get all logs from your microservices are aggregated in a single point for explotation and easiness to be read or analyzed. It depends of yoour needs, size of the system, etc.
• Service Map. We need know where they are, I mean, services must know where to access to new services. Refuse ideas as registries, "self discovery" and so on. A clusterable-scalable system is perfectly configurable. If needed, ask the CI server about the state of the system. It is fast, it is reliable. 
• Fast startup. Lightweight, easy, very robust services makes fast startup services. This is essential for autoscalation and resilience.
• Specialized Testing Well, we need to extend the services when interact between themselves from the very beginning.

The Communication Line

By using tools like Jenkins, uDeploy, Capistrano, Chief, Puppet or custom scripts should completely automate deployments of MSs. 

Each new deployment is an occurrence in the services map and triggers notifications. 
MSs communicates information between them. It's meta-information about the system. Under the business logic the blood of the system is metainformation.

That’s regarding SDLC. But what about the service in itself? By using Spring Boot, SBT, etc we can create auto-deployable components in order to one single action deployemnt operations. Naturally, these deployment actions, based on implicit properties of the services, are orchestrated by the CI Server (Jenkins, TeamCity, Bamboo) to keep continuously deployable the whole system.

Because each service talks to the others, it could be hard to pinpoint the source of performance problem. There is Apache log to get performance information. Solution is in performance monitoring of each individual MS. So, all the log information must be handled in a very specific way. There is so many different servers so just looking and finding problem in logs become in a big pain. Aggregated logs in a single location is the solution.

Synthetic monitoring (also known as active monitoring) is website monitoring that is done using a web browser emulation or scripted recordings of web transactions. Behavioral scripts (or paths) are created to simulate an action or path that a customer or end-user would take on a site. Those paths are then continuously monitored at specified intervals for performance, such as: functionality, availability, and response time measures. – Wikipedia

For example, one of our MSs suddenly does not perform well because of unexpected spike in traffic. By using “elasticity” provided by major cloud vendor, we can setup a simple rules that will automatically provision additional hardware when specific threshold are met. for example, if service response time increase to 500ms, deploy several more servers. Well, but how to notify about it to the rest of MSs? We need "intelligent" MSs based on good archetypes with all these common functionalities, specifically handling performance information for auto scaling out towards other MSs instances.

I call Communication Line the architectural part that describes how MSs communicates for the above ennumerated issues:

1. Log (Registry Information)
2. Concurrency pressure on the service instance (Concurrency Information)
3. Problems while working (Supervision Information)
4. Location of services and instances (Services Map and Notifications)

You can be tempted to alter your microservices with whatever mechanism for "autoregistration" or "automatic discovery". Did we return to 90s ideas? 

We have a pipeline, we have Continuous Delivery and DevOps. We do not need to add a useless new phase on build to insert alien things to our microservices. 

Moreover, these solutions use to be specific for a technology even a specific framework (!!??). Yes, I´m thinking of Spring Actuator and others. It´s really a very bad idea.

Trust your pipeline for dashboarding. Cont Delivery and devOps will give you needed information. Ask the correct questions to your CI Server. Be smart!

Towards a new ESB???

We all can see "specific" MS APIs that pretend to be a "panacea" for all og the issues with handling microservices. DO NOT BELIEVE IN THAT. 
Please, we are engineers ans Microservices give us an excellent opportunity to do the things in the right way. So, why blindly trust in a programmed downgrade of an API or a tool that seems to be programmed by two nerds in a dark room??? 
Most of these ideas in these types of "resources" slowly drive us to old 902 ideas. Yes, SOA world attacks again!
We do not need an ESB, we do not need RPC calls. Please, read technical bibliography from last 5 years!

We can define and design how Microservices communicates with Microservices. We can use Camel, Spring Data, pure HTTP, websockets (Everyboy has forgot the fantastic performance and robustness of JSON on STOMP and websockets??). Are we designing high performance Microservices in order they communicate with  others Microservices by invoking RPC calls?
Remember one of the needed features in Microservices: NON Blocking APIs. This is the foundamental goal. This is immcompatible with certain types of communication. 
Please, build somothing of your own if necessary. Feel Microservices as an opportunity! Beware of easiest-valid-for-all strange solutions!

Do not follow the "Spring way"!

The Spring project is trying the developer community in Java make Microservices based system only with Spring. It is normal. What is not so normal is the number of developers that believe in Microservices as a "springboot-springwhatever-netflix-tomcat" equation. This is simply bullshit.
Take a look on the Maven shade plugin. You can create a microservice in minutes...without Spring!!!

Spring is good, is useful. Spring is now closer to JEE specs with Spring 4. Do not forget this fact. JEE is light years away in non bloking apis development and thread management. They are allies, not enemies!!!

Finally

So, we have defined 4 main information channels in a MSA system.

We need MSs as clients and servers, as producers and consumers of meta-information, as living entities in a complex echosystem. Isn't exciting?

On next chapter: Transactionality, testing and the description of the PoC with (at last) specific technical details.

Thanks for your time!