Modern Web Architecture

Colocated Services

Before we get down to microservice, lets revisit the good stuff of "services colocated at single process" approach first. I try not to call it monolithic architecture as it may be confused with the webapp has no separate of concerns and its services are well coupled. In fact, it is not an architecture but an outcome of a bunch of tech debts.

colocated services

Pros of Service Colocation

single codebase
good IDE support
easy to develop/debug/deploy
all computation logic are in-memory
easy to scale horizontally but in undifferented manner)
a central ops team can efficiently handle

Cons of Service Colocation

But as the organization grows over time, the pros above could become a headache.

large codebase with many developers tends to introduce tight coupling between components and all components have to be coded in the same language.
no clean ownership unless you introduce an infrastructure team to deal with cross cutting features.
long deployment cycle (single change of a service needs a full release of everything. So, we can't iterate too frequent due to the burden of release cycle).
scaling is undifferentiated
a simple mistake in single service brought down the whole system - challenges in availability

These frustrations have led to the microservice architectural style: building applications as suites of services. As well as the fact that services are independently deployable and scalable, each service also provides a firm module boundary, even allowing for different services to be written in different programming languages. They can also be managed by different teams - Martin Fowler

Microservice Architecture

Microservices are a style of software architecture that involves delivering systems as a set of very small, granular, independent collaborating services.

The best way to see it in function can be visualized by looking at how unix command works. Each command serves its own purpose (ie. Single Responsibility Principle - SRP) while they can work together via chaining up together.

colocated services

Pros of Micorservice

Faster and simple deployments and rollbacks. It gives a sense of ownership and independency from other teams.
It is a great enabler for continuous delivery, allowing frequent releases whilst keeping the rest of the system available and stable.
You can release just the service you change rather than the whole app.
Each service can be built using the best and most appropriate tool for the job.
Systems built in this way are inherently loosely coupled
Greater resiliency due to fault isolation
Better availability if you architect right

Chanllenges and Solutions

Microservice = Freedom of language: Operational homogeneity is sacrificed. It can lead to chao if not designed right. Solution: You can provide a common homogenous operational/ infrastructure component for all your non-JVM based microservice.
Service Discovery: How to find out what services are available for the job? Solution: You need a service metadata registry/ discovery service to deal with that.
Chattiness and Fan Out: 1 request can turn into many internal remote calls. Solution: You can introduce server caching to avoid remote calls to some extents. The cached data could be a materialized view or a low level service cache. And you can config your TTL based on flexibility with data staleness.
Bottlenecks/ Hotspot: For example, a single request will end up calling the same user account service few times as they need the same info. Solution: You can pass the data via HTTP Headers to reduce dependency and load.
Remote call latency: There are 2 major areas that causes the latency for remote call. One is from the network and the other one is data serialization and deserialization.
Scaling microservice: it could be cumbersome to identify and scale individual service. Solution: use cloud based auto scaling can solve this headache for you..
Searching for error: it could be headache to grep all logs from 100s boxes of microservice. _Solution: try logevent pipeline like AWS kinesis, Netflix Suro, logentries, sumologic and etc).
Significant operation overhead
Substantial devops skills required
Implicit interfaces - simple cross cutting changes may end up requiring changes to many different components and all needing to be released in co-ordinated ways.
Distributed system complexity - this architecture imply a distribute system as each service could require different environments. Plus you now introduce lots of remote RPC calls or messaging to glue components together across different processes and servers. Once we have distributed a system, we have to consider a whole host of concerns that we didn't before. Network latency, fault tolerance, message serialisation, unreliable networks, asynchronicity, versioning, varying loads within our application tiers etc.
Asynchronicity is difficult
Testability challenges - With so many services all evolving at different paces and different services rolling out canary releases internally, it can be difficult to recreate environments in a consistent way for either manual or automated testing.
Polygot persistence could be good or bad. It really depends on how well your service decoupled.

Minimize the latency over Microservice Architecture
- Break down the monolithic app to a set of grouped services that function together function as a business unit.
- Replace the fine-grained communication with a coarser -grained approach.

Visualize it

microservice architecture

HTTP communication

Apache HttpClient

/**
 * set up a connection manager and client.
 * you'd normally only do this once in your module or project.
 */
val connManager: ClientConnectionManager = {
  val cm = new PoolingClientConnectionManager()
  cm.setDefaultMaxPerRoute(maxConnectionsPerRoute)
  cm.setMaxTotal(maxTotalConnections)
  cm
}
val httpClient: AbstractHttpClient = {
  val client = new DefaultHttpClient(connManager)
  val httpParams = client.getParams
  HttpConnectionParams.setConnectionTimeout(httpParams, connectionTimeout)
  HttpConnectionParams.setSoTimeout(httpParams, socketTimeout)
  client
}

/**
 * now make the actual GET request
 */
val req = new HttpGet
val url = new URL("http://paypal.com”)
req.setURI(url.toURI)
val headers: List[(String, String)] = ???
headers.foreach { tup: (String, String) =>
  if(!headerName.equalsIgnoreCase("Content-Type")) req.addHeader(tup._1, tup._2)
}
val body: Array[Byte] = Array(‘a’.toByte, ‘b’.toByte, ‘c’.toByte)
//oops, sending a request body with a GET request doesn't make sense
req.setEntity(new ByteArrayEntity(body)) 
val resp = httpClient.execute(req)

Technical Notes