How Istio, NSX Service Mesh and NSX Data Center Fit Together

How Istio, NSX Service Mesh and NSX Data Center Fit Together

This post was originally published on this site ---

This is the year of the service mesh. Service mesh solutions like Istio are popping up everywhere faster than you can say Kubernetes. Yet, with the exponential growth in interest also comes confusion. These are a few of the questions I hear out there:

  1. Where is the overlap between NSX service mesh (NSX-SM) with NSX-Datacenter (NSX-DC)?
  2. Is there synergy between the NSX-DC and Istio?
  3. Can service mesh be considered networking at all?

These are all excellent and valid questions. I will try to answer them at the end of the post, but to get there let’s first try and understand what each solution is trying to achieve and place both on the OSI layer to bring more clarity to this topic.

*Note – I focus this post on NSX-DC and Istio, to prevent confusion, Istio is an open source service mesh project, while NSX-SM is a VMware service delivering enterprise-grade service mesh, while it is built on top of Istio, it brings extensive capabilities beyond those that are offered by the Istio Open Source project.

 

Before we start, in a nutshell, what is Istio?

Istio (https://istio.io/) Is an Open Source service mesh project led by Google that addresses many of the challenges that come up with the rise of micro-services distributed architectures. A lot of attention is paid to networking, security and observability capabilities.  We will not explain everything about service mesh and Istio, as that would require a book 😊 but to really summarize it:

It is an abstraction layer that takes care of service to service communication (service discovery, encryption), observability (monitoring and tracing) and resiliency (circuit-breakers and retries), from the developer and onto a declarative configuration layer.

The data plane of Istio is based on Envoy (https://www.envoyproxy.io/) which is by itself a very successful Open source reverse proxy project started by Lyft. Istio adds a control plane on Envoy.

Why should you care? Istio solves a long-time problem of middleware management, and according to Gartner, “by 2020, all leading container management systems (delivered either as software or as a service) will include service mesh technology, up from less than 10% of generally available offerings today.”

Source: Innovation Insight for service mesh 3 December 2018

We at VMware see it even more than just containers (No reason it cannot be extended to VMs, but that is a different subject.

NSX-DC

NSX-DC delivers virtualized networking and security entirely in software, focusing its networking capabilities on layers 2-4 and some security and load balancing capabilities at layer 7. Here are the main components of NSX-DC placed in their respective place in the OSI model

Logical switching and routing

Switching is a layer 2 component and routing is a layer 3 component. That is basic networking instantiated in software.

 

Firewall

The NSX-DC Distributed Firewall (DFW) is the basis for achieving micro-segmentation on L4, which is the ability to inspect each packet flowing between all application endpoints irrespective of network topology against a security policy. Ports that are not defined as allowed will not be able to be accessed. When implemented on vSphere or KVM, it is enforced within the hypervisor kernel.

NSX-DC focuses on L3-L7 firewalling with added capabilities for L7 service insertion with Next Generation Firewall partners such as Checkpoint and the enforcement is L4 (ports).

L7 based application signatures (DPI)

NSX-T Data Center 2.4, NSX-DC added DPI (Deep packet inspection) capabilities which is the ability to define L7 based application signatures in Distributed Firewall rules. Users can either use a combination of L3/L4 rules with L7 Application Signatures or can just create L7 Application Signature-based rules only. We currently support Application signatures with various sub-attributes for Server-Server or Client-Server Communications only. For more information about NSX-DC DPI see https://blogs.vmware.com/networkvirtualization/2019/03/context-aware-micro-segmentation-with-nsx-t-2-4.html/

 

Load balancing

NSX-DC provides load balancing on layer 4 (TCP load balancing) and layer 7 (HTTP, HTTPS) with many enterprise level load balancing capabilities. This is a traditional edge load balancing implemented in software.

NSX-DC Observability

NSX-DC observability capabilities are significant to our enterprise customer base and focus on L2-L4 networking and security. Among other things, we can provide visibility into the traffic the flows through NSX-DC FW and provide physical FW traffic relatability into the NSX-DC logical networks and the platforms running with NSX-DC using SNAT tools. You can read more about that here: https://blogs.vmware.com/networkvirtualization/2018/06/the-value-of-nsx-data-center.html/

As you can see, most capabilities concentrate on L2-L4 networking instantiated in software with extensive enterprise grade monitoring and operational tools.

Let’s look at service mesh now. I am concentrating on Istio, mainly because it is the frontrunner service mesh and because it is the first service mesh to be supported by NSX-SM (SM=service mesh)*.

Another note about service mesh, Istio, and NSX-SM relationship. Istio is a solution for service mesh, NSX-SM is VMware’s enterprise service mesh solution that builds on top of Istio (as a first phase) which is a private beta currently. Read more about NSX-SM here: https://blogs.vmware.com/networkvirtualization/2018/12/nsx-service-mesh.html/

Traffic Management https://istio.io/docs/concepts/traffic-management/

Istio’s Traffic management capabilities are based on the envoy L7 proxy, which is a distributed load balancer that is attached to each micro-service, in the case of Kubernetes as a sidecar. The placement of that load balancer (close to the workload) and the fact that all traffic flows through it allows it to be programmed with very interesting rules. Before placing it on the OSI layer I will quickly cover some of the main traffic management capabilities.

Also, there is an ingress and egress proxy for edge load balancing in Istio that I will touch upon as well.

 

Traffic Splitting (sometimes called Traffic shifting)

Provides the ability to send a percentage of traffic from one version of a service to another version of a service. Sometimes referred to as “canary upgrade” or “rolling upgrade”. As you can see in the diagram below, 1% of traffic hitting the frontend service is directed to a new version of service B. If it looks good (no errors etc) more traffic will be shifted to “service B 2.0” until 100% of traffic has moved. The fact that it is L7 routing means that the decision where to send traffic is application based (service to service communications) and is unrelated to L3 routing which is IP based.  This will be a common theme as you will see.

Traffic Steering

Allows us to control where incoming traffic will be sent based on application attributes (some web attribute) such as authentication (send Jason to service A and Linda to service B), location (UK to service A and US to service B), device (watch to service A and mobile to service B) or anything else the application passes. This was achieved in the past in different ways from the application code to libraries but with Istio we can configure that declaratively using a yaml file. Cool stuff!

Ingress and egress traffic control

The Istio gateway is the same Envoy proxy, only this time it’s sitting at the edge. It controls traffic coming and going from the Mesh and allows us to apply monitoring and routing rules from Istio Pilot. This is very much like the traditional load balancing we know:

Now, let’s place Istio Traffic management on the OSI model

 

As we can see in the diagram above, all the traffic management capabilities are on the L7 traffic management and load balancing level. We can see some overlap here between Istio with NSX-DC LB on the edge load balancing in a purely functional sense, though we still need a load balancer in front of Envoy to send traffic to the Istio Gateway. The decision where to send traffic within the application would be for Istio. This is a common topology in container/micro-services platforms that is present in solutions like Pivotal Application services (PAS), where a reverse proxy is placed in front of the application and behind a traditional load balancer.

NSX-DC provides ingress L4 load balancing that is not achieved by Envoy and is required for service management on Kubernetes and traditional applications. NSX also provides the enterprise level operational tools needed by enterprise customers. The NSX-DC load balancer can send and manage traffic to the mesh through the ingress Envoy and to other entities that are not part of the mesh (not everything can be covered by service mesh, yet ). So, there is partial overlap, or more accurately, some of the intelligence is moved up the stack, however, a traditional edge LB is still very much needed.

 

Istio Observability and Telemetry

Istio provides significant capabilities for telemetry, again, all focused on L7. Let’s look at some of those:

service Discovery https://istio.io/docs/concepts/traffic-management/#discovery-and-load-balancing

While it is part of the load balancing act of Istio, I placed service discovery under observability as a category on its own. Istio’s Pilot component consumes information from the underlying platform service registry (e.g. Kubernetes) and provides a platform-independent service discovery interface. With this interface, tools like NSX-SM can provide service observability (only NSX-SM does that across Kubernetes clusters in multiple clouds and is not restricted to a site or a specific cluster).

here is a sneak peek to NSX-SM’s service discovery map:

Istio’s observability is focused on L7 and completes the picture with NSX-DC that focuses on L4. When troubleshooting a micro-services problem, especially cascading issues, one needs to visualize the service dependencies and flows. Problems can occur on different levels, can be a service application issue, compute issue or a network issue. Having the full picture is a powerful thing that reduces the diagnostics and troubleshooting significantly. No overlap here:

Istio security https://istio.io/docs/concepts/security/

Istio provides security features that are focused on the identity of the services. Even more so NSX-DC service mesh, extends that uniquely to users and data. When we talk about identity, we focus on L7 (did I say that Istio is all about L7?). Here are a few key capabilities:

Mutual TLS authentication https://istio.io/docs/tasks/security/mutual-tls/

Istio’s service to service communication all flow through the envoy proxies.  The Envoys can create mTLS tunnels between them where each service will have its certificate (identity) received from the Citadel component (the root CA of the mesh). The encryption is all happening on L7 meaning what gets encrypted is the payload, not the traffic. (See my earlier post about mTLS and the OSI layer here: http://cloud-abstract.com/istio-mtls-and-the-osi-layer/)

Istio Authorization https://istio.io/blog/2018/istio-authorization/

This is the micro-segmentation that Istio provides that I mentioned earlier. Istio authorization provides namespace-level, service-level, and method-level access control for services in an Istio mesh. It is called micro-segmentation because it is using the same notion that what is not defined as allowed to pass shall not pass, only this time the controls and enforcement is at L7 (service to service). The authorization service provides authorization for HTTP and provides micro-segmentation for services in an Istio mesh. Not to be confused with NSX-DC micro-segmentation which operates on L4 based ports access.

Now, NSX-Dc DPI does provide L7 controls based on application signatures. But it does not cover service to service communication, and enforcement is done on L3-L4, hence there is no overlap with NSX-DC micro-segmentation.

Istio can segment the services based on L7 constructs, such as:

RPC level authorization:

This method controls which service can access other services. Best described in the Istio docs:

Authorization is performed at the level of individual RPCs. Specifically, it controls “who can access my bookstore service”, or “who can access method getBook in my bookstore service”. It is not designed to control access to application-specific resource instances, like access to “storage bucket X” or access to “3rd book on 2nd shelf”. Today this kind of application-specific access control logic needs to be handled by the application itself.

Role-based access control with conditions

This is method controls access based on User identity, plus a group of additional attributes. It’s the combination of RBAC (role-based access) with the flexibility of ABAC (Attribute-based access)

 

When leveraging Istio “micro-segmentation” at L7 one may wonder “do I need L4 micro-segmentation?” The correct answer is “absolutely yes”. Not having both layers of protection is like leaving your house open when your front gate is locked, the network is a hostile environment, just as we did not drop the perimeter FW when we added East-West DFW with micro-segmentation, and it doesn’t make sense to do the same with L7 controls and enforcement.

As security attacks become much more sophisticated, a multi-layered security approach is important as each layer focuses specifically on an area where attacks could happen. (For more in-depth analysis of the layered approach with Istio see https://istio.io/blog/2017/0.1-using-network-policy/

 

To summarize, let’s go back to the original questions we asked:

  1. Where is the overlap between Istio and NSX-DC?

As you can see the amount of overlap between the solutions is minuscule, and even where it exists (edge LB) we still need both. Matter of fact when it comes to security and network intelligence, more is better.

We’ve seen the Istio solution and NSX-Dc on the OSI model. Let’s see now VMware’s NSX-SM solution with NSX-DC on a logical diagram

2. Is there synergy between the solutions?

You will see the discussed synergies being realized soon. We are working on some important integration services around NSX-SM and NSX-DC Data Center (NSX-T), bringing the two solutions closer together.

To see where each one fits check out this physical diagram:

 

Beyond what we have discussed in this post you can see that NSX-DC also covers a wider range of workloads, where NSX-SM is focused on Cloud native workloads (for now)

Lastly, can service mesh be considered networking at all?

In my view, since Istio lives in L7 where network and application boundaries start to blur, it can be considered networking and at the same time be an application platform.

What do you think? 

The post How Istio, NSX Service Mesh and NSX Data Center Fit Together appeared first on Network Virtualization.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.