I’ve done my google searching and I know what it is and I have a general idea of how it works but I’m hoping someone can give me an in-depth overview of how traffic flows, how the switches/routers negotiate the routes, and the protocols that are used. Thanks!
There are some pretty good explanations so far here, but I’m going to add the explanation I feel make the most sense. Since you ask for an in-depth one, I shall try to cover that too. I have to warn you. This post will be extremely long.
So far, I am sure you are aware, either from your own studies, or from the other fine posts here that MPLS is something about labels that are switched.
You might even be under the impression that this is for the sake of speed, as label switching can be done without your routers consult their routing tables.
You might also have read about MPLS VPNs, and that might have confused you slightly.
Let me clear up some of this.
While MPLS did provide some sort of speed benefit in the old days, it is certainly not the main cause for implementing it today, rather MPLS is all about tunnels.
In fact, you can view MPLS as a way of establishing uni-directional tunnels through a service provider network.
First off, a few terms that I am going to use here:
- LSP: Label Switched Path.
This is a path through the network as signalled by MPLS, or as mentioned before, a unidirectional tunnel.
- LER: Label Edge Router.
- LSR: Label Switching Router.
These are two roles a router can have in an MPLS router. The LER lives on the edge of the MPLS Service Provider router and is also known as a PE router (Provider Edge). It interfaces with customers and routers that do not run MPLS. The LSR interfaces with other MPLS-running routers.
- LDP: the Label Distribution Protocol.
- BGP: the Border Gateway Protocol.
I hope you’re somewhat familiar with BGP. It’s the routing protocol of the great internet, and heavily used with MPLS. LDP usually runs the MPLS signaling and I will dive more into that later.
Now, before we look into the operations of MPLS, let us first examine the core network of a Service Provider.
The service provider network is built up with edge routers and core routers, connected with high speed links and to convey information about routes, link addresses and similarly, they run an IGP (internal routing protocol). This is most often either OSPF or IS-IS.
When a prefix is learned in a network that does not run MPLS, only the IGP tells the neighboring routers about it, who then tells their neighboring routers about it, etc.
Now with MPLS each router assigns a label to each known prefix in their routing table, and communicates this downstream using LDP.
The neighboring router then notes in its label forwarding table that label X from router A corresponds to prefix Z which on that router has been given a label value of Y.
It is important to understand that labels are only locally significant, so label 114 can be used many times throughout an LSP.
Example:
In a topology like this: LER1—LSR1—LSR2—LER2
LER1 learns prefix 192.168.1.0/24. It assigns label 42 to it and through LDP it is communicated to its neighbors. LSR1 assigns label 30, LSR2 22 and finally LER2 112.
Now we have a LSP from LER2 to LER1 and it was automatically signalled for us.
You may be reading this and thinking about what the point of all of this is. LSPs are somewhat pointless by themselves. They are unidirectional tunnels, sure, but as long as they follow the routing table the point is somewhat lost.
Let us instead dive into two use cases: VPNs and Traffic Engineering. Both being the prime reasons to implement MPLS.
VPNs
Service Providers have customers connected over their backbones in closed networks, or VPNs. For corporate customers, an MPLS VPN give them a private network over the Service Provider network that is completely isolated from other customers’ networks.
The customers routes and traffic are isolated in the provider network in a VRF (Virtual Routing and Forwarding). This is nothing more than a separate routing table that is defined on the LER that the customers line is connected to.
The scalability of MPLS lies inherently in the fact that these VRFs are only defined on the actual LERs that the customer is connected to. The rest of the MPLS network is completely unaware of that customer’s routing and traffic.
To identify each customer’s traffic and routes and keep all the different customer prefixes unique, two techniques are used:
- BGP
- MPLS forwarding
BGP was extended to work for MPLS through what is known as MP-BGP or Multi Protocol BGP. A separate address family besides IPv4 (or v6) has been defined for VPN traffic called VPNv4. It works by distinguishing routes through a Route Distinguisher (RD) tied to the VRF. The RD is unique per LER and is prepended to the customer prefix.
Each VRF also have route targets attached in BGP. These are BGP communities that also identify the VRFs.
All LERs talk BGP with each other and through the use of Route Targets they can define what VPNs they are interested in knowing about.
In this way, only the LERs that actually needs routes for a particular customer VPN will receive them. This greatly conserves routing table spaces on all routers in the SP network.
So, how about MPLS? Well, besides the normal routes and labels exchanged between routers, the LER also assign labels per VPN prefix.
MPLS allows multiple labels in a stack, so besides the label needed to reach the LER, there will be an additional label indicating what VPN prefix that the traffic is going to.
That enables LSPs to be built without the upstream routers to know about what prefixes are.
A small detail:
To avoid that the terminal LER has to pop both the label meant for itself and the label for the VPN prefix, usually LERs signal prefixes that they terminate themselves with a special label value called implicit null. This tells the upstream LSRs to pop the label instead of swapping it before sending the packet to it. In this regard it means that packets to VPNs arrive only with the label assigned to the particular prefix and only a single lookup needs to be performed.
So in conclusion, using a mixture of BGP and MPLS allows the creation of VPNs deployed in a scalable way to only the required LERs while transportation across the MPLS core network works without knowledge of the VPNs.
Traffic Engineering
I realize I’ve already written quite a lot, but I feel I need to brush up on MPLS-TE just shortly.
MPLS-TE is the other great use-case of MPLS.
In very broad and short terms, it is possible to assign an LSP to other things than just prefixes. With the aptly named protocol RSVP (bandwidth ReSerVation Protocol) it is possible to define backbone traffic tunnels with bandwidth constrains or traffic class requirements, directing certain types or amounts of traffic other ways through the network. For example, you can define that your Voice over IP traffic needs to be transported on a different network path than the rest of your trafffic. You can even define two separate paths through your network, one active and one passive, and in case of network failure on the primary path, the traffic can be directed via the other paths with sub-second failover.
I hope this turns out to be informative to you, and not just a long wall of text. Feel free to ask me to elaborate anything.
MPLS is just the concept of forwarding based on labels (32bit numbers) rather than dissecting each packet and picking out the destination IP.
It’s less hardware-intensive for each hop to route, and protocol-independant (such as being able to encapsulate IPv6 traffic over a non-v6-aware core (6PE)
You can also stack labels (basically, taking a letter (IP packet), putting it in an envelope (first MPLS label) and putting that in another envelope (a second MPLS label)… and stack more labels if you like. Stacking labels allows other MPLS applications such as MPLS VPNs
someone can give me an in-depth overview of how traffic flows, how the switches/routers negotiate the routes, and the protocols that are used
From a service provider perspective which is where MPLS has become the killer application in practice here is the protocols involved and the flow of traffic. MPLS L3VPN example.
Example Topology
Cust-A----PE-1----P-----PE-2-----Cust-A
Control Plane
From PE-1, P, and PE-2 are the service provider routers, they run OSPF in the same area (they can run ISIS also) they exchange their internal loopbacks through OSPF. They also run a label exchange protocol called LDP (they can run RSVP, but thats for another discussion if TE is needed) LDP allows them to exchange the transport labels, this means that if PE-1 has a loopback of 1.1.1.1, it will send a label of 3 (implicit null, no need to know this yet) then P router will send PE-2 a label of 25 for 1.1.1.1 that maps to a label of 3 locally and its send to PE-1 (more on this during the forwarding plane) in summary, everyone has a label to each others loopback.
PE-1 and PE-2 also have a M-BGP peering which allows it to send MPLS/VPN routes with labels and special communities called route targets. Each router has an interface in a VRF, which is a separate routing table attach to customer A. This VRF also has a route-target (RT) configured and a route distingusher(RD) configured. During a BGP update, a route from the customer VRF is sent to the neighbor with a route target and an MPLS label. Lets say the route-target configured is 1:1 and the automatically generated label is 1000. When the other side receives a route target of 1:1, it knows which routing table to place this route into because it has the same route target configured. The RDs sole purpose is to keep the route unique so there isn’t any overlapping addressing if there is more than one customer (there always is)
From Cust-A routers to the PE routers there is a separate exchange of routes that stays in the VRF by a protocol not related to the global table (i.e. it can still run OSPF area 0 and not have its global OSPF area 0 affected), route redistribution is needed for the Customer to learn routes coming from the other side.
Forwarding Plane
Now that everyone has exchanged routes and all the pieces are in place, lets say a packet comes in to PE-1 from Cust-A destined to Cust-A attached to PE-1. PE-1 looks up the destination and sees that this destination has a VPN label of 1000, it then looks at the next-hop and it sees the next-hop of PE-1 with a label of 25, it then stacks this two labels and forwards the packet to the P router, the P router only looks at the top label which is 25, and swaps it with label 3 towards PE-1. PE-1 then sees that he has label 3 and pops the top label and sees label 1000 which is a local destination in VRF cust-A, it then takes that label off and forwards the IP packet towards Cust-A attached to it.
No editing was done, excuse grammar or spelling.
Some PE configurations:
http://inetpro.org/pastebin/11435/view/raw
In this case the PE is doing BGP towards the customer and its running RSVP instead of LDP.
NinjaEdit:
So in short the huge advantage to this is that a service provider only has to provision the Edges when a new customer needs an MPLS/VPN, the core devices do not need have a ton of RAM since they are not carrying any routes. This also allows for a BGP free core and nice and elegant networks to be configured.
Some good soul will probably answer you extensively. I was like you until a few years ago I read this book. It’s short and easily readable and doesn’t dwell too much into deployment considerations (it’s not very up to date anyway on this point). I find it particularly interesting in that it also covers the history behind MPLS and how decisions were made during its conception. It’s then much easier to understand very specific application of MPLS.
My experience with MPLS is from within a frame-relay meshed network, I honestly have no idea if there are different flavors of MPLS or how they would operate apart from my experience, but our cloud ran pretty well (when Verizon wasn’t screwing things up, but that’s a story for another time…)
We didn’t have a certain traffic flow from our cloud, we made everything reachable to everything else so traffic zipped around as needed. We did use QoS though, end-to-end on the router serial interfaces as best as possible (we ran VoIP over the WAN/LAN as well) Routing protocol was BGP all over. Verizon supplied us with BGP neighbors on the other end of the link, so we were able to re-use the AS number (64999 for local router, 65000 for neighbor) to redistribute BGP routes to all members of the mesh. Looking back on it now, that might not have been the best config in that it would be harder to determine BGP propagation with everything having the same AS number. For static routes, we picked the corporate router (the beefiest we had) and set up all the static routes on those which propagated out through BGP.
Hope that helps.
Without getting too much in depth, and not being sure of what you have read, I can give it to you in a simple explanation.
MPLS is the concept of having multiple routing tables so that you can separate out traffic on your network entirely. Let’s say we have three VRFs, “inside”, “dmz”, and “wireless” (you can do other things as well, I’ve used VRFs more than VPLS or VLLs). I have six VLANs:
1: Employee_PCs
2: Employee_Printers
3: DMZ_Servers
4: Internal_Servers
5: Public Wireless
6: Private Wireless
I know there would probably be more, but, let’s keep it as simple as we can. So, on a non-mpls network, you’d have a separate switch or router going from your firewall to your DMZ area (typically the data center), and thats how you segment out the traffic. With MPLS, I can use the same router with all six vlans defined on it to handle the traffic.
On the layer 3 interface for the VLANs, I set VLANs 1, 2, 3, and 6 to forward the traffic to the VRF ‘inside’, VLAN 4 forwards to the VRF ‘dmz’, and VLAN 5 I set to forward to VRF ‘wireless’. So the same router, which has all six VLANs defined on it, segments off the traffic into separate routing tables.
So what advantage does this give you? I work at a University. The Physics Department has their own servers which shares data between other Physics Departments at other Universities. This has to go in the DMZ, right? Right, but we have no VPN tunnel, and they don’t want to move their servers to our data center because of Politics. But instead of running a line from a switch or router in one building where my firewall is, to the building that has the physics department, I can MPLS both routers, create the ‘dmz’ VRFs in both routers, and have them share the routing tables. I can now have my DMZ in any router on my MPLS network, without having to move the servers to where I need them to be. It allows for a more robust and variable network, at a much more expensive cost.
The VRFs have a route import/export command that gets assigned, so that other VRFs on other routers know where to put the routes. For example:
On router ‘administrative’, we have VRF ‘inside’ and VRF ‘dmz’:
ip vrf dmz
rd 100:120
route-target export 100:2000
route-target import 100:2000
ip vrf inside
rd 100:110
route-target export 100:1000
route-target import 100:1000
Those route-target commands tell the router, “Hey, when you see the labels for routes coming in with those values, you put them in this table, got it?” So, on the administrative router, I could call the vrf ‘inside’ and on a different router, say, shipping, I could call the vrf ‘main_inside’, but as long as the route target export / import are the same values, they’ll get the same routes.
You can redistribute routes through a variety of means. For example, I use ISIS to redistribute the BGP routes. Other people use OSPF. ISIS and OSPF would be considered your backbone network, and would be used to deliver the route updates and information that your BGP network would use. This makes it nice and fancy, because your BGP configuration believes that all of your neighbors are right next to each other, and you don’t have to worry about things like route pattern maps or route mirrors.
That’s what I can think of off the top of my head without knowing a bit more of what you’d have questions about.
For anyone else reading, please, make sure you read this post:
http://www.reddit.com/r/networking/comments/z0mxf/can_someone_explain_how_mpls_works/c60klq7
This is what MPLS is, the concept of forwarding based on labels (Hell, it’s in the name, Multiprotocol Label Switching). Things like VRFs, VPLS, or VLLs are what we can do with the routing after we base it on labels.
Read dubcroster’s description, for it is good.
The tl;dr on mpls is that it’s a way of building virtual circuits across IP networks. It was originally intended to improve routing performance by decreasing the amount of work intermediate routers need to do, but thanks to inprovements in router performance, this isn’t necessary.
These days it’s mostly used for traffic engineering and resiliency purposes, as you can control traffic flows between routers with much greater ease and granularity than you can with vanilla IP routing, and can achieve rapid failover by building what amount to redundant virtual circuits.
Building on from this, I have a question about MPLS VPNs.
So, the traffic is tagged by the PE router and it crosses to some other PE and across the link to the CE router. How can it be encrypted/decrypted by CE routers to prevent snooping/interception over the MPLS cloud? Would you even do this?
I’ve no experience with traffic encryption, but I’ve labbed some MPLS and other VPN (DMVPN, etc) scenarios, so maybe this is simple and my Google-fu isn’t very good today?
Can I have the CE router encrypt all traffic heading out into the MPLS network using some pre-shared key which is common across the all CE routers? Is this bad practice, and if so, how what is a better approach? The goal would be permitting full-mesh connectivity without having to manually create all the CE-CE tunnels.
I mean, it is principal like VPN. Sorry, but i’m beginner in the MPLS.
I hate to be that guy, but:
This.
This is extremely helpful! Thank you
If speed is not necessarily an advantage of using MPLS, what would the advantage be, if any, of an enterprise having an MPLS circuit from an ISP between two sites vs having say a Point-to-Point VPN between two sites?
I’m not an expert on MPLS by any degree (in fact I’ve never worked with it), so my question to you comes from genuine curiosity: What makes you say it is less “hardware-intensive” as opposed to IP routing (given the capabilities of modern ASICs)?
This is a great answer… +1
They key point being: P devices (SP core routers) don’t have to worry about IP for routing, they only look at the labels. Much more trivial to process at insane packet rates.
I’m upboating this because I don’t understand a goddamn word of dem words but you sure sounds smart. (This system doesn’t have RES so i can’t save it) Also I keep pronouncing IS-IS as “Isis” instead of “I.S, I.S.” and always get corrected on it 
Would also recommend “MPLS Fundamentals” from cisco press and the original rfc as suggested elsewhere on this thread.
Don’t know why you’re being downvoted for linking the RFC, if you want in-depth details, I can’t think of a better place to go.
This! Quite understandable and provides a wealth of information.