How Does Internet Work

Hello World

This is an overview of what I think Cisco ACI actually is. It uses some examples from the lab environment to show you how the things look like when you start to work with ACI. There are other articles in the works which will be online soon and which will go in details through the real configuration of ACI and best practices while doing it.

What is this Cisco ACI Fabric?

Cisco ACI is a datacenter network Fabric. It actually means that it is a networking system of more networking L3 switches that have a modified, next-generation OS which enables them to be centrally provisioned and configured through APIC controller to work as one device from access port perspective.

The view at Cisco ACI APIC GUI where we see complete ACI Fabric Topology

Edson Erwin invented this highly scalable and optimized way of connecting network nodes in the 1930s and Charles Clos made the telephone nodes interconnection design using that solution. It was even before we had IP networks. He invented it in order to optimize the architecture of telephony network systems back then.

It was not used in IP based network for last few decades but it experienced a big comeback with new datacenter design in the last few years. It was first invented only for scalability requirements that it solved beautifully. In new datacenter design, CLOS topology of interconnecting network devices scalability is also the first requirement that gets solved, but it also greatly helps with improving resiliency and performance.

In today’s datacenters, CLOS topology is used to create Leaf’n’Spine system of interconnecting Leaf switches (datacenter access switches or ToR switches) together through Spine switches. It is created in a way that each Leaf switch is redundantly connected to all Spine switches directly.

As it is shown in the picture below, in this way, using CLOS topology, we are interconnecting Leaf switches in a way that they always have only two hops between each other and this done redundantly as two hops through each Spine switch. Spines are not directly connected and Leafs are also not directly connected.

Again I made it to the list of Cisco Champions, making this the second year in a row!

I am so glad that my effort to give back to the community and to all my networking fellows out there paid off again in the shape of this recognition from Cisco. This badge is only a small thing, relating to all the community connections and sharing that my involvement with networking community via social media and this blog, made possible. It only pushes me to get even more done in the future.

In 2018 I was involved in a few very challenging new projects, working with Cisco ACI and VMware NSX, and basically studying and researching continuously in order to get to know the products as much as it was possible, it resulted in not being so active on my blog or twitter this year.

If there’s no BFD

If you have two routers directly connected, like here:

In this case, it is normal that one of them will remove the routes learned from the other if the other one goes down completely. It is because the link will go to down state and the routing protocol adjacency will disappear.

If two routers are connected through an L2 device (switch) like down here:

In this case, when one of them goes down, it will not take down the interface of the L3 neighbour (other router) because the switch will still work fine and it will keep the other half of the like up:

If that’s the case, you will depend on routing protocol timers which are the failure detection mechanisms implemented in the routing protocol itself. Routing protocol timers will need to expire in order to bring the router adjacency down and start the convergence to some other path towards the destinations.

Routing protocols timers are not a bad mechanism and they can be tuned so that they detect the failure faster.

EIGRP hello and hold timers can be tuned to get you somewhere around 1 second for failure detection and the start of convergence. With IS-IS and OSPF you can enable fast hello option and this can get also to 1 second for failure detection.

You can probably guess by now that to speed things up the BFD from the title will be the best solution.

Whats is BFD?

To make failure detection fast, like really fast, like sub-second fast you should use BFD. BFD, which is a separate protocol for communication failure detection, uses small overhead probe packets (like smallish hello packets) that are sent many times in a second in order to get you to sub-second detection of communication failure.

Intro

The largest chunk of bytes that a transport protocol can forward across specific medium is called MTU – Maximum Transmission Unit. If we speak about Ethernet, which is today the most common, he has by default a frame size of 1522 bytes with the payload of 1500 bytes which is the amount of useful data he can carry in a single frame.

The story about MTU is that the MTU of specific protocol basically defines how much payload (or highest protocols headers + their payload) it can carry in its biggest packet, not counting his own headers. Putting more payload into a single packet than the MTU allows will result in fragmentation, the process of slicing the frame into more smaller frames so that they can get through the path.

Ethernet frame

So if we look at the wire and catch a standard Ethernet frame, we will see that he is able to carry 1500 bytes of data (Ethernet default MTU of 1500 bytes) and additional Ethernet header which is 18 bytes (14 bytes in header and 4 bytes in trailer) or 22 bytes when it has 802.1q VLAN tag added inside.

Different MTUs: Interface MTU, IP MTU and TCP MSS

Interface MTU (Ethernet MTU)

The image above is showing different MTU types. It is important for to note that Interface MTU (Ethernet MTU) is the maximum MTU size that a specific hardware port on our network device can forward. In the case of Ethernet (IEEE 802.3) the maximum frame size is 1522 bytes which was raised from 1518 bytes to allow VLAN tagging using 802.1q. Minimum Ethernet frame size can be 64 bytes. After some time, when Gigabit and faster ethernet ports started to appear, jumbo frames were introduced with the ability to increase the interface MTU to 9000 bytes for performance reasons (slightly less header overhead).

There was an old way of generating Interface IPv6 address using SLAAC process (Stateless autoconfiguration). You simply configured that you want SLAAC autoconfiguration and the interface IPv6 was generated by squeezing “FFFE” in hex (11111111 11111110 bits) between two parts of physical MAC address of that interface.

Then, after a while, several comments came to IETF about the use of predictable Interface Identifiers in IPv6 addresses. They were pointing to the ease of correlation of host activities within the same network and across multiple networks. If Interface Identifiers are constant across networks this is negatively affecting the privacy and security of users on that network.

There is always the option to edit a file locally on your computer in notepad++ and then upload it to Cisco device.

With this trick, you can write or edit a file from Flash memory directly from Cisco IOS console. If you need it for whatever reason, maybe to change something in the config file or something else, this is the way to do it.

I was using if for my next article lab where I needed a file on the flash with at least 1600 bytes so I can pull it from neighbouring router with HTTP and check some details in new TCP connection. It was faster to just create the file directly on the router flash and not creating it on my laptop and then transferring it with FTP or TFTP.