Configuring NDFC for Multisite Fabric
The first time opening up NDFC service after installation, you will be asked to decide which mode should NDFC operate in. For EVPN-VXLAN deployment, choose Fabric Controller to continue.
NDFC offers a heap of features to address different requirements and scenarios. In this lab, we are concerned with building the EVPN-VXLAN with automation, therefore only choose Fabric Builder and Apply. Note that if you haven't configured two (2) persistent IP addresses, you will be prompted with an error message. Please go back to Part 2 of this series to find out how to configure persistent IP addresses in ND.
Upon finishing the setup, you will see a warning message asking to reload the page to reflect new service. Then a prompt will pop up to ask for setting the credentials.
That will bring you to the Settings > LAN Credentials Management menu. This credentials configuration will be used by the NDFC to reach network switches. Hence, click set to configure the admin and passwords configured in the switch and also select the robot checker for confirming this credential will be used as a robot operation.
Go to LAN > Fabrics menu. This is the place to start building the network fabric.
The NDFC Fabric Hierarchy
In NDFC, fabric networks are managed through a logical container named as fabric. Each fabric corresponds to a specific network domain: whether it is a Cisco EVPN-VXLAN comprised of Nexus 9000 switches, or a traditional vPC network comprised of a mixed of Nexus 3000/7000 switches. These fabrics form the underlying skeleton of the data center networks managed by NDFC.
NDFC manages the fabric by storing a set of predefined templates and parameters according to the fabric characteristics. Configurations are then calculated and generated automatically in NDFC, and then be pushed to the switches - achieving the goal of automation. With NDFC, all configuration tasks are supposed to be managed and pushed from NDFC and there is no such need to configure via CLI.
Note:
NDFC has the capability to spot configuration differences even someone can still configure extra commands through CLI. In such scenario, the next time NDFC sync up configuration from the switches, NDFC will highlight those extra commands and turn the switch into config-out-of-sync status.
NDFC takes precedence over CLI commands and at the time of writing, there is no way to change this behavior meaning that NDFC will wipe extra commands that do NOT exist in its configuration.
For this reason, it is NOT recommended to manually add commands through CLI once NDFC becomes the source of configuration.
NDFC provides a Lego-like building block policy template, which can be highly customized. I will walkthrough those steps in a later stage, to cover scenarios where the predefined policy templates in NDFC do not meet our needs.
In EVPN-VXLAN multisite deployment, fabrics are managed per-site. Therefore, there will be two (2) fabrics, each representing the site network. In our case, each fabric consists of two (2) N9000v switches.
Multisite is also another form of fabric. However, it is a fabric type that contains the site fabrics. Therefore, in our lab, we are going to build a hierarchy of a multisite fabric having two child EVPN-VXLAN fabrics.
Building the Site Fabric and Adding Switches
Go back to the LAN > Fabrics menu, first create a new fabric using the Actions button.
Choose a name for the fabric, in this case, name it Site-1 for the lab.
Then a menu of different types of Fabric is prompted. Choose Data Center VXLAN EVPN.
Now here is the hard (it is not that hard, but tedious and require a bit of time for planning) part, is specifying all of the parameters presented and Save it. Remember NDFC generate configurations based on the pre-defined templates and parameters? This is where we are going to configure all of the required parameters for the fabric.
For the sake of time, I will not capture all of the screenshots but will provide the necessary parameters to build the lab as a reference. Items not listed can be assumed to take default values from NDFC.
Site-1 / Site-2
Section | Item | Item Value |
General Parameters | BGP ASN | 65001 / 65002 |
General Parameters | Anycast Gateway MAC | 2020.0000.00aa |
Replication | Replication Mode | Ingress |
vPC | vPC advertise-pip | Checked |
Advanced | Overlay-Mode | CLI |
Advanced | Site Id | 65001 / 65002 |
Advanced | Greenfield Cleanup | Enable |
Resources | Underlay Routing loopback | 10.2.0.0/22 / 20.2.0.0/22 |
Resources | Underlay VTEP loopback | 10.3.0.0/22 / 20.3.0.0/22 |
Resources | Underlay Subnet range | 10.4.0.0/16 / 20.4.0.0/16 |
Resources | VRF Lite Deployment | Manual |
Resources | VRF Lite subnet range and mask | 10.33.0.0/24 / 20.33.0.0/24 |
Configuration Backup | Hourly Fabric Backup | Checked |
Next is the multi-site fabric. Go to the Fabric menu and create the multi-site fabric.
Multisite-Fabric
Section | Item | Item Value |
DCI | Deployment Method | Direct_To_BGWS |
DCI | Multi-Site Underlay IFC Auto Deployment Flag | Checked |
Resources | Multisite VTEP VIP loopback | 10.100.0.0/24 |
Resources | DCI Subnet IP Range | 10.100.1.0/24 |
Note that the use of multi-site auto deployment. Without that option, you will need to build the multisite fabric manually instead of giving the tasks to NDFC.
Next step, we are going to move the site fabrics into multisite fabric.
Double click the multisite fabric name, and enter the Fabric Overview menu.
The Move Fabric into MSD option is under the Actions menu.
Select Site-1 and Site-2 fabric one by one.
Once the move is completed, the fabric overview should look like this. Following steps we will add the switches to the site fabric.
Navigate back to the LAN Fabric menu, double click to Site name (in this case, Site-1) will direct to the Fabric Overview menu.
Choose the Add Switches action under the Actions menu.
Configure the required IP and credentials, then click Discover Switches. Be sure to uncheck the Preserve Config as we are building a greenfield fabric.
All manageable switches will be presented.
Select the two switches in Site-1 and click Add Switches.
It will take a while for NDFC to finish the process. Once the switch status shows "Switch Added", you can safely close the discovery menu.
Now, going back to the Fabric Overview screen.
Select both switches, and choose Set Role action under Actions menu.
All switches in this lab are border gateways. Therefore, choose Border Gateway in the Select Role screen.
Once role is configured, an action to recalculate the topology and deploy configuration is required. You may experience switches rebooting but rest assured that the switches will come back online.
When all status shows green, go to the Actions menu at the top, and choose Recalculate and Deploy.
Note that this action triggers NDFC to recalculate based on the topology and configured parameters and come up with a proposed configuration. Therefore, it is recommended this action be used in caution and best to be executed only when NDFC asks for.
Notice the 321 Lines of config pending to push to the switches. You can also verify these configurations before deploying them by clicking the 321 Lines.
For now, it is safe to click the Deploy All button at the bottom right-hand corner.
This screen shows a successful deployment of the configurations.
Now, there is one more step required: which is setting up vPC pairing.
Go back to the Fabric Overview menu, select ONE switch and go to the Actions menu.
Choose the vPC Pairing action.
There is only one vPC for pairing in this lab, therefore select that peer and click Save.
Note that NDFC discovers vPC peer through CDP. It is a prerequisite for connecting links for vPC pairing to be automated. NDFC uses ALL available links between the peers as vPC port channel. Therefore, if a direct link keepalive is desired, a change to the default vPC keepalive link, and an extra step to explicitly configure the back-to-back keepalive link is required. However, we are not going to cover the step in this lab, as using the default management interface as keepalive is good enough.
After vPC pairing is saved, a warning message is prompted, for executing the Recalculate and Deploy action. Therefore, go ahead and hit that action one more time.
Notice that this time there are 75 Lines of config to be pushed. Simply Deploy All to finish the vPC pairing configurations.
Congratulations!! You have completed one DC site build. Before we move on to populate the multisite configuration, please take time to complete DC Site-2 using similar steps indicate in this part 3 guide.
Part 4 we will discuss populating multisite configurations.
- End of Part 3 -
Go back to Part 2
Gary Wong@Geelong, Australia. 2023.
Comments