Automate network topologies with CML, Ansible & Docker.
Intro
After many hours of reading Ansible books, looking at codes and testing, I'm continuing with the network project I started on my previous article.
On this post you will find how to create day-0 network topologies from scratch using Cisco Modelling Labs, Ansible and Docker.
Why Cisco Modelling Labs (CML)?, first and foremost, it is free (just reserve a lab). It comes with a default selection of Cisco images and won't put a strain on your local machine's resources. There is no need for installation or configuration. Furthermore, it is not limited to Cisco devices alone.
Prerequisites
To create the content of this post, you need:
- CML instance, either personal or from Devnet sandbox.
- If you don't have a CML instance:
- Reserve a lab in the Cisco Devnet Sandbox. Is free.
- Cisco Anyconnect in your laptop. You need the VPN client to reach the network where the devices are reachable.
- A mail will be sent with the details to connect or click in "output" in the sandbox page.
- Docker on the client machine.
- Git, to clone the repo.
TL;DR
To get your lab, just do.
- Clone the release v1.0.0 from Github.
- See below if you are not sure which command use to clone an specifc tag.
- The code in github is a living project, the main branch could have a different code than the one presented on the post, therefore all links presented in this article point to release
v1.0.0
- Start the container from the root directory of the project.
docker-compose -f ./docker/cml.docker-compose.yml up -d
- Create the lab from inside the container.
ansible-playbook cisco.cml.build -e startup='host'
CML Lab Created
With that, you will be ready to go and have a lab in CML. If you want to understand how this works, keep reading.
Why?
As I'm experimenting network automation, I need a baseline to start, this post automates this baseline so I can focus on trying new ideas. The lab created, contains the day-0 config to start.
High Level Overview
The benefits of using Docker, Ansible and CML are:
- Quickly create and destroy labs and containers.
- Ensure consistency in network topology creation.
- Write your network lab as code.
Not everything is perfect, there are a couple of manual pieces that have to be done. I would say this is more a limitation at the time of writing the post.
Prepare the Project
Go and grab the code at the GitHub repo.
git clone -b v1.0.0 --single-branch https://github.com/jillesca/network-automation.git
Project Structure
The project is divided in two main directories.
Ansible- This directory contains all files related to ansible, such as inventory, playbooks, variables, etc.
Docker- This directory has all files related to docker, such as dockerfiles, docker-compose files, environment variables.
╰─ tree -FL 2
./
├── LICENSE
├── README.md
├── ansible/
│ ├── ansible.cfg
│ ├── files/
│ ├── host_vars/
│ ├── inventory/
│ ├── playbooks/
│ └── templates/
└── docker/
├── ansible.docker-compose.yml
├── cml.docker-compose.yml
└── image/
8 directories, 5 files
Ansible image
Now is time to work with Ansible in docker, there are two options:
- You can build your own docker image.
- Or you can reuse the image I've created.
The main consideration is the version of CML you are using.
For this post, CML was running virl 2.2.2+build52. If your CML is running the same version, you can skip the building part.
If you don't know the virl version of your CML, you can do from your terminal.
curl https://{CML_IP}/api/v0/system_information -k
For example:
╰─ curl https://10.10.20.161/api/v0/system_information -k
{
"version": "2.2.2+build52",
"ready": true
}%
Build the Ansible container
Follow this section if you want to build your own image, otherwise you can use the image built from Docker Hub.
We start by reusing the Ansible image built in the previous post. This image will serve as base to add the additional libraries needed to manage CML.
To manage CML on Ansible, we use cisco.cml collection. which is available from Galaxy Collection. This library uses virl2-client under the hood.
If you want to know the details of this collection, check out the project in Github.
On the dockerfile we defined what is commented above; the image to use and which libraries to installed on pip and Ansible.
FROM jillesca/ansible:version1.1 as cml
RUN pip install virl2-client==2.2.1.post2 \
&& ansible-galaxy collection install cisco.cml
See virl2-client depends on the CML for more details.
To build the docker container use the command below from the root directory of the project. Otherwise adjust the path of the dockerfile.
docker build --target cml --file docker/image/ansible.dockerfile .
Add your environment variables
The project uses environment variables. This is a good way to manage secrets or special values you don't want to store in git.
The collection cisco.cml, requires some variables. We use env vars to define these values so your credentials are out of the repo.
The default values from the sandbox (except the lab name) are used:
CML_USERNAME=developer
CML_PASSWORD=C1sco12345
CML_HOST=10.10.20.161
CML_LAB="ansible_base_lab"
CML_VERIFY_CERT=false
In my case, the .env file is under the docker directory in the root project. Keep in mind the location where you place this file.
╰─ ls -lsa docker
total 24
0 drwxr-xr-x 6 jillesca staff 192 Dec 4 11:17 .
0 drwxr-xr-x 10 jillesca staff 320 Dec 1 12:59 ..
8 -rw-r--r-- 1 jillesca staff 117 Nov 1 15:43 .env
8 -rw-r--r-- 1 jillesca staff 851 Dec 4 13:37 ansible.docker-compose.yml
8 -rw-r--r-- 1 jillesca staff 420 Dec 4 13:43 cml.docker-compose.yml
0 drwxr-xr-x 4 jillesca staff 128 Dec 4 11:54 image
Start the container
To work locally with the container, docker compose is used. Below you can see the definition.
╰─ cat docker/cml.docker-compose.yml
version: "3.8"
services:
cml:
image: jillesca/ansible:cml1.1
container_name: cml
tty: true
volumes:
- ./../ansible:/home
environment:
CML_USERNAME: ${CML_USERNAME}
CML_PASSWORD: ${CML_PASSWORD}
CML_HOST: ${CML_HOST}
CML_LAB: ${CML_LAB}
CML_VERIFY_CERT: ${CML_VERIFY_CERT}
From the compose file, two sections are important:
volumes:- We say to docker, "copy everything that is under the
ansibledir all the time. - This allows to do changes on the code locally, and these changes will be replicated automatically on the container.
- We say to docker, "copy everything that is under the
environment:- This section injects the variables from the
.envfile to the container.
- This section injects the variables from the
Run the container from root directory:
docker-compose -f ./docker/cml.docker-compose.yml up -d
╰─ docker-compose -f ./docker/cml.docker-compose.yml up -d
[+] Running 1/1
⠿ Container cml Started 0.4s
╰─ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5af70918f7b8 jillesca/ansible:cml1.1 "python3" 3 seconds ago Up 2 seconds cml
Now you can connect to the container shell by using docker exec -it cml /bin/sh. If you use vscode, there are also extensions that will help you with some clicks:
╰─ docker exec -it cml /bin/sh
/home # ls
ansible.cfg files host_vars inventory playbooks templates
/home #
Verification of env vars
This is a good moment to verify your env vars are present on the container, otherwise the lab will fail. From inside the container do env | grep -i cml:
/home # env | grep -i cml
CML_USERNAME=developer
CML_PASSWORD=C1sco12345
CML_HOST=10.10.20.161
CML_VERIFY_CERT=false
CML_LAB=ansible_base_lab
/home #
Prepare the Bastion/Jump Host
For this lab I used a Jump Host as the connectivity between the network devices created in CML and the ansible container was not reliable. Even though IP connectivity was there, from time to time I had timeouts, so the best solution was to use a VM the sandbox provides.
VM Sandbox Details
All these steps are done from inside the container -> docker exec -it cml /bin/sh
Create the ssh keys
ssh-keygen -f /home/.ssh/cml -t ed25519 -C 'CML_Automation_key' -N ''
Create an authorized_keys file with the public ssh key created.
cp /home/.ssh/cml.pub /home/.ssh/authorized_keys
Copy the keys to the sandbox VM. This is the VM that will work as bastion/jump host. At the time of writing the VM on the Cisco sandbox uses the IP 10.10.20.50. Replace this IP with the address of the host you want to use as bastion.
Enter the password when prompted. At the time of writing the password was C1sco12345
scp /home/.ssh/authorized_keys developer@10.10.20.50:/home/developer/.ssh
Execute your ssh agent.
eval $(ssh-agent -s)
Add your keys to your agent
ssh-add /home/.ssh/cml
Test ssh is using your keys. The bastion/jump host should not ask for your password.
Below you can see an output of this process.
/home # ssh-keygen -f /home/.ssh/cml -t ed25519 -C 'CML_Automation_key' -N ''
Generating public/private ed25519 key pair.
Your identification has been saved in /home/.ssh/cml
Your public key has been saved in /home/.ssh/cml.pub
The key fingerprint is:
SHA256:4z4psi8nJ0hqnwXd05R5DuVeqPUcCu7YCpZnjUHjgwQ CML_Automation_key
The key's randomart image is:
+--[ED25519 256]--+
| E . |
| . = . |
| . o * = o |
| o = = O = . |
| . o *S+ + o |
| . . ..X. |
| o . = =.= |
|... O B.+ |
|. .o.X.o.. |
+----[SHA256]-----+
/home # cp /home/.ssh/cml.pub /home/.ssh/authorized_keys
/home # scp /home/.ssh/authorized_keys developer@10.10.20.50:/home/developer/.ssh
The authenticity of host '10.10.20.50 (10.10.20.50)' can't be established.
ED25519 key fingerprint is SHA256:MbXlsdtKy1J+Tj67hyVRPz5URQS/6eT2ILljoG1ihqA.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '10.10.20.50' (ED25519) to the list of known hosts.
developer@10.10.20.50's password:
authorized_keys 100% 100 0.5KB/s 00:00
/home # eval $(ssh-agent -s)
Agent pid 85
/home # ssh-add /home/.ssh/cml
Identity added: /home/.ssh/cml (CML_Automation_key)
/home # ssh developer@10.10.20.50
(py3venv) [developer@devbox ~]$
Create the lab
All these steps are done from inside the container.
With all pre-requisites done, is time to fire up the lab. To create the lab on CML, simply do:
ansible-playbook cisco.cml.build -e startup='host'
Below you can see the full output of the lab creation.
/home # ansible-playbook cisco.cml.build -e startup='host'
SSL Verification disabled
[WARNING]: running playbook inside collection cisco.cml
PLAY [Build the topology] ***************************************************************
TASK [Check for the lab file] ***********************************************************
ok: [localhost]
TASK [assert] ***************************************************************************
ok: [localhost] => {
"changed": false,
"msg": "All assertions passed"
}
TASK [Create the lab] *******************************************************************
[WARNING]: Both option username and its alias user are set.
changed: [localhost]
TASK [Check to see if the Lab is there] *************************************************
skipping: [localhost]
TASK [Refresh Inventory] ****************************************************************
SSL Verification disabled
PLAY [Start Individual Nodes] ***********************************************************
TASK [Check for the cml_config_file] ****************************************************
skipping: [bridge-to-sandbox]
skipping: [sandbox-backend]
ok: [spine2-20.12 -> localhost]
ok: [leaf1-20.21 -> localhost]
ok: [leaf3-20.23 -> localhost]
ok: [spine1-20.11 -> localhost]
ok: [leaf2-20.22 -> localhost]
ok: [host2-20.32 -> localhost]
ok: [host1-20.31 -> localhost]
ok: [host3-20.33 -> localhost]
TASK [Read in cml_config_file] **********************************************************
skipping: [bridge-to-sandbox]
skipping: [sandbox-backend]
ok: [spine1-20.11]
ok: [spine2-20.12]
ok: [leaf1-20.21]
ok: [leaf2-20.22]
ok: [leaf3-20.23]
ok: [host1-20.31]
ok: [host2-20.32]
ok: [host3-20.33]
TASK [Start Individual Nodes] ***********************************************************
changed: [sandbox-backend -> localhost]
changed: [bridge-to-sandbox -> localhost]
changed: [leaf1-20.21 -> localhost]
changed: [spine1-20.11 -> localhost]
changed: [spine2-20.12 -> localhost]
changed: [leaf2-20.22 -> localhost]
changed: [host2-20.32 -> localhost]
changed: [host1-20.31 -> localhost]
changed: [leaf3-20.23 -> localhost]
changed: [host3-20.33 -> localhost]
PLAY [Wait for Topology to BOOT] ********************************************************
TASK [Check to see if all hosts are BOOTED] *********************************************
skipping: [localhost]
PLAY RECAP ******************************************************************************
bridge-to-sandbox : ok=1 changed=1 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0
host1-20.31 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
host2-20.32 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
host3-20.33 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
leaf1-20.21 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
leaf2-20.22 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
leaf3-20.23 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
localhost : ok=3 changed=1 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0
sandbox-backend : ok=1 changed=1 unreachable=0 failed=0 skipped=2 rescued=0 ignored=0
spine1-20.11 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
spine2-20.12 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
/home #
From CML you can watch the progress of the lab created.
CML Lab Creation
If you examine the devices in the lab, you will see all the devices are running with their day-0 config. In our case, this day-0 config, is for the management of the device.
How the lab works?
If you are like me, you may wondering how the devices got their configs and how are the moving pieces inside Ansible. I consider this was the most difficult part for me to get my head around when starting with Ansible.
Ansible Inventory
The Ansible inventory is divided in three main components. Here is how the inventory looks like.
cml.yml
Defines the plugin and tags CML will use.
plugin: cisco.cml.cml_inventory
group_tags: clients, network
network.yml
Defines the network devices. Two main groups are defined network and iosxe_routers
all:
children:
network:
children:
iosxe_routers:
hosts:
spine1-20.11:
ansible_host: 10.10.20.11
spine2-20.12:
ansible_host: 10.10.20.12
leaf1-20.21:
ansible_host: 10.10.20.21
leaf2-20.22:
ansible_host: 10.10.20.22
leaf3-20.23:
ansible_host: 10.10.20.23
system.yml
Defines the clients. Two main groups are defined clients and linux_hosts
all:
children:
clients:
children:
linux_hosts:
hosts:
host1-20.31:
ansible_host: 10.10.20.31
host2-20.32:
ansible_host: 10.10.20.32
host3-20.33:
ansible_host: 10.10.20.33
Variable parsing in Ansible
Remember the lab structure presented before, now is time to see the Ansible part. Let's see the relevant parts from another perspective.
When you execute ansible-playbook cisco.cml.build -e startup='host' the following happens:
Variable Parsing Flow
- /inventory/group_vars/all/all.yml is parsed:
- Global variables are set, like ansible credentials, GW.
- ssh config for bastion is defined here.
- *.yml files under
/inventoryare parsed (this is not default behaviour, see the fileansible.cfgon the repo). - /inventory/cml.yml
- Gets variables from
/hosts_vars/cml.yml- The env vars for cml are parsed.
- Gets variable defined from
/inventory/group_vars/all/cml.ymlcml_lab_filetellscisco.cmlwhich file is use to create the topology in CML.- If you want to use your own lab, this is the file you need to point to.
- The hosts defined on
network.ymlandsystem.ymlmatch the hosts on this file.
- Gets variables from
- network.yml
- Gets variables from
/inventory/group_vars/iosxe_routers/iosxe_routers.ymlmgmt_interfacesets which interface will be used for managment.cml_config_filetellscisco.cmlwhich file will be used asday-0config.- This points to
ansible/templates/iosxe/bootstrap.j2 - The
day-0config is in jinja and takes several variables.
- This points to
- Gets variables from
- system.yml
- Gets variables from
/inventory/group_vars/linux_hosts/linux_hosts.ymlcml_config_filetellscisco.cmlwhich file will be used asday-0config.- This points to
/templates/linux_hosts/bootstrap.j2 - The
day-0config is in jinja and takes several variables.
- This points to
- Gets variables from
I would say this is the messy part, which at the beginning could be complex, but once you understand the flow it, becomes easier.
Once all variables are parsed and ready, cisco.cml proceeds to build the lab with its playbook cisco.cml.build.
Test the lab
For testing the lab we just created, we will be using cisco.ios.ios_facts The playbook below will print all facts gathered by this collection.
The goal is to be able to connect to the devices created with Ansible. And on this post, that will be through the Bastion/Jump host.
- name: Show cisco facts
hosts: network
connection: ansible.builtin.network_cli
gather_facts: no
tasks:
- name: Gather cisco facts
cisco.ios.ios_facts:
gather_subset: all
- name: Print cisco facts
ansible.builtin.debug:
var: ansible_facts
To run the playbook, simply do:
ansible-playbook playbooks/test_lab.yml
For example:
/home # ansible-playbook playbooks/test_lab.yml
SSL Verification disabled
PLAY [Show cisco facts] *****************************************************************
TASK [Gather cisco facts] ***************************************************************
ok: [spine1-20.11]
ok: [spine2-20.12]
ok: [leaf3-20.23]
ok: [leaf2-20.22]
ok: [leaf1-20.21]
TASK [Print cisco facts] ****************************************************************
ok: [leaf1-20.21] => {
. . . . .
ok: [spine1-20.11] => {
"ansible_facts": {
"net_all_ipv4_addresses": [
"10.10.20.11"
],
"net_all_ipv6_addresses": [],
"net_api": "cliconf",
"net_config": "Building configuration......."
"net_filesystems": [
"bootflash:"
],
"net_filesystems_info": {
"bootflash:": {
"spacefree_kb": 5308528.0,
"spacetotal_kb": 6139200.0
}
},
"net_gather_network_resources": [],
"net_gather_subset": [
"interfaces",
"config",
"hardware",
"default"
],
"net_hostname": "spine1-20.11",
"net_image": "bootflash:packages.conf",
"net_iostype": "IOS-XE",
"net_memfree_mb": 858103.16015625,
"net_memtotal_mb": 1105215.80859375,
"net_model": "CSR1000V",
"net_neighbors": {},
"net_python_version": "3.11.0",
"net_serialnum": "9TZF7KV77J9",
"net_system": "ios",
"net_version": "17.03.02",
"network_resources": {}
}
}
PLAY RECAP ******************************************************************************
leaf1-20.21 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
leaf2-20.22 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
leaf3-20.23 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
spine1-20.11 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
spine2-20.12 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
/home #
Delete the lab
To delete the lab on CML, use the playbook cisco.cml.clean, which uses the lab name defined from the env var:
ansible-playbook cisco.cml.clean
For example:
/home # ansible-playbook cisco.cml.clean
SSL Verification disabled
PLAY [localhost] ************************************************************************
TASK [Stop the lab] *********************************************************************
[WARNING]: Both option username and its alias user are set.
changed: [localhost]
TASK [Wipe the lab] *********************************************************************
changed: [localhost]
TASK [Erase the lab] ********************************************************************
changed: [localhost]
PLAY RECAP ******************************************************************************
localhost : ok=3 changed=3 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
/home #
If you want to specify a lab to delete, use -e cml_lab=LAB_NAME. And if you have spaces in your lab name use double quotations, for example: ' inside ""
ansible-playbook cisco.cml.clean -e cml_lab="'Small NXOS/IOSXE Network'"
From CML you can watch the progress of the lab deleted.
There are more options on the playbook cisco.cml.clean take a look
How can I add my own lab?
If you follow the structure layout in the project, the best way to create your own lab is the following:
- Create a baseline topology:
- Create a lab in CML by hand or use one of their examples.
- Export the lab from CML in
yamlformat.- In the GUI; Bottom bar > Simulate > Download Lab
- If you want to manage day-0 config, remove any configuration for the devices.
- Add the extension
.j2to the yaml file to make it a jinja file and place it at/files/topologies/base-setup/. For example. - You could have many labs stored there, but the lab that is picked up, is controlled by the variable cml_lab_file.
- Don't forget to adjust the Ansible inventory based on the hosts you added on the CML file.
- Day-0 config
- Day-0 config is managed based on the host group.
- the variable
cml_config_filetells CML which file has the day-0 config for a device. - Place you day-0 config on the file you defined in
cml_config_file. This file has to a jinja file.
If you follow these considerations, you should be able to add as many labs as you want.
If you want to create several labs, remember to update the corresponde variables.
Conclusion
This post took way longer that I expected when I had the idea. What I thought it could be short article, became way larger than what I was thinking.
But when trying to figure out, how to automate a lab in CML with Ansible, not many resources when into the detail about how it works, just assumed you already knew, that's why I wanted to add relevant details and try to be clear.
Additionally, once the lab process is automated, you don't need to repeat all the steps here, eventually becomes an easy process. But when there is an issue, you will be happy there is a detailed article explaining how it works and how you can use it.
Credits
I can't take credit for the Ansible structure of the project and how it works. When learning about cisco.cml, I found this awesome organization model-driven-devops. which is part of the book Model-Driven DevOps.
Take a look at the GitHub organization and book to learn more about how to manage networks using single source of truth and more interesting for me, working with structured data.
