1 Introduction

The Open RAN (O-RAN) architecture and the O-RAN Alliance specifications have the potential to promote virtualized and disaggregated Radio Access Networks (RANs), connected through open interfaces and optimized by intelligent controllers.

Understanding the O-RAN architecture, interfaces and workflows is critical to the future of the telecommunications ecosystem. The evolution of O-RAN has many challenges due to its complexity, requiring experimental platforms to evaluate programmable and virtualized protocol stacks with the new open interfaces, in conjunction with RAN optimization within the 5G architecture and beyond.

Our team at UnB spent quite some time looking for different platforms for experimentation and presented our findings, (Solis et al. 2022), at the Brazilian Symposium of Telecommunications (SBRT2022).

Since I didn’t have any free time up till now and we didn’t have enough space in the paper, lets review things here, were we can put any number of images.

2 What do we need to get a working Open-RAN setup?

It isn’t an official architecture diagram, but the diagram of the reference implementation made by the O-RAN Software Community (O-RAN SC). Figure 1 shows the main components of an O-RAN setup, which are:

1. the 3GPP core: 4G and 5G are supported in the O-RAN specifications (some companies are apparently adapting older networks too)

2. the RAN components: the eNB or gNB, or its disaggregated counterparts (CU, DU, RU)

3. the O-RAN components: near-RT RIC, non-RT RIC and the rest of the SMO, plus SDK for xApp/rApp development

Let’s look at each components alternatives.

2.1 5G Core implementations

We found two open-source implementations that are frequently used. The first one is the famous OpenAirInterface-5G Core Network (OAI-5G CN), which has been in the works for years and is used by a lot of researchers and companies, in both private and public networks.

The second one is the free5gc, which started as a fork of a 4G Core, NextEPC, and continues to evolve. Also seems to be used on public networks.

There are quite a lot of proprietary cores, but we can’t be certain those are not forks of the open-source ones. To name a few companies and their cores:

• Ericsson - 5GC

• Nokia - 5GC

• Tecore – iCore

• Altran/Capgemini Engineering - Virtualized Next Generation 5G Core (ViNGC)

• Mavenir – Mavenir Converged Package Core (MAVCore)

• Huawei – SDM/PCC

After we have a 3GPP core, we can then connect it to our RAN.

2.2 RAN implementations

Most likely due to complexity, performance issues due to signal processing overhead, and standards conformance and interoperability testing, there seems to be less options in terms of RAN.

The eNB/gNB/CU/DU are available in open-source projects such as:

• OpenAirInterface

• SrsRAN

• Radisys – gNB DU (part of the reference O-RAN SC implementation)

The O-RAN RICs and interfaces are available in open-source projects such as:

• OAI - FlexRic

• Open Networking Foundation (ONF) - SD-RAN

Other proprietary RAN implementations available:

• Radisys – ConnectRAN

• Huawei – SingleRAN

• Ericsson – Cloud RAN

• Nokia – AirScale Cloud RAN

And it is at this point someone asks me why I included Huawei in this list. The answer is: they can probably adapt their solution easily to support Open-RAN. As the time goes by, it will get even easier since all the tooling required for testing and conformance will be in place. If O-RAN really works and is set as a requirement in new spectrum auctions, they will certainly do it.

2.3 Service Management and Orquestration (SMO)

This is where things really start to get big and ugly.

We have great SMO services such as Magma, which was started by Facebook, then moved to Telecom Infrastructure Project (TIP), and finally to Linux Foundation (stop moving the damned project around).

Too bad it isn’t comparible with O-RAN. Even then, it probably is one of the reasons O-RAN even became a thing, since TIP is also responsible for pushing O-RAN forward, along with Telcos, software/hardware/cloud partners and some governments.

There is also Open-Source MANO, which doesn’t really seem to have gained much traction on the O-RAN space. But maybe I’m badly informed.

Kubernetes on the other hand, seems to be the popular choice, since both Trirematics, OAI’s SMO solution, and the SD-RAN, ONF’s SMO solution, are based on Kubernetes.

All of the services are implemented as Virtualized Network Functions (VNFs), that are executed within containers. The containers are glued together by software defined networks (SDNs). SD-RAN uses ONF’s in-house SDN/NFV solution named ONOS, while the reference O-RAN implementation uses Linux Foundation’s Open Network Automation Platform (ONAP).

2.4 A blueprint

With all components at hand, we then need to sewn them together.

This is when the problems really start appearing for a novice.

There are quite a lot of components, all of them with different configuration settings, syntaxes and semantics. You have no idea what most of the settinds do and how to properly set them up, but you’re still supposed to figure it out...

You probably won’t, because available documentation is pretty bad. I admit my documentation isn’t up to par either, but that’s how things are.

We are supposed to believe Akraino blueprints work, but I haven’t got a single one to work. Seems like another dead project from the Linux’s Foundation.

ONF is the most friendly one, with a complete and working blueprint, which they named SD-RAN-in-a-box , or just Riab.

3 SD-RAN-in-a-box: a not so hidden gem

By far, SD-RAN-in-a-box is the most user-friendly Open-RAN solution.

You just run a few commands and it setups everything up using the magic of good old Bash, Ansible, Kubernetes and ONOS.

The user can select if they want to emulate the user data-plane. Which of course we want.

It will create VM’s running OAI that act like the eNB and UE, connected by a virtual channel.

More surprisingly, since OAI is already running anyways, the user can choose to use a Software Defined Radio (SDR) to get a real network going.

Of course the settings are meant for a lab setup, with very few UEs.

There is a popular saying that says that the devil is in the details. I’d say that the devil is in the implementations. While the O-RAN standard look fairly simple, the implementation complexity can vary wildly depending on the implementer choices.

When I found SD-RAN-in-a-box, I started looking at the settings to understand how exactly it worked, what was connected to what, which standard defined the different interfaces.

Ended up writing a MermaidJS diagram. But since is too big to show, here is the link.

3.1 Stop the talking and show us some code!

3.1.1 Installation

Well, if you follow the manual, you will get there.

At the time I tested things, only Ubuntu 18.04 was supported. Later versions will fail with a nf_conntrack issue, so you may need to run everything in a VM.

Running things in a VM can also create problems, but more on that later.

# Check if you have virtualization support
sudo modprobe kvm
lsmod | grep kvm

# Check if you have AVX2 (a flavor of SIMD for floating point operations)
egrep "(avx2)" /proc/cpuinfo

# Check if you have virtualization instructions
egrep "(svm|vmx)" /proc/cpuinfo

# Don't even try to continue if one of the
# previous checks didn't return something

# Clone Riab
git clone "https://github.com/onosproject/sdran-in-a-box.git"

# Install Vagrant (VM manager) and the libvirt extension
apt install vagrant qemu-kvm libvirt-daemon-system

# Create a new Ubuntu 18.04 VM
vagrant init "generic/ubuntu1804" # bionic64

# Edit the vMemory and vCpus for the machine
# If you use the emulated data-plane, use >22GB of RAM
#
#  # -*- mode: ruby -*-
#  # vi: set ft=ruby :
#  Vagrant.configure("2") do |config|
#    config.vm.box = "generic/ubuntu1804" #
#    config.vm.synced_folder "./", "/vagrant" # sync the local directory to the VM's /vagrant directory
#    config.vm.provider "libvirt" do |vb|
#      #vb.gui = true
#      vb.cpus = 14
#      vb.memory = "21384"
#    end
#    config.ssh.insert_key=false # passwordless login
#  end

# Start the VM
vagrant up

# Log into the VM
vagrant ssh

# Update packages
sudo apt-get update
sudo apt-get upgrade

# Lock the kernel version. It might not be a problem on every cloud,
# but it was on mine.
sudo apt-mark hold linux-generic linux-image-generic linux-headers-generic

# Install extra kernel modules (nf_conntrack)
sudo apt install linux-modules-extra-*-generic
sudo modprobe nf_conntrack
sudo apt-get install make net-tools

# Nagivate to the cloned SD-RAN-in-a-box directory
cd /vagrant/sdran-in-a-box

Now, how do we do anything?

3.1.2 Basic Helm/Kubernetes background information

I’m definitely not an specialist in SD-RAN, Kubernetes or ONOS, so it took some experimentation to get anything done.

Helm charts are used as an encapsulation for kubernetes applications along which can be configured using templates for the different settings. It is just as bad as it sounds.

Helm charts can have subdirectories, which are considered subdependencies and are installed recursively.

A helm chart can be installed from a local repository (so you can work on a local clone)

helm install oai-enb-du sdran-in-a-box/workspace/sdran-helm-charts/oai-enb-du

The templated settings allow for the substitution of values via the command line or via yaml files (much preferred).

echo '{
 config.oai-enb-du.mode: "nfapi",
 config.oai-enb-du.radio.eutra_band: 42,
 config.oai-enb-du.radio.rbs: 100,
 config.oai-enb-du.radio.tx_gain: 90,
 config.oai-enb-du.radio.rx_gain: 125,
 config.oai-enb-du.radio.nb_antennas_rx: 2,
 config.oai-enb-du.radio.nb_antennas_tx: 2
}' > values.yaml
helm install -f values.yaml oai-enb-du sdran-in-a-box/workspace/sdran-helm-charts/oai-enb-du

In this case, one of the templates in sdran-in-a-box/workspace/sdran-helm-charts/oai-enb-du/templates/*.tpl will have their field values replaced with those in the yaml settings file.

For example, in the values file, we set "oai-enb-du.mode" to "nfapi", which will be processed by the following template:

{{ $duMode := index .Values "config" "oai-enb-du" "mode" }}

and transformed into the following line

duMode := "nfapi"

The helm charts used by the SD-RAN-in-a-box are installed by the makefiles em sdran-in-a-box/mk, using the following commands

helm upgrade --install $(HELM_ARGS) \
     --namespace $(RIAB_NAMESPACE) \
     --values $(HELM_VALUES) \
      ...

The values of HELM_VALUES are read from the default settings file sdran-in-a-box/sdran-in-a-box-values.yaml

To install from your own sdran-helm-charts fork, replace the repository link in sdran-in-a-box/Makefile. To do that, search for

            $(ONOS_GITHUB_URL)/sdran-helm-charts

The clone is only done if the directory does not exist, so you can modify it directly, then commit and push changes later. Just remember to do that before running "make clean-all", otherwise the changes will be lost.

3.1.3 Installing RIAB with data-plane emulation

If you do not have enough RAM, the VM or your computer will crash due to OOM.

# Install RIAB with OAI emulation
make OPT=oai

# Execute tests to check if the data-plane is actually working
# Tests are located in sdran-in-a-box/mk/test-cases
# E2E tests using pings in the user data-plane
make test-user-plane

# E2E tests using iperf in the user data-plane
make test-rsm-dataplane

Everything should be working now. You must be thinking on passing quite some traffic through that emulated network right now, but it isn’t that simple.

At least on my computer, due to using a normal kernel (not the low-latency one), and due to low CPU clocks (1st gen Ryzen 1700), the synchronization between the UEs and the eNB is lost quite frequently. I remember looking at the ONF Slack channel for answers, but it was still being investigated.

Due to that issue, I had to take everything down and up again, which took quite a lot of time. I tested the commands suggested by Woojoong Kim (I believe he is the lead developer for Riab), but didn’t really work for my specific case.

# I only really use clean-all, because it always works. Beware: it is super slow and deletes everything

# It deletes ONOS RIC services and RAN-Simulator but Kubernetes is still running
make reset-test

# Deletes Kubernetes environment; Eventually, all ONOS RIC and
# RAN-Simulator are terminated; The Helm chart directory is not deleted

make clean

# Deletes everything, including the Kubernetes environment,
# all componentes/PODs which RiaB deployed, and even the Helm chart directory

make clean-all

3.1.4 Other useful commands

Other useful commands I’ve learned while testing these.

# Lists all kubernetes namespaces and services
kubectl get po --all-namespaces

# Lists kubernetes services in the riab namespace
kubectl -n riab get pods

# Lists the status of sd-ran
helm -n riab status sd-ran

# Lists all sd-ran settings
helm -n riab get all sd-ran

# Check the status of different pods running SD-RAN services
watch kubectl -n riab get pods

# Comand to get the kubernetes pod shell to access binaries
kubectl -n riab exec -it $(kubectl -n riab get pods -l type=cli -o name) -- /bin/sh

3.2 That’s enough code. Now show us something interesting

I wish I had some banging xApps up and running... Maybe getting to work in real networks with people from Mavenir, Parallel, Radisys or Nokia...

But in my excitement about this, I ended up talking too much and getting a ton of people wanting to get involved, which ended up drastically slowing down the process.

A few partners offered us their clouds, which I’m very grateful for, but they were quite up to the task of running real-time services. Some failed to setup Riab due to missing AVX2 instructions, others due to disabled smx/vmx instructions for security reasons, others had very slow drives which timed out some of the services and prevented Riab from getting setup. Even if they worked, I’m not sure their clocks and number of cores would be able to keep up with the emulated network.

I also had my fair of issues trying to understand Helm packaging.

The best I managed to do was adapt our in-house traffic injection tool, Eros, and use it to inject some network traffic and see how things behaved. We produced the Figure 2, showing the behavior of the UE downlink when streaming a MPEG DASH-like traffic from the remote serve.

4 Next steps

For now, I’ve stopped playing with SD-RAN and started focusing on my own O-RAN implementation for the ns-3 simulator.

It is kind of sad that I decided to change course too late and missed the journal I had in mind.

Hopefully it will make it to the Brazilian Symposium of Computer Networks and Distributed Systems (SBRC), which funnily enough will be hosted by my alma mater, in the next year.

The source code for my O-RAN module should be published in a month or so, before I finish the paper for the tools section of the SBRC. It still needs some refactoring to look decent.

References