Note for developing XOS service synchronizer

Posted on

In this section, I will record each step that I implemented the custom service oaibbu from example service and took vHHS as an example.

In this section, I will record each step that I implemented the custom service oaibbu from example service and took vHHS as an example.

Synchronizer Design Guidelines

Synchronizer and data model

  • Synchronizers act as the link between the data model and the functional half of the system.
  • the data model contains a clean, abstract and declarative representation of the system

Earlier releases (3.0 and before) required additional files (mostly Python code) to on-board a service, including a REST API, a TOSCA API, and an Admin GUI. These components are now auto-generated from the models rather than coded by hand.

CiaB development loop

  • Make changes to your service code and propagate them to your CiaB host
  • Teardown the existing XOS installation and clean up OpenStack to remove any leftover instances or networks
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
cd cord/build

# clean environment
make xos-teardown
make clean-openstack 
make clean-profile

#Optional: Teardown ONOS. 
#Sometimes we find it helpful to reinstall the onos-cord and onos-fabric containers,
#to ensure that all state is wiped clean from ONOS
make clean-onos

# build
make -j4 build 
make compute-node-refresh
  • (Optional)
  • Test and verify your changes.

Create a XOS service based on your service

Synchronizers are processes that run continuously, checking for changes to the Tenant model and applying them to the running Instances. In this case, we’re using TenantWithContainer, which creates a Virtual Machine Instance for us.

XOS Synchronizers are located in the xos/synchronizers directory in the XOS source tree. It’s customary to name the synchronizer directory with the same name as your service. The example code given below is in the XOS repo at xos/synchronizers/exampleservice.

Define a model (xproto)

  • Define a model
  • XPROTO is a combination of Google Protocol Buffers and some XOS- specific annotations
  • The XPROTO Header, which contains options that are global to the rest of the file
  • The Service model, which manages the service as a whole
  • The ServiceInstance model, which manages tenant-specific (per-service-instance) state

name specifies a name for your service
app_label configures the internal xos database application that is attached to these models

A Service model extends (inherits from) the XOS base Service model. At its head is a set of option declarations such as verbose_name, which specifies a human-readable name for the service model. Then follows a set of field definitions.

oaibbu.xproto

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
option name = "oaiBBU";
option app_label = "oaibbu";

message OAIBBUService (Service){
    option verbose_name = "OAI Virtual Base Band Unit";
}

message OAIBBUVendor (XOSBase){
    option verbose_name = "OAI Vitrual Base Band Unit Vendor";

    required string name = 1 [help_text = "vendor name", max_length = 32, null = False, db_index = False, blank = False]; 
    required manytoone image->Image:+ = 2 [help_text = "select image for this vendor", db_index = True, null = False, blank = False];
    required manytoone flavor->Flavor:+ = 3 [help_text = "select openstack flavor for vendor image", db_index = True, null = False, blank = False];
}

message OAIBBUTenant (TenantWithContainer){
    option verbose_name = "OAI Vitrual Base Band Unit Instance";

    optional manytoone oaibbu_vendor->OAIBBUVendor:vendor_tenants = 1 [help_text = "select vendor of choice, leave blank for slice default", db_index = True, null = True, blank = True]; 
}

Define a syncrhronizer

The Synchronizer has three parts:

  • The synchronizer python program,
  • model policies which enact changes on the data model
  • a playbook (typically Ansible) that configures the underlying system. The following describes how to construct these.

model-deps

Note: Earlier versions included a tool to track model dependencies, but today it is sufficient to create a file named model-deps with the contents: {}.

=> Create a file named model-deps with the contents: {}.

synchronizer python program

It loads and runs the default xos-synchronizer module in it’s own Docker container.
To configure this module, create a file named exampleservice_config.yaml, which specifies various configuration and logging options:

exampleservice-synchronizer.py:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
#!/usr/bin/env python

# Runs the standard XOS synchronizer

import importlib
import os
import sys
from xosconfig import Config

# your custom config should be placed here
config_file = os.path.abspath(os.path.dirname(os.path.realpath(__file__)) + '/exampleservice_config.yaml')
Config.init(config_file, 'synchronizer-config-schema.yaml')

synchronizer_path = os.path.join(os.path.dirname(
    os.path.realpath(__file__)), "../../synchronizers/new_base")
sys.path.append(synchronizer_path)
mod = importlib.import_module("xos-synchronizer")
mod.main()

oaibbu-synchronizer.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#!/usr/bin/env python

# This imports and runs ../../xos-observer.py

import importlib
import os
import sys

from xosconfig import Config
config_file = os.path.abspath(os.path.dirname(os.path.realpath(__file__)) + '/oaibbu_config.yaml')

Config.init(config_file, 'synchronizer-config-schema.yaml')
observer_path = os.path.join(os.path.dirname(os.path.realpath(__file__)), "../../synchronizers/new_base")

sys.path.append(observer_path)
mod = importlib.import_module("xos-synchronizer")
mod.main()

config yaml:

  • list the path of your synchronizer component

exampleservice_config.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
name: exampleservice
accessor:
  username: xosadmin@opencord.org
  password: "@/opt/xos/services/exampleservice/credentials/xosadmin@opencord.org"
required_models:
  - ExampleService
  - ExampleServiceInstance
  - ServiceDependency
  - ServiceMonitoringAgentInfo
dependency_graph: "/opt/xos/synchronizers/exampleservice/model-deps"
steps_dir: "/opt/xos/synchronizers/exampleservice/steps"
sys_dir: "/opt/xos/synchronizers/exampleservice/sys"
model_policies_dir: "/opt/xos/synchronizers/exampleservice/model_policies"
models_dir: "/opt/xos/synchronizers/exampleservice/models"

oaibbu_config.yaml

1
2
3
4
5
6
7
8
name: oaibbu-synchronizer
accessor:
  username: xosadmin@opencord.org
  password: "@/opt/xos/services/oaibbu/credentials/xosadmin@opencord.org"
dependency_graph: "/opt/xos/synchronizers/oaibbu/model-deps"
steps_dir: "/opt/xos/synchronizers/oaibbu/steps"
sys_dir: "/opt/xos/synchronizers/oaibbu/sys"
model_policies_dir: "/opt/xos/synchronizers/oaibbu/model_policies"

/steps/sync_oaibbutenant.py

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
import os
import sys
from django.db.models import Q, F
# Bring in some basic prerequities, Q to perform complex queries, and F to get the value of the model field. Also include the models created earlier, and SyncInstanceUsingAnsible which will run the Ansible playbook in the Instance VM.

#from services.oaibbu.models import OAIBBUService, OAIBBUTenant
from synchronizers.new_base.modelaccessor import *
from synchronizers.new_base.SyncInstanceUsingAnsible import SyncInstanceUsingAnsible

parentdir = os.path.join(os.path.dirname(__file__), "..")
sys.path.insert(0, parentdir)

class SyncOAIBBUTenant(SyncInstanceUsingAnsible):

    # Used by XOSObserver : sync_steps to determine dependencies.
    provides = [OAIBBUTenant]

    # The Tenant that is synchronized.
    observes = OAIBBUTenant

    requested_interval = 0
    
    # Name of the ansible playbook to run.
    template_name = "oaibbutenant_playbook.yaml"

    # Path to the SSH key used by Ansible.
    service_key_name = "/opt/xos/configurations/mcord/mcord_private_key"

    def __init__(self, *args, **kwargs):
        super(SyncOAIBBUTenant, self).__init__(*args, **kwargs)

    def get_network_id(self, network_name):
        network = Network.objects.filter(name=network_name).first()

        return network.id

    def get_instance_object(self, instance_id):
        instance = Instance.objects.filter(id=instance_id).first()

        return instance

    def get_information(self, o):
        fields = {}

        # not sure if this part need to modify, need to check further network setting

        collect_network = [
           {'name': 'HSS_PRIVATE_IP', 'net_name': 'vhss_network'}
        ]

        instance = self.get_instance_object(o.instance_id)

        for data in collect_network:
            network_id = self.get_network_id(data['net_name'])
            port = filter(lambda x: x.network_id == network_id, instance.ports.all())[0]
            fields[data['name']] = port.ip

        return fields

    def get_extra_attributes(self, o):
        fields = self.get_information(o)

        return fields

files in Synchronizer

oaibbu.xproto

  • Define a model
  • XPROTO is a combination of Google Protocol Buffers and some XOS- specific annotations
  • The XPROTO Header, which contains options that are global to the rest of the file
  • The Service model, which manages the service as a whole
  • The ServiceInstance model, which manages tenant-specific (per-service-instance) state

oaibbu-synchonizer.py

  • replace config yaml file

oaibbu_config.yaml

  • Modify synchonizer path

sync_oaibbutenant.py

  • setting of the tenant information