Federating your testbed with Slices through the AM API

This document explains how to federate your testbed with Slices through the AM API. For this we will use jFed.

First of all, note that this document is about ading an AM, an Aggregate Manager (sometimes also know as CM, “Component Manager”). If you want to setup and use your own user and slice authority (SA and/or MA), there are some extra complications not mentioned here. (Feel free to ask us for additional info if you are in this case.) Note that if you federate your AM with the Slices SA and MA, there is not need to configure jFed to use the SA/MA of your testbed (if you have one), so there is also no need to configure it in jFed.

In short, this section assumes that you have a Slices account, and you will login to jFed with this account, in order to use your AM.

Requirements

You need a few things before you get started:

  • A server to run the AM software on.

  • A publicly reachable IP for that server. This needs to be either an IPv4 or an IPv6 address. We recommend both.

  • A DNS name for that server, that resolves to the publicly reachable IP addresses of the server. (Recommendation: It’s nice if the DNS name refers to your testbed and is specific for your AM. Example: am.mytestbed.example.com)

    Note

    A DNS name is not strictly required, but very highly recommended

  • Choose a URN for your AM. This is of the form: urn:publicid:IDN+DNSNAME+authority+am where your replace DNSNAME by the DNS name of your AM. (Example: urn:publicid:IDN+mytestbed.example.com+authority+am).

    If you don’t have a DNS name for your server use a globally unique name for your testbed instead. It’s not recommended to use an IP address instead, though that can work. Never use localhost or 127.0.0.1, and never use the default value of the AM software you use. (For more info, see Choosing your component manager urn)

  • Choose a port at which you server will run. There is no standard port in the specification, so a lot of different ports are used in practice (12369, 8010, …). We recommend using port 443, if that is not in use by anything else, as the AM does use https. The advantage of using the default https port is that it is reachable through most firewalls.

  • You need a X.509 Server Certificate, because the AM server uses https. This can be a self signed certificate (jFed stores a list of these to make it work safely). However, in that case, make sure you configure the fields in your self signed certificate correctly. See the next section for more details.

  • You probably have testbed resources that you want to make reachable to experimenters using SSH. There are 2 options:

    • You need public IP addresses that you can assign to these nodes when needed (IPv4 or IPv6)
    • You need to have one machine with a publically reachable IP address (IPv4 recommended) act as a gateway.

    In both cases, make sure you have these address(es) available. You can combine the gateway machine with the machine on which the AM runs, but for security reasons, this is not recommended.

An example of these choices for the imec Virtual Wall2 testbed:

  • IP: 193.191.148.179
  • DNS name: wall2.ilabt.iminds.be
  • URN: urn:publicid:IDN+wall2.ilabt.iminds.be+authority+cm So the URN including the DNS name as TLD/TLA part of the URN. “authority” as “type” of the URN, and “cm” as “name” part of the URN. (We actually recommend “am” as name instead of “cm”, but both are fine.)
  • Server certificate:
    • Not self signed in our case, but signed by a trusted root.
    • “Subject Name” contains CN=www.wall2.ilabt.iminds.be, as well as details about our address and organisation
    • “Subject Alternative Name” contains DNS:www.wall2.ilabt.iminds.be, DNS:boss.wall2.ilabt.iminds.be, DNS:authority.ilabt.iminds.be
  • Our server runs at port 12369, because emulab already uses port 443 for another webserver, with different SSL settings.
  • The full URL at which our AM is reachable is: https://www.wall2.ilabt.iminds.be:12369/protogeni/xmlrpc/am/3.0
  • Experimenters can reserve servers at our AM. All of these are then reachable with SSH using public IPv6 addresses. We also have a pool of IPv4 addresses which users can assign to servers if needed. In addition, we have a gateway (bastion.test.iminds.be, which has both an IPv4 and IPv6 address), which can be used by experimenters that do not have IPv6.

Server X.509 Certificate

An AM uses a message format specified in Geni AM specification, which is sent over XML-RPC, which is sent over HTTPS with client authentication. So an AM runs at a HTTPS server that is configured to require client authentication.

This means that the firsts step is to correctly setup the SSL server. That requires a server X509 certificate and matching private key. This certificate is used to identify the server to the users. Because only the server has the private key matching the certificate, users can setup a secure connection.

For HTTPS on the public internet, it is required that your server’s X509 certificate is signed by a trusted root. All browsers have a list of these trusted roots. This is in short how the internet is made secure.

jFed also uses the internet’s trusted roots, so if your server certificate is signed by these, that should be enough. However, for convenience, jFed also allows you to use a self signed X509 certificate. You can then add this self signed certificate to the list of certificates that jFed trusts, and everything will work without requiring the hassle of aquiring a “real” X509 certificate signed by a trusted root.

However, in recent years, thanks to “Let’s encrypt” and the excellent tool certbot it is now usually easier to get a real valid X509 certificate than it is to generate a decent self signed certificate. Because of this, we recommend setting up certbot on your server, and letting it generate the X509 certificate. Make sure that you setup certbot to automatically renew the certificate (using cron or a systemctl timer), and to restart the AM server when it does (this can be done using the certbot “post-renew hook”).

As mentioned, you can also generate a self signed certificate and register it to the central list of certificates that jFed trusts. When generating a self signed certicicate there are some best practices to take into account, regarding the self signed X509 certificate that you generate for your AM:

  • “Subject” field of the certificate must contain a “CN” that is the hostname of the server (NOT and IP, the DNS hostname!)
  • The “X509v3 Subject Alternative Name” section, must contain a “DNS” entry, which is the hostname of your server (NOT and IP, the DNS hostname!). Note that this means that your AM needs a DNS name, not just an IP address!
  • If the self-signed certificate is signed by another self-signed certificate, you need to include that other certificate in the “certificate chain” when configuring the AM’s HTTPS.

Note that you can also add your AM to jFed if the first 2 guidelines are not respected. However, try to at least have a meaningful “Subject” “CN”, and avoid a “Subject” “CN” like “localhost” at all cost. Also not that this will probably cause problems with non-jFed tools.

Allowing access to federated users

Federate with Slices User and Slice Authority

Your AM needs to be configured so it is federated with the Slices user and slice authority. How to do this depends on the AM software.

General instructions: You need to add the Slices root certificate to the list of trusted authorities.

AM specific instructions:

  • For GCF, you need to put the root certificate file in the gcf trusted_roots directory (this is configurable with rootcadir in gcf_config, check the default value in the gcf repo). Example command to do this: curl https://portal.slices-sc.eu/root_certificate > ~/.gcf/trusted_roots/slices.ilabt_imec_be.pem (and then restart the AM)

The result of configuring this is:

  • Your AM will allow (client-authenticated) HTTPS connections from Slices users. If not configured correctly, your AM might terminate the TLS handshake from Slices users (so no HTTP data is even sent). If correctly configured, your AM will know for each connection that a Slices user is connected. This is the “authentication” part.
  • Your AM will accept user and slice “credentials” that it receives from Slices users. It will know what Slices allows these users to do (typically, a slice credential will tell the AM a user can reserve resources). This is the “authorization” part.

Firewall

You’ll need to make sure some ports are reachable:

  • Your AM is listening on one or two ports, and these should be publicly reachable.
  • If your testbed nodes are reachable directly, the necessary ssh port on each node should be publicly reachable. If you use a testbed specific ssh proxy/gateway, only that proxy should be publically reachable.
  • Your AM should be ping-able from the Slices monitoring domain, so it should allow ICMP for at least that domain.

Note: The Slices monitoring connections will come from:

  • 2001:6a8:1d80:2031:225:90ff:fe1d:1c68
  • 193.191.148.194

Only for ICMP you could choose to allow only these (but you can also allow public access). For AM and SSH access, the ports need to be publicly reachable anyway. Don’t rely too much on these not changing, you probably need to allow the whole subnet.

Configuring the AM GetVersion call

Your AM needs to be configured to return the correct info in the GetVersion reply. This is not a critical step, and it can be skipped. But it is a good practice to get this right. It also makes the jFed scanner work better.

The GetVersion call is a call that the scanner uses to retrieve basic server info. It is defined in the AM specification. The results contains a lot of info, most of which will be automatically filled in correctly by the server software. Experience shows that the following fields have to be checked for correctness:

"urn": "urn:publicid:IDN+example.com+authority+am",
"geni_api_versions": {
    "2": "https://example.com/am/2",
    "3": "https://example.com/am/3"
}

You need to make sure that:

  • The URN is correct. That includes the hostname part (example.com in the example above), and the name part am in the example above. The name should be am, or cm. The URN must always match the expected URN in the component_manager_urn attribute in the request RSpec.

  • The geni_api_versions URLs must point to the correct server URL(s). This must be the URL at which the AM is publically available. Make sure that:

    • The protocol is https
    • The port is correct
    • The path is correct
    • The hostname is not localhost or 127.0.0.1.
    • Preferably, the hostname is not an IP address but a DNS hostname.
    • Multiple URLs are allowed if the server supports multiple AM versions. But this is not required. Do make sure however that at least the version contacted is listed. That is, if you connect to https://example.com/am/3 it is at least expected that "3": "https://example.com/am/3" is listed.

Adding to global Slices config

The jFed team can also add the same info about your AM, to the “global” jFed/Slices config. That enables the use of your AM for all jFed users. The global configuration is available via the Slices API, at https://flsmonitor-api.fed4fire.eu/testbed?embed=true

Note that this does not add your testbed to any of the “icons” in the jFed GUI. But you can test your testbed from jFed by manually editing the request RSpec once the testbed has been added to the global configuration.

Contact us to add your AM for all users.

You do not need to have a local working config, but that helps.

The minimal info you need to send us is:

  • The URL at which your AM runs (the full URL, including hostname, port and path)
  • The name you want to see for your testbed inside jFed and at our monitoring site
  • An example request RSpec (which shows us the sliver_type(s) used and other info)
  • Does your testbed support (or even require) specifying disk images in the request RSpec?
  • Does your testbed support (or even require) assigning specific nodes (component_id attibute) in the request RSpec?
  • Does your testbed support (or even require) specifying hardware types in the request RSpec?
  • Are links between nodes of your testbed supported?
  • Does your testbed support “stitched” links? (If you’re unsure what that is, your testbed doesn’t support them.)

Other info which is usefull to send us is:

  • Which “icon” in the jFed experiment GUI is the best match for your testbed nodes (ex: physical node, VM, wireless, …)?
  • For security reasons, it is recommended to also send your server certificate (in PEM format). We can also retrieve the certificate ourself, but theoretically, that could be intercepted.
  • The coordinates of the testbed (latitude, longitude and country)
  • Info on the organisation running the testbed (full name, latitude, longitude, country, address, link to logo, link to website)
  • Additional info about your testbed: link to documentation, link to testbed description and/or other general testbed information.
  • If your testbed has a web interface (where users can login and control the testbed) in addition to the AM, you can send us that link as well.
  • The email address(es) of the “primary” contact(s) of the testbed. If we need to contact someone about the testbed, we use these emails.
  • The email address(es) of the “technical” contact(s) of the testbed. For technical questions about the testbed, these emails are used. These emails also are used for automatic mails about the tests, and in the future, testbed specific bugreports will also be sent here. Let us know wether or not you want the technical contact(s) of the testbed to receive automated emails when the testbed goes down.
  • The email address(es) of the GDPR contact(s) of the testbed. Anything related to the GDPR we will send here.
  • The URN(s) of the users that may have admin access to some of the data we store about your testbed. This is currently not used, but we plan on using this later to allow you to restart tests, enable/disable tests or edit testbed info. This is typically the URN of the users you use to login to jFed.
  • Which software does your testbed AM use? Typical options are: emulab, gcf, openstack, …

(A lot of email addresses are mentioned above, but typically, they are all just the same single email address.)

Adding your AM to one of the experiment GUI “icons”

The jFed Experimenter GUI has a graphical editor, where you can drag “icons” to the canvas and in this way easily configure an experiment. Each of these icons has their own list of relevant testbeds.

Note that you do not need these icons to test your AM using the jFed Experimenter GUI: after you added your testbed to jFed, you can directly edit the XML Rspec, and everything will work.

It is currently not possible to manually add a testbed to one of these jFed icons in your local jFed install. Adding testbeds to the “icons” can only be done by the jFed developers. When you contact us to add your testbed to jFed for all users, we will discuss which icon(s) it needs to be added too, and add it for you.

Typically, to allow you to test without adding the testbed to the icon for all users, we add it to “dev-testbeds”. Icons added here are not available to all users. To show these testbeds in jFed:

  • Click the “jFed Preferences” button
  • On the left, choose “Icon Catalog & Gui Fine-tuning”
  • On the right, in the “Extra Flags” field, enter the text “dev-testbeds”
  • On the bottom of the dialog, press the “Save” button
  • Restart jFed
  • You will be able to select the testbed when using the appropriate icon

Note that any user (so not just you), can choose to see and use the dev testbeds icons in jFed in this way.

jFed and Testbed specific documentation

The jFed experimenter GUI shows links to the testbed documentation. This makes it easy for experiments to find testbed documentation.

Two links can be set in the central jFed config, which will both be shown in jFed:

  • “Testbed Info” button: a link to a (short) general description of the testbed
  • “Testbed Documentation” button: a link to a site that has step by step instructions on how to use the testbed from jFed. A short and simple “getting started” tutorial is ideal. It’s off course also nice if this site contains links to more info and more advanced tutorials.

These links are stored in the central jFed config, and testbed owners can ask us to add/update it. In future jFed versions, this link might be shown more prominently.