Skip to content

ENG-41944: Prune ConfD references #161

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
bbcsco wants to merge 1 commit into
base: 6.7
Choose a base branch
from
+74 −92
Open

ENG-41944: Prune ConfD references #161

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 52 additions & 69 deletions development/advanced-development/web-ui-development/json-rpc-api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ description: API documentation for JSON-RPC API.

# JSON-RPC API

## Protocol Overview
## Protocol Introduction

The [JSON-RPC 2.0 Specification](https://www.jsonrpc.org/specification) contains all the details you need to understand the protocol but a short version is given here:
The [JSON-RPC 2.0 Specification](https://www.jsonrpc.org/specification) contains all the details of the protocol. The following is only a quick introduction of the basics.

{% tabs %}
{% tab title="Request Payload" %}
Expand Down Expand Up @@ -70,53 +70,15 @@ With a possible response like (first result for `add`, the second result for `su
"result": 19}]
```

### Trace Context

JSON-RPC supports the Trace Context functionality corresponding to the IETF Draft [I-D.draft-ietf-netconf-restconf-trace-ctx-headers-00](https://www.ietf.org/archive/id/draft-ietf-netconf-restconf-trace-ctx-headers-00.html), that is an adaption of the [W3C Trace Context](https://www.w3.org/TR/2021/REC-trace-context-1-20211123/) standard. Trace Context makes it possible to follow a client's functionality via progress trace (logging) by `trace-id`, `span-id` and `tracestate`. Trace Context standardizes the format of `trace-id`, `span-id` and key-value pairs to be sent between distributed entities. The terms `span-id` and `parent-span-id` in NSO correspond to the naming of `parent-id` used in the Trace Context standard.

Trace Context consists of two HTTP headers `traceparent` and `tracestate`. Header `traceparent` must be of the format:

```
traceparent = <version>-<trace-id>-<parent-id>-<flags>
```

Where, `version = "00"` and `flags = "01"`. The support for the values of `version` and `flags` may change in the future depending on the extension of standard or functionality.

An example of header `traceparent` in use is:

```
traceparent: 00-100456789abcde10123456789abcde10-001006789abcdef0-01
```
## Common Concepts

Header `tracestate` is a vendor-specific list of key-value pairs. An example of header `tracestate` in use is:

```
tracestate: key1=value1,key2=value2
```

Where, a value may contain space characters but not end with a space.

Trace Context is handled by the progress trace functionality, see also [Progress Trace](../progress-trace.md).

The information in Trace Context will be presented by the progress trace output when invoking JSON-RPC methods `validate_commit`, `apply`, or `run_action`. Those methods will also generate a Trace Context if it has not already been given in a request.

The functionality a client aims to perform can consist of several JSON-RPC methods up to a transaction commit being executed. Those methods are carried out at the transaction commit and should share a common trace-id. Such a scenario calls for the need to store Trace Context in the transaction involved. For this reason JSON-RPC will only consider a Trace Context header for methods that take a transaction as parameter, with the exception of the method `commit`, which will ignore the Trace Context header.

{% hint style="info" %}
You can either let methods `validate_commit`, `apply`, or `run_action` automatically generate a Trace Context, or you can add a Trace Context header for one of the involved JSON-RPC methods sharing the same transaction.

If two methods, using the same transaction, are provided with different Trace Context, the latter Trace Context will be used - a procedure not recommended.
{% endhint %}

### Common Concepts <a href="#ug.jsonrpc.commonconcepts" id="ug.jsonrpc.commonconcepts"></a>

The URL for the JSON-RPC API is `` `/jsonrpc` ``. For logging and debugging purposes, you can add anything as a subpath to the URL, for example turning the URL into `` `/jsonrpc/<method>` `` which will allow you to see the exact method in different browsers' **Developer Tools** - **Network** tab - **Name** column, rather than just an opaque `jsonrpc`.
The URL for the JSON-RPC API is `/jsonrpc`. For logging and debugging purposes, you can add anything as a subpath to the URL, for example turning the URL into `/jsonrpc/<method>` which will allow you to see the exact method in different browsers' **Developer Tools** - **Network** tab - **Name** column, rather than just an opaque `jsonrpc`.

{% hint style="info" %}
For brevity, in the upcoming descriptions of each method, only the input `params` and the output `result` are mentioned, although they are part of a fully formed JSON-RPC payload.
{% endhint %}

* Authorization is based on HTTP cookies. The response to a successful call to `login` would create a session, and set an HTTP-only cookie, and even an HTTP-only secure cookie over HTTPS, named `sessionid`. All subsequent calls are authorized by the presence and the validity of this cookie.
* Authorization is based on HTTP cookies. The response to a successful call to `login` would create a session, and set an HTTP-only cookie (an HTTP-only secure cookie over HTTPS) named `sessionid`. All subsequent calls are authorized by the presence and the validity of this cookie.
* The `th` param is a transaction handle identifier as returned from a call to `new_trans`.
* The `comet_id` param is a unique ID (decided by the client) that must be given first in a call to the `comet` method, and then to upcoming calls which trigger comet notifications.
* The `handle` param needs to have a semantic value (not just a counter) prefixed with the `comet` ID (for disambiguation), and overrides the handle that would have otherwise been returned by the call. This gives more freedom to the client and sets semantic handles.
Expand Down Expand Up @@ -193,7 +155,45 @@ All methods may return one of the following JSON RPC or application-defined erro
{"type": "session.overload"}
```

### FAQs <a href="#ug.jsonrpc.faq" id="ug.jsonrpc.faq"></a>
### Trace Context

JSON-RPC supports the Trace Context functionality corresponding to the IETF Draft [I-D.draft-ietf-netconf-restconf-trace-ctx-headers-00](https://www.ietf.org/archive/id/draft-ietf-netconf-restconf-trace-ctx-headers-00.html), that is an adaption of the [W3C Trace Context](https://www.w3.org/TR/2021/REC-trace-context-1-20211123/) standard. Trace Context makes it possible to follow a client's functionality via progress trace (logging) by `trace-id`, `span-id` and `tracestate`. Trace Context standardizes the format of `trace-id`, `span-id` and key-value pairs to be sent between distributed entities. The terms `span-id` and `parent-span-id` in NSO correspond to the naming of `parent-id` used in the Trace Context standard.

Trace Context consists of two HTTP headers `traceparent` and `tracestate`. Header `traceparent` must be of the format:

```
traceparent = <version>-<trace-id>-<parent-id>-<flags>
```

Where, `version = "00"` and `flags = "01"`. The support for the values of `version` and `flags` may change in the future depending on the extension of standard or functionality.

An example of header `traceparent` in use is:

```
traceparent: 00-100456789abcde10123456789abcde10-001006789abcdef0-01
```

Header `tracestate` is a vendor-specific list of key-value pairs. An example of header `tracestate` in use is:

```
tracestate: key1=value1,key2=value2
```

Where, a value may contain space characters but not end with a space.

Trace Context is handled by the progress trace functionality, see also [Progress Trace](../progress-trace.md).

The information in Trace Context will be presented by the progress trace output when invoking JSON-RPC methods `validate_commit`, `apply`, or `run_action`. Those methods will also generate a Trace Context if it has not already been given in a request.

The functionality a client aims to perform can consist of several JSON-RPC methods up to a transaction commit being executed. Those methods are carried out at the transaction commit and should share a common trace-id. Such a scenario calls for the need to store Trace Context in the transaction involved. For this reason JSON-RPC will only consider a Trace Context header for methods that take a transaction as parameter, with the exception of the method `commit`, which will ignore the Trace Context header.

{% hint style="info" %}
You can either let methods `validate_commit`, `apply`, or `run_action` automatically generate a Trace Context, or you can add a Trace Context header for one of the involved JSON-RPC methods sharing the same transaction.

If two methods, using the same transaction, are provided with different Trace Context, the latter Trace Context will be used - a procedure not recommended.
{% endhint %}

### FAQs

<details>

Expand Down Expand Up @@ -2624,32 +2624,15 @@ curl \
{"jsonrpc": "2.0",
"id": 1,
"result": {
"cli-builtin": "cli-builtin",
"confd_cfg": "confd_cfg",
"iana-crypt-hash": "ianach",
"ietf-inet-types": "inet",
"ietf-netconf": "nc",
"ietf-netconf-acm": "nacm",
"ietf-netconf-monitoring": "ncm",
"ietf-netconf-notifications": "ncn",
"ietf-netconf-with-defaults": "ncwd",
"ietf-restconf": "rc",
"ietf-restconf-monitoring": "rcmon",
"ietf-yang-library": "yanglib",
"ietf-yang-types": "yang",
"tailf-aaa": "aaa",
"tailf-acm": "tacm",
"tailf-common-monitoring2": "tfcg2",
"tailf-confd-monitoring": "tfcm",
"tailf-confd-monitoring2": "tfcm2",
"tailf-ncs-rollback": "ncs-rollback",
"tailf-kicker": "kicker",
"tailf-netconf-extensions": "tfnce",
"tailf-netconf-monitoring": "tncm",
"tailf-netconf-query": "tfncq",
"tailf-rest-error": "tfrerr",
"tailf-rest-query": "tfrestq",
"tailf-rollback": "rollback",
"tailf-webui": "webui",
"tailf-ncs-webui": "ncs-webui",
"tailf-yang-patch-ncs": "ncsypatch",
"ietf-netconf-acm": "nacm",
"tailf-netconf-rollback": "netconf-rollback",
"tailf-restconf-error": "tfrcerr",
"tailf-ncs-kicker-extension": "ncs-kicker",
...
}
}
```
Expand Down
10 changes: 5 additions & 5 deletions development/connected-topics/web-server.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -121,7 +121,7 @@ admin@io% set tls certificate cert-data
[ok][2019-06-10 19:59:36]

[edit]
admin@confd% set tls private-key key-data
admin@ncs% set tls private-key key-data
(<unknown>):
[Multiline mode, exit with ctrl-D.]
> -----BEGIN RSA PRIVATE KEY-----
Expand All @@ -132,12 +132,12 @@ admin@confd% set tls private-key key-data
[ok][2019-06-10 20:00:27]

[edit]
admin@confd% set tls private-key passphrase
admin@ncs% set tls private-key passphrase
(<AES encrypted string>): ********
[ok][2019-06-10 20:00:39]

[edit]
admin@confd% set tls ca-certificates ca-cert-1 cert-data
admin@ncs% set tls ca-certificates ca-cert-1 cert-data
(<unknown>):
[Multiline mode, exit with ctrl-D.]
> -----BEGIN CERTIFICATE-----
Expand All @@ -147,7 +147,7 @@ admin@confd% set tls ca-certificates ca-cert-1 cert-data
[ok][2019-06-10 20:02:22]

[edit]
admin@confd% set tls ca-certificates ca-cert-2 cert-data
admin@ncs% set tls ca-certificates ca-cert-2 cert-data
(<unknown>):
[Multiline mode, exit with ctrl-D.]
> -----BEGIN CERTIFICATE-----
Expand All @@ -157,7 +157,7 @@ admin@confd% set tls ca-certificates ca-cert-2 cert-data
[ok][2019-06-10 20:03:07]

[edit]
admin@confd% commit
admin@ncs% commit
Commit complete.
[ok][2019-06-10 20:03:11]

Expand Down
9 changes: 4 additions & 5 deletions development/core-concepts/northbound-apis/restconf-api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -162,8 +162,7 @@ Accept: application/yang-data+xml
<netconf-state xmlns="urn:ietf:params:xml:ns:yang:ietf-netconf-monitoring"/>
<restconf-state xmlns="urn:ietf:params:xml:ns:yang:ietf-restconf-monitoring"/>
<aaa xmlns="http://tail-f.com/ns/aaa/1.1"/>
<confd-state xmls="http://tail-f.com/yang/confd-monitoring"/>
<last-logins xmlns="http://tail-f.com/yang/last-login"/>
...
</data>
```
{% endcode %}
Expand All @@ -174,7 +173,7 @@ Let's assume we are interested in the `dhcp/subnet` resource in our configuratio

{% code title="Example: The dhcp.yang Resource" %}
```cli
> yanger -f tree examples.confd/restconf/basic/dhcp.yang
> yanger -f tree dhcp.yang
module: dhcp
+--rw dhcp
+--rw max-lease-time? uint32
Expand Down Expand Up @@ -1463,7 +1462,7 @@ Accept: application/yang-data+json

This functionality is supported if the `http://tail-f.com/ns/restconf/unhide/1.0` capability is presented. See also [How to View the Capabilities of the RESTCONF Server](restconf-api.md#ncs.northbound.restconf.capabilities).

By default, hidden nodes are not visible in the RESTCONF interface. To unhide hidden nodes for retrieval or editing, clients can use the query parameter `unhide` or set parameter `showHidden` to `true` under `/confdConfig/restconf` in `confd.conf` file. The query parameter `unhide` is supported for RPC and action invocation.
By default, hidden nodes are not visible in the RESTCONF interface. To unhide hidden nodes for retrieval or editing, clients can use the query parameter `unhide` or set parameter `show-hidden` to `true` under `/ncs-config/restconf` in `ncs.conf` file. The query parameter `unhide` is supported for RPC and action invocation.

The format of the `unhide` parameter is a comma-separated list of

Expand All @@ -1477,7 +1476,7 @@ As an example:
unhide=extra,debug;secret
```

This example unhides the unprotected group _extra_ and the password-protected group `debug` with the password `secret;`.
This example unhides the unprotected group _extra_ and the password-protected group `debug` with the password `secret`.

## Trace Context

Expand Down
26 changes: 13 additions & 13 deletions development/core-concepts/yang.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -523,7 +523,7 @@ $ ncsc -c test.yang

The above command creates an output file `test.fxs` that is a compiled schema that can be loaded into the system. The `ncsc` compiler with all its flags is fully described in [ncsc(1)](../../resources/man/ncsc.1.md) in Manual Pages.

There exist several standards-based auxiliary YANG modules defining various useful data types. These modules, as well as their accompanying `.fxs` files can be found in the `${NCS_DIR}/src/confd/yang` directory in the distribution.
There exist several standards-based auxiliary YANG modules defining various useful data types. These modules, as well as their accompanying `.fxs` files can be found in the `${NCS_DIR}/src/ncs/yang` directory in the distribution.

The modules are:

Expand Down Expand Up @@ -684,7 +684,7 @@ leaf b {

This data model snippet says that `b` can only exist if `a` is true. If `a` is true, and `b` has a value, and `a` is set to false, `b` will automatically be deleted.

Since the XPath expression in theory can refer to any node in the data tree, it has to be re-evaluated when any node in the tree is modified. But this would have a disastrous performance impact, so to avoid this, NSO keeps track of dependencies for each when expression. In many cases, the **confdc** can figure out these dependencies by itself. In the example above, NSO will detect that `b` is dependent on `a`, and evaluate `b`'s XPath expression only if `a` is modified. If `confdc` cannot detect the dependencies by itself, it requires a `tailf:dependency` statement in the `when` statement. See `tailf:dependency` in [tailf\_yang\_extensions(5)](../../resources/man/tailf_yang_extensions.5.md) in Manual Pages for details.
Since the XPath expression in theory can refer to any node in the data tree, it has to be re-evaluated when any node in the tree is modified. But this would have a disastrous performance impact, so to avoid this, NSO keeps track of dependencies for each when expression. In many cases, the **ncsc** can figure out these dependencies by itself. In the example above, NSO will detect that `b` is dependent on `a`, and evaluate `b`'s XPath expression only if `a` is modified. If **ncsc** cannot detect the dependencies by itself, it requires a `tailf:dependency` statement in the `when` statement. See `tailf:dependency` in [tailf\_yang\_extensions(5)](../../resources/man/tailf_yang_extensions.5.md) in Manual Pages for details.

## Using the Tail-f Extensions with YANG <a href="#d5e2188" id="d5e2188"></a>

Expand Down Expand Up @@ -791,10 +791,10 @@ module test-ann {
}
```

To compile the module with annotations, use the `-a` parameter to `confdc`:
To compile the module with annotations, use the `-a` parameter to `ncsc`:

```
confdc -c -a test-ann.yang test.yang
ncsc -c -a test-ann.yang test.yang
```

## Custom Help Texts and Error Messages <a href="#d5e2219" id="d5e2219"></a>
Expand Down Expand Up @@ -1350,10 +1350,10 @@ module links {
}
```

If the above YANG file is saved on disk, as `links.yang`, we can compile and link it using the `confdc` compiler:
If the above YANG file is saved on disk, as `links.yang`, we can compile and link it using the `ncsc` compiler:

```bash
$ confdc -c links.yang
$ ncsc -c links.yang
```

We now have a ready-to-use schema file named `links.fxs` on disk. To run this example, we need to copy the compiled `links.fxs` to a directory where NSO can find it.
Expand Down Expand Up @@ -1445,7 +1445,7 @@ leaf server-port {
}
```

Note that using the `deref` function is syntactic sugar for the basic syntax. The translation between the two formats is trivial. Also note that `deref()` is an extension to YANG, and third-party tools might not understand this syntax. To make sure that only plain YANG constructs are used in a module, the parameter `--strict-yang` can be given to `confdc -c`.
Note that using the `deref` function is syntactic sugar for the basic syntax. The translation between the two formats is trivial. Also note that `deref()` is an extension to YANG, and third-party tools might not understand this syntax. To make sure that only plain YANG constructs are used in a module, the parameter `--strict-yang` can be given to `ncsc -c`.

## Using Multiple Namespaces <a href="#d5e2425" id="d5e2425"></a>

Expand Down Expand Up @@ -1474,7 +1474,7 @@ module datatypes {
We compile and link `datatypes.yang` into a final schema file representing the `http://example.com/ns/dt` namespace:

```bash
$ confdc -c datatypes.yang
$ ncsc -c datatypes.yang
```

To reuse our user defined `countersType`, we must import the `datatypes` module.
Expand All @@ -1494,13 +1494,13 @@ module test {
}
```

When compiling this new module that refers to another module, we must indicate to `confdc` where to search for the imported module:
When compiling this new module that refers to another module, we must indicate to `ncsc` where to search for the imported module:

```bash
$ confdc -c test.yang --yangpath /path/to/dt
$ ncsc -c test.yang --yangpath /path/to/dt
```

`confdc` also searches for referred modules in the colon (:) separated path defined by the environment variable `YANG_MODPATH` and . (dot) is implicitly included.
`ncsc` also searches for referred modules in the colon (:) separated path defined by the environment variable `YANG_MODPATH` and . (dot) is implicitly included.

## Module Names, Namespaces, and Revisions <a href="#ug.yang.names_namespaces_and_revisions" id="ug.yang.names_namespaces_and_revisions"></a>

Expand Down Expand Up @@ -1560,11 +1560,11 @@ We have three different entities that define our configuration data.

## Hash Values and the `id-value` Statement <a href="#ug.yang.id_value" id="ug.yang.id_value"></a>

Internally and in the programming APIs, NSO uses integer values to represent YANG node names and the namespace URI. This conserves space and allows for more efficient comparisons (including `switch` statements) in the user application code. By default, `confdc` automatically computes a hash value for the namespace URI and for each string that is used as a node name.
Internally and in the programming APIs, NSO uses integer values to represent YANG node names and the namespace URI. This conserves space and allows for more efficient comparisons (including `switch` statements) in the user application code. By default, `ncsc` automatically computes a hash value for the namespace URI and for each string that is used as a node name.

Conflicts can occur in the mapping between strings and integer values - i.e. the initial assignment of integers to strings is unable to provide a unique, bi-directional mapping. Such conflicts are extremely rare (but possible) when the default hashing mechanism is used.

The conflicts are detected either by `confdc` or by the NSO daemon when it loads the `.fxs` files.
The conflicts are detected either by `ncsc` or by the NSO daemon when it loads the `.fxs` files.

If there are any conflicts reported they will pertain to XML tags (or the namespace URI),

Expand Down