ONVIF Profile S, Actions API, and PTZ Controls Implementation

[kaburagisec]

TL;DR

1. The PTZ component has now been switched to ONVIF component.
2. The Telegram component (the PTZ control section) has been refactored to accommodate the changes to the ONVIF component.
3. Implementation of the ONVIF client requirements for Profile S (see Profile S Specs) in Viseron (starting point of ONVIF Profile G support #1166).
4. Implementation of the Actions API to connect backend and frontend component functions.
5. PTZ Controls are available on the Live View page as a PlayerMenu (closed [FEATURE] PTZ controls #449).
6. ONVIF settings can be configured through the Camera Tuning page (see PR Add Camera Tuning Implementation (Tuning Interface, Zone/Mask Drawer Tool, Labels Editor, OSD Texts, and Miscellaneous UI)  #1165).
7. This ONVIF component can be used with all types of ONVIF cameras (even if the camera does not support PTZ).
8. Using the onvif-python library as a bridge for ONVIF communication with the camera.
9. Closed Can't configure PTZ with go2rtc #1264
10. Small docs improvements

PR Overview

PR TODO:

• ONVIF Component backend implementation
• Actions API backend implementation for ONVIF Component
• Refactor Telegram PTZ Control for ONVIF Component
• Implementing API Hooks in Frontend for ONVIF Component
• Implementing PTZ Controls in Frontend for ONVIF Component
• Implementing Camera Tuning handlers in the backend for ONVIF Component
• Implementing helper configuration UI in Camera Tuning page for ONVIF Component
• Several UI style improvements on the Camera Tuning page
• Several refactors for Camera Tuning API & Camera Tuning UI
• Small docs improvements

ONVIF Profile S?

ONVIF consists of a set of profiles; ranging from A to T, that define compatibility, services, and mandatory functional requirements for both clients and devices. Further details can be found at https://www.onvif.org/profiles.

In this PR, I am only implementing client compatibility for Profile S (https://www.onvif.org/profiles/profile-s/). It is certainly possible to implement other ONVIF profiles in Viseron in the future.

What is the Actions API?

The Actions API is a set of APIs that allow frontend components to perform specific actions on specific component functions defined in the backend that directly impact or communicate with the camera. The implementation code must conform to the following folder pattern: viseron/components/webserver/api/v1/actions/{component_name}.

The naming of the Actions API is simply a separation of focus, as the majority of the REST APIs in the webserver components are APIs for Viseron's built-in systems.

Custom Decorators

There are two main decorators introduced in this PR:

1. @operation : found in viseron/components/onvif/utils.py and used to wrap all methods that implement ONVIF operations for a service. Any method that calls ONVIF operations within it is required to use this decorator.
2. @action_handler : found in viseron/components/webserver/api/v1/actions/onvif/base.py and used to wrap methods used by the Actions API. Because the context is an ONVIF component, the @action_handler decorator also communicates to determine which service will process the API request.

How PTZ Controls works?

On the frontend, PTZ Controls will only be rendered on cameras with a non-null ptz_support key. This means the PTZ Controls button will not appear in the player on cameras that don't support PTZ. The current implementation is to match the ptz_support key with the available PTZ controller (referred to here as the UI component). Currently, PTZ Controls can only be used with ONVIF components. It is possible (in the future) to implement PTZ Controls other than ONVIF components, by integrating custom components with specific brands (for example, Dahua).

All PTZ operation communications must go through the Actions API defined in frontend/src/lib/api/actions/{component_name} for frontend and in viseron/components/webserver/api/v1/actions/{component_name} for backend. The UI component for PTZ Controls must be located in frontend/src/components/actions/ptz.

Tested Devices

No	Model	Behaviours	Results	ONVIF Version
1	TAPO C210	Conformant product. Works well for all operations. But ignore any Imaging service settings.	no delay, PTZ work properly	ONVIF Version 2.20
2	EZVIZ H8C	Conformant product. Works well for all operations. But ignore speed parameters for Continuous Move operation.	no delay, PTZ work properly	ONVIF Version 18.12
3	UNIVIEW Uho-S2E	Conformant product. Works well for all operations.	no delay, PTZ work properly	ONVIF Version 19.12
4	Prolink DS-310	Non-conformant product. Most basic services—such as Device, PTZ, and Media—function as expected; however, this camera ignores many advanced parameters because it does not use official ONVIF firmware.	no delay, PTZ work with constraint	The ONVIF firmware uses unofficial firmware from Happytimesoft; consequently, many parameters do not work with the official ONVIF API.
5	BIOTA PTZ Indoor	Non-conformant product. Most basic services—such as Device, PTZ, and Media—function as expected; however, this camera ignores many advanced parameters because it does not use official ONVIF firmware.	no delay, PTZ work with constraint	The ONVIF firmware uses unofficial firmware from Happytimesoft; consequently, many parameters do not work with the official ONVIF API.
6	Reolink E1 Pro	Conformant product. Works well for all operations. But ignore speed parameters for Continuous Move operation.	no delay, PTZ work properly	ONVIF Version 21.6

Notes:

1. I've tried my best to implement all ONVIF Profile S requirements in this PR, but it's still far from perfect, so any improvements and feedback are welcome.
2. Never use the term "ONVIF conformant," as this label is only for NVRs registered as members of the ONVIF consortium. The label "ONVIF compatible" is permitted, as this label does not violate ONVIF's trademark policy.
3. All limitations and common errors in ONVIF operation are explained in detail in the ONVIF component documentation.

Notes (dev):

1. Because I was a bit rushed to complete PR Add Camera Tuning Implementation (Tuning Interface, Zone/Mask Drawer Tool, Labels Editor, OSD Texts, and Miscellaneous UI)  #1165 yesterday, there are a lot of things that need to be fixed in Camera Tuning. This PR addresses several things related to the workflow and logic of Camera Tuning, both on the backend and frontend.
2. Some ONVIF API actions in POST operations don't require a complete JSON file in the request body, but instead require an empty object {}. For example, the stop API actions in the PTZ Service and reboot in the Device Service.
3. In Camera Tuning, both backend and frontend, ONVIF will be treated like a domain, and each service will be treated like a component. The main (non-service) configuration will be treated as a client component. This is the approach I took to adapt the existing and established workflow.
4. We can now configure ONVIF cameras directly in Camera Tuning, but why do we still need the auto_config mechanism and service key in config.yaml in the ONVIF component? So many cameras will reset all settings if the camera restarts/reboots, suppose there is a scenario where there is a power outage where the Viseron server and ONVIF cameras are in the same location, then the service key is very useful so that the ONVIF component in Viseron configures each camera based on the configuration that has been written when restarting (Viseron server) without the need to configure each camera one by one.

ONVIF Profile S Client Requirements Checklist:

For transparency and reference of ONVIF component development, the following is a checklist based on the official requirements for Profile S client (https://www.onvif.org/wp-content/uploads/2019/12/ONVIF_Profile_-S_Specification_v1-3.pdf) implementation in ONVIF:

no	scope	requirements	backend	frontend
1	User Authentication	Client shall implement WS-Usernametoken according to WS-security as covered by the core specification.	✅ yes	❌ no
2	User Authentication	Client shall implement HTTP Digest as covered by the core specification.	❌ no	❌ no
3	Capabilities	Client shall be able to get Capabilities of a device using GetCapabilities.	✅ yes	✅ yes
4	Discovery	Client shall be able to discover a device using WS-Discovery as specified in the core specification.	❌ no	❌ no
5	Network Configuration	If configuring a device’s network configuration is supported in any way by the client, the client shall be able to list and configure the device network interface using the GetNetworkInterfaces and SetNetworkInterfaces operations. If configuring a device’s network configuration is supported in any way by the client, the client shall be able to list and set the default gateway of the device using the GetNetworkDefaultGateway and SetNetworkDefaultGateway operations.	✅ yes	✅ yes
6	System	Client shall be able to get device information such as manufacturer, model and firmware version using the GetDeviceInformation operation.	✅ yes	✅ yes
7	User Handling	Client shall be able to create, list, modify and delete users from the device using the CreateUsers, GetUsers, SetUser and DeleteUsers operations.	✅ yes	✅ yes
8	Event handling	Client shall be able to use at least one way to retrieve events out of the following: - “Pull” using the CreatePullPointSubscription and PullMessage operations. - “Push” using Notify, Subscribe and Renew operations from WSBaseNotification.	❌ no	❌ no
9	Video Streaming	- Client shall be able to list media profiles from the device using the GetProfiles operation. - Client shall be able to get the stream URI for the selected profile using the GetStreamURI operation. - Client shall be able to stream video on RTP/UDP or RTP/RTSP/HTTP/TCP using the selected profile over RTSP.	✅ yes and 🚫 out of scope	⚠️ partial and 🚫 out of scope
10	Video Streaming – MJPEG	- Client shall be able to receive a stream and decode MJPEG video using the selected profile over RTSP. - Client shall be able to understand RTSP – JPEG RTP header extension.	🚫 out of scope	🚫 out of scope
11	Video Encoder Configuration	Client shall be able to modify video encoder configurations using the GetVideoEncoderConfigurationOptions and SetVideoEncoderConfiguration operations.	✅ yes	✅ yes
12	Media Profile Configuration	- Client shall be able to list available profiles using the GetProfiles operation. - Client shall be able to create a media profile using the CreateProfile operation.	✅ yes	⚠️ partial
13	Video Source Configuration	Client shall be able to modify and add video source configurations using the GetCompatibleVideoSourceConfigurations, GetVideoSourceConfigurationOptions, SetVideoSourceConfiguration and AddVideoSourceConfiguration operations.	❌ no	❌ no
14	Metadata Configuration	Client shall be able to modify metadata configurations using the, GetMetaDataConfigurationOptions and SetMetaDataConfiguration operations.	❌ no	❌ no
15	Video Streaming – MPEG4 (if supported)	Client shall be able to receive a stream and decode MPEG4 video using the selected profile over RTSP.	🚫 out of scope	🚫 out of scope
16	Video Streaming – H264 (if supported)	Client shall be able to receive a stream and decode H264 video using the selected profile over RTSP.	🚫 out of scope	🚫 out of scope
17	PTZ (if supported)	- Client shall be able to read PTZ capabilities from a PTZ node using GetNodes or GetNode operations. - Client shall be able to add a PTZ configuration to a profile using GetConfigurations and AddPTZConfiguration. - Client shall be able to move a PTZ device using the ContinuousMove operation. - Client shall be able to stop a PTZ device using the Stop operation.	✅ absolutely yes! ++	✅ absolutely yes! ++
18	PTZ – Absolute Positioning (if supported)	Client shall be able to move a PTZ device using the AbsoluteMove operation.	✅ yes	✅ yes
19	PTZ – Relative Positioning (if supported)	Client shall be able to move a PTZ device using the RelativeMove operation.	✅ yes	❌ no
20	PTZ – Presets (if supported)	- Client shall be able to list the presets using the GetPresets operation. - Client shall be able to move a PTZ device to a specific preset using the GotoPreset operation.	✅ yes	✅ yes
21	PTZ – Home Position (if supported)	Client shall be able to move a PTZ device to its home position using the GotoHomePosition operation.	✅ yes	✅ yes
22	PTZ – Auxiliary Command (if supported)	Client shall be able to send auxiliary command to a PTZ device using the SetAuxiliaryCommand operation.	❌ no	❌ no
23	Audio Streaming (if supported)	- Client shall be able to receive a stream and playback audio in G.711μ Law (SimplexCamera Microphone Only, 1ch) codec. - Client shall be able to configure a media profile for audio streaming using the GetCompatibleAudioSourceConfigurations, AddAudioSourceConfiguration, GetCompatibleAudioEncoderConfigurations, and AddAudioEncoderConfiguration operations.	🚫 out of scope	🚫 out of scope
24	Audio Streaming – G726 (if supported)	Client shall be able to receive a stream and decode G726 audio using the selected profile over RTSP.	🚫 out of scope	🚫 out of scope
25	Audio Streaming – AAC (if supported)	Client shall be able to receive a stream and decode AAC audio using the selected profile over RTSP.	🚫 out of scope	🚫 out of scope
26	Multicast Streaming (if supported)	- Client shall be able to control multicast streaming using RTSP or the StartMultiCastStreaming and StopMultiCastStreaming operations. - Client shall be able to receive a multicast stream sent by a device.	🚫 out of scope	🚫 out of scope
27	Relay Outputs (if supported)	- Client shall be able to list available relay outputs using the GetRelayOutputs operation. - Client shall be able to set the settings of a relay output using the SetRelayOutputSettings operation. - Client shall be able to trigger a relay output using the SetRelayOutputState operation.	❌ no	❌ no
28	NTP (if supported)	Client shall be able to configure NTP servers on a device using GetNTP and SetNTP operations.	✅ yes	✅ yes
29	Dynamic DNS (if supported)	Client shall be able to configure Dynamic DNS on device using GetDynamicDNS and SetDynamicDNS operations.	❌ no	❌ no
30	Zero Configuration (if supported)	Client shall be able to configure Zero Configuration on device using GetZeroConfiguration and SetZeroConfiguration operations.	❌ no	❌ no
31	IP Address Filtering (if supported)	Client shall be able to configure, add and remove IP Address Filters on device using GetIPAddressFilter, SetIPAddressFilter, AddIPAddressFilter and RemoveIPAddressFilter operations.	❌ no	❌ no

• ✅ yes : Implemented
• ⚠️ partial : Partially implemented
• ❌ no : Not/not yet implemented
• 🚫 out of scope : Outside the scope of the ONVIF component or already implemented by Viseron

[netlify]

✅ Deploy Preview for viseron ready!

Name	Link
🔨 Latest commit	901b4d8
🔍 Latest deploy log	https://app.netlify.com/projects/viseron/deploys/6982256ba15b6b0008f524cf
😎 Deploy Preview	https://deploy-preview-1205--viseron.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[roflcoopter]

Finally got through everything, i have a few comments (inline and in this comment)

The ONVIF calls are blocking the Tornado API event loop, slowing down other requests. I think this can be solved by changing the wrapper @operation() to use run_in_executor, and changing the operations to be sync instead.

[kaburagisec]

Alright, I'll check and refactor it; thanks for the review.

[roflcoopter]

Great, thanks for the quick fixes. I think some logging.getLogger(COMPONENT).setLevel(logging.CRITICAL) accidentally snuck into your commits? 😄

If you remove those i am happy!

[kaburagisec]

Great, thanks for the quick fixes. I think some logging.getLogger(COMPONENT).setLevel(logging.CRITICAL) accidentally snuck into your commits? 😄

If you remove those i am happy!

I intentionally included that line: logging.getLogger(COMPONENT).setLevel(logging.CRITICAL), not to remove logging from the ONVIF component, but simply to set the level to 'CRITICAL' for the onvif-python library. Without it, there would be duplicate logs coming from both the ONVIF component and the onvif-python library.

[roflcoopter]

Great, thanks for the quick fixes. I think some logging.getLogger(COMPONENT).setLevel(logging.CRITICAL) accidentally snuck into your commits? 😄
If you remove those i am happy!

I intentionally included that line: logging.getLogger(COMPONENT).setLevel(logging.CRITICAL), not to remove logging from the ONVIF component, but simply to set the level to 'CRITICAL' for the onvif-python library. Without it, there would be duplicate logs coming from both the ONVIF component and the onvif-python library.

Right i see, that is handled in viseron/__init__.py for other similar cases, but i am currently refactoring that so no point that you change that now. I can make that transition later on

[kaburagisec]

Ok I will update it

[kaburagisec]

Great, thanks for the quick fixes. I think some logging.getLogger(COMPONENT).setLevel(logging.CRITICAL) accidentally snuck into your commits? 😄
If you remove those i am happy!

I intentionally included that line: logging.getLogger(COMPONENT).setLevel(logging.CRITICAL), not to remove logging from the ONVIF component, but simply to set the level to 'CRITICAL' for the onvif-python library. Without it, there would be duplicate logs coming from both the ONVIF component and the onvif-python library.

Right i see, that is handled in viseron/__init__.py for other similar cases, but i am currently refactoring that so no point that you change that now. I can make that transition later on

Yes, the current logger configuration in Viseron will capture all logs from the libraries, not just onvif-python.

[roflcoopter]

Great work, i am OK with merging this now, anything else you wanna change before i do that?

[kaburagisec]

No, enough for now. Any improvements or fix will be on different PR. Thanks for the review 🙏

[roflcoopter]

Merged, thank you! 🚀