deprecation-process.md

Deprecation process

Motivation

Deprecation of code is a difficult topic, which can become a point of friction.

On one hand, maintainers have to maintain the code base for many platforms, many compilers, while making bug fixes and adding new features over time.

To do so, sometime code cleanup, minor refactoring, or removal of previous engineering debt is required to maintain a healthy code base, to allow for further improvements.

In short, maintainers may want to, or may have to, remove some apis during the lifetime of a project.

On the other hand, users who have a working application built using opentelemetry-cpp, for a given platform, a given compiler, and a given configuration, prefer to have stability and as few changes as possible.

Doing code changes in the application imply a new build / test / deploy cycle, which induce costs, and risks.

In short, users typically do now want changes, if the code works for them.

The following extreme behaviors are what needs to be avoided for a project to stay healthy:

• constant changes in the code base, done without notice, introducing incompatibilities, causing adoption of each new release to be costly with code rewrite. While this allows maintainers to make more releases, it ultimately slows down adoption from users.

• impossibility to make any changes in the code base, or having to support many different flavors of the same api for compatibility reasons, which introduces cost, technical risks, and code bloat, ultimately slowing innovation down.

The whole idea of a deprecation and removal process is to provide some mitigation, that will satisfy both maintainers and users, while allowing both to move forward with their own areas or responsibilities.

Also, note that some areas of the code are considered stable, while others are maturing (feature preview), or experimental and in active development.

The deprecation and removal process is primarily for stable parts of the code. Code in feature preview and experimental is expected to change in a faster pace, possibly with short of no notice in case of experimental code.

This process attempts to provide:

• visibility on the delivery pipeline itself, so that deprecations are visible and documented in advance,
• sufficient time between the deprecation and the removal of an API, so that users can adapt the application at a time of their choice, planned, instead of having to do it in emergency,
• tooling to verify if a given code base is exposed to deprecated code or not, to help assess how much effort is required to stay up to date,
• migration and mitigations, to document not only that some api is going away, but to also provide instructions to adjust the code with an alternate solution.

Artifacts

Issues

Deprecation issues should be tagged with the "Deprecated" label, for example like issue #1892.

Removal issues should be tagged with the "Removal" label, for example like issue #1938.

Discoverability of ongoing deprecations and removals is the key here, to avoid surprises.

The DEPRECATED document

At any given time, file DEPRECATED lists all the ongoing deprecations, no matter how long ago these were announced.

A user using release N should find, when reading DEPRECATED, if a given api was deprecated in release N-5 or N-10, without having to go back to release notes from past versions.

When a removal is finally implemented, the DEPRECATED file is expunged, so it contains only the currently active deprecations.

Main sections in that document are organized as follows:

• [Platforms]
• [Compilers]
• [Third party dependencies]
• [Build tools]
  • This is for CMake, Bazel, Doxygen, ...
  • Changes of tooling go here.
• [Build scripts]
  • This is for CMakeLists.txt, BUILD, Doxyfile, ...
  • Changes of build options go here.
• [opentelemetry-cpp API]
• [opentelemetry-cpp SDK]
• [opentelemetry-cpp Exporter]
• [Documentation]

Please keep main sections as is, with a content of "N/A", instead of removing sections with no current deprecations.

Workflow

Decide deprecation

The first step is to discuss, typically in SIG C++ meeting, the need to deprecate a given feature or api.

Upon agreement, file an issue in GitHub so it becomes visible to all parties following closely development.

Use [DEPRECATION] in the title, and tag with label "Deprecation".

For highly visible features, consider other ways to improve awareness, such as:

• pin the deprecation issue in the GitHub UI,
• discussions in the slack channels
• discussions with other teams in open-telemetry
• blog posts

Prepare deprecation

Implement the deprecation issue with a PR.

Deprecation consist of two parts, communication and tooling.

For the communication, the PR should add an entry in file DEPRECATED, detailing the following sections:

• Announcement
  • set Version: to TO-BE-RELEASED-VERSION
  • set Date: to TO-BE-RELEASED-DATE
  • set PR: to the deprecation PR
  • If applicable, add references to any other relevant document (slack discussion, blog, meeting minutes)
• Motivation
  • add justifications for the need to deprecate.
  • this step is critical to get support from users that will be affected, instead of getting push back.
• Scope:
  • details which part are affected.
  • when applicable, give the exact name of CMake options, C++ header files, classes, functions, etc, so that users can easily search in their code base for references to the deprecated items, and assess the exposure to deprecated code.
• Mitigation:
  • indicate any tooling available to identify the deprecated code still in use,
  • indicate technical mitigations if known,
  • name replacement apis, libraries, compiling options, etc, and describe how to adjust the user code in general, to remove any dependencies on deprecated code.
• Planned removal:
  • Indicate an estimated date and/or version, if known, by which the deprecated code will be removed,
  • this allows users to plan accordingly, to minimize disruption.

For the tooling, the PR should implement any code change necessary to help users transition to a clean code base, where all dependencies to deprecated code are removed.

This imply these dependencies can be identified, in a reliable way.

The goal is to, at the same time:

• not break existing functionality in any way, in "normal" mode,
• find all references to deprecated code, in "verify" mode.

See Technical guidelines for examples.

Once both parts are addressed, get the PR reviewed and merged to the main branch.

File a different issue, to remove the deprecated code later.

Use [REMOVAL] in the title, and tag with label "Removal".

In the removal issue, refer to the corresponding deprecation issue.

Release deprecation

When making a new opentelemetry-cpp release, in addition to the regular release tasks:

• Replace every occurrences of TO-BE-RELEASED-VERSION in file DEPRECATED with the new release version,
• Replace every occurrences of TO-BE-RELEASED-DATE in file DEPRECATED with the new release date,
• In the CHANGELOG and the release notes, add references to the new deprecated items for this release, pointing to the DEPRECATED document.

Wait

Do not implement the removal right away.

First, if a removal is implemented and merged to main just after a new release, it exposes itself to a full revert, should another release be necessary to fix any regressions just caused recently.

Second, some people will only notice the deprecation when discovering it in the release, no matter how many previous announcements were done. Allow some time for people to raise issues or concerns, especially if there are special usage patterns that were not anticipated.

Once things are stable, proceed with the removal. Depending on the change, it can be implemented:

• in the next minor release,
• after a few minor releases,
• in the next major release,
• after a few major releases

following the deprecation.

The more disruptive the change is, the more time users will need to adjust, so trivial deprecations can be removed right away, while complex ones can take much longer.

In any case, never implement the deprecation and the removal in the same release: wait.

Prepare removal

Implement the removal issue with a PR.

Remove all the deprecated code.

Remove all the tooling (compiling options) related to the deprecated code, if any.

Remove all the relevant entries in DEPRECATED.

Add a CHANGELOG entry for the removal.

Get the removal PR reviewed and merged to the main branch.

Release removal

When making a new opentelemetry-cpp release, in addition to the regular release tasks:

• In the CHANGELOG and the release notes, add references to the new removed items for this release.

Technical guidelines

CMake options deprecation

Assume the option WITH_FOO needs to be deprecated.

Code using WITH_FOO=OFF or WITH_FOO=ON must build as before, yet users should be notified if they use WITH_FOO in their build.

CMake defines a OTELCPP_WITH_NO_DEPRECATED_CODE option, set to OFF by default.

In a normal build, used in production, code is compiled with OTELCPP_WITH_NO_DEPRECATED_CODE=OFF.

In a verification build, code is compiled with OTELCPP_WITH_NO_DEPRECATED_CODE=ON. This verification also defines OPENTELEMETRY_NO_DEPRECATED_CODE, for code level checks.

Implement the following logic in CMake:

  option(WITH_FOO "DEPRECATED - With the foo feature" OFF)

  if(WITH_FOO)
    if(OTELCPP_WITH_NO_DEPRECATED_CODE)
      message(FATAL_ERROR "WITH_FOO is deprecated")
    else()
      message(WARNING "WITH_FOO is deprecated")
    endif()
  endif

The verification build is not meant to be used in production. It is meant to find all references of deprecated items.

If the verification build succeeds, the user code is guaranteed to be clean, and will not be impacted by the removal to come.

When implementing such logic, document it in the mitigation section, in file DEPRECATED.

CMake package component deprecation

The installed opentelemetry-cpp CMake package supports the COMPONENTS argument to find_package(opentelemetry-cpp CONFIG ...). CMake components may need to be deprecated over time.

There are three cases to distinguish:

• a component name is changed,
• a component is merged (with all its targets) into another component,
• a component and its targets are deprecated and set for future removal from the package.

Component name change

For a component name change, keep the CMake target set unchanged and add the deprecated name under the DEPRECATED_NAMES list.

Example:

otel_add_component(
  COMPONENT component_old_name
  TARGETS opentelemetry_some_target)

The new component receives the new name but must retain the targets unchanged.

otel_add_component(
  COMPONENT component_new_name
  DEPRECATED_NAMES component_old_name
  TARGETS opentelemetry_some_target)

When a user requests component_old_name, the installed package config emits a CMake deprecation warning and resolves the request to component_new_name.

# the following will now resolve `component_old_name` to `component_new_name`
find_package(opentelemetry-cpp CONFIG COMPONENTS component_old_name)

Component merge

For a component merge, add component_B to component_A's DEPRECATED_NAMES list and add component_B's targets to component_A's TARGETS list:

Example:

Before the merge, two components are defined.

otel_add_component(
  COMPONENT component_A
  TARGETS component_A_target)

otel_add_component(
  COMPONENT component_B
  TARGETS component_B_target)

After the merge, component_B's definition is removed. Its name and targets are added to the deprecated names and targets for component_A respectively.

otel_add_component(
  COMPONENT component_A
  DEPRECATED_NAMES component_B
  TARGETS
  component_A_target
  component_B_target)

When a user requests component_B, the installed package config emits a CMake deprecation warning and resolves the request to component_A importing the merged set of targets.

# the following will now resolve deprecated `component_B` to `component_A`
# all targets from the merged component are imported and remain unchanged
find_package(opentelemetry-cpp CONFIG COMPONENTS component_B)

target_link_libraries(my_library PRIVATE
        opentelemetry-cpp::component_A_target
        opentelemetry-cpp::component_B_target)

Note: The set of targets provided by the independent components before merge and from the merged component must not change. Changing target names risks breaking the user's CMake configuration. Deprecating and renaming targets remains a TODO and is currently not supported.

Deprecating a component for removal

[TODO] This use case is not fully implemented in otel_add_component

When a component and its targets are set for removal from the package, the component must be marked deprecated with the DEPRECATED tag and remain in the package for a deprecation period. During this time the component and its targets must be installed with the package and be imported by find_package.

Example:

otel_add_component(
  COMPONENT component_to_remove
  DEPRECATED
  TARGETS component_target
  )

# the following will import the deprecated component's targets and emit a
# CMake deprecation warning
find_package(opentelemetry-cpp CONFIG COMPONENTS component_to_remove)

Validate the deprecated component

Add the deprecated component name to the ./ci/do_ci.sh script's cmake.install.test target's DEPRECATED_COMPONENTS variable.

The install test ./install/test/cmake/usage_tests/deprecated_components verifies that requesting deprecated component names with find_package configures successfully and that CMake emits a deprecation warning.

C++ deprecation

C++ header deprecation

When a header is deprecated, it will be removed, so users should no longer include the header.

Add the following code in the header file

#ifdef OPENTELEMETRY_NO_DEPRECATED_CODE
#  error "header <opentelemetry/foo.h> is deprecated."
#endif

macro deprecation

For macros, there are no [[deprecated]] annotations.

Replace the macro with something that is sure to fail at build time, so it gets noticed when used.

#ifndef OPENTELEMETRY_NO_DEPRECATED_CODE
  #define OTEL_MACRO_FOO(X, Y) ... (normal implementation) ...
#else
  #define OTEL_MACRO_FOO(X, Y) { this macro foo is deprecated }
#endif

C++ code deprecation

Assume a C++ item needs to be deprecated.

For example:

  struct some_options
  {
    int x;
    int y; // TODO: deprecate me
    int z;
  };

Code using y must build as before, and yet users should be notified if still using y.

First, there is a way in C++ to flag deprecated code:

  struct some_options
  {
    int x;
    OPENTELEMETRY_DEPRECATED int y; // deprecated, to be removed
    int z;
  };

This will cause deprecation warnings. Some users will notice, but others will not, because warnings are most of the time ignored.

A better solution is to provide a way to forcefully remove the deprecated code:

  struct some_options
  {
    int x;
#ifndef OPENTELEMETRY_NO_DEPRECATED_CODE
    OPENTELEMETRY_DEPRECATED int y; // deprecated, to be removed
#endif
    int z;
  };

A regular build, to use in production, will build with the regular, unchanged, configuration, in which OPENTELEMETRY_NO_DEPRECATED_CODE is not defined.

A verification build, to not use in production, will build with OPENTELEMETRY_NO_DEPRECATED_CODE defined, removing the deprecated member entirely.

A verification build that succeeds is a proof that the application does not use the deprecated member, and needs no change.

A verification build that fails will list all the application code that needs to be fixed before the deprecated code is finally removed.

Even if the verification build succeeds, it really should not be used in production, because the memory layout just changed, breaking the ABI.

This verification build is used to enforce:

• the API (Application Programming Interface) subset is compatible, aka the source code will build before and after the removal.

By the time the removal is implemented:

  struct some_options
  {
    int x;
    int z;
  };

the new release will have:

• a reduced API
• an ABI (Application Binary Interface) change

The reduced API will be:

• a subset of the original API,
• compatible with applications which no longer use the removed code
• incompatible with applications which still uses the removed code

When documenting the deprecation, document this logic in the mitigation section, so that users know how to find and remove old references to deprecated code.

When documenting the removal, clarify in the release notes that the release is API compatible (when application cleanup was done), but not ABI compatible (when sizeof() or virtual tables of classes changed), and therefore a full build must be done again.

This example used a member in a struct, but the same applies to other constructs as well (classes, structs, functions, methods, etc).