Skip to content

[BUG] tidb 6 backup failed: unknown service meta_storagepb.MetaStorage #10724

Description

@JashBook

Describe the bug
A clear and concise description of what the bug is.

helm get notes -n kb-system kb-addon-tidb
NOTES:
Grafana dashboards are copied from pingcap repos according to https://docs.pingcap.com/tidb/stable/deploy-monitoring-services#step-2-import-a-grafana-dashboard

Backup/restore only tested in minio and aliyun oss.


Release Information:
  Commit ID: "e5783506ef3a9d3684fcac52a18e730738b1d3d1"
  Commit Time: "2026-07-20 11:14:39 +0800"
  Release Branch: "v1.1.0-beta.8"
  Release Time:  "2026-07-20 17:04:23 +0800"
  Enterprise: "false"

➜  ~ kbcli version
Kubernetes: v1.30.4-vke.4
KubeBlocks: 1.1.0-beta.8
kbcli: 1.0.2

To Reproduce
Steps to reproduce the behavior:

  1. create cluster
apiVersion: apps.kubeblocks.io/v1
kind: Cluster
metadata:
  name: tidb-ncyxag
  namespace: default
spec:
  clusterDef: tidb
  topology: cluster
  terminationPolicy: WipeOut
  componentSpecs:
    - name: tidb
      replicas: 2
      serviceVersion: 6.5.12
      disableExporter: false
      resources:
        requests:
          cpu: 500m
          memory: 3Gi
        limits:
          cpu: 500m
          memory: 3Gi
    - name: tikv
      replicas: 1
      serviceVersion: 6.5.12
      disableExporter: false
      resources:
        requests:
          cpu: 500m
          memory: 3Gi
        limits:
          cpu: 500m
          memory: 3Gi
      persistentVolumeClaimRetentionPolicy:
        whenDeleted: Delete
      volumeClaimTemplates:
        - name: data
          spec:
            storageClassName: 
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
    - name: tidb-pd
      replicas: 2
      serviceVersion: 6.5.12
      disableExporter: false
      resources:
        requests:
          cpu: 500m
          memory: 3Gi
        limits:
          cpu: 500m
          memory: 3Gi
      persistentVolumeClaimRetentionPolicy:
        whenDeleted: Delete
      volumeClaimTemplates:
        - name: data
          spec:
            storageClassName: 
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 20Gi
  1. backup
kbcli cluster backup tidb-ncyxag --method br --namespace default
Backup backup-default-tidb-ncyxag-20260730124021 created successfully, you can view the progress:
	kbcli cluster list-backups --names=backup-default-tidb-ncyxag-20260730124021 -n default

  1. See error
kubectl get backup backup-default-tidb-ncyxag-20260730124021 
NAME                                        POLICY                           METHOD   REPO                    STATUS    TOTAL-SIZE   DURATION   DELETION-POLICY   CREATION-TIME          COMPLETION-TIME   EXPIRATION-TIME
backup-default-tidb-ncyxag-20260730124021   tidb-ncyxag-tidb-backup-policy   br       backuprepo-kbcli-test   Running                           Delete            2026-07-30T04:40:21Z                     
➜  ~ 
➜  ~ kubectl get pod | grep tidb-ncyxag
dp-backup-0-backup-default-tidb-ncyxag-20260730124021-b3848xf5f   0/2     Error      0          5m49s
dp-backup-0-backup-default-tidb-ncyxag-20260730124021-b384wr9lt   2/2     Running    0          92s
dp-backup-0-backup-default-tidb-ncyxag-20260730124021-b384xrrm5   0/2     Error      0          10m
tidb-ncyxag-tidb-0                                                2/2     Running    0          11m
tidb-ncyxag-tidb-1                                                2/2     Running    0          11m
tidb-ncyxag-tidb-pd-0                                             3/3     Running    0          16m
tidb-ncyxag-tidb-pd-1                                             3/3     Running    0          14m
tidb-ncyxag-tikv-0                                                2/2     Running    0          13m

bacckup yaml

kubectl get backup backup-default-tidb-ncyxag-20260730124021 -oyaml
apiVersion: dataprotection.kubeblocks.io/v1alpha1
kind: Backup
metadata:
  annotations:
    kubeblocks.io/cluster-snapshot: '{"metadata":{"name":"tidb-ncyxag","namespace":"default","creationTimestamp":null,"annotations":{"kubeblocks.io/crd-api-version":"apps.kubeblocks.io/v1"}},"spec":{"clusterDef":"tidb","topology":"cluster","terminationPolicy":"WipeOut","componentSpecs":[{"name":"tidb-pd","componentDef":"tidb-pd-7-1.1.0-alpha.0","serviceVersion":"6.5.12","replicas":2,"resources":{"limits":{"cpu":"500m","memory":"3Gi"},"requests":{"cpu":"500m","memory":"3Gi"}},"volumeClaimTemplates":[{"name":"data","spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"20Gi"}}}}],"persistentVolumeClaimRetentionPolicy":{"whenDeleted":"Delete"},"podUpdatePolicy":"PreferInPlace","ordinals":{},"disableExporter":false},{"name":"tikv","componentDef":"tikv-7-1.1.0-alpha.0","serviceVersion":"6.5.12","replicas":1,"resources":{"limits":{"cpu":"500m","memory":"3Gi"},"requests":{"cpu":"500m","memory":"3Gi"}},"volumeClaimTemplates":[{"name":"data","spec":{"accessModes":["ReadWriteOnce"],"resources":{"requests":{"storage":"20Gi"}}}}],"persistentVolumeClaimRetentionPolicy":{"whenDeleted":"Delete"},"podUpdatePolicy":"PreferInPlace","ordinals":{},"disableExporter":false},{"name":"tidb","componentDef":"tidb-7-1.1.0-alpha.0","serviceVersion":"6.5.12","replicas":2,"resources":{"limits":{"cpu":"500m","memory":"3Gi"},"requests":{"cpu":"500m","memory":"3Gi"}},"podUpdatePolicy":"PreferInPlace","ordinals":{},"disableExporter":false}]},"status":{}}'
    kubeblocks.io/encrypted-system-accounts: '{"tidb":{"kbadmin":"o5o39zd1ljJJ0vg1V3gGMNc58kUQZCw9pzX4Xk0wjT4Aot0uAfWCfbmaFVo=","root":"uAs6nnZces+/7+2+ZHhx/P1hL3Ph3+QQRs0xesAr0jZoZR/1cG76oPbCj70="}}'
  creationTimestamp: "2026-07-30T04:40:21Z"
  finalizers:
  - dataprotection.kubeblocks.io/finalizer
  generation: 1
  labels:
    app.kubernetes.io/instance: tidb-ncyxag
    app.kubernetes.io/managed-by: kubeblocks-dataprotection
    apps.kubeblocks.io/component-name: tidb
    dataprotection.kubeblocks.io/backup-policy: tidb-ncyxag-tidb-backup-policy
    dataprotection.kubeblocks.io/backup-repo-name: backuprepo-kbcli-test
    dataprotection.kubeblocks.io/backup-type: Full
    dataprotection.kubeblocks.io/cluster-uid: b21c4ecd-497e-460f-bb5a-23d942b57f0a
    operations.kubeblocks.io/ops-name: backup-default-tidb-ncyxag-20260730124021
    operations.kubeblocks.io/ops-type: Backup
  name: backup-default-tidb-ncyxag-20260730124021
  namespace: default
  resourceVersion: "258136332"
  uid: b384cb34-073b-409e-b915-bc8b11c59f25
spec:
  backupMethod: br
  backupPolicyName: tidb-ncyxag-tidb-backup-policy
  deletionPolicy: Delete
status:
  actions:
  - actionType: Job
    name: dp-backup-0
    objectRef:
      apiVersion: batch/v1
      kind: Job
      name: dp-backup-0-backup-default-tidb-ncyxag-20260730124021-b384cb34
      namespace: default
      resourceVersion: "258136330"
      uid: cb35fd8e-1f8c-409a-8872-bf776b5b99c1
    phase: Running
    startTimestamp: "2026-07-30T04:40:21Z"
    targetPodName: tidb-ncyxag-tidb-0
  backupMethod:
    actionSetName: tidb-br
    env:
    - name: IMAGE_TAG
      value: v7.5.2
    name: br
    snapshotVolumes: false
  backupRepoName: backuprepo-kbcli-test
  formatVersion: 0.1.0
  path: /default/tidb-ncyxag-b21c4ecd-497e-460f-bb5a-23d942b57f0a/tidb/backup-default-tidb-ncyxag-20260730124021
  phase: Running
  startTimestamp: "2026-07-30T04:40:21Z"
  target:
    podSelector:
      matchLabels:
        app.kubernetes.io/instance: tidb-ncyxag
        app.kubernetes.io/managed-by: kubeblocks
        apps.kubeblocks.io/component-name: tidb
      strategy: Any
    selectedTargetPods:
    - tidb-ncyxag-tidb-0

describe pod

kubectl describe pod dp-backup-0-backup-default-tidb-ncyxag-20260730124021-b384xrrm5
Name:             dp-backup-0-backup-default-tidb-ncyxag-20260730124021-b384xrrm5
Namespace:        default
Priority:         0
Service Account:  kubeblocks-dataprotection-worker
Node:             192.168.0.194/192.168.0.194
Start Time:       Thu, 30 Jul 2026 12:40:21 +0800
Labels:           app.kubernetes.io/instance=tidb-ncyxag
                  app.kubernetes.io/managed-by=kubeblocks-dataprotection
                  batch.kubernetes.io/controller-uid=cb35fd8e-1f8c-409a-8872-bf776b5b99c1
                  batch.kubernetes.io/job-name=dp-backup-0-backup-default-tidb-ncyxag-20260730124021-b384cb34
                  controller-uid=cb35fd8e-1f8c-409a-8872-bf776b5b99c1
                  dataprotection.kubeblocks.io/backup-name=backup-default-tidb-ncyxag-20260730124021
                  dataprotection.kubeblocks.io/backup-policy=tidb-ncyxag-tidb-backup-policy
                  dataprotection.kubeblocks.io/backup-repo-name=backuprepo-kbcli-test
                  dataprotection.kubeblocks.io/backup-type=Full
                  dataprotection.kubeblocks.io/cluster-uid=b21c4ecd-497e-460f-bb5a-23d942b57f0a
                  job-name=dp-backup-0-backup-default-tidb-ncyxag-20260730124021-b384cb34
                  operations.kubeblocks.io/ops-name=backup-default-tidb-ncyxag-20260730124021
                  operations.kubeblocks.io/ops-type=Backup
Annotations:      vke.volcengine.com/cello-pod-evict-policy: allow
Status:           Failed
IP:               192.168.0.162
IPs:
  IP:           192.168.0.162
Controlled By:  Job/dp-backup-0-backup-default-tidb-ncyxag-20260730124021-b384cb34
Init Containers:
  dp-copy-datasafed:
    Container ID:  containerd://a43baac857c68bbe7b1d7f6e264d857256f00f4c14f0e70b1d07e87e1d7408e9
    Image:         apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/datasafed:0.2.3
    Image ID:      apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/datasafed@sha256:7775e8184fbc833ee089b33427c4981bd7cd7d98cce5aeff1a9856b5de966b0f
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/sh
      -c
      /scripts/install-datasafed.sh /bin/datasafed
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Thu, 30 Jul 2026 12:40:26 +0800
      Finished:     Thu, 30 Jul 2026 12:40:26 +0800
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:        0
      memory:     0
    Environment:  <none>
    Mounts:
      /bin/datasafed from dp-datasafed-bin (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-7bsfj (ro)
Containers:
  backupdata:
    Container ID:  containerd://f970d59fea5f62c1e85f81e82fc3b5db9b5a28ad1b0ee8e4ac9df9827de1eebd
    Image:         apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/br:v7.5.2
    Image ID:      apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/br@sha256:14b2dd40f10c02f2ce6ba7d854ceb41744a9894aa8a3ec8eafebaa53c1a1e6f4
    Port:          <none>
    Host Port:     <none>
    Command:
      bash
      -c
      #!/bin/bash
      
      set -eo pipefail
      
      function getToolConfigValue() {
        local var=$1
        grep "$var" < "$TOOL_CONFIG" | awk '{print $NF}'
      }
      
      # shellcheck disable=SC2034
      function setStorageVar() {
        TOOL_CONFIG=/etc/datasafed/datasafed.conf
      
        ACCESS_KEY_ID=$(getToolConfigValue access_key_id)
        SECRET_ACCESS_KEY=$(getToolConfigValue secret_access_key)
        ENDPOINT=$(getToolConfigValue endpoint)
        BUCKET=$(getToolConfigValue "root =")
        PROVIDER=$(getToolConfigValue provider)
      
        BR_EXTRA_ARGS=""
        if [[ "$PROVIDER" == "Alibaba" ]]; then
          ENDPOINT="https://${ENDPOINT}"
          BR_EXTRA_ARGS="--s3.provider alibaba"
        fi
      
        export PATH="$PATH:$DP_DATASAFED_BIN_PATH"
        export DATASAFED_BACKEND_BASE_PATH=${DP_BACKUP_BASE_PATH}
      }
      
      # Save backup status info file for syncing progress.
      # timeFormat: %Y-%m-%dT%H:%M:%SZ
      function DP_save_backup_status_info() {
        local totalSize=$1
        local startTime=$2
        local stopTime=$3
        local timeZone=$4
        local extras=$5
        local timeZoneStr=""
        if [ -n "${timeZone}" ]; then
          timeZoneStr=$(printf ',"timeZone":"%s"' "${timeZone}")
        fi
        if [ -z "${stopTime}" ]; then
          printf '{"totalSize":"%s"}' "${totalSize}" > "${DP_BACKUP_INFO_FILE}"
        elif [ -z "${startTime}" ]; then
          printf '{"totalSize":"%s","extras":[%s],"timeRange":{"end":"%s"%s}}' "${totalSize}" "${extras}" "${stopTime}" "${timeZoneStr}" > "${DP_BACKUP_INFO_FILE}"
        else
          printf '{"totalSize":"%s","extras":[%s],"timeRange":{"start":"%s","end":"%s"%s}}' "${totalSize}" "${extras}" "${startTime}" "${stopTime}" "${timeZoneStr}" > "${DP_BACKUP_INFO_FILE}"
        fi
      }
      
      #!/bin/bash
      
      # if the script exits with a non-zero exit code, touch a file to indicate that the backup failed,
      # the sync progress container will check this file and exit if it exists
      function handle_exit() {
        exit_code=$?
        if [ $exit_code -ne 0 ]; then
          echo "failed with exit code $exit_code"
          touch "${DP_BACKUP_INFO_FILE}.exit"
          exit $exit_code
        fi
      }
      
      trap handle_exit EXIT
      setStorageVar
      
      echo "start full backup"
      # shellcheck disable=SC2086
      /br backup full --pd "$PD_ADDRESS" --storage "s3://$BUCKET$DP_BACKUP_BASE_PATH?access-key=$ACCESS_KEY_ID&secret-access-key=$SECRET_ACCESS_KEY" --s3.endpoint "$ENDPOINT" $BR_EXTRA_ARGS
      
      # use datasafed to get backup size
      # if we do not write into $DP_BACKUP_INFO_FILE, the backup job will stuck
      TOTAL_SIZE=$(datasafed stat / | grep TotalSize | awk '{print $2}')
      DP_save_backup_status_info "$TOTAL_SIZE"
      
    State:          Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Thu, 30 Jul 2026 12:40:36 +0800
      Finished:     Thu, 30 Jul 2026 12:44:29 +0800
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:                        0
      memory:                     0
      vke.volcengine.com/eni-ip:  1
    Requests:
      cpu:                        0
      memory:                     0
      vke.volcengine.com/eni-ip:  1
    Environment Variables from:
      tidb-ncyxag-tidb-env  ConfigMap  Optional: false
    Environment:
      TIDB_ROOT_PASSWORD:     <set to the key 'password' in secret 'tidb-ncyxag-tidb-account-root'>  Optional: false
      CURRENT_POD_NAME:       dp-backup-0-backup-default-tidb-ncyxag-20260730124021-b384xrrm5 (v1:metadata.name)
      SERVICE_PORT:           10080
      DP_BACKUP_NAME:         backup-default-tidb-ncyxag-20260730124021
      DP_TARGET_POD_NAME:     tidb-ncyxag-tidb-0
      DP_TARGET_POD_ROLE:     
      DP_BACKUP_BASE_PATH:    /default/tidb-ncyxag-b21c4ecd-497e-460f-bb5a-23d942b57f0a/tidb/backup-default-tidb-ncyxag-20260730124021
      DP_BACKUP_INFO_FILE:    /dp-manager/backup.info
      DP_TTL:                 
      DP_DB_HOST:             tidb-ncyxag-tidb-0.tidb-ncyxag-tidb-headless
      DP_DB_PORT:             4000
      KB_CLUSTER_UID:         b21c4ecd-497e-460f-bb5a-23d942b57f0a
      KB_CLUSTER_NAME:        tidb-ncyxag
      KB_COMP_NAME:           tidb
      KB_NAMESPACE:           default
      IMAGE_TAG:              v7.5.2
      DP_DATASAFED_BIN_PATH:  /bin/datasafed
    Mounts:
      /bin/datasafed from dp-datasafed-bin (rw)
      /dp-manager from manager-shared-volume (rw)
      /etc/datasafed from dp-datasafed-config (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-7bsfj (ro)
  manager:
    Container ID:  containerd://2e5a200ce42ef499a00e7fa6f9c45b64d5ca8a02dc256d890072d4774f023051
    Image:         apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:1.1.0-beta.8
    Image ID:      apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools@sha256:04defca698854665909ce7a411eb1b4da805afaefcbbbdf1ad89ec65c9755d2c
    Port:          <none>
    Host Port:     <none>
    Command:
      sh
      -c
    Args:
      
      set -o errexit
      set -o nounset
      
      export PATH="$PATH:$DP_DATASAFED_BIN_PATH"
      export DATASAFED_BACKEND_BASE_PATH="$DP_BACKUP_BASE_PATH"
      
      backup_info_file="${DP_BACKUP_INFO_FILE}"
      sleep_seconds="${DP_CHECK_INTERVAL}"
      namespace="default"
      backup_name="backup-default-tidb-ncyxag-20260730124021"
      
      if [ "$sleep_seconds" -le 0 ]; then
        sleep_seconds=30
      fi
      
      exit_file="${backup_info_file}.exit"
      while true; do
        if [ -f "$exit_file" ]; then
          echo "exit file $exit_file exists, exit"
          exit 1
        fi
        if [ -f "$backup_info_file" ]; then
          break
        fi
        echo "backup info file not exists, wait for ${sleep_seconds}s"
        sleep "$sleep_seconds"
      done
      
      backup_info=$(cat "$backup_info_file")
      echo "backupInfo:${backup_info}"
      
      status="{\"status\":${backup_info}}"
      kubectl -n "$namespace" patch backups.dataprotection.kubeblocks.io "$backup_name" --subresource=status --type=merge --patch "${status}"
      
      # save the backup CR object to the backup repo
      kubectl -n "$namespace" get backups.dataprotection.kubeblocks.io "$backup_name" -o json | datasafed push - "/kubeblocks-backup.json"
      
    State:          Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Thu, 30 Jul 2026 12:40:36 +0800
      Finished:     Thu, 30 Jul 2026 12:44:31 +0800
    Ready:          False
    Restart Count:  0
    Limits:
      cpu:     0
      memory:  0
    Requests:
      cpu:     0
      memory:  0
    Environment Variables from:
      tidb-ncyxag-tidb-env  ConfigMap  Optional: false
    Environment:
      TIDB_ROOT_PASSWORD:     <set to the key 'password' in secret 'tidb-ncyxag-tidb-account-root'>  Optional: false
      CURRENT_POD_NAME:       dp-backup-0-backup-default-tidb-ncyxag-20260730124021-b384xrrm5 (v1:metadata.name)
      SERVICE_PORT:           10080
      DP_BACKUP_NAME:         backup-default-tidb-ncyxag-20260730124021
      DP_TARGET_POD_NAME:     tidb-ncyxag-tidb-0
      DP_TARGET_POD_ROLE:     
      DP_BACKUP_BASE_PATH:    /default/tidb-ncyxag-b21c4ecd-497e-460f-bb5a-23d942b57f0a/tidb/backup-default-tidb-ncyxag-20260730124021
      DP_BACKUP_INFO_FILE:    /dp-manager/backup.info
      DP_TTL:                 
      DP_DB_HOST:             tidb-ncyxag-tidb-0.tidb-ncyxag-tidb-headless
      DP_DB_PORT:             4000
      KB_CLUSTER_UID:         b21c4ecd-497e-460f-bb5a-23d942b57f0a
      KB_CLUSTER_NAME:        tidb-ncyxag
      KB_COMP_NAME:           tidb
      KB_NAMESPACE:           default
      IMAGE_TAG:              v7.5.2
      DP_DATASAFED_BIN_PATH:  /bin/datasafed
      DP_CHECK_INTERVAL:      5
    Mounts:
      /bin/datasafed from dp-datasafed-bin (rw)
      /dp-manager from manager-shared-volume (rw)
      /etc/datasafed from dp-datasafed-config (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-7bsfj (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   False 
  Initialized                 True 
  Ready                       False 
  ContainersReady             False 
  PodScheduled                True 
Volumes:
  manager-shared-volume:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  dp-datasafed-config:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  tool-config-backuprepo-kbcli-test-bnfj2s
    Optional:    false
  dp-datasafed-bin:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  kube-api-access-7bsfj:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 kb-controller=true:NoSchedule
                             node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  10m   default-scheduler  Successfully assigned default/dp-backup-0-backup-default-tidb-ncyxag-20260730124021-b384xrrm5 to 192.168.0.194
  Normal  Pulling    10m   kubelet            Pulling image "apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/datasafed:0.2.3"
  Normal  Pulled     10m   kubelet            Successfully pulled image "apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/datasafed:0.2.3" in 3.238s (3.238s including waiting). Image size: 26256737 bytes.
  Normal  Created    10m   kubelet            Created container dp-copy-datasafed
  Normal  Started    10m   kubelet            Started container dp-copy-datasafed
  Normal  Pulling    10m   kubelet            Pulling image "apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/br:v7.5.2"
  Normal  Pulled     10m   kubelet            Successfully pulled image "apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/br:v7.5.2" in 9.182s (9.182s including waiting). Image size: 158057063 bytes.
  Normal  Created    10m   kubelet            Created container backupdata
  Normal  Started    10m   kubelet            Started container backupdata
  Normal  Pulled     10m   kubelet            Container image "apecloud-registry.cn-zhangjiakou.cr.aliyuncs.com/apecloud/kubeblocks-tools:1.1.0-beta.8" already present on machine
  Normal  Created    10m   kubelet            Created container manager
  Normal  Started    10m   kubelet            Started container manager

logs error pod

kubectl logs dp-backup-0-backup-default-tidb-ncyxag-20260730124021-b384xrrm5
Defaulted container "backupdata" out of: backupdata, manager, dp-copy-datasafed (init)
start full backup
Detail BR log in /tmp/br.log.2026-07-30T04.40.36Z 
[2026/07/30 04:44:29.052 +00:00] [INFO] [collector.go:77] ["Full Backup failed summary"] [total-ranges=0] [ranges-succeed=0] [ranges-failed=0]
Error: rpc error: code = Unimplemented desc = unknown service meta_storagepb.MetaStorage
failed with exit code 1

Expected behavior
A clear and concise description of what you expected to happen.

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: [e.g. iOS]
  • Browser [e.g. chrome, safari]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

Labels

kind/bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions