Skip to content

Commit 2fa09b7

Browse files
authored
Added metrics for immutable folder usage (#7168)
## Changes Added metrics for immutable folder usage ## Why Capturing usage and how long it takes to upload ## Tests Updated acceptance test golden files <!-- If your PR needs to be included in the release notes for next release, add a changelog fragment: create .nextchanges/<section>/<name>.md with a one-line description (e.g. .nextchanges/cli/quickstart.md). See .nextchanges/README.md. -->
1 parent 9c74505 commit 2fa09b7

19 files changed

Lines changed: 113 additions & 0 deletions

File tree

‎acceptance/bundle/resource_deps/job_tasks/output.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ has_classic_interactive_compute false
1818
has_classic_job_compute false
1919
has_python_wheel_task true
2020
has_serverless_compute true
21+
immutable_snapshot_used false
2122
local.cache.attempt true
2223
local.cache.miss true
2324
permissions_section_set false

‎acceptance/bundle/resource_deps/resources_var/output.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -47,6 +47,7 @@ experimental.use_legacy_run_as false
4747
has_classic_interactive_compute false
4848
has_classic_job_compute false
4949
has_serverless_compute false
50+
immutable_snapshot_used false
5051
local.cache.attempt true
5152
local.cache.hit true
5253
permissions_section_set false

‎acceptance/bundle/resource_deps/tf_path_only_error/out.telemetry.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
experimental.use_legacy_run_as false
22
has_tf_only_references true
3+
immutable_snapshot_used false
34
local.cache.attempt true
45
local.cache.hit true
56
presets_name_prefix_is_set false

‎acceptance/bundle/telemetry/deploy-app-lifecycle-started/output.txt‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -93,6 +93,10 @@ Resources: 1 created, 0 changed, 0 deleted, 0 unchanged
9393
"key": "deployment_history_enabled",
9494
"value": false
9595
},
96+
{
97+
"key": "immutable_snapshot_used",
98+
"value": false
99+
},
96100
{
97101
"key": "app_lifecycle_started",
98102
"value": true

‎acceptance/bundle/telemetry/deploy-compute-type/output.txt‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,8 @@ has_serverless_compute true
3333
has_serverless_compute true
3434
has_spark_python_task true
3535
has_spark_python_task true
36+
immutable_snapshot_used false
37+
immutable_snapshot_used false
3638
local.cache.attempt true
3739
local.cache.attempt true
3840
local.cache.hit true

‎acceptance/bundle/telemetry/deploy-experimental/output.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -20,6 +20,7 @@ has_classic_interactive_compute true
2020
has_classic_job_compute false
2121
has_serverless_compute false
2222
has_spark_python_task true
23+
immutable_snapshot_used false
2324
local.cache.attempt true
2425
local.cache.miss true
2526
permissions_section_set false
Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
bundle:
2+
name: test-bundle
3+
4+
workspace:
5+
sync_mode: immutable
6+
7+
resources:
8+
jobs:
9+
foo:
10+
name: foo
11+
tasks:
12+
- task_key: task1
13+
spark_python_task:
14+
python_file: ./src/main.py
15+
environment_key: env
16+
17+
environments:
18+
- environment_key: env
19+
spec:
20+
environment_version: "4"
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Cloud = false
2+
EnvMatrix.DMS = [""]
Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
2+
=== First deploy uploads the snapshot
3+
4+
>>> [CLI] bundle deploy
5+
Created internal_immutable_snapshots.immutable
6+
Created jobs.foo
7+
Files: 0 uploaded, 0 deleted
8+
Resources: 2 created, 0 changed, 0 deleted, 0 unchanged
9+
10+
>>> print_telemetry_bool_values
11+
immutable_snapshot_used true
12+
13+
>>> print_snapshot_upload_duration_included
14+
true
15+
16+
=== Second deploy finds the snapshot already uploaded
17+
18+
>>> [CLI] bundle deploy
19+
Files: 0 uploaded, 0 deleted
20+
Resources: 0 created, 0 changed, 0 deleted, 2 unchanged
21+
22+
>>> print_telemetry_bool_values
23+
immutable_snapshot_used true
24+
25+
>>> print_snapshot_upload_duration_included
26+
false
Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# The snapshot upload duration is reported next to the files.(upload) mutator time. Its value
2+
# varies, so only assert that it is reported.
3+
print_snapshot_upload_duration_included() {
4+
jq -r 'select(.path? == "/telemetry-ext") | (.body.protoLogs // [])[] | fromjson | (.entry // .) | .databricks_cli_log.bundle_deploy_event.experimental.bundle_mutator_execution_time_ms // [] | any(.key == "snapshot.upload" and .value >= 0)' "$OUT_REQUESTS"
5+
}
6+
7+
title "First deploy uploads the snapshot\n"
8+
trace $CLI bundle deploy
9+
trace print_telemetry_bool_values | grep immutable_snapshot_used
10+
trace print_snapshot_upload_duration_included
11+
rm "$OUT_REQUESTS"
12+
13+
title "Second deploy finds the snapshot already uploaded\n"
14+
trace $CLI bundle deploy
15+
trace print_telemetry_bool_values | grep immutable_snapshot_used
16+
trace print_snapshot_upload_duration_included
17+
rm "$OUT_REQUESTS"

0 commit comments

Comments
 (0)