Skip to content

Commit 270b2d0

Browse files
denikIsaac
andauthored
acc: split vector search tests between direct and DMS (#7176)
## Changes Each vector search test now runs a single DMS variant instead of both: `EnvMatrix.DMS = [""]` in `vector_search_indexes/test.toml` and `vector_search_endpoints/test.toml`, overridden to `["true"]` in 8 of the 16 tests. Of the 6 tests that run on cloud, 3 use each variant (endpoints: `basic` direct, `drift/recreated_same_name` DMS; indexes: `basic` and `recreate/embedding_dimension` direct, `grants/select` and `schema_normalization` DMS). ## Why Most of the cloud time in these tests is endpoint/index provisioning (e.g. `recreate/embedding_dimension` spends ~20-25 min on AWS/Azure in the initial deploy; the recreate itself takes ~10s). Running every test under both variants doubled that cost; splitting keeps both paths covered for each resource type. ## Tests Regenerated `out.test.toml` files; all 16 pass locally against the testserver. This pull request and its description were written by Isaac. --------- Co-authored-by: Isaac <no-reply@databricks.com>
1 parent c6bb214 commit 270b2d0

26 files changed

Lines changed: 50 additions & 16 deletions

File tree

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
Cloud = true
2-
EnvMatrix.DMS = ["", "true"]
2+
EnvMatrix.DMS = [""]
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
Cloud = false
2-
EnvMatrix.DMS = ["", "true"]
2+
EnvMatrix.DMS = ["true"]
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,5 @@
11
RecordRequests = false
22
Cloud = false
3+
4+
# DMS=true half of the split; see the vector_search_* test.toml.
5+
EnvMatrix.DMS = ["true"]
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
Cloud = true
2-
EnvMatrix.DMS = ["", "true"]
2+
EnvMatrix.DMS = ["true"]
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,6 @@
11
Badness = "After deleting and recreating a vector search endpoint remotely with the same name but a different UUID, bundle plan/deploy treats it as unchanged instead of deleting and recreating it."
22

33
RecordRequests = false
4+
5+
# DMS=true half of the split; see the vector_search_* test.toml.
6+
EnvMatrix.DMS = ["true"]
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
Cloud = false
2-
EnvMatrix.DMS = ["", "true"]
2+
EnvMatrix.DMS = [""]
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
Cloud = false
2-
EnvMatrix.DMS = ["", "true"]
2+
EnvMatrix.DMS = [""]
Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,2 @@
11
Cloud = false
2-
EnvMatrix.DMS = ["", "true"]
2+
EnvMatrix.DMS = ["true"]
Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,4 @@
11
Cloud = false
2+
3+
# DMS=true half of the split; see the vector_search_* test.toml.
4+
EnvMatrix.DMS = ["true"]

‎acceptance/bundle/resources/vector_search_endpoints/test.toml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,3 +6,8 @@ Ignore = [
66
"databricks.yml",
77
".databricks",
88
]
9+
10+
# Endpoint provisioning takes ~20 min on cloud, so running every test under both DMS variants doubles
11+
# the cost. Each test runs one variant instead: direct by default, DMS=true for half of
12+
# them (set in their own test.toml), so both paths stay covered.
13+
EnvMatrix.DMS = [""]

0 commit comments

Comments
 (0)