Skip to content

Commit c8d4223

Browse files
fix(testkit): reject #[should_panic] on testkit tests
A testkit test panics on any failure, including an unreachable gateway or a failed cleanup, so #[should_panic] would let such a failure pass. Reject it and point to checking the expected failure in the body. Signed-off-by: Matthew Grossman <mgrossman@nvidia.com>
1 parent af674ad commit c8d4223

2 files changed

Lines changed: 29 additions & 2 deletions

File tree

‎tests/testkit/README.md‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,10 @@ async fn long_running_exec_delete(cx: &mut Context) -> testkit::Result {
3232
- **The doc comment is required.** It is the test's description.
3333
- **The function name is the nextest test name.**
3434
- **Other attributes apply to the test.** For example, `#[ignore = "reason"]`
35-
skips it unless run with `--run-ignored`.
35+
skips it unless run with `--run-ignored`. `#[should_panic]` is rejected,
36+
because a testkit test panics on any failure. To check that something is
37+
refused, assert the failure in the body, for example with
38+
`cx.cli([...]).await?.failure()?` and `testkit::ensure!`.
3639

3740
Cleanup runs after every test, whether it passes, fails, or panics. It only
3841
deletes sandboxes created through `cx.sandbox(...)`. Delete anything else a

‎tests/testkit/testkit-macros/src/lib.rs‎

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,9 @@
1616
//! The attribute expands to a `#[tokio::test]` wrapper with the same name, so
1717
//! the nextest test name is the function name. The original function is nested
1818
//! in the wrapper and passed to `testkit::run`. Other attributes, such as
19-
//! `#[ignore]` or `#[cfg(...)]`, move to the wrapper.
19+
//! `#[ignore]` or `#[cfg(...)]`, move to the wrapper. `#[should_panic]` is
20+
//! rejected: a testkit test panics on any failure, so it would pass for the
21+
//! wrong reason. Check the expected failure in the body instead.
2022
2123
use proc_macro::TokenStream;
2224
use proc_macro2::TokenStream as TokenStream2;
@@ -56,6 +58,18 @@ fn expand(attr: TokenStream2, item: TokenStream2) -> syn::Result<TokenStream2> {
5658
"`#[testkit::test]` already makes this a test; remove this attribute",
5759
));
5860
}
61+
if let Some(attr) = body
62+
.attrs
63+
.iter()
64+
.find(|attr| attr.path().is_ident("should_panic"))
65+
{
66+
return Err(syn::Error::new_spanned(
67+
attr,
68+
"testkit tests panic on any failure, so `#[should_panic]` would pass for the \
69+
wrong reason; check the expected failure in the body, for example with \
70+
`CliOutput::failure` and `testkit::ensure!`",
71+
));
72+
}
5973
let description = doc_comment(&body.attrs);
6074
if description.is_empty() {
6175
return Err(syn::Error::new(
@@ -200,6 +214,16 @@ mod tests {
200214
}
201215
}
202216

217+
#[test]
218+
fn rejects_should_panic() {
219+
let err = expand_str(
220+
"",
221+
"/// Doc.\n#[should_panic(expected = \"denied\")]\nasync fn t(cx: &mut Context) -> testkit::Result { Ok(()) }",
222+
)
223+
.unwrap_err();
224+
assert!(err.to_string().contains("`#[should_panic]`"), "{err}");
225+
}
226+
203227
#[test]
204228
fn rejects_arguments() {
205229
let err = expand_str(

0 commit comments

Comments
 (0)