Skip to content

Commit 61f70d1

Browse files
committed
Enhance cache stats, add DB helpers & benches
Several improvements across the codebase: - Query/Prepared-statement cache: add atomic counters and fast-path flags (hits, misses, entries, size_bytes, evictions, invalidations), track approximate serialized size per entry, update eviction/invalidation accounting, optimize common read paths, and expose snapshot/reset semantics. Add tests covering replacements, reset semantics and clearing behavior. - Benchmarks: add end-to-end query-cache and uncached-concurrency benchmarks (BenchCacheUser model + new benchmark functions), register them in the criterion group, and minor bench formatting fixes. - Database API: add convenience async helpers (raw_with_params, execute_with_params, raw_json_with_params) and internal __* variants that operate on a given Database instance to run raw SQL with typed params. - Error context: box ErrorContext in Error variants, add rendered conditions and operator_chain fields and helper builders, and improve formatting and tests. - Docs/manifest: clarify that attachments/translations/fulltext are compile-time feature gates (do not add deps) in Cargo.toml and README; add a sqlite CI smoke test to Cargo.toml. - Misc: small fixes and formatting in attachments, fulltext query formatting and parameter pushes, CRUD benchmark formatting, and move config tests to an external testing module. These changes improve cache observability, concurrency behavior in benchmarks, and convenience for raw SQL usage while keeping feature flags and docs clearer.
1 parent abf0e9d commit 61f70d1

35 files changed

Lines changed: 2881 additions & 734 deletions

Cargo.toml

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,8 @@ mysql = ["sea-orm/sqlx-mysql"]
2222
sqlite = ["sea-orm/sqlx-sqlite"]
2323
runtime-tokio = ["sea-orm/runtime-tokio-rustls", "dep:tokio"]
2424
runtime-async-std = ["sea-orm/runtime-async-std-rustls"]
25+
# These module flags intentionally gate TideORM's public API surface via #[cfg(feature = ...)]
26+
# and feature-specific tests/benches only. They do not enable extra dependencies.
2527
attachments = []
2628
translations = []
2729
fulltext = []
@@ -103,3 +105,8 @@ harness = false
103105
[[bench]]
104106
name = "tokenization_benchmarks"
105107
harness = false
108+
109+
[[test]]
110+
name = "sqlite_ci_smoke_test"
111+
path = "tests/sqlite_ci_smoke_test.rs"
112+
required-features = ["sqlite", "runtime-tokio"]

README.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -192,15 +192,15 @@ tideorm = { version = "0.8.0", features = ["postgres", "fulltext"] }
192192
| `sqlite` | SQLite support |
193193
| `runtime-tokio` | Tokio runtime (default) |
194194
| `runtime-async-std` | async-std runtime |
195-
| `attachments` | Enables the attachments API and attachment-specific benchmarks/tests |
196-
| `translations` | Enables the translations API and translation-specific benchmarks/tests |
197-
| `fulltext` | Enables the full-text search API and fulltext-specific benchmarks/tests |
195+
| `attachments` | Compile-time-only feature gate for the attachments API and attachment-specific benchmarks/tests; adds no extra dependencies |
196+
| `translations` | Compile-time-only feature gate for the translations API and translation-specific benchmarks/tests; adds no extra dependencies |
197+
| `fulltext` | Compile-time-only feature gate for the full-text search API and fulltext-specific benchmarks/tests; adds no extra dependencies |
198198

199-
Attachments are opt-in. Enable the `attachments` feature when you want to use `tideorm::attachments`, `HasAttachments`, or attachment URL generation helpers.
199+
Attachments are opt-in. Enable the `attachments` feature when you want to use `tideorm::attachments`, `HasAttachments`, or attachment URL generation helpers. This is a compile-time API gate only; it does not pull in additional crates.
200200

201-
Translations are opt-in. Enable the `translations` feature when you want to use `tideorm::translations`, `HasTranslations`, or `ApplyTranslations`.
201+
Translations are opt-in. Enable the `translations` feature when you want to use `tideorm::translations`, `HasTranslations`, or `ApplyTranslations`. This is a compile-time API gate only; it does not pull in additional crates.
202202

203-
Full-text search is opt-in. Enable the `fulltext` feature when you want to use `tideorm::fulltext`, `FullTextSearch`, or the highlighting helpers.
203+
Full-text search is opt-in. Enable the `fulltext` feature when you want to use `tideorm::fulltext`, `FullTextSearch`, or the highlighting helpers. This is a compile-time API gate only; it does not pull in additional crates.
204204

205205
## Relation Types
206206

@@ -241,7 +241,7 @@ For runnable applications and broader demos, see **[tideorm-examples](https://gi
241241

242242
## Testing
243243

244-
TideORM ships with unit, integration, and feature-specific test coverage. The repository CI runs `cargo check` plus `cargo test --lib` across PostgreSQL, MySQL, and SQLite feature sets.
244+
TideORM ships with unit, integration, and feature-specific test coverage. The repository CI runs cargo fmt, cargo clippy, cargo check, cargo test --lib across PostgreSQL, MySQL, and SQLite feature sets, plus a SQLite end-to-end smoke test.
245245

246246
Common local commands:
247247

benches/attachments_translations_benchmarks.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -301,7 +301,7 @@ fn bench_files_data_serialization(c: &mut Criterion) {
301301
{"key": "img3.jpg", "filename": "img3.jpg", "created_at": "2024-01-01T00:00:00Z"},
302302
]
303303
});
304-
304+
305305
b.iter(|| {
306306
black_box(FilesData::from_json(&json))
307307
});
@@ -435,7 +435,7 @@ fn bench_translation_input(c: &mut Criterion) {
435435
"meta_title": {"en": "Title", "ar": "عنوان", "fr": "Titre", "es": "Título", "de": "Titel"},
436436
"meta_desc": {"en": "Meta", "ar": "ميتا", "fr": "Méta", "es": "Meta", "de": "Meta"}
437437
});
438-
438+
439439
b.iter(|| {
440440
black_box(TranslationInput::from_json(&json).unwrap())
441441
});

benches/cache_benchmarks.rs

Lines changed: 229 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,23 @@
66
77
use criterion::{BenchmarkId, Criterion, Throughput, criterion_group, criterion_main};
88
use std::hint::black_box;
9+
use std::sync::Arc;
10+
use std::thread;
11+
use std::time::Duration;
12+
use tideorm::Database;
913
use tideorm::cache::{CacheKeyBuilder, CacheStrategy, PreparedStatementCache, QueryCache};
14+
use tideorm::internal::{ActiveModelTrait, ConnectionTrait, InternalModel};
15+
use tideorm::prelude::*;
16+
17+
#[derive(Model, PartialEq)]
18+
#[tideorm(table = "bench_cache_users")]
19+
struct BenchCacheUser {
20+
#[tideorm(primary_key, auto_increment)]
21+
id: i64,
22+
email: String,
23+
name: String,
24+
active: bool,
25+
}
1026

1127
// =============================================================================
1228
// QUERY CACHE BENCHMARKS
@@ -448,6 +464,217 @@ fn benchmark_cache_with_serialization(c: &mut Criterion) {
448464
group.finish();
449465
}
450466

467+
fn benchmark_end_to_end_query_cache_paths(c: &mut Criterion) {
468+
let rt = tokio::runtime::Runtime::new().expect("failed to create tokio runtime");
469+
let db = rt
470+
.block_on(Database::connect("sqlite::memory:"))
471+
.expect("failed to connect to benchmark sqlite database");
472+
473+
rt.block_on(async {
474+
db.__internal_connection()
475+
.execute_unprepared(
476+
r#"
477+
CREATE TABLE bench_cache_users (
478+
id INTEGER PRIMARY KEY AUTOINCREMENT,
479+
email TEXT NOT NULL,
480+
name TEXT NOT NULL,
481+
active INTEGER NOT NULL DEFAULT 1
482+
)
483+
"#,
484+
)
485+
.await
486+
.expect("failed to create benchmark table");
487+
488+
for i in 0..100 {
489+
BenchCacheUser {
490+
id: 0,
491+
email: format!("user_{i}@example.com"),
492+
name: format!("User {i}"),
493+
active: i % 2 == 0,
494+
}
495+
.into_active_model()
496+
.insert(db.__internal_connection())
497+
.await
498+
.expect("failed to seed benchmark row");
499+
}
500+
});
501+
502+
let cache = QueryCache::global();
503+
cache.disable();
504+
cache.clear();
505+
cache.reset_stats();
506+
507+
let mut group = c.benchmark_group("end_to_end_query_cache");
508+
509+
group.bench_function("uncached_cache_disabled", |b| {
510+
b.to_async(&rt).iter(|| async {
511+
let results = BenchCacheUser::query_with(&db)
512+
.where_eq("email", "user_42@example.com")
513+
.get()
514+
.await
515+
.expect("uncached query should succeed");
516+
black_box(results)
517+
});
518+
});
519+
520+
cache.enable();
521+
cache.clear();
522+
cache.reset_stats();
523+
524+
group.bench_function("uncached_cache_enabled", |b| {
525+
b.to_async(&rt).iter(|| async {
526+
let results = BenchCacheUser::query_with(&db)
527+
.where_eq("email", "user_42@example.com")
528+
.get()
529+
.await
530+
.expect("uncached query should succeed");
531+
black_box(results)
532+
});
533+
});
534+
535+
cache.clear();
536+
cache.reset_stats();
537+
538+
group.bench_function("cached_query_enabled", |b| {
539+
b.to_async(&rt).iter(|| async {
540+
let results = BenchCacheUser::query_with(&db)
541+
.where_eq("email", "user_42@example.com")
542+
.cache(Duration::from_secs(60))
543+
.get()
544+
.await
545+
.expect("cached query should succeed");
546+
black_box(results)
547+
});
548+
});
549+
550+
cache.disable();
551+
cache.clear();
552+
cache.reset_stats();
553+
group.finish();
554+
}
555+
556+
fn benchmark_uncached_query_concurrency(c: &mut Criterion) {
557+
let db_url = "sqlite://target/bench_cache_concurrency.db?mode=rwc";
558+
let rt = tokio::runtime::Runtime::new().expect("failed to create tokio runtime");
559+
let db = rt
560+
.block_on(Database::connect(db_url))
561+
.expect("failed to connect to concurrency benchmark sqlite database");
562+
563+
rt.block_on(async {
564+
db.__internal_connection()
565+
.execute_unprepared("DROP TABLE IF EXISTS bench_cache_users")
566+
.await
567+
.expect("failed to drop benchmark table");
568+
db.__internal_connection()
569+
.execute_unprepared(
570+
r#"
571+
CREATE TABLE bench_cache_users (
572+
id INTEGER PRIMARY KEY AUTOINCREMENT,
573+
email TEXT NOT NULL,
574+
name TEXT NOT NULL,
575+
active INTEGER NOT NULL DEFAULT 1
576+
)
577+
"#,
578+
)
579+
.await
580+
.expect("failed to create benchmark table");
581+
582+
for i in 0..200 {
583+
BenchCacheUser {
584+
id: 0,
585+
email: format!("concurrent_{i}@example.com"),
586+
name: format!("Concurrent User {i}"),
587+
active: i % 2 == 0,
588+
}
589+
.into_active_model()
590+
.insert(db.__internal_connection())
591+
.await
592+
.expect("failed to seed concurrency benchmark row");
593+
}
594+
});
595+
596+
let db = Arc::new(db);
597+
let cache = QueryCache::global();
598+
let mut group = c.benchmark_group("uncached_query_concurrency");
599+
let threads = 4;
600+
let queries_per_thread = 50;
601+
let total_queries = (threads * queries_per_thread) as u64;
602+
group.throughput(Throughput::Elements(total_queries));
603+
604+
cache.disable();
605+
cache.clear();
606+
cache.reset_stats();
607+
608+
group.bench_function("cache_disabled", |b| {
609+
b.iter(|| {
610+
let handles: Vec<_> = (0..threads)
611+
.map(|thread_id| {
612+
let db = Arc::clone(&db);
613+
thread::spawn(move || {
614+
let rt = tokio::runtime::Runtime::new()
615+
.expect("failed to create per-thread runtime");
616+
rt.block_on(async move {
617+
for query_id in 0..queries_per_thread {
618+
let user_index = (thread_id * queries_per_thread + query_id) % 200;
619+
let email = format!("concurrent_{user_index}@example.com");
620+
let results = BenchCacheUser::query_with(db.as_ref())
621+
.where_eq("email", email)
622+
.get()
623+
.await
624+
.expect("uncached concurrent query should succeed");
625+
black_box(results);
626+
}
627+
});
628+
})
629+
})
630+
.collect();
631+
632+
for handle in handles {
633+
handle.join().expect("benchmark thread should join");
634+
}
635+
});
636+
});
637+
638+
cache.enable();
639+
cache.clear();
640+
cache.reset_stats();
641+
642+
group.bench_function("cache_enabled_opt_out", |b| {
643+
b.iter(|| {
644+
let handles: Vec<_> = (0..threads)
645+
.map(|thread_id| {
646+
let db = Arc::clone(&db);
647+
thread::spawn(move || {
648+
let rt = tokio::runtime::Runtime::new()
649+
.expect("failed to create per-thread runtime");
650+
rt.block_on(async move {
651+
for query_id in 0..queries_per_thread {
652+
let user_index = (thread_id * queries_per_thread + query_id) % 200;
653+
let email = format!("concurrent_{user_index}@example.com");
654+
let results = BenchCacheUser::query_with(db.as_ref())
655+
.where_eq("email", email)
656+
.get()
657+
.await
658+
.expect("uncached concurrent query should succeed");
659+
black_box(results);
660+
}
661+
});
662+
})
663+
})
664+
.collect();
665+
666+
for handle in handles {
667+
handle.join().expect("benchmark thread should join");
668+
}
669+
});
670+
});
671+
672+
cache.disable();
673+
cache.clear();
674+
cache.reset_stats();
675+
group.finish();
676+
}
677+
451678
// =============================================================================
452679
// CRITERION GROUPS
453680
// =============================================================================
@@ -473,6 +700,8 @@ criterion_group!(
473700
complex_benches,
474701
benchmark_realistic_workload,
475702
benchmark_cache_with_serialization,
703+
benchmark_end_to_end_query_cache_paths,
704+
benchmark_uncached_query_concurrency,
476705
);
477706

478707
criterion_main!(

benches/crud_benchmarks.rs

Lines changed: 4 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -135,7 +135,8 @@ fn bench_single_insert(c: &mut Criterion) {
135135
b.iter(|| {
136136
let unique_id = COUNTER.fetch_add(1, Ordering::SeqCst);
137137
rt.block_on(async {
138-
let user = BenchUser::new(format!("bench_{unique_id}@example.com"), "Benchmark User");
138+
let user =
139+
BenchUser::new(format!("bench_{unique_id}@example.com"), "Benchmark User");
139140
user.save().await.expect("Insert failed")
140141
})
141142
});
@@ -191,11 +192,8 @@ fn bench_find_by_id(c: &mut Criterion) {
191192
let user_ids: Vec<i64> = rt.block_on(async {
192193
let mut ids = Vec::new();
193194
for i in 0..100 {
194-
let user = BenchUser::new(
195-
format!("find_{i}@example.com"),
196-
format!("Find User {i}"),
197-
)
198-
.with_age(25 + (i % 30));
195+
let user = BenchUser::new(format!("find_{i}@example.com"), format!("Find User {i}"))
196+
.with_age(25 + (i % 30));
199197
let saved = user.save().await.expect("Insert failed");
200198
ids.push(saved.id);
201199
}

src/attachments.rs

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -434,8 +434,8 @@ pub trait HasAttachments {
434434
if Self::is_has_one_relation(relation) {
435435
if attachments.is_empty() {
436436
files.remove_one(relation);
437-
} else if let Some(first) = attachments.into_iter().next() {
438-
files.set_one(relation, first);
437+
} else if let Some(first) = attachments.into_iter().next() {
438+
files.set_one(relation, first);
439439
}
440440
} else {
441441
files.clear_many(relation);

0 commit comments

Comments
 (0)