@@ -20,15 +20,33 @@ info:
2020 description : |
2121 The metric compares the number of cell type specific marker expression peaks between unintegrated and batch integrated data.
2222 The number of peaks is calculated using the `scipy.signal.find_peaks` function.
23- The metric is calculated as the absolute difference between the number of peaks in the unintegrated and batch-normalized data.
2423 The (cell type) marker expression profiles are first smoothed using kernel density estimation (KDE) (`scipy.stats.gaussian_kde`),
25- and then peaks are then identified using the `scipy.signal.find_peaks` function.
24+ and then peaks are identified using the `scipy.signal.find_peaks` function.
2625 For peak calling, the `prominence` parameter is set to 0.1 and the `height` parameter is set to 0.05*max_density.
27- Ratio of inconsistent peaks is defined as number of cases where the number of peaks differ between the two splits in the batch
28- normalized data divided by the total number of cases.
29- Cases where there are different number of peaks between the two splits in the unintegrated data are ignored from the denominator.
30- A lower score indicates better performance, means there are less cases with inconsistent peaks after batch integration.
3126
27+ Case Definitions:
28+ - Case 1: Consistent peaks in both unintegrated AND integrated data (ideal outcome).
29+ Example: A marker shows 2 peaks for both splits in unintegrated AND integrated data.
30+ - Case 2: Inconsistent peaks in unintegrated data, consistent in integrated data (excluded from metric).
31+ Example: A marker shows 1 peaks in unintegrated split 1 but 3 peaks in unintegrated split 2,
32+ and the same for integrated (1 peak in split 1 and 3 peaks in split 2).
33+ Excluded from calculation because there are differences in unintegrated that cannot be accounted for.
34+ - Case 3: Consistent peaks in unintegrated data but inconsistent in integrated data (problematic - batch integration introduced inconsistency).
35+ Example: A marker shows 2 peaks in both unintegrated splits, but after integration shows 2 peaks in split 1 and 1 peak in split 2.
36+ Batch correction broke the consistency.
37+ - Case 4: Inconsistent peaks in both unintegrated AND integrated data (excluded from metric).
38+ Example: A marker is inconsistent in unintegrated data (2 and 3 peaks in split 1 and split 2) and
39+ remains inconsistent after integration (1 and 2 peaks in split 1 and split 2).
40+ Excluded from calculation because there are differences in unintegrated that cannot be accounted for.
41+
42+ Ratio of inconsistent peaks is defined as the number of Case 3 occurrences divided by the total of Case 1 and Case 3 occurrences.
43+ Cases 2 and 4 (where unintegrated data has inconsistent peaks) are excluded from the denominator, but is reported
44+ in the final anndata output for diagnostic.
45+ A lower score indicates better performance, meaning there are fewer cases with inconsistent peaks after batch integration.
46+
47+ For edge cases where methods return only zero values for a given marker/donor/cell type, but is not the
48+ case in the unintegrated data, it is automatically assigned as Case 3.
49+
3250 references :
3351 doi :
3452 - 10.1038/s41592-019-0686-2
0 commit comments