Repository navigation
Commit 91e3121
fix(pipeline): take the stage lock before the stage file is visible (close DeusData#2111 create→register race)
create_staging_path() minted the stage's main file with mkstemp and only then
took its sidecar .lock via stage_owner_register(). A second, concurrent
cbm_pipeline_run() against the same final_path (auto_index, or the
stale-rendezvous-recovery retry path) could land its sweep_orphan_stages() in
that window: the main file was visible but not yet locked, so the sweep treated
it like a genuinely orphaned stage and deleted it out from under the live writer
(POSIX), or collided on the not-yet-created sidecar and surfaced EACCES/errno=13
(Windows). That is the DeusData#2111 red on test-windows-guards and both test-windows
CLANG64 x86_64 shards (index_format / simhash / index_resilience, with
pipeline.stage action=lock_failed errno=13).
Fix: mint the six-char alphanumeric suffix in create_staging_path() itself,
create and hold the sidecar lock FIRST, then create the main file with O_EXCL.
The stage's main file is therefore never present on disk without its lock
already held, so a racing sweep can only ever find the stage lock-held: it
reaches the stage through the .lock entry too (stage_entry_stage_length()
matches it), probes the lock, sees a live holder, and keeps it. The
create->register TOCTOU is closed by construction.
DeusData#1839 is preserved: sweep_one_stage() still removes a stage whose sidecar is
absent (ENOENT). With live stages now always carrying their sidecar, ENOENT is
only ever a genuine pre-lock-era orphan minted by an older binary, which DeusData#1839
requires be swept.
stage_owner_register() becomes stage_owner_adopt(), recording the already-held
descriptor; re-taking the lock would self-conflict, since flock() and Windows
_SH_DENYRW deny a second acquire of the same sidecar even from this process.
A new after_stage_created test seam and pipeline_concurrent_sweep_must_not_-
remove_inflight_stage drive a real concurrent sweep in that window; the test is
RED without the lock-first ordering.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>1 parent 7b4eb4a commit 91e3121
3 files changed
Lines changed: 258 additions & 28 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
37 | 37 | | |
38 | 38 | | |
39 | 39 | | |
| 40 | + | |
40 | 41 | | |
41 | 42 | | |
42 | 43 | | |
| |||
73 | 74 | | |
74 | 75 | | |
75 | 76 | | |
| 77 | + | |
| 78 | + | |
76 | 79 | | |
77 | 80 | | |
78 | 81 | | |
| |||
106 | 109 | | |
107 | 110 | | |
108 | 111 | | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
109 | 135 | | |
110 | 136 | | |
111 | 137 | | |
| |||
125 | 151 | | |
126 | 152 | | |
127 | 153 | | |
| 154 | + | |
| 155 | + | |
128 | 156 | | |
129 | 157 | | |
130 | 158 | | |
| |||
1630 | 1658 | | |
1631 | 1659 | | |
1632 | 1660 | | |
1633 | | - | |
1634 | | - | |
1635 | | - | |
1636 | | - | |
1637 | | - | |
1638 | | - | |
1639 | | - | |
1640 | | - | |
1641 | | - | |
| 1661 | + | |
| 1662 | + | |
| 1663 | + | |
| 1664 | + | |
| 1665 | + | |
| 1666 | + | |
| 1667 | + | |
| 1668 | + | |
| 1669 | + | |
| 1670 | + | |
1642 | 1671 | | |
1643 | 1672 | | |
1644 | 1673 | | |
1645 | 1674 | | |
1646 | 1675 | | |
1647 | | - | |
1648 | 1676 | | |
1649 | 1677 | | |
1650 | 1678 | | |
1651 | | - | |
| 1679 | + | |
1652 | 1680 | | |
1653 | 1681 | | |
1654 | 1682 | | |
| |||
2641 | 2669 | | |
2642 | 2670 | | |
2643 | 2671 | | |
2644 | | - | |
2645 | | - | |
2646 | | - | |
2647 | | - | |
2648 | | - | |
2649 | | - | |
2650 | | - | |
2651 | | - | |
2652 | | - | |
| 2672 | + | |
| 2673 | + | |
| 2674 | + | |
| 2675 | + | |
| 2676 | + | |
| 2677 | + | |
| 2678 | + | |
| 2679 | + | |
| 2680 | + | |
| 2681 | + | |
| 2682 | + | |
| 2683 | + | |
| 2684 | + | |
| 2685 | + | |
| 2686 | + | |
| 2687 | + | |
| 2688 | + | |
| 2689 | + | |
| 2690 | + | |
| 2691 | + | |
| 2692 | + | |
| 2693 | + | |
| 2694 | + | |
| 2695 | + | |
| 2696 | + | |
| 2697 | + | |
| 2698 | + | |
| 2699 | + | |
| 2700 | + | |
| 2701 | + | |
| 2702 | + | |
| 2703 | + | |
| 2704 | + | |
| 2705 | + | |
| 2706 | + | |
| 2707 | + | |
| 2708 | + | |
| 2709 | + | |
| 2710 | + | |
| 2711 | + | |
| 2712 | + | |
| 2713 | + | |
| 2714 | + | |
| 2715 | + | |
| 2716 | + | |
| 2717 | + | |
| 2718 | + | |
| 2719 | + | |
| 2720 | + | |
| 2721 | + | |
| 2722 | + | |
| 2723 | + | |
| 2724 | + | |
| 2725 | + | |
| 2726 | + | |
| 2727 | + | |
2653 | 2728 | | |
2654 | | - | |
2655 | | - | |
2656 | | - | |
2657 | | - | |
2658 | | - | |
2659 | | - | |
| 2729 | + | |
| 2730 | + | |
| 2731 | + | |
| 2732 | + | |
| 2733 | + | |
| 2734 | + | |
| 2735 | + | |
| 2736 | + | |
| 2737 | + | |
| 2738 | + | |
| 2739 | + | |
| 2740 | + | |
| 2741 | + | |
| 2742 | + | |
| 2743 | + | |
| 2744 | + | |
2660 | 2745 | | |
2661 | 2746 | | |
2662 | 2747 | | |
| |||
2876 | 2961 | | |
2877 | 2962 | | |
2878 | 2963 | | |
2879 | | - | |
2880 | | - | |
| 2964 | + | |
| 2965 | + | |
| 2966 | + | |
| 2967 | + | |
| 2968 | + | |
| 2969 | + | |
| 2970 | + | |
| 2971 | + | |
| 2972 | + | |
| 2973 | + | |
| 2974 | + | |
| 2975 | + | |
2881 | 2976 | | |
2882 | 2977 | | |
2883 | 2978 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
908 | 908 | | |
909 | 909 | | |
910 | 910 | | |
| 911 | + | |
| 912 | + | |
| 913 | + | |
| 914 | + | |
| 915 | + | |
| 916 | + | |
| 917 | + | |
| 918 | + | |
| 919 | + | |
911 | 920 | | |
912 | 921 | | |
913 | 922 | | |
| |||
917 | 926 | | |
918 | 927 | | |
919 | 928 | | |
| 929 | + | |
920 | 930 | | |
921 | 931 | | |
922 | 932 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3707 | 3707 | | |
3708 | 3708 | | |
3709 | 3709 | | |
| 3710 | + | |
| 3711 | + | |
| 3712 | + | |
| 3713 | + | |
| 3714 | + | |
| 3715 | + | |
| 3716 | + | |
| 3717 | + | |
| 3718 | + | |
| 3719 | + | |
| 3720 | + | |
| 3721 | + | |
| 3722 | + | |
| 3723 | + | |
| 3724 | + | |
| 3725 | + | |
| 3726 | + | |
| 3727 | + | |
| 3728 | + | |
| 3729 | + | |
| 3730 | + | |
| 3731 | + | |
| 3732 | + | |
| 3733 | + | |
| 3734 | + | |
| 3735 | + | |
| 3736 | + | |
| 3737 | + | |
| 3738 | + | |
| 3739 | + | |
| 3740 | + | |
| 3741 | + | |
| 3742 | + | |
| 3743 | + | |
| 3744 | + | |
| 3745 | + | |
| 3746 | + | |
| 3747 | + | |
| 3748 | + | |
| 3749 | + | |
| 3750 | + | |
| 3751 | + | |
| 3752 | + | |
| 3753 | + | |
| 3754 | + | |
| 3755 | + | |
| 3756 | + | |
| 3757 | + | |
| 3758 | + | |
| 3759 | + | |
| 3760 | + | |
| 3761 | + | |
| 3762 | + | |
| 3763 | + | |
| 3764 | + | |
| 3765 | + | |
| 3766 | + | |
| 3767 | + | |
| 3768 | + | |
| 3769 | + | |
| 3770 | + | |
| 3771 | + | |
| 3772 | + | |
| 3773 | + | |
| 3774 | + | |
| 3775 | + | |
| 3776 | + | |
| 3777 | + | |
| 3778 | + | |
| 3779 | + | |
| 3780 | + | |
| 3781 | + | |
| 3782 | + | |
| 3783 | + | |
| 3784 | + | |
| 3785 | + | |
| 3786 | + | |
| 3787 | + | |
| 3788 | + | |
| 3789 | + | |
| 3790 | + | |
| 3791 | + | |
| 3792 | + | |
| 3793 | + | |
| 3794 | + | |
| 3795 | + | |
| 3796 | + | |
| 3797 | + | |
| 3798 | + | |
| 3799 | + | |
| 3800 | + | |
| 3801 | + | |
| 3802 | + | |
| 3803 | + | |
| 3804 | + | |
| 3805 | + | |
| 3806 | + | |
| 3807 | + | |
| 3808 | + | |
| 3809 | + | |
| 3810 | + | |
| 3811 | + | |
| 3812 | + | |
| 3813 | + | |
| 3814 | + | |
| 3815 | + | |
| 3816 | + | |
| 3817 | + | |
| 3818 | + | |
| 3819 | + | |
| 3820 | + | |
| 3821 | + | |
| 3822 | + | |
| 3823 | + | |
| 3824 | + | |
| 3825 | + | |
| 3826 | + | |
| 3827 | + | |
| 3828 | + | |
| 3829 | + | |
| 3830 | + | |
| 3831 | + | |
| 3832 | + | |
| 3833 | + | |
3710 | 3834 | | |
3711 | 3835 | | |
3712 | 3836 | | |
| |||
14473 | 14597 | | |
14474 | 14598 | | |
14475 | 14599 | | |
| 14600 | + | |
14476 | 14601 | | |
14477 | 14602 | | |
14478 | 14603 | | |
| |||
0 commit comments