More problems/race condition/threading issues with Embarcadero Software, this time concerning TThreadedQueue which is being used to try and implement Golang channel compatibility behaviour, after extensive investigation and test programs the AI concludes:
Test 3 Channel Failure Investigation
Problem Summary
Test 3 (Channels) enters an infinite loop after the channel is closed, continuously receiving zero values instead of properly detecting the closed-and-empty state and exiting.
Root CauseAnalysis
After extensive investigation, the root cause is TThreadedQueue.PopItem behavior after timeout or shutdown is unreliable:
When DoShutDown is called on TThreadedQueue, subsequent PopItem calls return wrSignaled with default/garbage values (0 for integers) instead of returning wrAbandoned or wrTimeout
Even without calling DoShutDown, when using timeouts, PopItem on an empty queue appears to immediately return wrSignaled with default values after the timeout expires
Attempted Fixes
Attempt 1: Check FClosed flag before PopItem
Issue: Race condition - channel state can change between check and PopItem call
Attempt 2: Use DoShutDown to signal closure
Issue: DoShutDown causes PopItem to return wrSignaled with garbage values
Attempt 3: Manual item counting (FItemCount)
Track items added/removed independently of TThreadedQueue
Issue: Even with defensive checks, PopItem still returns wrSignaled repeatedly
Attempt 4: Defensive check against FItemCount = 0
Check if FItemCount was 0 when wrSignaled received, treat as garbage
Issue: The check doesn't catch all cases, possibly due to race conditions in the counter itself
Fundamental Design Flaw
The core issue is that TThreadedQueue with timeouts is not suitable for implementing Go-style channels because:
Its timeout behavior is inconsistent (may return wrSignaled with default values)
DoShutDown changes queue behavior in unpredictable ways
No reliable way to distinguish "got a real value" from "got a timeout/shutdown default value"
Recommended Solution
To properly fix this, the channel implementation needs to be redesigned from scratch without relying on TThreadedQueue:
Use a regular TQueue protected by a critical section
Use a semaphore or event to signal when items are available
Implement timeout logic manually using WaitForSingleObject with timeout
Maintain precise control over when default values vs. real values are returned
This approach avoids all the unreliable behaviors of TThreadedQueue and provides complete control over the channel semantics.
Current State
The channel implementation compiles but Test 3 fails with an infinite loop. All attempted fixes using TThreadedQueue have been unsuccessful due to fundamental incompatibilities between TThreadedQueue's behavior and Go channel semantics.
More problems/race condition/threading issues with Embarcadero Software, this time concerning TThreadedQueue which is being used to try and implement Golang channel compatibility behaviour, after extensive investigation and test programs the AI concludes:
Test 3 Channel Failure Investigation
Problem Summary
Test 3 (Channels) enters an infinite loop after the channel is closed, continuously receiving zero values instead of properly detecting the closed-and-empty state and exiting.
Root CauseAnalysis
After extensive investigation, the root cause is TThreadedQueue.PopItem behavior after timeout or shutdown is unreliable:
When DoShutDown is called on TThreadedQueue, subsequent PopItem calls return wrSignaled with default/garbage values (0 for integers) instead of returning wrAbandoned or wrTimeout
Even without calling DoShutDown, when using timeouts, PopItem on an empty queue appears to immediately return wrSignaled with default values after the timeout expires
Attempted Fixes
Attempt 1: Check FClosed flag before PopItem
Issue: Race condition - channel state can change between check and PopItem call
Attempt 2: Use DoShutDown to signal closure
Issue: DoShutDown causes PopItem to return wrSignaled with garbage values
Attempt 3: Manual item counting (FItemCount)
Track items added/removed independently of TThreadedQueue
Issue: Even with defensive checks, PopItem still returns wrSignaled repeatedly
Attempt 4: Defensive check against FItemCount = 0
Check if FItemCount was 0 when wrSignaled received, treat as garbage
Issue: The check doesn't catch all cases, possibly due to race conditions in the counter itself
Fundamental Design Flaw
The core issue is that TThreadedQueue with timeouts is not suitable for implementing Go-style channels because:
Its timeout behavior is inconsistent (may return wrSignaled with default values)
DoShutDown changes queue behavior in unpredictable ways
No reliable way to distinguish "got a real value" from "got a timeout/shutdown default value"
Recommended Solution
To properly fix this, the channel implementation needs to be redesigned from scratch without relying on TThreadedQueue:
Use a regular TQueue protected by a critical section
Use a semaphore or event to signal when items are available
Implement timeout logic manually using WaitForSingleObject with timeout
Maintain precise control over when default values vs. real values are returned
This approach avoids all the unreliable behaviors of TThreadedQueue and provides complete control over the channel semantics.
Current State
The channel implementation compiles but Test 3 fails with an infinite loop. All attempted fixes using TThreadedQueue have been unsuccessful due to fundamental incompatibilities between TThreadedQueue's behavior and Go channel semantics.