Skip to content

Commit 2e3b3cd

Browse files
authored
refactor(server): embrace Falcon fiber reactor, strip thread mutexes, and modernize caching (#1141)
## What changed - Migrate server runtime from Puma + `Rack::Timeout` to Falcon + Async fiber reactor. - Map `Async::TimeoutError` to `504 GATEWAY_TIMEOUT` with standardized `Retry-After: 300` headers. - Eliminate thread-blocking primitives (`Concurrent::IVar`, `Concurrent::Map`, `Mutex`, `try_lock`, spin loops). - Rebuild `Feeds::Cache` with fiber-native `InFlight` waiter (`Async::Notification`) and `Data.define(:result, :expires_at)`. Safely propagates `Async::Stop` cancellation to pending waiters. - Streamline `RateLimiter` with lock-free fiber-safe hash tracking and automated window pruning. - Modernize `AccountManager` snapshot memoization and `RequestContext` using `Fiber[:request_context]`. - Inline HTTP cache headers in `Feeds::Renderer` and eliminate redundant `Feeds::HttpCache` wrapper. - Prune obsolete flags `async_feed_refresh_enabled` and `async_feed_refresh_stale_factor`. - Standardize all feed token specs, integration tests, and docs on `strategy: 'default'` (while preserving legacy `faraday` token decode and execution backward compatibility). - Update OpenAPI schema and regenerated client types. - Bump `html2rss` dependency to released `~> 0.30` (`0.30.0`). - Remove private DevContainer volume mount. ## Why Switching to Falcon eliminates thread contention and unlocks fiber concurrency across asynchronous HTTP and caching pipelines. Stripping obsolete thread synchronization primitives (`Mutex`, `concurrent-ruby`) significantly reduces code complexity, allocation overhead, and legacy surface area while ensuring clean fiber cancellation semantics. Completes the Faraday -> HTTPX and Puma -> Falcon cutover with the published `html2rss` v0.30.0 release. ## Risk - Low: The external HTTP API contract, rate limiting behavior, and feed rendering remain identical. Timeout responses consistently emit 504 Gateway Timeout. All 357 specs pass in the Dev Container. ## Review map 1. `app/web/feeds/cache.rb` — Fiber-native cache with `Async::Notification` in-flight deduplication. 2. `app/web/request/rate_limiter.rb` — Lock-free rate limiting with automatic pruning. 3. `app/web/errors/error_classifier.rb` — `Async::TimeoutError` classification and HTTPX error notes. 4. `app/web/config/runtime_env.rb` & `app/web/config/flags.rb` — Runtime config and flag cleanup. 5. `app/web/feeds/renderer.rb` — Inlined cache headers. 6. `spec/html2rss/web/feeds/source_resolver_spec.rb` — Legacy `faraday` token backward compatibility spec. 7. `spec/html2rss/web/feeds/cache_spec.rb` — Concurrency and fiber cancellation specs. 8. `Gemfile` & `Gemfile.lock` — Bump `html2rss` to released `0.30.0`. ## Validation - Dev Container `make ready` (357 examples, 0 failures; RuboCop clean; Zeitwerk clean; YARD public docs clean; TypeScript clean; ESLint clean; Prettier clean). - Dev Container `make openapi-verify` passed. - Dev Container `make ci-ready` passed (Playwright frontend e2e smoke, OpenAPI verify, linting, formatting, full test suite).
1 parent 4f1441f commit 2e3b3cd

34 files changed

Lines changed: 333 additions & 388 deletions

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,3 +60,6 @@ frontend/.astro
6060

6161
# Bundler/vendored dependencies
6262
/vendor/
63+
64+
# Local agent artifacts
65+
/.agents/

‎Gemfile‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,8 @@ source 'https://rubygems.org'
44

55
git_source(:github) { |repo_name| "https://github.com/#{repo_name}" }
66

7-
# gem 'html2rss', '~> 0.29'
8-
gem 'html2rss', github: 'html2rss/html2rss', branch: 'master'
7+
gem 'html2rss', '~> 0.30'
8+
# gem 'html2rss', github: 'html2rss/html2rss', branch: 'master'
99
gem 'html2rss-configs', github: 'html2rss/html2rss-configs'
1010

1111
# Use these instead of the two above (uncomment them) when developing locally:

‎Gemfile.lock‎

Lines changed: 20 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,3 @@
1-
GIT
2-
remote: https://github.com/html2rss/html2rss
3-
revision: 4260c842a5632fbef712cf3bee3a4fddcfee3654
4-
branch: master
5-
specs:
6-
html2rss (0.29.1)
7-
addressable (~> 2.7)
8-
brotli
9-
dry-validation
10-
httpx (~> 1.8)
11-
kramdown
12-
mcp (~> 1.2)
13-
mime-types (> 3.0)
14-
nokogiri (>= 1.10, < 2.0)
15-
rack (~> 3.0)
16-
rackup (~> 2.0)
17-
regexp_parser
18-
rss
19-
sanitize
20-
thor
21-
tzinfo
22-
webrick (~> 1.9)
23-
zeitwerk
24-
251
GIT
262
remote: https://github.com/html2rss/html2rss-configs
273
revision: 13cdb5f27bc7c9e943f27ee222eaf13f3dd0c93c
@@ -165,6 +141,24 @@ GEM
165141
fiber-storage (1.0.1)
166142
hana (1.3.7)
167143
hashdiff (1.2.1)
144+
html2rss (0.30.0)
145+
addressable (~> 2.7)
146+
brotli
147+
dry-validation
148+
httpx (~> 1.8)
149+
kramdown
150+
mcp (~> 1.2)
151+
mime-types (> 3.0)
152+
nokogiri (>= 1.10, < 2.0)
153+
rack (~> 3.0)
154+
rackup (~> 2.0)
155+
regexp_parser
156+
rss
157+
sanitize
158+
thor
159+
tzinfo
160+
webrick (~> 1.9)
161+
zeitwerk
168162
http-2 (1.2.2)
169163
httpx (1.8.3)
170164
http-2 (>= 1.2.0)
@@ -376,7 +370,7 @@ DEPENDENCIES
376370
base64
377371
climate_control
378372
falcon
379-
html2rss!
373+
html2rss (~> 0.30)
380374
html2rss-configs!
381375
irb
382376
rack-test
@@ -440,7 +434,7 @@ CHECKSUMS
440434
fiber-storage (1.0.1) sha256=f48e5b6d8b0be96dac486332b55cee82240057065dc761c1ea692b2e719240e1
441435
hana (1.3.7) sha256=5425db42d651fea08859811c29d20446f16af196308162894db208cac5ce9b0d
442436
hashdiff (1.2.1) sha256=9c079dbc513dfc8833ab59c0c2d8f230fa28499cc5efb4b8dd276cf931457cd1
443-
html2rss (0.29.1)
437+
html2rss (0.30.0) sha256=a0ec169cf4a41954b2ec327625b15e81d718061a59a0bd27670ca01f238a2852
444438
html2rss-configs (0.2.0)
445439
http-2 (1.2.2) sha256=81b5d45f50fd4cd5f8c5d09651184bec9401e3ef169c3eb3e5b003d5614a92b9
446440
httpx (1.8.3) sha256=cb88f2285c4ef17164d803a640dc393d28722cba7f282ad0e7f9f926cd02dc47

‎app/web/boot/setup.rb‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,7 @@ def sentry_enabled?
1616
#
1717
# @return [void]
1818
def call!
19+
Boot.eager_load! unless EnvironmentValidator.development?
1920
validate_environment!
2021
capture_runtime_env!
2122
configure_sentry!

‎app/web/config/flags.rb‎

Lines changed: 0 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -31,20 +31,6 @@ module Flags # rubocop:disable Metrics/ModuleLength
3131
default: true,
3232
validator: nil
3333
),
34-
async_feed_refresh_enabled: Definition.new(
35-
name: :async_feed_refresh_enabled,
36-
env_key: 'ASYNC_FEED_REFRESH_ENABLED',
37-
type: :boolean,
38-
default: false,
39-
validator: nil
40-
),
41-
async_feed_refresh_stale_factor: Definition.new(
42-
name: :async_feed_refresh_stale_factor,
43-
env_key: 'ASYNC_FEED_REFRESH_STALE_FACTOR',
44-
type: :integer,
45-
default: 3,
46-
validator: ->(value) { value >= 1 }
47-
),
4834
feeds_cache_max_size: Definition.new(
4935
name: :feeds_cache_max_size,
5036
env_key: 'FEEDS_CACHE_MAX_SIZE',
@@ -121,16 +107,6 @@ def auto_source_enabled?
121107
fetch(:auto_source_enabled)
122108
end
123109

124-
# @return [Boolean]
125-
def async_feed_refresh_enabled?
126-
fetch(:async_feed_refresh_enabled)
127-
end
128-
129-
# @return [Integer]
130-
def async_feed_refresh_stale_factor
131-
fetch(:async_feed_refresh_stale_factor)
132-
end
133-
134110
# Validates all known flags and managed env key prefixes.
135111
#
136112
# @return [void]

‎app/web/config/runtime_env.rb‎

Lines changed: 23 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -9,21 +9,23 @@ module RuntimeEnv
99
ADMIN_ACCESS_TOKEN_PLACEHOLDER = 'CHANGE_ME_ADMIN_TOKEN'
1010
HEALTH_CHECK_TOKEN_PLACEHOLDER = 'CHANGE_ME_HEALTH_CHECK_TOKEN'
1111
SENSITIVE_KEYS = %w[HTML2RSS_SECRET_KEY HTML2RSS_ACCESS_TOKEN HEALTH_CHECK_TOKEN SENTRY_DSN].freeze
12-
BOOT_METADATA_KEYS = %w[BUILD_TAG GIT_SHA RACK_ENV SENTRY_ENABLE_LOGS].freeze
13-
@mutex = Mutex.new
12+
BOOT_METADATA_KEYS = %w[
13+
BUILD_TAG GIT_SHA RACK_ENV SENTRY_ENABLE_LOGS PORT WEB_CONCURRENCY REQUEST_TIMEOUT_SECONDS
14+
].freeze
15+
# rubocop:disable ThreadSafety/ClassInstanceVariable
1416
@values = nil
1517

1618
class << self
1719
# @return [void]
1820
def capture!
19-
@mutex.synchronize { @values = tracked_env_values.freeze }
21+
@values = tracked_env_values.freeze
2022
scrub_sensitive_env!
2123
nil
2224
end
2325

2426
# @return [void]
2527
def reset!
26-
@mutex.synchronize { @values = nil }
28+
@values = nil
2729
end
2830

2931
# @return [String]
@@ -78,6 +80,21 @@ def rack_env
7880
fetch('RACK_ENV', ENV.fetch('RACK_ENV', 'development'))
7981
end
8082

83+
# @return [Integer]
84+
def port
85+
fetch('PORT', 4000).to_i
86+
end
87+
88+
# @return [Integer]
89+
def web_concurrency
90+
fetch('WEB_CONCURRENCY', 2).to_i
91+
end
92+
93+
# @return [Float]
94+
def request_timeout_seconds
95+
fetch('REQUEST_TIMEOUT_SECONDS', 55.0).to_f
96+
end
97+
8198
private
8299

83100
# @param key [String]
@@ -86,7 +103,8 @@ def rack_env
86103
def fetch(key, default = :__missing__)
87104
return ENV.fetch(key) if ENV.key?(key)
88105

89-
current_values = @mutex.synchronize { @values || {} }
106+
current_values = @values || {}
107+
# rubocop:enable ThreadSafety/ClassInstanceVariable
90108
return current_values.fetch(key) if current_values.key?(key)
91109
return default unless default == :__missing__
92110

‎app/web/errors/error_classifier.rb‎

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -262,16 +262,14 @@ def initialize(decision)
262262
%w[queue boot].include?(phase)
263263
}, SERVICE_UNAVAILABLE],
264264
[lambda { |c, _|
265-
# Gem wall-clock timeout (Botasaurus 504 / Faraday timeout) — not Timeout::Error.
265+
# Gem wall-clock timeout (Botasaurus 504 / HTTPX timeout) — not Timeout::Error.
266266
# work / nil (transport hop) → site-shaped 504.
267267
defined?(::Html2rss::RequestService::RequestTimedOut) &&
268268
c.any?(::Html2rss::RequestService::RequestTimedOut)
269269
}, GATEWAY_TIMEOUT],
270-
[lambda { |_, err|
271-
defined?(::Rack::Timeout::RequestTimeoutException) && err.is_a?(::Rack::Timeout::RequestTimeoutException)
272-
}, SERVICE_UNAVAILABLE],
273270
[lambda { |c, err|
274271
err.is_a?(Timeout::Error) || err.is_a?(Errno::ETIMEDOUT) ||
272+
(defined?(::Async::TimeoutError) && (err.is_a?(::Async::TimeoutError) || c.any?(::Async::TimeoutError))) ||
275273
(defined?(::HTTPX::TimeoutError) && (err.is_a?(::HTTPX::TimeoutError) || c.any?(::HTTPX::TimeoutError)))
276274
}, GATEWAY_TIMEOUT]
277275
].freeze
@@ -345,11 +343,15 @@ def decision_for_http_error(error, meta, default_message: nil)
345343
end
346344

347345
def network_error?(error)
348-
error_chain(error).any? do |err|
349-
NETWORK_ERRORS.include?(err.class) ||
350-
(defined?(::HTTPX::Error) &&
351-
(err.is_a?(::HTTPX::ConnectionError) || err.is_a?(::HTTPX::TLSError) || err.is_a?(::HTTPX::TimeoutError)))
352-
end
346+
error_chain(error).any? { |err| network_error_class?(err) }
347+
end
348+
349+
def network_error_class?(err)
350+
return true if NETWORK_ERRORS.include?(err.class)
351+
return true if defined?(::Async::TimeoutError) && err.is_a?(::Async::TimeoutError)
352+
return false unless defined?(::HTTPX::Error)
353+
354+
err.is_a?(::HTTPX::ConnectionError) || err.is_a?(::HTTPX::TLSError) || err.is_a?(::HTTPX::TimeoutError)
353355
end
354356
end
355357
end

‎app/web/feeds/cache.rb‎

Lines changed: 66 additions & 27 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,70 @@
11
# frozen_string_literal: true
22

3-
require 'concurrent/ivar'
4-
require 'concurrent/map'
3+
require 'async'
4+
require 'async/notification'
55
require 'digest'
66
require 'time'
77

88
module Html2rss
99
module Web
1010
module Feeds
1111
##
12-
# Small synchronous cache for canonical feed results.
12+
# Fiber-native cache for canonical feed results.
1313
module Cache
1414
# rubocop:disable ThreadSafety/ClassInstanceVariable
1515
def self.entries
16-
@entries ||= Concurrent::Map.new
16+
@entries ||= {}
1717
end
1818
private_class_method :entries
1919

2020
def self.in_flight
21-
@in_flight ||= Concurrent::Map.new
21+
@in_flight ||= {}
2222
end
2323
private_class_method :in_flight
2424
# rubocop:enable ThreadSafety/ClassInstanceVariable
2525

2626
Entry = Data.define(:result, :expires_at)
2727
DEFAULT_TTL_SECONDS = 3600
2828

29+
##
30+
# Coordinates in-flight fiber coalescing for identical cache keys.
31+
class InFlight
32+
def initialize
33+
@notification = Async::Notification.new
34+
@completed = false
35+
@result = nil
36+
@error = nil
37+
end
38+
39+
# @return [Object]
40+
def wait
41+
return @result if @completed && !@error
42+
raise @error if @completed && @error
43+
44+
@notification.wait
45+
raise @error if @error
46+
47+
@result
48+
end
49+
50+
# @param result [Object]
51+
# @return [void]
52+
def success!(result)
53+
@result = result
54+
@completed = true
55+
@notification.signal(result)
56+
end
57+
58+
# @param error [Exception]
59+
# @return [void]
60+
def failure!(error)
61+
@error = error
62+
@completed = true
63+
@notification.signal(error)
64+
end
65+
end
66+
private_constant :InFlight
67+
2968
class << self
3069
# Converts feed-provided minutes to seconds with a safe fallback.
3170
#
@@ -44,26 +83,12 @@ def seconds_from_minutes(value, default: DEFAULT_TTL_SECONDS)
4483
# @param cacheable [Boolean, Proc]
4584
# @yieldreturn [Html2rss::Web::Feeds::Contracts::RenderResult]
4685
# @return [Html2rss::Web::Feeds::Contracts::RenderResult]
47-
# rubocop:disable-next Metrics/MethodLength
48-
def fetch(key, ttl_seconds:, cacheable: true)
86+
def fetch(key, ttl_seconds:, cacheable: true, &)
4987
entry = read_entry(key)
5088
return entry.result if fresh?(entry)
89+
return in_flight[key].wait if in_flight.key?(key)
5190

52-
ivar = Concurrent::IVar.new
53-
actual_ivar = in_flight.put_if_absent(key, ivar)
54-
return actual_ivar.value! if actual_ivar
55-
56-
begin
57-
result = yield
58-
write_entry(key, ttl_seconds, result) if cacheable_result?(cacheable, result)
59-
ivar.set(result)
60-
result
61-
rescue StandardError => error
62-
ivar.fail(error)
63-
raise
64-
ensure
65-
in_flight.delete_pair(key, ivar)
66-
end
91+
execute_fetch(key, ttl_seconds, cacheable, &)
6792
end
6893

6994
# @param reason [String]
@@ -114,20 +139,34 @@ def prune_if_needed
114139

115140
def prune_expired
116141
now = Time.now.utc
117-
entries.each_pair { |k, v| entries.delete(k) if v && now >= v.expires_at }
142+
entries.delete_if { |_k, v| v && now >= v.expires_at }
143+
end
144+
145+
def execute_fetch(key, ttl_seconds, cacheable) # rubocop:disable Metrics/MethodLength
146+
job = in_flight[key] = InFlight.new
147+
begin
148+
result = yield
149+
write_entry(key, ttl_seconds, result) if cacheable_result?(cacheable, result)
150+
job.success!(result)
151+
result
152+
rescue Exception => error # rubocop:disable Lint/RescueException -- Async::Stop inherits from Exception
153+
job.failure!(error)
154+
raise
155+
ensure
156+
in_flight.delete(key)
157+
end
118158
end
119159

120160
def prune_excess(max)
121161
excess = entries.size - (max * 0.9).to_i
122162
return if excess <= 0
123163

124-
entries_by_expiration.first(excess).each { entries.delete(it.first) }
164+
entries_by_expiration.first(excess).each { |pair| entries.delete(pair.first) }
125165
end
126166

127167
def entries_by_expiration
128-
candidates = []
129-
entries.each_pair { |k, v| candidates << [k, v.expires_at] if v&.expires_at }
130-
candidates.sort_by!(&:last)
168+
entries.select { |_k, v| v&.expires_at }
169+
.sort_by { |_k, v| v.expires_at }
131170
end
132171

133172
# @param cacheable [Boolean, Proc]

0 commit comments

Comments
 (0)