Skip to content

Commit 642454c

Browse files
Add syslog stream and datagram listeners (#9)
* Add syslog stream and datagram listeners Adds two syslog transports to the server alongside HTTP, both delivering complete messages to a Ruby handler on the existing worker threads: - a stream listener (Unix socket or TCP) framed with RFC 6587 octet counting and a newline fallback, with optional PROXY v2 parsing to recover the peer address, and - a UDP listener with SO_REUSEPORT where one datagram is one message, with receive-buffer truncation detected and dropped. Undelivered messages are bounded per connection and overall; a refused message stalls stream reads and is retried, while a refused datagram is dropped and counted. Listener readiness and transport counters are exposed as syslog_listening? and syslog_stats. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Share worker threads fairly between requests and syslog The worker fast path always drained requests first, so sustained request load could starve syslog delivery indefinitely and stall every sender behind the per-connection backpressure. The two channels now alternate which is asked first, and the blocking select already picks uniformly between them. Workers with no syslog listener configured keep the original single-channel path. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Apply syslog listener review fixes Licence: the framing module, including its test data, is a port of Vector 0.48.0's octet counting decoder, so the file, the README and the licence file record that it is MPL-2.0 rather than MIT. Framing: a read boundary landing on a digit inside an oversize body no longer restarts octet counting and hands that body over as messages. Delivery: the per-connection queue is bounded separately from the listener-wide one, so a stalled sender cannot take every credit; the handler is given a stable message id and attempt number, its exceptions are counted apart from refusals, and retries end when shutdown starts. Messages the drain could not admit are counted instead of vanishing. Lifecycle: a cancellation token replaces the per-connection broadcast so a connection accepted in the same round as the shutdown still sees it, and stop() takes owned handles before releasing the GVL. Connection and idle limits bound stream resources, and datagram receive errors back off. A PROXY header naming a Unix or unspecified family falls back to the transport peer, with read failures counted separately. Workers may take several syslog messages per request, so a busy request queue cannot pace syslog delivery. The handler now reports :stream or :datagram, a peerless stream gives nil, the datagram buffer keys are named for what they bound, and the handler stays reachable through the server object rather than a permanent GC root. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Yield syslog messages to the worker block Requests and syslog messages now reach Ruby the same way: the block given to run_worker receives either a Request, which it answers with a Response, or a SyslogMessage, which it answers with an admission verdict. The separate configured handler, and the GC root it needed, are gone; delivery, limits, retries and accounting are unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Declare both of the gem's licences The framing module is MPL-2.0 while the rest is MIT, so the gem metadata lists both. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Tighten how a syslog message is answered and held A block that answers a syslog message with a response was not giving a verdict, but its response read as truthy and admitted the message. That now logs once and refuses, counted as a handler error, and the failure log names the message and its transport. The message object is freed as soon as it is collected and reports its payload to the GC, its string is built without revalidating bytes the framer already checked, and inspect prints the transport as the reader returns it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * Correct the syslog and licence notes The README described the removed callable, and the extension manifest named only one of the two licences the gem declares. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
1 parent 13390c8 commit 642454c

12 files changed

Lines changed: 2826 additions & 24 deletions

File tree

‎Cargo.lock‎

Lines changed: 11 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎LICENSE.txt‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,3 +19,12 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
1919
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
2020
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
2121
THE SOFTWARE.
22+
23+
Third-party components
24+
----------------------
25+
26+
ext/hyper_ruby/src/syslog/framing.rs is a port of Vector 0.48.0's
27+
lib/codecs/src/decoding/framing/octet_counting.rs
28+
(https://github.com/vectordotdev/vector) and is licensed under the Mozilla
29+
Public License 2.0 (https://www.mozilla.org/en-US/MPL/2.0/), not the MIT
30+
licence above.

‎README.md‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,23 @@ After checking out the repo, run `bin/setup` to install dependencies. Then, run
1010

1111
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
1212

13+
## Syslog listeners
14+
15+
Alongside HTTP, the server can accept syslog over a stream socket (Unix socket
16+
or TCP, optionally behind a PROXY v2 header) and over UDP. Each complete message
17+
is yielded to the same `Server#run_worker` block as requests, as a
18+
`HyperRuby::SyslogMessage` the block answers with an admission verdict. See the
19+
configuration keys documented at the top of `lib/hyper_ruby.rb`.
20+
1321
## License
1422

1523
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
1624

25+
`ext/hyper_ruby/src/syslog/framing.rs` is an exception: it is a port of
26+
Vector 0.48.0's `lib/codecs/src/decoding/framing/octet_counting.rs` and is
27+
licensed under the [Mozilla Public License 2.0](https://www.mozilla.org/en-US/MPL/2.0/),
28+
which is why the gem's metadata lists both licences.
29+
1730
## Code of Conduct
1831

1932
Everyone interacting in the HyperRuby project's codebases, issue trackers, chat rooms and mailing lists is expected to follow the [code of conduct](https://github.com/[USERNAME]/hyper_ruby/blob/master/CODE_OF_CONDUCT.md).

‎ext/hyper_ruby/Cargo.toml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ name = "hyper_ruby"
33
version = "0.1.0"
44
edition = "2021"
55
authors = ["alistairjevans <alistair@betterstack.com>"]
6-
license = "MIT"
6+
license = "MIT AND MPL-2.0"
77
publish = false
88

99
[lib]
@@ -26,3 +26,5 @@ async-stream = "0.3.5"
2626
env_logger = "0.11"
2727
log = "0.4"
2828
form_urlencoded = "1.2.1"
29+
tokio-util = { version = "0.7", features = ["codec", "rt"] }
30+
socket2 = "0.5"

0 commit comments

Comments
 (0)