Move monitoring services to flume-ng-instrumentation modules - #445
Merged
Merged
Conversation
Keeping HTTPSource in flume-ng-core forced the heavy Jetty/Gson HTTP stack onto every core consumer. Extract it into a dedicated optional source module under flume-ng-sources, like taildir, so it ships only when needed. The package (org.apache.flume.source.http) is unchanged, so the SourceType.HTTP reflective mapping and existing "http" configs keep working. Core retains Jetty/Gson for its metrics server and HTTPServerConstraintUtil. Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
HTTPSource was reaching into flume-ng-core for HTTPServerConstraintUtil. Move that helper into the module next to its only caller and make it package-private, so the HTTP source no longer depends on a core internal for its Jetty constraint handling. Declare jetty-security directly here (managed in flume-parent) since the helper uses it. Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Split the Ganglia, HTTP and Prometheus MonitorService implementations out of flume-ng-core into three modules under a new flume-ng-instrumentation parent, each in its own package so there are no split packages. Replace the hardcoded MonitoringType enum with ServiceLoader discovery: MonitorService gains a getType() default method, each provider declares itself via META-INF/services, and the node selects one by matching flume.monitoring.type against getType() (FQCN fallback preserved). The modules are wired through the BOM and bundled by flume-ng-dist only. Core keeps the MonitorService interface and JMXPollUtil, and drops the now unused gson and prometheus dependencies. Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ppkarwasz
marked this pull request as draft
June 10, 2026 13:34
The taildir source uses gson for its position file but was getting it transitively from flume-ng-core, which no longer provides it after the monitoring split. Without an explicit dependency the position file handling fails at runtime with NoClassDefFoundError. Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Finish the Prometheus and Jetty upgrade for the modules split out of flume-ng-core: port HTTPSource and its handlers and tests from javax.servlet / Jetty 9 to jakarta.servlet / Jetty 12 (ee11), and align the flume-http-monitor, flume-prometheus-monitor and flume-http-source poms with the new artifacts. The HTTPS tests now generate their TLS material with Bouncy Castle (the server keystore in a temporary folder, the client truststore in memory) instead of a checked-in keystore, keeping Jetty 12's SNI host checking enabled. Assisted-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ppkarwasz
marked this pull request as ready for review
June 11, 2026 11:51
rgoers
approved these changes
Jun 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split the Ganglia, HTTP and Prometheus MonitorService implementations out of flume-ng-core into three modules under a new flume-ng-instrumentation parent, each in its own package so there are no split packages.
Replace the hardcoded MonitoringType enum with ServiceLoader discovery: MonitorService gains a getType() default method, each provider declares itself via META-INF/services, and the node selects one by matching flume.monitoring.type against getType() (FQCN fallback preserved). The modules are wired through the BOM and bundled by flume-ng-dist only.
Core keeps the MonitorService interface and JMXPollUtil, and drops the now unused gson and prometheus dependencies.
Assisted-By: Claude Opus 4.8 (1M context) noreply@anthropic.com
Note
This PR is stacked on top of #444, which should be reviewed first.