Repository navigation
Commit 409a33f
authored
feat(api): bound field occurrences of all gRPC API requests (#7052)
* feat(api): bound transaction field occurrences before parsing
Scan the wire bytes before parsing on the API entry points that accept
a binary Transaction, and reject the request once it exceeds 50,000
field occurrences:
- TransactionAdmissionGuard counts every tag with CodedInputStream,
recurses into known sub-messages by descriptor, and recurses into the
contract payload with the type the node unpacks it into, following
protobuf merge semantics. Group encoding and nesting deeper than 32
are rejected. Block, database and P2P parsing do not use it.
- For gRPC, every method whose request is a Transaction gets a request
marshaller that reads the request with the inbound size bound, runs
the guard and hands the same bytes to the original marshaller.
A rejection returns a sentinel instead of throwing, and a handler
wrapper closes the call with INVALID_ARGUMENT before the service
method runs. This affects BroadcastTransaction,
GetTransactionSignWeight, GetTransactionApprovedList,
GetShieldTransactionHash and CreateCommonTransaction.
- /wallet/broadcasthex runs the guard before Transaction.parseFrom.
- Admission rejections use one fixed client message and one DEBUG
line per rejected request, without a stack trace.
Across the full mainnet history (15.79 billion transactions), the
maximum observed count was 3,926 field occurrences. This is an API
admission policy, not a consensus-validity limit: previously tolerated
encodings and unusually complex transactions may now be rejected at
these entry points. Consensus, execution and block validation are
unchanged.
* perf(api): preserve KnownLength in transaction admission
Use the exact size of KnownLength request streams to allocate the
bounded read buffer once, including zero-length requests.
Pass checked bytes to the original marshaller through a KnownLength
stream to restore its array-backed protobuf parsing fast path.
Keep admission budgets, size limits, rejection handling and the
original delegate unchanged. Compressed requests retain the bounded
growing read path and also use the KnownLength handoff after checking.
* feat(api): bound field occurrences across gRPC API requests
Apply the 50,000-field-occurrence admission budget to requests of
the Wallet, WalletSolidity, Database and Monitor services before
the original request parser runs. Previously only Transaction
requests had an occurrence budget.
Resolve the original marshaller's protobuf type once when wrapping
the method and use it as the scan root. Fall back to the proto
schema when no protobuf instance is available. Normal rejections
use per-method, per-reason sentinels; the schema-only fallback counts
and throws the rejection before delegate parsing.
Preserve the original marshaller and KnownLength handoff.
Transaction requests keep their rejection message; other requests
use "request rejected by admission check". Count admission
rejections by method and reason through the existing metrics system.
Newly guarded methods also reject group encoding and nesting deeper
than 32, and report decoding errors as INVALID_ARGUMENT instead of
UNKNOWN. Reflection and P2P are unchanged.
* fix(api): reject oversized compressed requests without an error log
Map the RESOURCE_EXHAUSTED that the grpc decompressing stream throws
while the admission marshaller reads a request over the inbound size
limit to the TOO_LARGE rejection, and close the call with
RESOURCE_EXHAUSTED, the status grpc already gives an uncompressed
message over the limit, instead of UNKNOWN. Other rejections keep
INVALID_ARGUMENT.
* refactor(api): fail at wiring time without a request prototype
Every guarded gRPC method must now have a request marshaller that
gives a protobuf prototype. GuardedRequestMarshaller.guard takes the
scan root from that prototype and throws IllegalStateException with
the full method name otherwise, so such a method fails the
registration instead of being kept unguarded or guarded through a
separate path.
Remove the schema-only Plain marshaller, the empty-bytes probe for
non-prototype marshallers and the schema lookup in RpcService, and
replace their tests with wiring failure tests.
* refactor(api): register gRPC services through RpcService
Add an abstract services() to RpcService. addServices guards every
definition it returns and registers it, so the subclasses only list
their services and no longer register them directly. Remove the
one-argument guardRequestMethods.
Keep the registration order, the per-port service sets and the cursor
interceptor that wraps the SOLIDITY and PBFT services before the
admission wrapper. The reflection service is still registered in
initServerBuilder and is not wrapped.
Make the registration and cursor wiring tests call addServices, and
make the shared registration check also fail a method that is guarded
twice.
* refactor(api): tighten gRPC service wiring and admission helpers
Make initServerBuilder, addInterceptor and initServer of RpcService
protected final, so subclasses declare their services through
services() instead of adding them in these steps.
GuardedRequestMarshaller.guard now throws IllegalStateException when
the request marshaller is already guarded. ServerInterceptors.intercept
preserves the request marshaller, so this also detects a guarded
definition intercepted before another guard.
Turn parseChecked into an instance method, fold rejectionMessage,
make prototypeOf private and move the rejection counter into
AdmissionCheckedHandler. Remove the test of a plain delegate, which
guard no longer accepts, and rename leftover test variables.
* refactor(api): rename the admission guard to ProtobufAdmissionGuard
The guard scans the requests of every guarded gRPC method and of
/wallet/broadcasthex, not only transactions. Rename
TransactionAdmissionGuard to ProtobufAdmissionGuard,
TransactionAdmissionException to ProtobufAdmissionException and
MAX_TX_OCCURRENCES to MAX_OCCURRENCES, with the test class.
checkTransaction and countTransaction keep their names. Behavior,
messages, status codes and metric labels are unchanged.
* refactor(api): clarify admission sentinel naming and buffer growth
Rename the rejection map of GuardedRequestMarshaller to
rejectionSentinels and its accessor to rejectionSentinel, to
distinguish the per-reason sentinel objects from rejection state.
Split the buffer growth in readBounded into a doubling step and a cap,
and compute the cap once before the loop. The growth is unchanged.
Add tests for a negative max and for reading at most one byte past
max.1 parent d7df62b commit 409a33f
21 files changed
Lines changed: 3562 additions & 36 deletions
File tree
- common/src/main/java/org/tron/common/prometheus
- framework/src
- main/java/org/tron
- common/application
- core
- admission
- services
- http/servlets
- interfaceOnPBFT
- interfaceOnSolidity
- test/java/org/tron
- common/application
- core
- admission
- services
- filter
- http/servlets
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
26 | 26 | | |
27 | 27 | | |
28 | 28 | | |
| 29 | + | |
29 | 30 | | |
30 | 31 | | |
31 | 32 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
| 33 | + | |
| 34 | + | |
33 | 35 | | |
34 | 36 | | |
35 | 37 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
19 | 22 | | |
20 | 23 | | |
| 24 | + | |
| 25 | + | |
21 | 26 | | |
22 | 27 | | |
23 | 28 | | |
24 | 29 | | |
25 | 30 | | |
26 | 31 | | |
27 | 32 | | |
| 33 | + | |
| 34 | + | |
28 | 35 | | |
29 | 36 | | |
30 | 37 | | |
| |||
84 | 91 | | |
85 | 92 | | |
86 | 93 | | |
87 | | - | |
| 94 | + | |
88 | 95 | | |
89 | 96 | | |
90 | 97 | | |
91 | 98 | | |
92 | 99 | | |
93 | 100 | | |
94 | | - | |
| 101 | + | |
95 | 102 | | |
96 | 103 | | |
97 | 104 | | |
| |||
116 | 123 | | |
117 | 124 | | |
118 | 125 | | |
119 | | - | |
| 126 | + | |
| 127 | + | |
120 | 128 | | |
121 | | - | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
| 144 | + | |
| 145 | + | |
| 146 | + | |
| 147 | + | |
| 148 | + | |
| 149 | + | |
| 150 | + | |
| 151 | + | |
| 152 | + | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
| 156 | + | |
| 157 | + | |
| 158 | + | |
| 159 | + | |
122 | 160 | | |
123 | 161 | | |
124 | 162 | | |
| |||
134 | 172 | | |
135 | 173 | | |
136 | 174 | | |
137 | | - | |
| 175 | + | |
138 | 176 | | |
139 | 177 | | |
140 | 178 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
0 commit comments