You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Storage] Add Expect: 100-continue support - #50094
Adds support for the HTTP header Expect: 100-continue on requests that carry a body. When the transport honours it, the service can reject a request before a single byte of the body is sent, which cuts the bandwidth wasted re-uploading against a throttling service.
Adds ExpectContinueOptions and ExpectContinueMode (APPLY_ON_THROTTLE, ON, OFF) to azure-storage-common, along with the two policies. APPLY_ON_THROTTLE applies the header only for a window after the service responds 429, 500, or 503, so the extra round trip is only paid when it earns its keep.
Exposed on the blob client builders as expectContinueBehavior. Blob only, matching the Go SDK, which ships this in azblob and not in azqueue, azfile or azdatalake.
The policy is added below the retry policy so a throttling response takes effect on the very next attempt rather than only on a later call, and above the credential policies since headers may affect the string to sign.
Not applied unless configured. The handshake needs the HTTP client to withhold the request body until the service responds, and the default transport does not do that (see below), so a default client would pay for the header and save nothing.
Can also be turned off without a code change via the system property or environment variable AZURE_STORAGE_DISABLE_EXPECT_CONTINUE_HEADER, matching .NET and Go.
Transport support
ExpectContinueTests drives each HTTP client against a real socket and asserts on what reached the wire and when:
Transport
Behaviour
azure-core-http-okhttp
Withholds the body until 100 Continue — the handshake works
azure-core-http-netty (default)
Sends the header, streams the body immediately
azure-core-http-vertx
Same as netty
azure-core-http-jdk-httpclient
Drops the header; Expect is restricted by java.net.http.HttpClient
This is an azure-core gap rather than a Storage one. .NET and Go enable the feature by default because their transports perform the handshake — SocketsHttpHandler in .NET, and Go's net/http with azcore setting ExpectContinueTimeout: 1s. Once an equivalent lands for netty, flipping the default back to APPLY_ON_THROTTLE is a one-line change.
Title of the pull request is clear and informative.
There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.
Applies the HTTP header Expect: 100-continue to requests that carry a
body, so that a body is not uploaded just to be rejected while the
service is under load. The service can reject the request before a
single byte of body is sent, which cuts the bandwidth wasted retrying
large uploads against a throttling service.
- Adds Request100ContinueOptions and Request100ContinueMode (AUTO,
ALWAYS, NEVER) to azure-storage-common. AUTO, the default, applies the
header only for a window after the service responds 429, 500, or 503,
so the extra round trip is only paid when it earns its keep.
- The policies are added below the retry policy so a throttling response
takes effect on the very next attempt rather than only on a later
call, and above the credential policies since headers may affect the
string to sign.
- Wired into the blob, data lake, file share, and queue pipelines, and
exposed on the blob client builders via request100ContinueOptions.
- Can be turned off without a code change by setting the system property
or environment variable AZURE_STORAGE_DISABLE_EXPECT_CONTINUE_HEADER.
Ported from azure-sdk-for-net #40611 and #52438.
Azure Pipelines:
Successfully started running 1 pipeline(s).
35 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.
- Rename Request100ContinueOptions/Request100ContinueMode to
ExpectContinueOptions/ExpectContinueMode, the mode values to
APPLY_ON_THROTTLE/ON/OFF, autoInterval to throttleInterval, and the
builder method to expectContinueBehavior, matching the .NET names.
- Make ExpectContinueOnThrottlePolicy extend HttpPipelineSyncPolicy
rather than ExpectContinuePolicy. Both policies are now final and
share their logic through a package private helper, so neither
advertises an extension point that callers outside the package cannot
actually use.
- Allow the Expect header in the blob default log options, so it is
visible when debugging whether a request negotiated the handshake.
- Replace the unrelated timeouts link in the options javadoc with RFC
9110.
- Add ExpectContinueTransportTests, which drives each HTTP client against
a real socket and asserts on what reached the wire and when.
The transport tests show that setting the header does not by itself make
a client wait. Only okhttp performs the handshake. netty and vertx send
the header and stream the body immediately, and jdk-httpclient drops the
header entirely since Expect is restricted by java.net.http.HttpClient.
That is recorded in the test matrix and documented on ExpectContinueOptions
and in the changelog rather than left implicit.
Cover the default log options change with an assertion, and keep the
changelog to the feature itself. The transport caveat stays on
ExpectContinueOptions, where the behavior it describes lives.
…everywhere
Use the .NET casing for the mode values, ApplyOnThrottle / On / Off,
rather than Java's upper snake case. Checkstyle accepts it.
Allow the Expect header in the default log options of the queue, file
share and data lake builders too, not only blob, as the review comment
was a general one.
Restructure ExpectContinueTests so the first section mirrors
ExpectContinueTests.cs test for test, with the same names and the same
case values, making the two suites directly comparable. Java specific
coverage, including the per retry behaviour that .NET has no test for,
moves to a clearly separated second section.
ThrottlePolicyRevertsAfterBackoff is disabled in .NET
(Azure/azure-sdk-for-net#41368); the Java mirror is enabled, since the
window is a monotonic nanoTime deadline and oversleeping it can only
close the window.
Extend the transport test to transport x sync/async, as those are
separate code paths in every client. All four transports behave the same
on both paths, so the earlier finding is unchanged.
Revert the queue, file share and data lake packages entirely, so the
header is applied only where a customer can configure it.
Drop the explanatory comments that were not carrying their weight,
including the cross references to the .NET implementation.
Keep them in the class the review comment was on rather than a separate
file. Each one opens and closes its own server, so the socket fixture
does not run for the tests that do not need it.
A negative interval is accepted here and later converted to zero, silently making ApplyOnThrottle behave like Off. Reject negative durations at this public API boundary so configuration errors are diagnosable, and cover the validation in the options tests.
- Drop the azure-storage-common changelog entry; the changes there are
implementation level and the customer facing option is on the blob
builders.
- Remove the okhttp, vertx and jdk test dependencies. They already come
transitively from azure-core-test, so they were redundant.
- Move ExpectContinuePolicy and ExpectContinueOnThrottlePolicy into
com.azure.storage.common.policy alongside the other storage policies,
and rename the shared helper to ExpectContinuePolicyHelper.
- Use upper snake case for the mode constants.
- Reject a negative content length threshold or throttle interval rather
than silently changing behaviour, with boundary tests for both.
The handshake needs the HTTP client to withhold the request body until
the service responds, and the default transport does not do that, so a
default client would pay for the header and save nothing. The policy is
now added only when expectContinueBehavior is set.
…-100-continue
The only conflict was in the blob BuilderHelper, where both sides added
policies at the same point. Both are kept. The expect-continue policy is
placed after the content validation policies, as the encoding policy
rewrites the body and resets Content-Length, and the threshold check
needs to read the encoded length that is actually sent.
Although marked internal, this public class and its constructors become SDK API because azure-storage-common/src/main/java/module-info.java:9 exports com.azure.storage.common.policy, creating compatibility obligations for implementation details. Move both policies and their shared helper into com.azure.storage.common.implementation.policy, updating imports and tests while keeping ExpectContinueOptions and ExpectContinueMode public. The same issue applies to ExpectContinueOnThrottlePolicy.java:24.
Azure Pipelines:
Successfully started running 1 pipeline(s).
35 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Isabelle (ibrandes)
left a comment
browndav-msft
left a comment
Adds support for the HTTP header
Expect: 100-continueon requests that carry a body. When the transport honours it, the service can reject a request before a single byte of the body is sent, which cuts the bandwidth wasted re-uploading against a throttling service.ExpectContinueOptionsandExpectContinueMode(APPLY_ON_THROTTLE,ON,OFF) toazure-storage-common, along with the two policies.APPLY_ON_THROTTLEapplies the header only for a window after the service responds 429, 500, or 503, so the extra round trip is only paid when it earns its keep.expectContinueBehavior. Blob only, matching the Go SDK, which ships this inazbloband not inazqueue,azfileorazdatalake.AZURE_STORAGE_DISABLE_EXPECT_CONTINUE_HEADER, matching .NET and Go.Transport support
ExpectContinueTestsdrives each HTTP client against a real socket and asserts on what reached the wire and when:azure-core-http-okhttp100 Continue— the handshake worksazure-core-http-netty(default)azure-core-http-vertxazure-core-http-jdk-httpclientExpectis restricted byjava.net.http.HttpClientThis is an azure-core gap rather than a Storage one. .NET and Go enable the feature by default because their transports perform the handshake —
SocketsHttpHandlerin .NET, and Go'snet/httpwith azcore settingExpectContinueTimeout: 1s. Once an equivalent lands for netty, flipping the default back toAPPLY_ON_THROTTLEis a one-line change.Ported from azure-sdk-for-net #40611 and #52438.
Description
Please add an informative description that covers that changes made by the pull request and link all relevant issues.
If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
All SDK Contribution checklist:
General Guidelines and Best Practices
Testing Guidelines