[BUG] DFS Path Update (append) silently drops written bytes — content lost after create/append/flush

[spuru9]

Service

Azure Data Lake Storage Gen2 (DFS host, hierarchical-namespace-style writes)

Azure API Action

Path Update — action=append followed by Path Update — action=flush

Expected behavior

Azure's create → append → flush sequence against PUT https://{account}.dfs.core.windows.net/{filesystem}/{path} persists the bytes sent in the append request body. A subsequent GET/HEAD against the same path (via either the DFS or Blob endpoint) reports the real content length and returns the appended bytes unchanged.

Actual behavior

Floci-az accepts all three calls (201 Created for each), but the appended body is never persisted. Both a subsequent Blob-endpoint HEAD and GET report the file as 0 bytes:

HTTP/1.1 200 OK
Content-Length: 0
x-ms-blob-content-length: 0
x-ms-blob-type: BlockBlob

$ curl -s "http://localhost/flink/manual-gen2-test.txt" -H "Host: devstoreaccount1.dfs.core.windows.net" -w '\n[size_download: %{size_download}]\n'

[size_download: 0]

This reproduces with a hand-built request (no SDK involved), so it isn't specific to any particular client's request shape (confirmed against Hadoop's hadoop-azure ABFS driver first, then isolated with plain curl).

Reproduction

curl -sS -X POST http://localhost:4577/_admin/reset

curl -sS -D - -o /dev/null \
  -X PUT 'http://localhost:4577/devstoreaccount1/flink?restype=container'

H='Host: devstoreaccount1.dfs.core.windows.net'
BASE='http://localhost:4577/flink/manual-gen2-test.txt'

# create
curl -sS -D - -o /dev/null -X PUT "$BASE?resource=file" -H "$H"

# append 19 bytes
curl -sS -D - -o /dev/null -X PUT "$BASE?position=0&action=append" -H "$H" \
  -H 'Content-Type: application/octet-stream' \
  --data-binary 'hello-world-content'

# flush
curl -sS -D - -o /dev/null -X PUT "$BASE?position=19&action=flush" -H "$H"

# check what actually got stored
curl -sS -D - "$BASE" -H "$H"

All four calls return 2xx, but the final GET returns an empty body and Content-Length: 0 instead of the 19 bytes written in the append step.

Environment

• Floci-az version / image tag: floci/floci-az:latest, digest sha256:0c673d49bb75b502ea0750f1c1347777483ffc33945539e1d9254438cb441a03
• Java SDK version (if applicable): reproduced via Hadoop's hadoop-azure (org.apache.hadoop.fs.azurebfs, bundled with flink-azure-fs-hadoop) and independently via plain curl
• How you're running Floci-az (Docker / native / mvn quarkus:dev): Docker (docker run floci/floci-az:latest) and in-cluster as a Kubernetes Deployment

Impact

This makes Floci-az unusable for any workload that round-trips real file content through the DFS/Gen2 create-append-flush write path (e.g. Hadoop's ABFS driver, which always uses this sequence regardless of fs.azure.account.hns.enabled). Metadata-only operations against the same path (create, delete, list) all work correctly — only the actual byte content is lost.

[github-actions]

👋 Thanks for opening your first issue in floci-az!

A maintainer will triage this and add the right service label. If you included a minimal reproduction and the floci-az version, that is everything we need — no action required from you right now.

Come say hi in Slack if you want to talk it through with maintainers and other contributors. Questions about Azure behaviour, design tradeoffs, and rough proposals are all welcome there.