Keep native insulin uploads pending when historical profile is missing

.gitattributes

@@ -1,2 +1,3 @@
 tools/session_store/integration.patch whitespace=-blank-at-eol
 tools/aaps4_compat/master-admission.patch whitespace=-blank-at-eol
+tools/aaps4_compat/upload-profile-retry.patch whitespace=-blank-at-eol,-blank-at-eof

AI_NOTES.md

@@ -1,3 +1,48 @@
+# Native Missing-Profile Upload Retry - 2026-10-23
+
+## Change And Evidence
+
+- Reproduced three failures in the pinned native uploader: missing-profile TBR
+  and EB advanced cursors; direct transport acknowledged without a network call.
+- Added a separate minimal retry patch: hold the unconverted record until its
+  historical profile is available. Preserve imported/identifier-only skips.
+- Initial two native suites86passed;16recipe checks passed. Extended combined
+  verification covers emulated delivery and HTTP failures followed by recovery.
+- Combined native recipe passed in1m4s:485tests, no failures/errors/skips,
+  including87uploader cases. Unchanged suites reused verified Gradle task outputs;
+  source cleanup returned the upstream checkout to its exact pristine pin.
+- All22recipe/JSON-isolation checks passed; shell syntax and whitespace pass.
+- The recipe applies/reverses only owned patches; unknown changes are preserved.
+  No Copilot runtime code, therapy, phone, server or existing APK modification.
+
+## Limits And Checks
+
+See AAPS4_UPLOAD_PROFILE_RETRY.md. An unavailable profile can hold that collection
+indefinitely; no reset or current-profile substitution is permitted. Historical
+skips, destination binding, physical delivery and clinical forecast validation
+remain separate. Exact-head CI and reviewed integration remain required; no new
+clinical admission follows from source verification.
+
+# Native Insulin Form Audit - 2026-10-23
+
+- Five pure regressions reproduced hidden basal/extended fields in carbohydrate,
+  unrelated, instantaneous and emulated-child branches. A sixth regression used
+  accepted wire data and the native Room source. Original failing logs retained.
+- Audit explicit delivery evidence before native mapper precedence. Preserve
+  legitimate bolus/basal/extended forms and one native emulated child; competing
+  forms refuse a partial inventory. Unrelated durations/glucose/profile context
+  and null fields do not create insulin. Storage remains lossless and unchanged.
+- Retained-cache refresh, database reopen and valid correction recovery pass;
+  no clinical/action rows are written. All164focused tests pass without skips.
+  Frozen full test/lint/compile/assemble passed in6m48s:6800passed/six existing
+  skips, zero failures/errors, real pristine SDK TLS2.142s and zero lint errors
+  with382existing warnings/eight hints. All1431source hashes stayed unchanged
+  and APK retention passed. Mainfa6f22b7 CI39906923561 passed all three jobs.
+  Exact-head feature CI and main integration remain separate post-commit gates.
+- Existing0.1.11 release, AAPS, live onset clock, targets, doses and alarms remain
+  unchanged. This stage does not attest complete upstream/pump history or enable
+  a clinically recommended meal waiting time.
+
 # Insulin History Knowledge Boundary And 0.1.11 - 2026-10-23
 
 - Main6a0a7894 CI39881365894 passed all three jobs. Native basal/profile source

android-app/app/src/main/kotlin/io/aaps/copilot/service/AapsV3InsulinInventory.kt

@@ -111,9 +111,14 @@ internal class AapsV3InsulinInventoryTimeline(snapshot: LocalV3RetainedSnapshot)
 
     private fun dose(body: JsonObject): AapsV3InsulinDose? {
         val insulin = number(body, "insulin")
+        val event = string(body, "eventType")
+        val intervalFields = listOf("enteredinsulin", "absolute", "percent", "rate", "extendedEmulated")
+        val intervalEvidence = event in setOf("Temp Basal", "Combo Bolus") ||
+            intervalFields.any { present(body, it) } || boolean(body, "isEmulatingTempBasal") == true
         require(insulin == null || insulin >= 0)
-        // The native mapper chooses insulin before eventType and before carbohydrate fields.
+        // Native branch precedence is not proof that competing delivery fields are absent.
         if (insulin != null && insulin > 0) {
+            require(!intervalEvidence)
             require((number(body, "carbs") ?: 0.0) == 0.0)
             require((number(body, "duration") ?: 0.0) == 0.0 && (number(body, "durationInMilliseconds") ?: 0.0) == 0.0)
             require(body["extendedEmulated"] == null || body["extendedEmulated"] == JsonNull)
@@ -122,10 +127,20 @@ internal class AapsV3InsulinInventoryTimeline(snapshot: LocalV3RetainedSnapshot)
             val config = body["icfg"]?.takeUnless { it == JsonNull }?.let { AapsV3ProfileJson.insulin(HistoryJson.obj(it)) }
             return AapsV3InsulinDose.Bolus(insulin, type, boolean(body, "isBasalInsulin") ?: false, config)
         }
-        if ((number(body, "carbs") ?: 0.0) != 0.0) return null
-        return when (string(body, "eventType")) {
+        require(boolean(body, "isBasalInsulin") != true)
+        if ((number(body, "carbs") ?: 0.0) != 0.0) {
+            require(!intervalEvidence)
+            return null
+        }
+        return when (event) {
             "Temp Basal" -> {
-                body["extendedEmulated"]?.takeUnless { it == JsonNull }?.let { return extended(HistoryJson.obj(it)) }
+                require(!present(body, "enteredinsulin"))
+                body["extendedEmulated"]?.takeUnless { it == JsonNull }?.let {
+                    val child = HistoryJson.obj(it)
+                    require(boolean(child, "isEmulatingTempBasal") == true)
+                    return extended(child)
+                }
+                require(boolean(body, "isEmulatingTempBasal") != true)
                 val duration = duration(body)
                 val absolute = number(body, "absolute")?.also { require(it >= 0) }
                 val percent = number(body, "percent")?.also { require(it >= -100) }
@@ -137,11 +152,18 @@ internal class AapsV3InsulinInventoryTimeline(snapshot: LocalV3RetainedSnapshot)
             }
             "Combo Bolus" -> extended(body)
             "Correction Bolus", "Meal Bolus", "Snack Bolus" -> throw IllegalArgumentException("missing_native_insulin")
-            else -> null
+            else -> {
+                require(!intervalEvidence)
+                null
+            }
         }
     }
 
     private fun extended(body: JsonObject): AapsV3InsulinDose.Extended {
+        require((number(body, "insulin") ?: 0.0) == 0.0 && (number(body, "carbs") ?: 0.0) == 0.0)
+        require(boolean(body, "isBasalInsulin") != true)
+        require(listOf("absolute", "percent", "extendedEmulated").none { present(body, it) })
+        number(body, "rate")?.let { require(it >= 0) }
         val units = checkNotNull(number(body, "enteredinsulin")).also { require(it >= 0) }
         return AapsV3InsulinDose.Extended(units, duration(body), boolean(body, "isEmulatingTempBasal") ?: false)
     }
@@ -176,4 +198,5 @@ internal class AapsV3InsulinInventoryTimeline(snapshot: LocalV3RetainedSnapshot)
     private fun number(body: JsonObject, key: String) = body[key]?.takeUnless { it == JsonNull }?.let(HistoryJson::double)
     private fun string(body: JsonObject, key: String) = body[key]?.takeUnless { it == JsonNull }?.let(HistoryJson::string)
     private fun boolean(body: JsonObject, key: String) = body[key]?.takeUnless { it == JsonNull }?.let(HistoryJson::bool)
+    private fun present(body: JsonObject, key: String) = body[key]?.let { it != JsonNull } == true
 }

android-app/app/src/test/kotlin/io/aaps/copilot/service/AapsV3EffectiveProfileSourceTest.kt

@@ -137,6 +137,41 @@ class AapsV3EffectiveProfileSourceTest {
         Unit
     }
 
+    @Test fun acceptedMixedWirePayloadCannotHideDeliveryAcrossCacheAndReopen() = runBlocking {
+        val from = at - 60_000
+        store().create(input("profile", from))
+        val api = LocalNightscoutV3EntriesApi(db) { at + 100_000 }
+        val body = buildJsonObject {
+            put("app", "AAPS"); put("eventType", "Temp Basal"); put("date", from)
+            put("duration", 1); put("absolute", 0.8); put("carbs", 24)
+        }.toString()
+        val accepted = api.handle("POST", "/api/v3/treatments", emptyMap()) { body.toByteArray() }
+        assertEquals(201, accepted.status)
+        val id = Json.parseToJsonElement(accepted.body).jsonObject.getValue("identifier").jsonPrimitive.content
+        suspend fun rejected(reader: AapsV3EffectiveProfileSource) {
+            val result = reader.readEvidence(from, at, at + 200_000)
+            assertEquals(AapsV3ObservedInsulinStatus.INVALID, result.insulin.status)
+            assertTrue(result.insulin.records.isEmpty())
+            assertEquals(AapsV3ReportedBasalStatus.INPUT_UNAVAILABLE, result.reportedBasal.status)
+            assertTrue(result.reportedBasal.segments.isEmpty())
+        }
+        val source = AapsV3EffectiveProfileSource(store())
+        rejected(source)
+        db.close(); db = open()
+        val reopened = AapsV3EffectiveProfileSource(store())
+        rejected(reopened)
+        val original = checkNotNull(store().get(LocalV3Collection.TREATMENTS, id))
+        val corrected = JsonObject(Json.parseToJsonElement(original.payloadJson).jsonObject - "carbs")
+        store().update(LocalV3DocumentInput(LocalV3Collection.TREATMENTS, id, "local-v3-owner", from,
+            corrected.toString()), original.modifiedRevision)
+        val result = reopened.readEvidence(from, at, at + 200_000)
+        assertEquals(AapsV3ObservedInsulinStatus.OBSERVED, result.insulin.status)
+        assertEquals(AapsV3ReportedBasalStatus.OBSERVED_WINDOW, result.reportedBasal.status)
+        assertFalse(result.insulin.upstreamCompletenessVerified)
+        assertTrue(db.therapyDao().since(0).isEmpty())
+        assertTrue(db.actionCommandDao().latest(10).isEmpty())
+    }
+
     @Test fun profileHistorySharesRevisionAndRetainsCorrectionsAndDeletionAcrossReopen() = runBlocking {
         store().create(input("old", at - 60_000))
         val original = store().create(input("new", at - 10_000, 6.5)).document

android-app/app/src/test/kotlin/io/aaps/copilot/service/AapsV3InsulinInventoryTest.kt

@@ -229,4 +229,93 @@ class AapsV3InsulinInventoryTest {
         assertFalse(result.records.single().toString().contains("dose-1"))
         assertFalse(result.records.single().dose.toString().contains("historical"))
     }
+
+    @Test fun unrecognizedDeliveryFieldsCannotBecomeAnEmptyOrPartialInventory() {
+        val indicators = listOf(
+            buildJsonObject { put("enteredinsulin", 1.0) },
+            buildJsonObject { put("absolute", 0.0) },
+            buildJsonObject { put("percent", -100) },
+            buildJsonObject { put("rate", 0.8) },
+            buildJsonObject { put("isEmulatingTempBasal", true) },
+            buildJsonObject { put("isBasalInsulin", true) },
+            buildJsonObject { put("extendedEmulated", buildJsonObject {
+                put("enteredinsulin", 1.0); put("duration", 30); put("isEmulatingTempBasal", true)
+            }) }
+        )
+        for (indicator in indicators) for (event in listOf("Note", "Future Delivery")) {
+            val fields = JsonObject(indicator + ("eventType" to JsonPrimitive(event)))
+            val hidden = row("hidden", fields = fields)
+            for (result in listOf(read(hidden), read(row(), hidden))) {
+                assertEquals("$event $indicator", AapsV3ObservedInsulinStatus.INVALID, result.status)
+                assertTrue(result.records.isEmpty())
+            }
+            // Unknown interval support cannot be discarded as expired.
+            assertEquals(AapsV3ObservedInsulinStatus.INVALID,
+                read(row("old", time = at - 7_200_000, fields = fields)).status)
+        }
+    }
+
+    @Test fun carbohydrateBranchCannotHideBasalOrExtendedDelivery() {
+        for (delivery in listOf(
+            buildJsonObject { put("eventType", "Temp Basal"); put("absolute", 0.8); put("duration", 30) },
+            buildJsonObject { put("eventType", "Combo Bolus"); put("enteredinsulin", 1.0); put("duration", 30) },
+            buildJsonObject { put("eventType", "Carb Correction"); put("enteredinsulin", 1.0) }
+        )) {
+            val result = read(row(fields = JsonObject(delivery + ("carbs" to JsonPrimitive(24)))))
+            assertEquals(AapsV3ObservedInsulinStatus.INVALID, result.status)
+            assertTrue(result.records.isEmpty())
+        }
+    }
+
+    @Test fun instantaneousInsulinCannotHideAdditionalDeliveryForms() {
+        for (extra in listOf(
+            buildJsonObject { put("enteredinsulin", 1.0) },
+            buildJsonObject { put("absolute", 0.0) },
+            buildJsonObject { put("percent", -100) },
+            buildJsonObject { put("rate", 0.8) },
+            buildJsonObject { put("isEmulatingTempBasal", true) }
+        )) {
+            assertEquals(AapsV3ObservedInsulinStatus.INVALID, read(row(fields = JsonObject(extra +
+                ("insulin" to JsonPrimitive(2.2))))).status)
+        }
+    }
+
+    @Test fun basalAndExtendedBranchesCannotHideAnotherForm() {
+        val basal = buildJsonObject { put("eventType", "Temp Basal"); put("duration", 30); put("absolute", 0.8) }
+        val extended = buildJsonObject { put("eventType", "Combo Bolus"); put("duration", 30); put("enteredinsulin", 1.0) }
+        for (fields in listOf(JsonObject(basal + ("enteredinsulin" to JsonPrimitive(1.0))),
+            JsonObject(extended + ("absolute" to JsonPrimitive(0.8))),
+            JsonObject(extended + ("percent" to JsonPrimitive(0.0))))) {
+            assertEquals(AapsV3ObservedInsulinStatus.INVALID, read(row(fields = fields)).status)
+        }
+    }
+
+    @Test fun emulatedChildCannotSilentlyDropMixedOrRecursivePayloads() {
+        val child = buildJsonObject { put("enteredinsulin", 1.0); put("duration", 30); put("isEmulatingTempBasal", true) }
+        for (extra in listOf(
+            buildJsonObject { put("insulin", 2.0) },
+            buildJsonObject { put("carbs", 24) },
+            buildJsonObject { put("absolute", 0.8) },
+            buildJsonObject { put("isEmulatingTempBasal", false) },
+            buildJsonObject { put("extendedEmulated", child) }
+        )) {
+            val result = read(row(fields = buildJsonObject {
+                put("eventType", "Temp Basal"); put("duration", 30); put("absolute", 0.8)
+                put("extendedEmulated", JsonObject(child + extra))
+            }))
+            assertEquals(AapsV3ObservedInsulinStatus.INVALID, result.status)
+            assertTrue(result.records.isEmpty())
+        }
+    }
+
+    @Test fun nonDeliveryDurationsGlucoseAndInsulinConfigurationDoNotBecomeDoses() {
+        for (fields in listOf(
+            buildJsonObject { put("eventType", "Carb Correction"); put("carbs", 24); put("duration", 120) },
+            buildJsonObject { put("eventType", "Exercise"); put("duration", 30) },
+            buildJsonObject { put("eventType", "Note"); put("glucose", 5.5); put("icfg", config()) },
+            buildJsonObject { put("eventType", "Note"); put("enteredinsulin", JsonNull); put("rate", JsonNull) }
+        )) {
+            assertEquals(AapsV3ObservedInsulinStatus.NOT_OBSERVED, read(row(fields = fields)).status)
+        }
+    }
 }

docs/AAPS4_INSULIN_OBSERVATIONS.md

@@ -28,6 +28,15 @@ earlier absence. Latest retained versions are not a historical version archive.
 Queries include intervals that start before the requested window and overlap it.
 Malformed records cannot be declared expired using an unknown duration.
 
+Native mapper precedence is not completeness evidence. A carbohydrate branch or
+unknown event cannot hide entered insulin, basal rate/percentage or an emulated
+child. An instantaneous bolus cannot silently discard interval fields. Conflicting
+forms reject the whole observational inventory, including old records with unknown
+support. Null fields, ordinary carbohydrate/exercise durations and glucose/profile
+configuration alone are not delivery. Emulated wrappers still select one child,
+which must explicitly be emulating basal and cannot contain another delivery form.
+Transport retention stays lossless; successful storage is separate from admission.
+
 The existing native connection owner publishes observation status and record count.
 The count includes invalidated retained records and is not delivered insulin or IOB.
 Stop/revocation/offline ownership clears the published observation status. No new
@@ -42,6 +51,11 @@ and ignores changes that only add that ID. These counters are not in treatment
 payloads. A zero queue therefore cannot prove that a new destination received the
 old history. Similar collection-specific selectors must be considered separately.
 
+Missing historical profiles can also advance native interval-upload cursors
+without a network request. AAPS4_UPLOAD_PROFILE_RETRY.md records the minimal
+retry patch and its limits. It prevents future silent skips in the patched source;
+it cannot attest or recover already missing records or lift the completeness gate.
+
 Initial history admission needs a source/destination-bound generation and complete
 window with explicit revisions, deletions, record kinds and acknowledgements.
 Existing foreign Nightscout IDs cannot simply be assumed present in the local

docs/AAPS4_UPLOAD_PROFILE_RETRY.md

@@ -0,0 +1,57 @@
+# Historical Profile Upload Retry
+
+Pinned upstream: AAPS4.0.0-rc1 at
+`d5e7f5bdbb4aaefa4eaaa89a46fba257ef396fa0`.
+
+## Failure
+
+The native uploader converts temporary basal and extended bolus records using
+the profile at the record timestamp. Without that profile, the stock transport
+returns success before calling the network client. The extended-bolus selector
+also counts a missing-profile record as ignored and advances its synchronization
+cursor. An apparently drained queue therefore can omit insulin history.
+
+## Minimal Patch
+
+`tools/aaps4_compat/upload-profile-retry.patch` changes only the two uploader
+classes and adds synthetic tests to their existing native suites:
+
+- A missing historical profile returns false for both CREATE and UPDATE of a
+  temporary basal or extended bolus, including an emulated basal and invalidation.
+- The extended-bolus selector stops that collection at the unconverted record
+  without advancing its cursor. Its next normal upload retries the same record.
+- Existing imported-record and identifier-only skip rules remain unchanged.
+- No current-profile fallback, clinical record rewrite, cursor reset, new worker,
+  endpoint, credential, dose calculation or pump command is added.
+
+Missing profile is not an HTTP error to ignore. Even when the upstream
+IGNORE_NS_V3_ERRORS option is enabled, it cannot turn an unsent conversion into
+an acknowledgement. Other native HTTP-error policies remain unchanged.
+
+## Limits
+
+If the historical profile never becomes available, that collection remains
+blocked. Newer records in that collection cannot bypass the gap; other collections
+keep their existing upload behavior. This is deliberate retention, not an automatic
+repair. Do not clear synchronization IDs or substitute today's profile to advance
+the queue. An eventual explicit gap-aware transfer protocol is a separate stage.
+
+This does not recover records already skipped by an older build, verify foreign
+Nightscout identifiers at a new destination, attest a stable full-history window,
+or confirm physical pump delivery. Copilot still keeps
+`upstreamCompletenessVerified=false` and the live meal recommendation gate closed.
+No AAPS APK or phone update is part of this source stage.
+
+## Checks
+
+Three native failures were reproduced before the fix. Tests exercise real selector
+and plugin methods against a mocked SDK client: missing-profile CREATE, recovery,
+HTTP failure without cursor advancement, successful retry, then missing-profile
+invalidation and recovery. Ordinary extended bolus, emulated basal and temporary
+basal use separate cases. Imported and identifier-only records remain skipped.
+
+The existing verification recipe applies all reviewed patches to the exact clean
+pin, runs the native suites, and reverses only its unchanged owned diff. Missing
+patch files and concurrent changes fail without overwriting unknown work. CI
+then rebuilds the pristine SDK before Copilot TLS tests. Passing these tests is
+software evidence, not clinical recommendation admission.