Skip to content

README.md

Latest commit

 

History

History
371 lines (279 loc) · 16.5 KB

README.md

File metadata and controls

371 lines (279 loc) · 16.5 KB
Raw

Third-party integration handbook

This guide defines how Sahaay should connect to identity, KYC, credit bureau, financial-data, signature, document, and communication providers without coupling the core loan workflow to one vendor.

It is an implementation and onboarding guide. It does not grant access to any provider and does not replace the provider's contract, current technical specification, security audit, or legal review.

Current implementation status: The optional OpenAI application-assessment adapter, private S3 document-storage adapter, SMTP password-recovery adapter, and secondary generic HTTP SMS adapter are implemented. AI setup is documented in AI_MODELS.md, identity/notification setup in IDENTITY_AND_EMAIL.md, and S3 setup in the root README.md. KYC, PAN, bureau, CKYC, AA, e-sign, OCR, and general-purpose notification variables define planned contracts only; setting those values alone will not initiate a provider request.

Contents

Use ONBOARDING_CHECKLIST.md to track a provider from commercial approval through production activation. Use .env.example as the local configuration template.

Integration principles

  1. The loan workflow calls a Sahaay-owned interface, never a provider SDK directly.
  2. Each provider has its own adapter for authentication, payload mapping, signatures, errors, and webhooks.
  3. Canonical Sahaay requests and responses remain stable when a provider changes.
  4. Provider credentials are loaded from a secret manager in production.
  5. Raw PII, Aadhaar data, bureau reports, OTPs, biometrics, and access tokens must never appear in application logs.
  6. Every external request is correlated to a loan, borrower, consent record, purpose, and audit event.
  7. Automatic failover is disabled for regulated data providers unless compliance has approved equivalent consent, permissible purpose, contracts, and customer disclosures for the fallback provider.
  8. Provider timeouts must not leave a loan in an ambiguous state. Reconciliation handles unknown outcomes.

Supported integration categories

Category Example selection value Typical capability Production prerequisites
Aadhaar eKYC uidai_partner OTP/biometric authentication and eKYC through an authorised partner AUA/KUA or Sub-AUA/Sub-KUA arrangement, ASA connectivity, security approval
PAN verification protean PAN status and identity-field matching Provider registration, approved purpose, credentials and signing configuration
CKYC cersai Search, download, upload, and update KYC records Reporting-entity registration, institution code, certificate, static IP allowlist
Credit bureau cibil, crif, experian Consumer/commercial report and score Membership or authorised-user agreement, permissible purpose and borrower consent
Account Aggregator aa_partner Consent-based bank and financial information FIU eligibility/onboarding, signing/encryption keys, consent artefact implementation
DigiLocker digilocker Customer-authorised document retrieval Requester onboarding and approved use cases
E-sign/e-stamp esign_partner Agreement signing and stamp workflow Licensed/approved provider contract and callback validation
OCR/IDP ocr_partner Classification, extraction and confidence scoring India-region deployment, DPA, retention and model-use approval
Private document storage s3 Encrypted document objects and short-lived signed access Private bucket, least-privilege workload role, CORS, retention and malware scanning
Notifications notification_partner SMS, email and WhatsApp delivery Sender/template approval, consent and opt-out handling

Provider slugs are internal configuration names. They do not imply that credentials are publicly obtainable.

Credential configuration

Local development

cp .env.example .env

Fill only the sandbox values for the provider being tested. .env is ignored by Git.

Production

Use a managed secret service such as AWS Secrets Manager, Azure Key Vault, GCP Secret Manager, or Vault. Environment variables should contain secret references or injected runtime values—not secrets embedded in images, Kubernetes manifests, CI files, or source code.

Recommended secret paths:

sahaay/{environment}/{provider}/client-secret
sahaay/{environment}/{provider}/api-key
sahaay/{environment}/{provider}/signing-key
sahaay/{environment}/{provider}/webhook-secret

Required metadata for each credential:

Field Purpose
Owner Team responsible for approval and rotation
Environment Sandbox, UAT, pre-production, or production
Provider account Contract/member/institution identifier
Created and expiry date Rotation and certificate monitoring
Allowed source IP, workload identity, service account, or mTLS client
Permitted operations Verification, report pull, upload, webhook verification, etc.
Rotation procedure Dual-key or maintenance-window instructions

Never put actual keys in documentation, tickets, screenshots, chat, email, or Git history.

Provider switching

Provider selection is environment-driven:

EKYC_PROVIDER=uidai_partner
PAN_PROVIDER=protean
BUREAU_PRIMARY_PROVIDER=cibil
BUREAU_FALLBACK_PROVIDER=crif
BUREAU_FAILOVER_ENABLED=false

The backend should resolve the selected adapter at startup:

registry = {
    "cibil": CibilBureauAdapter,
    "crif": CrifBureauAdapter,
    "experian": ExperianBureauAdapter,
}

adapter = registry[settings.bureau_primary_provider](settings)

Before switching production providers:

  1. Complete the provider onboarding checklist.
  2. Validate canonical contract tests against the new sandbox.
  3. Confirm consent text and permissible-purpose mapping.
  4. Confirm data-field parity and decision-rule impact.
  5. Run a shadow comparison using approved synthetic or masked cases.
  6. Obtain business, security, compliance, and model-risk approval.
  7. Change the configuration through the controlled deployment pipeline.
  8. Monitor error rate, match rate, latency, score distribution, and reconciliation.

Do not silently fail over a bureau or identity request. A retry with another provider can create a new regulated enquiry, require separate consent, affect cost, and return materially different data.

Canonical adapter contract

Every adapter should implement a stable contract similar to the following:

class VerificationAdapter(Protocol):
    async def create_request(self, request: VerificationRequest) -> ProviderReference: ...
    async def get_status(self, provider_reference: str) -> VerificationStatus: ...
    async def cancel(self, provider_reference: str, reason: str) -> None: ...
    async def verify_webhook(self, headers: dict[str, str], body: bytes) -> VerifiedEvent: ...
    async def health(self) -> ProviderHealth: ...

Canonical request envelope:

{
  "request_id": "uuid",
  "loan_id": "HL-2026-0842",
  "borrower_id": "uuid",
  "consent_id": "uuid",
  "purpose_code": "CREDIT_UNDERWRITING",
  "requested_at": "2026-08-18T10:30:00Z",
  "data": {}
}

Canonical result envelope:

{
  "request_id": "uuid",
  "provider": "provider-slug",
  "provider_reference": "opaque-provider-reference",
  "status": "completed",
  "result_code": "VERIFIED",
  "completed_at": "2026-08-18T10:30:04Z",
  "normalized_data": {},
  "raw_payload_object_ref": "encrypted-object-store-reference",
  "warnings": []
}

Store raw regulated payloads only when legally and contractually permitted. Keep them encrypted, access-controlled, retention-limited, and separate from general application logs.

Canonical error classes

Error Retry Workflow action
invalid_request No Send to data-correction queue
consent_missing No Obtain fresh borrower consent
authentication_failed No Alert integration operations
rate_limited Yes, with provider guidance Queue with bounded backoff
provider_unavailable Yes Retry, then reconcile; do not assume failure
unknown_outcome Status check only Reconcile before resubmission
no_hit No Continue using approved no-hit policy
multiple_match No Send to manual verification
webhook_invalid No Reject and alert security operations

Provider-specific onboarding

Aadhaar eKYC

Aadhaar authentication/eKYC is not integrated using a generic public API key. Use an authorised AUA/KUA or Sub-AUA/Sub-KUA relationship and the assigned ASA/partner connectivity.

Configuration commonly includes:

  • Partner base URL and client identity
  • AUA/KUA and, where applicable, Sub-AUA/Sub-KUA codes
  • ASA licence key supplied through the authorised arrangement
  • Signing key reference and current encryption certificate
  • Registered callback URLs and source IPs

Security requirements:

  • Capture explicit, purpose-specific consent.
  • Encrypt the PID block at capture according to the current UIDAI specification.
  • Do not permanently store OTP, biometric data, or decrypted PID data.
  • Do not use Aadhaar numbers as application-domain identifiers.
  • Verify signed responses and record non-sensitive audit metadata.
  • Mask Aadhaar values in the LOS and logs.

PAN verification

For a Protean adapter, complete Online PAN Verification registration and obtain the environment-specific integration document and credentials from Protean.

The adapter should normalize:

  • PAN validity/status
  • Name match result
  • Date-of-birth or incorporation match result, when contracted
  • Entity/category response, when contracted
  • Provider reference and response timestamp

Do not infer identity solely from a fuzzy name match. Store the supplied consent/purpose, provider reference, response code, and reviewer outcome.

CKYC / CERSAI

CKYC integration is for registered reporting entities. Plan for:

  • Institution code and authorised users
  • Public-key upload and certificate rotation
  • Registered static source IP
  • Search/download/upload/update permissions
  • Encrypted file/payload exchange and response reconciliation
  • CKYC identifier masking and access logging

Keep search and download permission separate from upload/update permission where the provider supports it.

TransUnion CIBIL

Complete membership/authorised-user onboarding and product enablement before technical integration. Credentials and schemas depend on the contracted product.

The adapter should capture:

  • Member/subscriber identifiers
  • Permissible-purpose code
  • Borrower consent reference and timestamp
  • Enquiry/reference number
  • Report version, score type, score, no-hit/thin-file state, and reason codes
  • Raw report location and normalized tradeline summary

CRIF High Mark

Complete CRIF membership and product onboarding. CRIF provides real-time credit reports to member institutions through its portal or API for supported products.

Map CRIF responses into the same canonical bureau result used by CIBIL. Preserve provider-specific reason codes separately; do not assume scores or match logic are interchangeable between bureaus.

Account Aggregator

The LOS typically acts through an eligible/onboarded Financial Information User (FIU) arrangement with a licensed Account Aggregator.

Persist the full consent lifecycle:

  • Consent handle and consent artefact identifier
  • Purpose and requested financial-information types
  • Data range, frequency, use count, creation and expiry
  • FIU and recipient identity
  • Status transitions, notifications, fetch references, and revocation

Never collect a customer's banking password, PIN, or private key. Do not fetch data outside the granted consent scope.

DigiLocker

Onboard the organisation as an approved Requester and configure the issued client credentials and callback URLs. Preserve the customer authorisation record, document URI/issuer metadata, retrieval timestamp, and integrity verification result.

E-sign and e-stamp

The adapter should support document hash creation, signer identity, consent, signing session, stamp details, callback verification, final signed-document hash, certificate chain, and evidence package.

OCR and document intelligence

Use an India-region endpoint approved by security and legal teams. The adapter must expose document type, extracted fields, bounding references, confidence, model/version metadata, and manual-review requirements.

Provider contracts must prohibit unapproved training on borrower documents and define deletion, retention, subprocessors, breach reporting, and data location.

Consent and audit requirements

Before a regulated provider request, record:

consent_id
borrower_id
loan_id
provider_category
provider
purpose_code
consent_text_version
consented_at
expires_at
channel
ip/device metadata where appropriate
revoked_at

For every request and response, record:

internal request ID
provider reference
actor/service identity
request timestamp
response timestamp
status and normalized result code
payload fingerprint, not the raw PII payload
credential version, never the credential value
retry/reconciliation history
human review or override

Webhook requirements

  1. Accept HTTPS only.
  2. Verify signature or mTLS identity before parsing the event.
  3. Retain the exact request bytes until verification completes.
  4. Enforce timestamp tolerance and replay protection.
  5. Deduplicate using the provider event ID plus provider name.
  6. Return quickly and process asynchronously.
  7. Make processing idempotent.
  8. Quarantine invalid signatures and alert security operations.
  9. Reconcile missed callbacks through the provider's status endpoint.

Testing and go-live

Required test suites:

  • Adapter contract tests for every provider
  • Provider sandbox certification cases
  • Signature, certificate, webhook, and replay tests
  • Timeout, retry, rate-limit, and unknown-outcome tests
  • No-hit, multiple-match, deceased/fraud alert, and partial-response cases where applicable
  • Consent expiry and revocation tests
  • PII redaction and log scanning
  • Key/certificate rotation test
  • Reconciliation and duplicate-request test
  • Load and provider quota test

Production activation should use a feature flag and a small controlled cohort. Define rollback as a configuration change to stop new requests; never discard in-flight provider references.

Operational security

  • Use separate credentials for sandbox, UAT, and production.
  • Restrict outbound traffic to approved provider endpoints.
  • Prefer mTLS and workload identity where supported.
  • Rotate secrets and certificates before expiry with an overlap period.
  • Redact request/response bodies by default.
  • Restrict raw bureau/KYC access by role and purpose.
  • Encrypt raw reports and identity documents with tenant-aware keys.
  • Apply provider-specific retention and deletion rules.
  • Alert on authentication failures, signature failures, unusual volumes, and repeated no-hit patterns.
  • Maintain a provider incident and borrower-grievance runbook.

Official references

Always retrieve current specifications from the contracted provider during onboarding.