feat: Massive real-time flag synchronization for clients

[t-kikuc]

Describe the feature you'd like

Enable synchronization of flags in real-time for a massive number of clients, more than 3M concurrent clients.

We need to consider some ways, such as WebSocket.
This feature might cost so much.

Background

Currently, the real-time flag synchronization for clients is supported via Firebase Cloud Messaging (FCM).
https://docs.bucketeer.io/integration/pushes

However, the synchronization pace is not so fast for massive clients due to the FCM quota.

Alternative solutions

No response

[Ubisoft-potato]

We need to consider some ways, such as WebSocket.

Like you said, using WebSocket could cost too much resource, maybe we can use HTTP SSE since the user only need to receive server's update, it's single way communication.

[t-kikuc]

Tasks:

• Proto: Define SSE payload types feat(realtime-sync): add SSE stream evaluations messages #2592
• Backend:
  • Domain topic subscription, connection pooling feat(sse): prepare the fan-out dispatcher #2630
  • Evaluation, Metrics feat(sse): implemented the evaluation logic and add 7 metrics #2665
  • Graceful shutdown feat(stream): add graceful shutdown for SSE connections #2675
  • tag propagation feat(stream): propagate prerequisite-dependent tags in SSE dispatch #2679
  • max conns limitation feat(stream): add per-pod SSE connection limit #2676
• Infra:
  • Gateway config for allowing SSE
  • Resize memory of API pods
  • HPA on connection count
• Load Test
• Client SDKs:
  • JavaScript
  • iOS
  • Android
  • later:
    • Flutter,
    • React,
    • React Native

[t-kikuc]

note: we need to add safety against the cache gap (like batch per 1min)

#2630