298 lines
12 KiB
Markdown
298 lines
12 KiB
Markdown
# notice-service 架构
|
||
|
||
## 目标
|
||
|
||
`notice-service` 是外部通知投递服务，不是新的业务状态 owner。它从各 owner service 的 outbox 或只读事件源读取已经提交的事实，再投递到用户私有实时通道、后续 push 或站内 inbox。
|
||
|
||
当前已实现的链路是：
|
||
|
||
- `wallet-service` 在同一账务事务里写 `wallet_outbox.WalletBalanceChanged`。
|
||
- `notice-service` 抢占该事件，写自己的 `notice_delivery_events` 投递位点。
|
||
- `notice-service` 通过腾讯云 IM C2C `TIMCustomElem` 给用户发送 `wallet_notice`。
|
||
- 客户端收到通知后按 `asset_type + balance_version` 更新本地余额；乱序或重复事件直接丢弃。
|
||
- `room-service` 写 `room_outbox.RoomUserKicked` 后，`notice-service` 可通过 MySQL 扫描或 RocketMQ `hyapp_room_outbox` fanout 消费该事实，只给被踢用户发送 `room_notice` C2C 私有通知；房间群系统消息和 IM 群成员移除仍由 room-service 的房间 IM bridge 负责。
|
||
- RocketMQ 模式下 `notice-service` 不反写 `room_outbox.status`，只写 `notice_delivery_events`；room outbox 的发布状态属于 room-service，下游通知投递状态属于 notice-service。
|
||
|
||
## 服务边界
|
||
|
||
| 服务 | 职责 |
|
||
| --- | --- |
|
||
| `wallet-service` | 余额、流水、交易幂等、`wallet_outbox` 事实源；不调用 IM、不关心客户端在线状态 |
|
||
| `notice-service` | 通知投递位点、抢占锁、退避重试、死信、腾讯云 IM 单聊投递 |
|
||
| `room-service` | 房间状态、房间群系统消息 outbox、腾讯 IM 群组/成员桥接；不直接发送用户私有通知 |
|
||
| `activity-service` | 活动、任务、系统消息业务事实；后续可新增 activity notice 模块 |
|
||
| 腾讯云 IM | 客户端长连接、单聊/群消息、离线/漫游投递 |
|
||
|
||
`notice-service` 不直接修改钱包、房间、用户或活动业务表。它只读取 owner outbox，并写自己的投递状态表。
|
||
|
||
房间 IM 被拆成两种通道：
|
||
|
||
- 房间群通道：`CreateRoom` 主链路同步建群，`RoomCreated` outbox 只做幂等补偿；`RoomUserKicked` 移除群成员、`room_user_kicked` 群系统消息由 room-service outbox worker direct 投递，或由 room-service 的 `hyapp-room-im-bridge` consumer group 从 `hyapp_room_outbox` 消费后投递。它依赖房间群生命周期和群成员控制，不进入 notice-service。
|
||
- 用户私有通道：被踢本人可能已经先被移出 IM 群，不能保证收到群消息；因此 notice-service 监听同一条 `RoomUserKicked` 事实，发送 C2C `room_notice` 给 `target_user_id`。
|
||
|
||
## 当前目录结构
|
||
|
||
```text
|
||
services/notice-service/
|
||
cmd/server/ 进程入口
|
||
configs/ 本地、Docker、腾讯云示例配置
|
||
deploy/mysql/initdb/ notice-service 自己的库表
|
||
internal/app/ 进程装配、health、worker 生命周期
|
||
internal/config/ 配置解析和默认值
|
||
internal/modules/roomnotice/ 房间私有通知模块
|
||
internal/modules/walletnotice/ 钱包余额通知模块
|
||
internal/platform/mysql/ notice-service MySQL 连接池
|
||
```
|
||
|
||
后续新增通知类型时按模块扩展，例如：
|
||
|
||
```text
|
||
internal/modules/activitynotice/
|
||
internal/modules/systeminbox/
|
||
internal/modules/pushnotice/
|
||
```
|
||
|
||
不要把不同业务域的 SQL、payload 和 worker 都塞进一个通用目录。
|
||
|
||
## 钱包余额通知链路
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant Client as App Client
|
||
participant Gateway as gateway-service
|
||
participant Room as room-service
|
||
participant Wallet as wallet-service
|
||
participant WalletDB as hyapp_wallet
|
||
participant Notice as notice-service
|
||
participant NoticeDB as hyapp_notice
|
||
participant TencentIM as Tencent IM
|
||
|
||
Client->>Gateway: POST /api/v1/rooms/gift/send
|
||
Gateway->>Room: SendGift(command_id)
|
||
Room->>Wallet: DebitGift(command_id)
|
||
Wallet->>WalletDB: tx: debit/credit entries + wallet_outbox
|
||
Wallet-->>Room: balance_after, heat_value
|
||
Room->>Room: Room Cell command + command log + room outbox
|
||
Room-->>Gateway: success
|
||
Gateway-->>Client: success response
|
||
Notice->>WalletDB: claim WalletBalanceChanged
|
||
Notice->>NoticeDB: notice_delivery_events=delivering
|
||
Notice->>TencentIM: C2C TIMCustomElem(wallet_notice)
|
||
TencentIM-->>Client: private balance notice
|
||
Notice->>NoticeDB: delivered / retryable / failed
|
||
```
|
||
|
||
主请求链路只等待钱包扣费和房间状态提交，不等待腾讯云 IM 私有通知。因此送礼请求不会因为 IM 抖动变慢。IM 投递失败时，用户仍可以通过钱包接口拉取最终余额；notice worker 会按退避策略补偿。
|
||
|
||
## 房间踢人私有通知链路
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant Owner as Room Owner App
|
||
participant Gateway as gateway-service
|
||
participant Room as room-service
|
||
participant RoomDB as hyapp_room
|
||
participant RocketMQ as RocketMQ
|
||
participant Notice as notice-service
|
||
participant NoticeDB as hyapp_notice
|
||
participant TencentIM as Tencent IM
|
||
participant Target as Kicked User App
|
||
|
||
Owner->>Gateway: POST /api/v1/rooms/user/kick
|
||
Gateway->>Room: KickUser(command_id, target_user_id)
|
||
Room->>Room: remove presence, release mic, write ban
|
||
Room->>RoomDB: command log + snapshot + room_outbox.RoomUserKicked
|
||
Room-->>Gateway: success
|
||
Room-->>TencentIM: direct/IM bridge deletes group member and sends group event
|
||
Room->>RocketMQ: publish hyapp_room_outbox when MQ enabled
|
||
Notice->>RocketMQ: consume RoomUserKicked by hyapp-notice-room-outbox
|
||
Notice->>RoomDB: or claim RoomUserKicked by MySQL polling
|
||
Notice->>NoticeDB: notice_delivery_events=delivering
|
||
Notice->>TencentIM: C2C TIMCustomElem(room_notice)
|
||
TencentIM-->>Target: private kicked notice
|
||
Notice->>NoticeDB: delivered / retryable / failed
|
||
```
|
||
|
||
房间群消息和 C2C 私有通知使用同一个 `event_id`，客户端按 `event_id` 去重即可。notice-service 不修改 `room_outbox.status`，它只用 `notice_delivery_events` 记录自己的 C2C 投递状态，避免和 room-service outbox worker 或 IM bridge consumer 互相覆盖。
|
||
|
||
MQ 和 MySQL 扫描是同一事实的两种入口：
|
||
|
||
| 入口 | 使用场景 | 幂等边界 |
|
||
| --- | --- | --- |
|
||
| MySQL `room_notice_worker` | 本地或未接 MQ 的环境，直接从 `hyapp_room.room_outbox` claim `RoomUserKicked` | `notice_delivery_events(source_name, app_code, source_event_id, channel)` |
|
||
| RocketMQ `hyapp_room_outbox` | 线上推荐，notice 使用独立 consumer group `hyapp-notice-room-outbox` 解耦消费 | 同一张 `notice_delivery_events`，重复 MQ 消息不会重复投 C2C |
|
||
|
||
## Payload 约定
|
||
|
||
`WalletBalanceChanged` 发送给客户端的核心字段：
|
||
|
||
```json
|
||
{
|
||
"event_type": "WalletBalanceChanged",
|
||
"event_id": "tx_xxx:WalletBalanceChanged:10001:COIN",
|
||
"app_code": "lalu",
|
||
"transaction_id": "tx_xxx",
|
||
"command_id": "gift_cmd_001",
|
||
"user_id": "10001",
|
||
"asset_type": "COIN",
|
||
"available_delta": -9900,
|
||
"frozen_delta": 0,
|
||
"available_after": 1123123,
|
||
"frozen_after": 0,
|
||
"balance_version": 18,
|
||
"created_at_ms": 1710000000000,
|
||
"source_created_at_ms": 1710000000000,
|
||
"metadata": {}
|
||
}
|
||
```
|
||
|
||
客户端处理规则：
|
||
|
||
- 按 `event_id` 去重。
|
||
- 按 `asset_type` 分桶维护本地余额。
|
||
- 如果本地已有该资产 `balance_version >= event.balance_version`，丢弃这条通知。
|
||
- 如果发现版本跳跃较大或本地余额不可确认，调用钱包摘要接口重新拉取权威余额。
|
||
|
||
这套规则同时覆盖普通送礼扣费、幸运礼物中奖返奖、VIP 购买扣费、资源组发金币、币商转币等余额变化。
|
||
|
||
`VipActivated` 用于通知客户端用户已获得或续期 VIP，C2C 自定义消息 `Ext=vip_notice`：
|
||
|
||
```json
|
||
{
|
||
"event_type": "VipActivated",
|
||
"event_id": "wev_xxx",
|
||
"app_code": "lalu",
|
||
"transaction_id": "wtx_xxx",
|
||
"command_id": "vip_purchase_10001_6_1790000000000",
|
||
"user_id": "10001",
|
||
"asset_type": "VIP",
|
||
"source": "vip_purchase",
|
||
"action": "upgrade",
|
||
"previous_level": 3,
|
||
"level": 6,
|
||
"vip_name": "VIP6",
|
||
"started_at_ms": 1790000000000,
|
||
"expires_at_ms": 1792592000000,
|
||
"reward_resource_group_id": 101,
|
||
"resource_grant_id": "rgr_xxx",
|
||
"reward_items": [],
|
||
"source_created_at_ms": 1790000000000
|
||
}
|
||
```
|
||
|
||
客户端按 `event_id` 去重，收到后刷新全局 VIP 状态；`source` 取值为 `vip_purchase`、`admin_grant` 或 `activity_grant`。
|
||
|
||
## 多实例和重启语义
|
||
|
||
`notice_delivery_events` 是 worker 的投递位点表：
|
||
|
||
- 主键：`source_name + app_code + source_event_id + channel`。
|
||
- `delivering` 通过 `locked_by + lock_until_ms` 表示抢占锁。
|
||
- worker 重启后，锁过期的 `delivering` 会被其它实例重新抢占。
|
||
- 投递失败进入 `retryable`，按 `next_retry_at_ms` 退避重试。
|
||
- 超过 `max_retry_count` 进入 `failed`，不再自动重试，需要后台或运维手动查看死信。
|
||
|
||
因此部署两台 `notice-service` 不会重复抢同一条事件；即使腾讯云 IM 返回成功后本地标记 delivered 失败，下一轮最多重复投递一次，客户端用 `event_id` 去重。
|
||
|
||
## 配置
|
||
|
||
本地默认关闭 worker，避免误发真实腾讯云 IM：
|
||
|
||
```yaml
|
||
wallet_database: "hyapp_wallet"
|
||
room_database: "hyapp_room"
|
||
|
||
tencent_im:
|
||
enabled: false
|
||
|
||
wallet_notice_worker:
|
||
enabled: false
|
||
|
||
room_notice_worker:
|
||
enabled: false
|
||
|
||
rocketmq:
|
||
enabled: false
|
||
room_outbox:
|
||
enabled: false
|
||
topic: "hyapp_room_outbox"
|
||
consumer_group: "hyapp-notice-room-outbox"
|
||
```
|
||
|
||
线上启用私有通知必须同时开启腾讯云 IM，并配置对应 owner outbox 所在库名：
|
||
|
||
```yaml
|
||
wallet_database: "hyapp_wallet"
|
||
room_database: "hyapp_room"
|
||
|
||
tencent_im:
|
||
enabled: true
|
||
sdk_app_id: 1400000000
|
||
secret_key: "${TENCENT_IM_SECRET_KEY}"
|
||
admin_identifier: "administrator"
|
||
|
||
wallet_notice_worker:
|
||
enabled: true
|
||
poll_interval: 500ms
|
||
batch_size: 100
|
||
lock_ttl: 30s
|
||
max_retry_count: 10
|
||
|
||
room_notice_worker:
|
||
enabled: true
|
||
poll_interval: 500ms
|
||
batch_size: 100
|
||
lock_ttl: 30s
|
||
max_retry_count: 10
|
||
|
||
rocketmq:
|
||
enabled: true
|
||
name_servers:
|
||
- "TENCENT_ROCKETMQ_NAMESERVER:9876"
|
||
access_key: "TENCENT_ROCKETMQ_ACCESS_KEY"
|
||
secret_key: "TENCENT_ROCKETMQ_SECRET_KEY"
|
||
room_outbox:
|
||
enabled: true
|
||
topic: "hyapp_room_outbox"
|
||
consumer_group: "hyapp-notice-room-outbox"
|
||
consumer_max_reconsume_times: 16
|
||
```
|
||
|
||
`notice-service` 使用 `13009` 作为 gRPC health 端口，`13109` 作为 HTTP health 端口。
|
||
|
||
启用 MQ 消费后，可以按部署阶段选择是否保留 MySQL 扫描 worker。两者同时开启时不会重复发通知，因为 `notice_delivery_events` 的主键会挡住同一 `event_id + channel` 的重复投递；长期线上推荐以 MQ consumer 作为实时入口，MySQL 扫描只作为补偿或独立验证工具。
|
||
|
||
## 后续扩展顺序
|
||
|
||
1. 钱包余额通知：已按 `walletnotice` 模块落地。
|
||
2. 房间踢人私有通知：已按 `roomnotice` 模块落地，只消费 `RoomUserKicked` 的 C2C 通知。
|
||
3. 幸运礼物中奖：仍由 `wallet-service` 写余额 outbox；notice 不需要新增特殊账务逻辑，只要客户端按 `balance_version` 更新。
|
||
4. 语音房宝箱私有通知：如产品需要私聊提醒，可在 `roomnotice` 中消费 `RoomTreasureRewardGranted` 或由活动/系统消息模块转发；奖励入账事实仍由 room-service 调 wallet 完成，notice 只投通知。
|
||
5. 活动/系统私有通知：新增 `activitynotice` 模块，读取 activity outbox，复用 `notice_delivery_events` 或新增同语义位点表。
|
||
6. 站内 inbox：新增 inbox 模块，写 notice-service 自己的 inbox 表，再按客户端分页接口读取。
|
||
7. Push：新增 push 模块，和 IM 投递使用不同 `channel`，独立重试和死信。
|
||
|
||
## 真实库验证
|
||
|
||
默认测试不会访问本地 MySQL。需要验证真实表链路时，显式传入 DSN：
|
||
|
||
```bash
|
||
NOTICE_REAL_MYSQL_DSN='hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_notice?parseTime=true&charset=utf8mb4&loc=UTC' \
|
||
NOTICE_REAL_WALLET_DATABASE=hyapp_wallet \
|
||
go test ./services/notice-service/internal/modules/walletnotice \
|
||
-run TestMySQLRepositoryProcessesRealWalletOutbox -count=1 -v
|
||
```
|
||
|
||
如果需要保留本次验证行用于查库，加：
|
||
|
||
```bash
|
||
NOTICE_REAL_MYSQL_KEEP_ROWS=true
|
||
```
|
||
|
||
验证内容：
|
||
|
||
- 往 `hyapp_wallet.wallet_outbox` 插入一条真实 `WalletBalanceChanged`。
|
||
- `walletnotice` 从真实 MySQL 抢占该事件。
|
||
- fake publisher 代替腾讯云 IM，避免误发真实消息。
|
||
- `hyapp_notice.notice_delivery_events` 写入 `delivered`，并保存最终客户端 payload。
|