新增部署相关
This commit is contained in:
parent
8955a0d118
commit
1e542ba563
@ -143,15 +143,18 @@ MySQL 是本地和线上运行的持久化底座:
|
|||||||
`room-service` uses Redis for current room ownership:
|
`room-service` uses Redis for current room ownership:
|
||||||
|
|
||||||
```text
|
```text
|
||||||
room:route:{room_id}
|
room:route:{app_code + "\0" + room_id}
|
||||||
|
room:node:{node_id}
|
||||||
```
|
```
|
||||||
|
|
||||||
Redis lease 只保存当前执行权,不能作为恢复来源。恢复来源是 MySQL snapshot + command log。
|
Redis lease 只保存当前执行权和节点转发地址,不能作为恢复来源。恢复来源是 MySQL snapshot + command log。
|
||||||
|
|
||||||
## Room Recovery
|
## Room Recovery
|
||||||
|
|
||||||
A room is not permanently bound to one process. A node is only the current executor while its Redis lease is valid.
|
A room is not permanently bound to one process. A node is only the current executor while its Redis lease is valid.
|
||||||
|
|
||||||
|
在多机部署下,gateway 可以把 room gRPC 请求打到 `room-service` 内网 CLB。非 owner 节点会读取 `room:route:*` 和 `room:node:*`,把房间命令、IM guard 和完整快照查询转发到当前 owner 实例。每个 room 节点必须配置唯一 `node_id` 和实例级 `advertise_addr`;`advertise_addr` 不能写负载均衡地址。无 Kubernetes 的部署步骤见 `docs/多机部署与无感发布.md`。
|
||||||
|
|
||||||
Recovery path:
|
Recovery path:
|
||||||
|
|
||||||
1. A request reaches a node that can own the room.
|
1. A request reaches a node that can own the room.
|
||||||
|
|||||||
71
deploy/standalone/README.md
Normal file
71
deploy/standalone/README.md
Normal file
@ -0,0 +1,71 @@
|
|||||||
|
# Standalone CVM Deployment Templates
|
||||||
|
|
||||||
|
本目录提供不依赖 Kubernetes 的生产部署模板。目标形态是 systemd 托管 Docker 容器,公网 CLB 只接 gateway,内部服务通过内网 CLB/DNS 或固定私网地址访问。
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
- `systemd/*.service`: 每个后端服务一份 systemd unit,负责启动、停止和故障重启 Docker 容器。
|
||||||
|
- `docker/*.env.example`: 每个服务一份 Docker 运行参数示例,只保存镜像名、容器名、配置路径和停止等待时间。
|
||||||
|
- 服务业务配置仍使用 `services/<service>/configs/config.tencent.example.yaml` 作为线上模板,渲染后放到 `/etc/hyapp/<service>/config.yaml`。
|
||||||
|
|
||||||
|
## Install One Service
|
||||||
|
|
||||||
|
以 `gateway-service` 为例:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo mkdir -p /etc/hyapp/gateway-service
|
||||||
|
sudo cp deploy/standalone/docker/gateway-service.env.example /etc/hyapp/gateway-service/docker.env
|
||||||
|
sudo cp services/gateway-service/configs/config.tencent.example.yaml /etc/hyapp/gateway-service/config.yaml
|
||||||
|
sudo vi /etc/hyapp/gateway-service/docker.env
|
||||||
|
sudo vi /etc/hyapp/gateway-service/config.yaml
|
||||||
|
|
||||||
|
sudo cp deploy/standalone/systemd/hyapp-gateway-service.service /etc/systemd/system/
|
||||||
|
sudo systemctl daemon-reload
|
||||||
|
sudo systemctl enable --now hyapp-gateway-service
|
||||||
|
sudo systemctl status hyapp-gateway-service
|
||||||
|
sudo docker inspect --format '{{json .State.Health}}' hyapp-gateway-service
|
||||||
|
```
|
||||||
|
|
||||||
|
`docker.env` 不承载业务密钥;密钥仍在最终 `config.yaml` 中,或者由发布系统先从密钥系统渲染到 `config.yaml`。当前代码只读取单个 YAML,不会自动展开 `${VAR}`。
|
||||||
|
|
||||||
|
## Network
|
||||||
|
|
||||||
|
模板默认使用 `--network=host`,让容器直接监听项目约定的 `13xxx` 端口。这样 CLB、安全组和本机排障都以同一个端口工作:
|
||||||
|
|
||||||
|
- `13000`: gateway HTTP
|
||||||
|
- `13001`: room gRPC
|
||||||
|
- `13004`: wallet gRPC
|
||||||
|
- `13005`: user gRPC
|
||||||
|
- `13006`: activity gRPC
|
||||||
|
- `13007`: cron gRPC
|
||||||
|
- `13008`: game gRPC
|
||||||
|
|
||||||
|
同一台 CVM 上不要启动两个相同端口的同服务实例。做蓝绿发布时优先使用新 CVM 或新容器节点,健康通过后再挂载到 CLB。
|
||||||
|
|
||||||
|
## Rolling Update
|
||||||
|
|
||||||
|
普通服务的无感更新流程:
|
||||||
|
|
||||||
|
1. 在新 CVM 上准备 `/etc/hyapp/<service>/docker.env` 和 `/etc/hyapp/<service>/config.yaml`。
|
||||||
|
2. `systemctl start hyapp-<service>`,等待 Docker health 进入 `healthy`。
|
||||||
|
3. 将新实例加入对应 CLB 或内部服务入口。
|
||||||
|
4. 观察错误率、延迟和服务日志。
|
||||||
|
5. 从 CLB 摘除旧实例,等待连接保护或业务 drain 窗口结束。
|
||||||
|
6. `systemctl stop hyapp-<service>`,Docker 会先发送 SIGTERM,服务内的 graceful shutdown 会进入 draining 并等待已进入请求结束。
|
||||||
|
|
||||||
|
`room-service` 当前还没有 gateway owner-routing 或非 owner 自动转发能力。没有补完该能力前,不要把多个 `room-service` active 实例直接放到普通负载均衡后面承接同一批房间命令;生产先保持单 active,或者只把备用实例作为故障接管目标。
|
||||||
|
|
||||||
|
## Service Discovery
|
||||||
|
|
||||||
|
不使用 Kubernetes 时,第一阶段推荐用内网 CLB/DNS 提供稳定入口,例如:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
user_service_addr: "user.service.internal:13005"
|
||||||
|
wallet_service_addr: "wallet.service.internal:13004"
|
||||||
|
activity_service_addr: "activity.service.internal:13006"
|
||||||
|
room_service_addr: "room.service.internal:13001"
|
||||||
|
game_service_addr: "game.service.internal:13008"
|
||||||
|
```
|
||||||
|
|
||||||
|
如果后续引入 Consul/Nacos,仍建议让业务配置只保存稳定服务名,不把实例 IP 写进每个服务配置。
|
||||||
|
|
||||||
6
deploy/standalone/docker/activity-service.env.example
Normal file
6
deploy/standalone/docker/activity-service.env.example
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
# Image is immutable per release; publish scripts pull this tag before restarting systemd.
|
||||||
|
IMAGE=ccr.ccs.tencentyun.com/hyapp/activity-service:REPLACE_WITH_RELEASE_TAG
|
||||||
|
CONTAINER_NAME=hyapp-activity-service
|
||||||
|
CONFIG_PATH=/etc/hyapp/activity-service/config.yaml
|
||||||
|
STOP_TIMEOUT_SEC=60
|
||||||
|
|
||||||
6
deploy/standalone/docker/cron-service.env.example
Normal file
6
deploy/standalone/docker/cron-service.env.example
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
# cron-service already uses MySQL task leases, but keep one replica until every task is verified under multi-node scheduling.
|
||||||
|
IMAGE=ccr.ccs.tencentyun.com/hyapp/cron-service:REPLACE_WITH_RELEASE_TAG
|
||||||
|
CONTAINER_NAME=hyapp-cron-service
|
||||||
|
CONFIG_PATH=/etc/hyapp/cron-service/config.yaml
|
||||||
|
STOP_TIMEOUT_SEC=90
|
||||||
|
|
||||||
5
deploy/standalone/docker/game-service.env.example
Normal file
5
deploy/standalone/docker/game-service.env.example
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
IMAGE=ccr.ccs.tencentyun.com/hyapp/game-service:REPLACE_WITH_RELEASE_TAG
|
||||||
|
CONTAINER_NAME=hyapp-game-service
|
||||||
|
CONFIG_PATH=/etc/hyapp/game-service/config.yaml
|
||||||
|
STOP_TIMEOUT_SEC=60
|
||||||
|
|
||||||
6
deploy/standalone/docker/gateway-service.env.example
Normal file
6
deploy/standalone/docker/gateway-service.env.example
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
# gateway instances are stateless HTTP entries and should be attached behind the public CLB only after /healthz/ready passes.
|
||||||
|
IMAGE=ccr.ccs.tencentyun.com/hyapp/gateway-service:REPLACE_WITH_RELEASE_TAG
|
||||||
|
CONTAINER_NAME=hyapp-gateway-service
|
||||||
|
CONFIG_PATH=/etc/hyapp/gateway-service/config.yaml
|
||||||
|
STOP_TIMEOUT_SEC=60
|
||||||
|
|
||||||
6
deploy/standalone/docker/room-service.env.example
Normal file
6
deploy/standalone/docker/room-service.env.example
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
# Keep room-service single-active until owner routing or non-owner forwarding is implemented.
|
||||||
|
IMAGE=ccr.ccs.tencentyun.com/hyapp/room-service:REPLACE_WITH_RELEASE_TAG
|
||||||
|
CONTAINER_NAME=hyapp-room-service
|
||||||
|
CONFIG_PATH=/etc/hyapp/room-service/config.yaml
|
||||||
|
STOP_TIMEOUT_SEC=90
|
||||||
|
|
||||||
6
deploy/standalone/docker/user-service.env.example
Normal file
6
deploy/standalone/docker/user-service.env.example
Normal file
@ -0,0 +1,6 @@
|
|||||||
|
# id_generator.node_id in config.yaml must be unique for every active user-service instance.
|
||||||
|
IMAGE=ccr.ccs.tencentyun.com/hyapp/user-service:REPLACE_WITH_RELEASE_TAG
|
||||||
|
CONTAINER_NAME=hyapp-user-service
|
||||||
|
CONFIG_PATH=/etc/hyapp/user-service/config.yaml
|
||||||
|
STOP_TIMEOUT_SEC=60
|
||||||
|
|
||||||
5
deploy/standalone/docker/wallet-service.env.example
Normal file
5
deploy/standalone/docker/wallet-service.env.example
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
IMAGE=ccr.ccs.tencentyun.com/hyapp/wallet-service:REPLACE_WITH_RELEASE_TAG
|
||||||
|
CONTAINER_NAME=hyapp-wallet-service
|
||||||
|
CONFIG_PATH=/etc/hyapp/wallet-service/config.yaml
|
||||||
|
STOP_TIMEOUT_SEC=60
|
||||||
|
|
||||||
24
deploy/standalone/systemd/hyapp-activity-service.service
Normal file
24
deploy/standalone/systemd/hyapp-activity-service.service
Normal file
@ -0,0 +1,24 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Hyapp activity-service container
|
||||||
|
Requires=docker.service
|
||||||
|
After=network-online.target docker.service
|
||||||
|
Wants=network-online.target
|
||||||
|
StartLimitIntervalSec=120
|
||||||
|
StartLimitBurst=3
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
EnvironmentFile=/etc/hyapp/activity-service/docker.env
|
||||||
|
ExecStartPre=-/usr/bin/docker rm -f ${CONTAINER_NAME}
|
||||||
|
ExecStart=/usr/bin/docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --health-cmd="/app/grpc-health-probe -addr=127.0.0.1:13006 -service=activity-service" --health-interval=5s --health-timeout=3s --health-retries=6 --health-start-period=10s ${IMAGE}
|
||||||
|
ExecStop=/usr/bin/docker stop --time=${STOP_TIMEOUT_SEC} ${CONTAINER_NAME}
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=5s
|
||||||
|
TimeoutStartSec=60
|
||||||
|
TimeoutStopSec=90
|
||||||
|
KillMode=none
|
||||||
|
LimitNOFILE=1048576
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
|
||||||
24
deploy/standalone/systemd/hyapp-cron-service.service
Normal file
24
deploy/standalone/systemd/hyapp-cron-service.service
Normal file
@ -0,0 +1,24 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Hyapp cron-service container
|
||||||
|
Requires=docker.service
|
||||||
|
After=network-online.target docker.service
|
||||||
|
Wants=network-online.target
|
||||||
|
StartLimitIntervalSec=120
|
||||||
|
StartLimitBurst=3
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
EnvironmentFile=/etc/hyapp/cron-service/docker.env
|
||||||
|
ExecStartPre=-/usr/bin/docker rm -f ${CONTAINER_NAME}
|
||||||
|
ExecStart=/usr/bin/docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --health-cmd="/app/grpc-health-probe -addr=127.0.0.1:13007 -service=cron-service" --health-interval=5s --health-timeout=3s --health-retries=6 --health-start-period=10s ${IMAGE}
|
||||||
|
ExecStop=/usr/bin/docker stop --time=${STOP_TIMEOUT_SEC} ${CONTAINER_NAME}
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=5s
|
||||||
|
TimeoutStartSec=60
|
||||||
|
TimeoutStopSec=120
|
||||||
|
KillMode=none
|
||||||
|
LimitNOFILE=1048576
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
|
||||||
24
deploy/standalone/systemd/hyapp-game-service.service
Normal file
24
deploy/standalone/systemd/hyapp-game-service.service
Normal file
@ -0,0 +1,24 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Hyapp game-service container
|
||||||
|
Requires=docker.service
|
||||||
|
After=network-online.target docker.service
|
||||||
|
Wants=network-online.target
|
||||||
|
StartLimitIntervalSec=120
|
||||||
|
StartLimitBurst=3
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
EnvironmentFile=/etc/hyapp/game-service/docker.env
|
||||||
|
ExecStartPre=-/usr/bin/docker rm -f ${CONTAINER_NAME}
|
||||||
|
ExecStart=/usr/bin/docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --health-cmd="/app/grpc-health-probe -addr=127.0.0.1:13008 -service=game-service" --health-interval=5s --health-timeout=3s --health-retries=6 --health-start-period=10s ${IMAGE}
|
||||||
|
ExecStop=/usr/bin/docker stop --time=${STOP_TIMEOUT_SEC} ${CONTAINER_NAME}
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=5s
|
||||||
|
TimeoutStartSec=60
|
||||||
|
TimeoutStopSec=90
|
||||||
|
KillMode=none
|
||||||
|
LimitNOFILE=1048576
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
|
||||||
24
deploy/standalone/systemd/hyapp-gateway-service.service
Normal file
24
deploy/standalone/systemd/hyapp-gateway-service.service
Normal file
@ -0,0 +1,24 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Hyapp gateway-service container
|
||||||
|
Requires=docker.service
|
||||||
|
After=network-online.target docker.service
|
||||||
|
Wants=network-online.target
|
||||||
|
StartLimitIntervalSec=120
|
||||||
|
StartLimitBurst=3
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
EnvironmentFile=/etc/hyapp/gateway-service/docker.env
|
||||||
|
ExecStartPre=-/usr/bin/docker rm -f ${CONTAINER_NAME}
|
||||||
|
ExecStart=/usr/bin/docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --health-cmd="wget -q -O - http://127.0.0.1:13000/healthz/ready >/dev/null" --health-interval=5s --health-timeout=3s --health-retries=6 --health-start-period=10s ${IMAGE}
|
||||||
|
ExecStop=/usr/bin/docker stop --time=${STOP_TIMEOUT_SEC} ${CONTAINER_NAME}
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=5s
|
||||||
|
TimeoutStartSec=60
|
||||||
|
TimeoutStopSec=90
|
||||||
|
KillMode=none
|
||||||
|
LimitNOFILE=1048576
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
|
||||||
24
deploy/standalone/systemd/hyapp-room-service.service
Normal file
24
deploy/standalone/systemd/hyapp-room-service.service
Normal file
@ -0,0 +1,24 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Hyapp room-service container
|
||||||
|
Requires=docker.service
|
||||||
|
After=network-online.target docker.service
|
||||||
|
Wants=network-online.target
|
||||||
|
StartLimitIntervalSec=120
|
||||||
|
StartLimitBurst=3
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
EnvironmentFile=/etc/hyapp/room-service/docker.env
|
||||||
|
ExecStartPre=-/usr/bin/docker rm -f ${CONTAINER_NAME}
|
||||||
|
ExecStart=/usr/bin/docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --health-cmd="/app/grpc-health-probe -addr=127.0.0.1:13001 -service=room-service" --health-interval=5s --health-timeout=3s --health-retries=6 --health-start-period=10s ${IMAGE}
|
||||||
|
ExecStop=/usr/bin/docker stop --time=${STOP_TIMEOUT_SEC} ${CONTAINER_NAME}
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=5s
|
||||||
|
TimeoutStartSec=60
|
||||||
|
TimeoutStopSec=120
|
||||||
|
KillMode=none
|
||||||
|
LimitNOFILE=1048576
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
|
||||||
24
deploy/standalone/systemd/hyapp-user-service.service
Normal file
24
deploy/standalone/systemd/hyapp-user-service.service
Normal file
@ -0,0 +1,24 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Hyapp user-service container
|
||||||
|
Requires=docker.service
|
||||||
|
After=network-online.target docker.service
|
||||||
|
Wants=network-online.target
|
||||||
|
StartLimitIntervalSec=120
|
||||||
|
StartLimitBurst=3
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
EnvironmentFile=/etc/hyapp/user-service/docker.env
|
||||||
|
ExecStartPre=-/usr/bin/docker rm -f ${CONTAINER_NAME}
|
||||||
|
ExecStart=/usr/bin/docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --health-cmd="/app/grpc-health-probe -addr=127.0.0.1:13005 -service=user-service" --health-interval=5s --health-timeout=3s --health-retries=6 --health-start-period=10s ${IMAGE}
|
||||||
|
ExecStop=/usr/bin/docker stop --time=${STOP_TIMEOUT_SEC} ${CONTAINER_NAME}
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=5s
|
||||||
|
TimeoutStartSec=60
|
||||||
|
TimeoutStopSec=90
|
||||||
|
KillMode=none
|
||||||
|
LimitNOFILE=1048576
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
|
||||||
24
deploy/standalone/systemd/hyapp-wallet-service.service
Normal file
24
deploy/standalone/systemd/hyapp-wallet-service.service
Normal file
@ -0,0 +1,24 @@
|
|||||||
|
[Unit]
|
||||||
|
Description=Hyapp wallet-service container
|
||||||
|
Requires=docker.service
|
||||||
|
After=network-online.target docker.service
|
||||||
|
Wants=network-online.target
|
||||||
|
StartLimitIntervalSec=120
|
||||||
|
StartLimitBurst=3
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=simple
|
||||||
|
EnvironmentFile=/etc/hyapp/wallet-service/docker.env
|
||||||
|
ExecStartPre=-/usr/bin/docker rm -f ${CONTAINER_NAME}
|
||||||
|
ExecStart=/usr/bin/docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --health-cmd="/app/grpc-health-probe -addr=127.0.0.1:13004 -service=wallet-service" --health-interval=5s --health-timeout=3s --health-retries=6 --health-start-period=10s ${IMAGE}
|
||||||
|
ExecStop=/usr/bin/docker stop --time=${STOP_TIMEOUT_SEC} ${CONTAINER_NAME}
|
||||||
|
Restart=on-failure
|
||||||
|
RestartSec=5s
|
||||||
|
TimeoutStartSec=60
|
||||||
|
TimeoutStopSec=90
|
||||||
|
KillMode=none
|
||||||
|
LimitNOFILE=1048576
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=multi-user.target
|
||||||
|
|
||||||
@ -39,6 +39,7 @@ services:
|
|||||||
TZ: UTC
|
TZ: UTC
|
||||||
ports:
|
ports:
|
||||||
- "13001:13001"
|
- "13001:13001"
|
||||||
|
- "13101:13101"
|
||||||
depends_on:
|
depends_on:
|
||||||
mysql:
|
mysql:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
@ -47,7 +48,7 @@ services:
|
|||||||
wallet-service:
|
wallet-service:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD-SHELL", "/app/grpc-health-probe -addr=127.0.0.1:13001 -service=room-service"]
|
test: ["CMD-SHELL", "wget -q -O - http://127.0.0.1:13101/healthz/ready >/dev/null"]
|
||||||
interval: 5s
|
interval: 5s
|
||||||
timeout: 3s
|
timeout: 3s
|
||||||
retries: 20
|
retries: 20
|
||||||
@ -63,6 +64,7 @@ services:
|
|||||||
TZ: UTC
|
TZ: UTC
|
||||||
ports:
|
ports:
|
||||||
- "13004:13004"
|
- "13004:13004"
|
||||||
|
- "13104:13104"
|
||||||
depends_on:
|
depends_on:
|
||||||
mysql:
|
mysql:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
@ -71,7 +73,7 @@ services:
|
|||||||
activity-service:
|
activity-service:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD-SHELL", "/app/grpc-health-probe -addr=127.0.0.1:13004 -service=wallet-service"]
|
test: ["CMD-SHELL", "wget -q -O - http://127.0.0.1:13104/healthz/ready >/dev/null"]
|
||||||
interval: 5s
|
interval: 5s
|
||||||
timeout: 3s
|
timeout: 3s
|
||||||
retries: 20
|
retries: 20
|
||||||
@ -87,11 +89,12 @@ services:
|
|||||||
TZ: UTC
|
TZ: UTC
|
||||||
ports:
|
ports:
|
||||||
- "13005:13005"
|
- "13005:13005"
|
||||||
|
- "13105:13105"
|
||||||
depends_on:
|
depends_on:
|
||||||
mysql:
|
mysql:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD-SHELL", "/app/grpc-health-probe -addr=127.0.0.1:13005 -service=user-service"]
|
test: ["CMD-SHELL", "wget -q -O - http://127.0.0.1:13105/healthz/ready >/dev/null"]
|
||||||
interval: 5s
|
interval: 5s
|
||||||
timeout: 3s
|
timeout: 3s
|
||||||
retries: 20
|
retries: 20
|
||||||
@ -107,13 +110,14 @@ services:
|
|||||||
TZ: UTC
|
TZ: UTC
|
||||||
ports:
|
ports:
|
||||||
- "13006:13006"
|
- "13006:13006"
|
||||||
|
- "13106:13106"
|
||||||
depends_on:
|
depends_on:
|
||||||
mysql:
|
mysql:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
user-service:
|
user-service:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD-SHELL", "/app/grpc-health-probe -addr=127.0.0.1:13006 -service=activity-service"]
|
test: ["CMD-SHELL", "wget -q -O - http://127.0.0.1:13106/healthz/ready >/dev/null"]
|
||||||
interval: 5s
|
interval: 5s
|
||||||
timeout: 3s
|
timeout: 3s
|
||||||
retries: 20
|
retries: 20
|
||||||
@ -129,6 +133,7 @@ services:
|
|||||||
TZ: UTC
|
TZ: UTC
|
||||||
ports:
|
ports:
|
||||||
- "13007:13007"
|
- "13007:13007"
|
||||||
|
- "13107:13107"
|
||||||
depends_on:
|
depends_on:
|
||||||
mysql:
|
mysql:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
@ -137,7 +142,7 @@ services:
|
|||||||
activity-service:
|
activity-service:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD-SHELL", "/app/grpc-health-probe -addr=127.0.0.1:13007 -service=cron-service"]
|
test: ["CMD-SHELL", "wget -q -O - http://127.0.0.1:13107/healthz/ready >/dev/null"]
|
||||||
interval: 5s
|
interval: 5s
|
||||||
timeout: 3s
|
timeout: 3s
|
||||||
retries: 20
|
retries: 20
|
||||||
@ -153,6 +158,7 @@ services:
|
|||||||
TZ: UTC
|
TZ: UTC
|
||||||
ports:
|
ports:
|
||||||
- "13008:13008"
|
- "13008:13008"
|
||||||
|
- "13108:13108"
|
||||||
depends_on:
|
depends_on:
|
||||||
mysql:
|
mysql:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
@ -161,7 +167,7 @@ services:
|
|||||||
user-service:
|
user-service:
|
||||||
condition: service_healthy
|
condition: service_healthy
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD-SHELL", "/app/grpc-health-probe -addr=127.0.0.1:13008 -service=game-service"]
|
test: ["CMD-SHELL", "wget -q -O - http://127.0.0.1:13108/healthz/ready >/dev/null"]
|
||||||
interval: 5s
|
interval: 5s
|
||||||
timeout: 3s
|
timeout: 3s
|
||||||
retries: 20
|
retries: 20
|
||||||
|
|||||||
@ -126,7 +126,8 @@ SIGINT
|
|||||||
- 对已经进入处理中的命令,允许在 deadline 内完成提交。
|
- 对已经进入处理中的命令,允许在 deadline 内完成提交。
|
||||||
- 命令提交后必须完成必要 command log、snapshot dirty 标记和 outbox 写入。
|
- 命令提交后必须完成必要 command log、snapshot dirty 标记和 outbox 写入。
|
||||||
- 对本节点持有的 Room Cell 执行 snapshot flush。
|
- 对本节点持有的 Room Cell 执行 snapshot flush。
|
||||||
- 停止 lease 续约,或按实现安全释放 lease。
|
- 停止 node registry 心跳;gRPC drain 完成后删除 `room:node:{node_id}`。
|
||||||
|
- gRPC drain 完成后释放本节点仍持有的已加载房间 lease,让其他节点无需等待 `lease_ttl` 才能接管。
|
||||||
- outbox worker 处理完当前批次,未处理事件保持在 MySQL 中等待其他 worker 或下次启动补投。
|
- outbox worker 处理完当前批次,未处理事件保持在 MySQL 中等待其他 worker 或下次启动补投。
|
||||||
|
|
||||||
Room Cell 下线顺序:
|
Room Cell 下线顺序:
|
||||||
@ -137,7 +138,9 @@ Room Cell 下线顺序:
|
|||||||
3. command log/outbox transaction completes
|
3. command log/outbox transaction completes
|
||||||
4. snapshot flush runs with timeout
|
4. snapshot flush runs with timeout
|
||||||
5. lease renew stops
|
5. lease renew stops
|
||||||
6. cell is removed from local manager
|
6. owner route is released after in-flight commands finish
|
||||||
|
7. node registry is removed
|
||||||
|
8. cell is removed from local manager
|
||||||
```
|
```
|
||||||
|
|
||||||
如果 shutdown timeout 到期:
|
如果 shutdown timeout 到期:
|
||||||
@ -145,6 +148,7 @@ Room Cell 下线顺序:
|
|||||||
- 不能删除未安全释放的持久状态。
|
- 不能删除未安全释放的持久状态。
|
||||||
- 不能清空 outbox。
|
- 不能清空 outbox。
|
||||||
- 可以让 Redis lease 自然过期,由其他节点通过 snapshot + command log 恢复。
|
- 可以让 Redis lease 自然过期,由其他节点通过 snapshot + command log 恢复。
|
||||||
|
- 如果主动释放 lease 未完成,不能强删 MySQL 状态;让 Redis TTL 自然过期是安全兜底。
|
||||||
|
|
||||||
禁止行为:
|
禁止行为:
|
||||||
|
|
||||||
|
|||||||
@ -147,6 +147,7 @@ activity-service
|
|||||||
- 当前节点不处于 draining 状态。
|
- 当前节点不处于 draining 状态。
|
||||||
- gRPC listener 已启动。
|
- gRPC listener 已启动。
|
||||||
- Redis lease 操作能力正常,至少能执行一次轻量读写或 Lua 脚本探测。
|
- Redis lease 操作能力正常,至少能执行一次轻量读写或 Lua 脚本探测。
|
||||||
|
- 节点启动时必须能写入 `room:node:{node_id}`;运行期心跳失败要记录告警,避免其他 room 节点无法转发到当前 owner。
|
||||||
|
|
||||||
`wallet-service` 建议作为 `SendGift` 链路的强依赖监控,但不要默认作为 `room-service` 整体 ready 的硬失败条件;否则 wallet 短暂异常会导致房间基础命令也被摘流。
|
`wallet-service` 建议作为 `SendGift` 链路的强依赖监控,但不要默认作为 `room-service` 整体 ready 的硬失败条件;否则 wallet 短暂异常会导致房间基础命令也被摘流。
|
||||||
|
|
||||||
|
|||||||
112
docs/多机部署与无感发布.md
Normal file
112
docs/多机部署与无感发布.md
Normal file
@ -0,0 +1,112 @@
|
|||||||
|
# 多机部署与无感发布
|
||||||
|
|
||||||
|
本文定义不使用 Kubernetes 时的生产部署边界。目标是让 gateway 和内部微服务可以多机滚动发布,并让 `room-service` 在普通内网负载均衡命中非 owner 节点时自动转发到当前 Room Cell owner。
|
||||||
|
|
||||||
|
## 网络拓扑
|
||||||
|
|
||||||
|
推荐拓扑:
|
||||||
|
|
||||||
|
```text
|
||||||
|
公网 CLB
|
||||||
|
-> gateway-service 多实例
|
||||||
|
-> room-service 内网入口
|
||||||
|
-> user-service 内网入口
|
||||||
|
-> wallet-service 内网入口
|
||||||
|
-> activity-service 内网入口
|
||||||
|
-> game-service 内网入口
|
||||||
|
|
||||||
|
room-service 实例之间通过 Redis owner route 和 node registry 直连转发。
|
||||||
|
```
|
||||||
|
|
||||||
|
规则:
|
||||||
|
|
||||||
|
- 公网只暴露 `gateway-service`。
|
||||||
|
- 内部 gRPC 服务只监听私网,安全组只允许服务网段访问 `13xxx` 端口。
|
||||||
|
- `user-service`、`wallet-service`、`activity-service`、`game-service` 可以直接挂内网 CLB 或 DNS。
|
||||||
|
- `room-service` 可以挂内网 CLB/DNS 给 gateway 使用,但 owner 转发必须使用实例级 `advertise_addr`,不能使用 CLB 地址。
|
||||||
|
|
||||||
|
## Room Owner 路由
|
||||||
|
|
||||||
|
`room-service` 使用 Redis 维护两个短租约:
|
||||||
|
|
||||||
|
```text
|
||||||
|
room:route:{app_code + "\0" + room_id} -> owner node_id / lease_token / expires_at_ms
|
||||||
|
room:node:{node_id} -> 实例 advertise_addr / expires_at_ms
|
||||||
|
```
|
||||||
|
|
||||||
|
写命令、IM guard 和完整房间快照查询的处理顺序:
|
||||||
|
|
||||||
|
1. 请求先到任意 `room-service` 节点。
|
||||||
|
2. 节点按 `app_code + "\0" + room_id` 查询 Redis owner route。
|
||||||
|
3. 如果当前节点就是 owner,直接进入本地 Room Cell。
|
||||||
|
4. 如果 owner 是其他节点,按 `room:node:{owner_node_id}` 找到实例私网地址并转发 gRPC。
|
||||||
|
5. 如果 route 不存在或已过期,当前节点按原有 `EnsureOwner` 逻辑接管并从 MySQL snapshot + command log 恢复。
|
||||||
|
|
||||||
|
转发只做一跳。`advertise_addr` 写错成 CLB 地址时,请求会在一跳后停止递归,并暴露真实 owner 冲突或 unavailable。
|
||||||
|
|
||||||
|
## room-service 配置
|
||||||
|
|
||||||
|
每个实例必须有唯一配置:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
node_id: "room-node-prod-1"
|
||||||
|
grpc_addr: ":13001"
|
||||||
|
advertise_addr: "10.0.1.12:13001"
|
||||||
|
health_http_addr: ":13101"
|
||||||
|
node_registry_ttl: "30s"
|
||||||
|
node_registry_heartbeat_interval: "10s"
|
||||||
|
owner_forward_timeout: "2s"
|
||||||
|
lease_ttl: "10s"
|
||||||
|
```
|
||||||
|
|
||||||
|
配置边界:
|
||||||
|
|
||||||
|
- `grpc_addr` 是本进程监听地址,可以是 `:13001`。
|
||||||
|
- `advertise_addr` 是其他 room 节点能直连的实例私网地址,必须是单实例地址。
|
||||||
|
- `node_id` 参与 room owner lease、节点注册和日志排障,不能在多个实例间复用。
|
||||||
|
- `node_registry_ttl` 必须大于心跳周期;默认 `30s/10s`。
|
||||||
|
- `lease_ttl` 是节点故障后的自动接管窗口;正常滚动发布会在 gRPC drain 后主动释放本节点已加载房间 lease,减少等待 TTL 的窗口。
|
||||||
|
|
||||||
|
## 无感发布步骤
|
||||||
|
|
||||||
|
单个服务滚动发布顺序:
|
||||||
|
|
||||||
|
1. 在新机器或新端口启动新版本实例。
|
||||||
|
2. 等 `health_http_addr` 的 `/readyz` 或 gRPC health 通过。
|
||||||
|
3. 把新实例加入对应 CLB 后端或内部 DNS。
|
||||||
|
4. 观察错误率、延迟和日志。
|
||||||
|
5. 从 CLB 摘除一个旧实例,等待 CLB 健康检查和连接排空周期。
|
||||||
|
6. 对旧实例发送 `SIGTERM`。
|
||||||
|
7. 旧实例进入 draining,gRPC 完成 in-flight 请求后释放已加载房间 lease,并删除 `room:node:{node_id}`。
|
||||||
|
8. 新请求命中新节点后,如果 Redis route 已释放,会立即接管并从 MySQL 恢复;如果旧实例异常退出,则等待 `lease_ttl` 到期后接管。
|
||||||
|
|
||||||
|
发布约束:
|
||||||
|
|
||||||
|
- 不要先杀进程再摘 CLB。
|
||||||
|
- 不要把 `room-service` 的 `advertise_addr` 写成内网 CLB。
|
||||||
|
- 不要多个实例复用同一个 `node_id`。
|
||||||
|
- 数据库变更要先执行兼容当前发布窗口的迁移,再滚动服务。
|
||||||
|
|
||||||
|
## 配置拆分
|
||||||
|
|
||||||
|
不用统一配置中心也可以上线。推荐发布系统渲染最终 `config.yaml`:
|
||||||
|
|
||||||
|
- 公共环境配置:MySQL/Redis 私网地址、日志等级、腾讯云 endpoint。
|
||||||
|
- 服务配置:端口、worker 周期、批量大小、超时。
|
||||||
|
- 实例配置:`node_id`、`advertise_addr`、`user-service.id_generator.node_id`。
|
||||||
|
- Secret:MySQL/Redis 密码、JWT secret、腾讯 IM/RTC/COS secret、callback token。
|
||||||
|
|
||||||
|
只有当需要动态配置、灰度下发、审计回滚时,再引入 Consul/Nacos/Apollo。当前代码的启动配置仍以单服务 YAML 为边界。
|
||||||
|
|
||||||
|
## 真实数据验证
|
||||||
|
|
||||||
|
本地真实 MySQL/Redis 验证:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose up -d mysql redis
|
||||||
|
ROOM_SERVICE_MYSQL_TEST_DSN='hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_room?parseTime=true&charset=utf8mb4&loc=UTC' \
|
||||||
|
ROOM_SERVICE_REDIS_TEST_ADDR='127.0.0.1:13379' \
|
||||||
|
go test -count=1 ./services/room-service/internal/transport/grpc -run TestOwnerForwardingUsesRedisRouteAndNodeRegistry
|
||||||
|
```
|
||||||
|
|
||||||
|
该测试会启动两个真实 gRPC `room-service` 实例,共用真实 MySQL schema 和 Redis route/node registry:在 node A 创建房间,通过 node B 调 `JoinRoom`、`CheckSpeakPermission` 和 `GetRoomSnapshot`,验证请求会转发到当前 owner。
|
||||||
@ -96,12 +96,15 @@ stateDiagram-v2
|
|||||||
规则:
|
规则:
|
||||||
|
|
||||||
- Redis runtime route key 统一使用 `app_code + "\0" + room_id`,包括默认 App;不能再使用裸 `room_id` 作为兼容 key。
|
- Redis runtime route key 统一使用 `app_code + "\0" + room_id`,包括默认 App;不能再使用裸 `room_id` 作为兼容 key。
|
||||||
|
- `room:node:{node_id}` 保存当前 room-service 实例的 `advertise_addr`,只用于 owner 转发;它不是服务发现的全局配置中心。
|
||||||
|
- gateway 可以连接 room-service 内网负载均衡;命中非 owner 节点时,gRPC 边界必须按 Redis owner route 转发到当前 owner 实例。
|
||||||
|
- `advertise_addr` 必须是实例私网地址,不能是 room-service 内网 CLB 地址,否则转发会命中任意节点并触发递归保护。
|
||||||
- 命令开始前通过 `EnsureOwner` 获得执行权。
|
- 命令开始前通过 `EnsureOwner` 获得执行权。
|
||||||
- 命令真正写 command log/outbox 前必须用 `lease_token` 再做一次 `VerifyOwner`。
|
- 命令真正写 command log/outbox 前必须用 `lease_token` 再做一次 `VerifyOwner`。
|
||||||
- 如果提交前 lease 已经过期或 token 已变化,旧节点必须返回 `UNAVAILABLE`,不能写房间状态。
|
- 如果提交前 lease 已经过期或 token 已变化,旧节点必须返回 `UNAVAILABLE`,不能写房间状态。
|
||||||
- `room_command_log` 保存 `owner_node_id` 和 `lease_token`,用于排查谁在什么 lease 下提交了命令。
|
- `room_command_log` 保存 `owner_node_id` 和 `lease_token`,用于排查谁在什么 lease 下提交了命令。
|
||||||
- stale presence worker 和 mic publish timeout worker 只能扫描本节点已加载 Cell;清理前必须确认当前 Redis lease 仍属于本节点,并在读完快照后再次用相同 `lease_token` 校验,最后由命令提交链路再做 fencing。
|
- stale presence worker 和 mic publish timeout worker 只能扫描本节点已加载 Cell;清理前必须确认当前 Redis lease 仍属于本节点,并在读完快照后再次用相同 `lease_token` 校验,最后由命令提交链路再做 fencing。
|
||||||
- 节点进入 draining 后不能再接收新的 Room Cell 命令,也不能恢复或创建新的 Cell。
|
- 节点进入 draining 后不能再接收新的 Room Cell 命令,也不能恢复或创建新的 Cell;gRPC drain 完成后应主动释放本节点已加载房间的 owner lease,降低滚动发布等待 TTL 的窗口。
|
||||||
|
|
||||||
## Outbox
|
## Outbox
|
||||||
|
|
||||||
|
|||||||
166
pkg/grpcclient/client.go
Normal file
166
pkg/grpcclient/client.go
Normal file
@ -0,0 +1,166 @@
|
|||||||
|
// Package grpcclient provides one production dial policy for internal gRPC
|
||||||
|
// clients. The policy keeps transport behavior out of business handlers.
|
||||||
|
package grpcclient
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"google.golang.org/grpc"
|
||||||
|
"google.golang.org/grpc/backoff"
|
||||||
|
"google.golang.org/grpc/credentials/insecure"
|
||||||
|
"google.golang.org/grpc/keepalive"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Config controls connection backoff, keepalive, retry and per-RPC deadlines.
|
||||||
|
type Config struct {
|
||||||
|
DefaultTimeout time.Duration
|
||||||
|
ConnectTimeout time.Duration
|
||||||
|
KeepaliveTime time.Duration
|
||||||
|
KeepaliveTimeout time.Duration
|
||||||
|
RetryMaxAttempts int
|
||||||
|
RetryInitialBackoff time.Duration
|
||||||
|
RetryMaxBackoff time.Duration
|
||||||
|
RetryBackoffMultiplier float64
|
||||||
|
RetryableStatusCodes []string
|
||||||
|
}
|
||||||
|
|
||||||
|
// DefaultConfig is conservative: retries are limited to one retry on
|
||||||
|
// UNAVAILABLE and must complete inside the caller's deadline.
|
||||||
|
func DefaultConfig() Config {
|
||||||
|
return Config{
|
||||||
|
DefaultTimeout: 5 * time.Second,
|
||||||
|
ConnectTimeout: 2 * time.Second,
|
||||||
|
KeepaliveTime: 30 * time.Second,
|
||||||
|
KeepaliveTimeout: 10 * time.Second,
|
||||||
|
RetryMaxAttempts: 2,
|
||||||
|
RetryInitialBackoff: 100 * time.Millisecond,
|
||||||
|
RetryMaxBackoff: 500 * time.Millisecond,
|
||||||
|
RetryBackoffMultiplier: 2,
|
||||||
|
RetryableStatusCodes: []string{"UNAVAILABLE"},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Normalize fills safe transport defaults and clamps retry attempts to grpc-go's
|
||||||
|
// supported range. A value below two disables explicit retry.
|
||||||
|
func Normalize(cfg Config) Config {
|
||||||
|
defaults := DefaultConfig()
|
||||||
|
if cfg.DefaultTimeout <= 0 {
|
||||||
|
cfg.DefaultTimeout = defaults.DefaultTimeout
|
||||||
|
}
|
||||||
|
if cfg.ConnectTimeout <= 0 {
|
||||||
|
cfg.ConnectTimeout = defaults.ConnectTimeout
|
||||||
|
}
|
||||||
|
if cfg.KeepaliveTime <= 0 {
|
||||||
|
cfg.KeepaliveTime = defaults.KeepaliveTime
|
||||||
|
}
|
||||||
|
if cfg.KeepaliveTimeout <= 0 {
|
||||||
|
cfg.KeepaliveTimeout = defaults.KeepaliveTimeout
|
||||||
|
}
|
||||||
|
if cfg.RetryMaxAttempts < 0 {
|
||||||
|
cfg.RetryMaxAttempts = 0
|
||||||
|
}
|
||||||
|
if cfg.RetryMaxAttempts == 0 {
|
||||||
|
cfg.RetryMaxAttempts = defaults.RetryMaxAttempts
|
||||||
|
}
|
||||||
|
if cfg.RetryMaxAttempts > 5 {
|
||||||
|
cfg.RetryMaxAttempts = 5
|
||||||
|
}
|
||||||
|
if cfg.RetryInitialBackoff <= 0 {
|
||||||
|
cfg.RetryInitialBackoff = defaults.RetryInitialBackoff
|
||||||
|
}
|
||||||
|
if cfg.RetryMaxBackoff <= 0 {
|
||||||
|
cfg.RetryMaxBackoff = defaults.RetryMaxBackoff
|
||||||
|
}
|
||||||
|
if cfg.RetryMaxBackoff < cfg.RetryInitialBackoff {
|
||||||
|
cfg.RetryMaxBackoff = cfg.RetryInitialBackoff
|
||||||
|
}
|
||||||
|
if cfg.RetryBackoffMultiplier <= 0 {
|
||||||
|
cfg.RetryBackoffMultiplier = defaults.RetryBackoffMultiplier
|
||||||
|
}
|
||||||
|
if len(cfg.RetryableStatusCodes) == 0 {
|
||||||
|
cfg.RetryableStatusCodes = defaults.RetryableStatusCodes
|
||||||
|
}
|
||||||
|
return cfg
|
||||||
|
}
|
||||||
|
|
||||||
|
// Dial creates a ClientConn with deadlines, TCP reconnect backoff, keepalive and
|
||||||
|
// a minimal retry policy. Startup does not block on the remote endpoint; ready
|
||||||
|
// probes are responsible for proving the connection can become READY.
|
||||||
|
func Dial(target string, cfg Config, opts ...grpc.DialOption) (*grpc.ClientConn, error) {
|
||||||
|
cfg = Normalize(cfg)
|
||||||
|
dialOptions := []grpc.DialOption{
|
||||||
|
grpc.WithTransportCredentials(insecure.NewCredentials()),
|
||||||
|
grpc.WithConnectParams(grpc.ConnectParams{
|
||||||
|
Backoff: backoff.Config{
|
||||||
|
BaseDelay: 100 * time.Millisecond,
|
||||||
|
Multiplier: 1.6,
|
||||||
|
Jitter: 0.2,
|
||||||
|
MaxDelay: 5 * time.Second,
|
||||||
|
},
|
||||||
|
MinConnectTimeout: cfg.ConnectTimeout,
|
||||||
|
}),
|
||||||
|
grpc.WithKeepaliveParams(keepalive.ClientParameters{
|
||||||
|
Time: cfg.KeepaliveTime,
|
||||||
|
Timeout: cfg.KeepaliveTimeout,
|
||||||
|
PermitWithoutStream: true,
|
||||||
|
}),
|
||||||
|
grpc.WithUnaryInterceptor(timeoutUnaryInterceptor(cfg.DefaultTimeout)),
|
||||||
|
}
|
||||||
|
if serviceConfig := defaultServiceConfig(cfg); serviceConfig != "" {
|
||||||
|
// The retry budget is intentionally transport-level and narrow. Business
|
||||||
|
// handlers still rely on command IDs/idempotency for mutation safety.
|
||||||
|
dialOptions = append(dialOptions, grpc.WithDefaultServiceConfig(serviceConfig))
|
||||||
|
}
|
||||||
|
dialOptions = append(dialOptions, opts...)
|
||||||
|
return grpc.Dial(strings.TrimSpace(target), dialOptions...)
|
||||||
|
}
|
||||||
|
|
||||||
|
func timeoutUnaryInterceptor(timeout time.Duration) grpc.UnaryClientInterceptor {
|
||||||
|
return func(ctx context.Context, method string, req any, reply any, conn *grpc.ClientConn, invoker grpc.UnaryInvoker, opts ...grpc.CallOption) error {
|
||||||
|
if timeout > 0 {
|
||||||
|
if _, ok := ctx.Deadline(); !ok {
|
||||||
|
var cancel context.CancelFunc
|
||||||
|
ctx, cancel = context.WithTimeout(ctx, timeout)
|
||||||
|
defer cancel()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return invoker(ctx, method, req, reply, conn, opts...)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func defaultServiceConfig(cfg Config) string {
|
||||||
|
if cfg.RetryMaxAttempts < 2 {
|
||||||
|
return `{"methodConfig":[{"name":[{}],"waitForReady":true}]}`
|
||||||
|
}
|
||||||
|
|
||||||
|
return fmt.Sprintf(
|
||||||
|
`{"methodConfig":[{"name":[{}],"waitForReady":true,"retryPolicy":{"maxAttempts":%d,"initialBackoff":"%s","maxBackoff":"%s","backoffMultiplier":%.2f,"retryableStatusCodes":[%s]}}]}`,
|
||||||
|
cfg.RetryMaxAttempts,
|
||||||
|
durationString(cfg.RetryInitialBackoff),
|
||||||
|
durationString(cfg.RetryMaxBackoff),
|
||||||
|
cfg.RetryBackoffMultiplier,
|
||||||
|
statusCodeList(cfg.RetryableStatusCodes),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
func durationString(value time.Duration) string {
|
||||||
|
return fmt.Sprintf("%.3fs", value.Seconds())
|
||||||
|
}
|
||||||
|
|
||||||
|
func statusCodeList(codes []string) string {
|
||||||
|
quoted := make([]string, 0, len(codes))
|
||||||
|
for _, code := range codes {
|
||||||
|
code = strings.ToUpper(strings.TrimSpace(code))
|
||||||
|
if code == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
quoted = append(quoted, fmt.Sprintf("%q", code))
|
||||||
|
}
|
||||||
|
if len(quoted) == 0 {
|
||||||
|
return `"UNAVAILABLE"`
|
||||||
|
}
|
||||||
|
return strings.Join(quoted, ",")
|
||||||
|
}
|
||||||
52
pkg/grpcclient/client_test.go
Normal file
52
pkg/grpcclient/client_test.go
Normal file
@ -0,0 +1,52 @@
|
|||||||
|
package grpcclient
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"google.golang.org/grpc"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestNormalizeRestoresTransportDefaults(t *testing.T) {
|
||||||
|
cfg := Normalize(Config{})
|
||||||
|
|
||||||
|
if cfg.DefaultTimeout != 5*time.Second || cfg.ConnectTimeout != 2*time.Second {
|
||||||
|
t.Fatalf("deadline defaults mismatch: %+v", cfg)
|
||||||
|
}
|
||||||
|
if cfg.KeepaliveTime != 30*time.Second || cfg.KeepaliveTimeout != 10*time.Second {
|
||||||
|
t.Fatalf("keepalive defaults mismatch: %+v", cfg)
|
||||||
|
}
|
||||||
|
if cfg.RetryMaxAttempts != 2 || cfg.RetryInitialBackoff != 100*time.Millisecond || cfg.RetryMaxBackoff != 500*time.Millisecond {
|
||||||
|
t.Fatalf("retry defaults mismatch: %+v", cfg)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestTimeoutInterceptorPreservesExistingDeadline(t *testing.T) {
|
||||||
|
parent, cancel := context.WithTimeout(context.Background(), time.Minute)
|
||||||
|
defer cancel()
|
||||||
|
before, _ := parent.Deadline()
|
||||||
|
|
||||||
|
interceptor := timeoutUnaryInterceptor(time.Millisecond)
|
||||||
|
err := interceptor(parent, "/svc/Method", nil, nil, nil, func(ctx context.Context, method string, req any, reply any, conn *grpc.ClientConn, opts ...grpc.CallOption) error {
|
||||||
|
after, ok := ctx.Deadline()
|
||||||
|
if !ok || !after.Equal(before) {
|
||||||
|
t.Fatalf("deadline should be preserved: ok=%t before=%s after=%s", ok, before, after)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("interceptor returned error: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDefaultServiceConfigIncludesUnavailableRetry(t *testing.T) {
|
||||||
|
serviceConfig := defaultServiceConfig(DefaultConfig())
|
||||||
|
if !strings.Contains(serviceConfig, `"retryableStatusCodes":["UNAVAILABLE"]`) {
|
||||||
|
t.Fatalf("retry policy missing UNAVAILABLE: %s", serviceConfig)
|
||||||
|
}
|
||||||
|
if !strings.Contains(serviceConfig, `"waitForReady":true`) {
|
||||||
|
t.Fatalf("service config should enable waitForReady: %s", serviceConfig)
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -19,9 +19,9 @@ const (
|
|||||||
|
|
||||||
// Check 描述一项 readiness 子检查;gRPC health 只返回总状态,子项用于服务内聚合。
|
// Check 描述一项 readiness 子检查;gRPC health 只返回总状态,子项用于服务内聚合。
|
||||||
type Check struct {
|
type Check struct {
|
||||||
Name string
|
Name string `json:"name"`
|
||||||
Status string
|
Status string `json:"status"`
|
||||||
Message string
|
Message string `json:"message,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// Checker 把服务自己的 ready 语义适配到标准 gRPC health protocol。
|
// Checker 把服务自己的 ready 语义适配到标准 gRPC health protocol。
|
||||||
|
|||||||
@ -2,6 +2,7 @@ package grpchealth
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"errors"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
healthpb "google.golang.org/grpc/health/grpc_health_v1"
|
healthpb "google.golang.org/grpc/health/grpc_health_v1"
|
||||||
@ -37,3 +38,18 @@ func TestServingCheckerMapsReadiness(t *testing.T) {
|
|||||||
t.Fatalf("status after draining mismatch: got %s want NOT_SERVING", resp.GetStatus())
|
t.Fatalf("status after draining mismatch: got %s want NOT_SERVING", resp.GetStatus())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestServingCheckerDependencyFailureAffectsReady(t *testing.T) {
|
||||||
|
checker := NewServingChecker("wallet-service", Dependency{
|
||||||
|
Name: "mysql",
|
||||||
|
Check: func(context.Context) error {
|
||||||
|
return errors.New("mysql down")
|
||||||
|
},
|
||||||
|
})
|
||||||
|
checker.MarkServing()
|
||||||
|
|
||||||
|
checks := checker.Ready(context.Background())
|
||||||
|
if Healthy(checks) {
|
||||||
|
t.Fatalf("dependency failure should make checker not ready: %+v", checks)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@ -2,19 +2,31 @@ package grpchealth
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"strings"
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
const defaultDependencyBudget = 300 * time.Millisecond
|
||||||
|
|
||||||
|
// Dependency 是 readiness 中必须可用的外部依赖。
|
||||||
|
type Dependency struct {
|
||||||
|
Name string
|
||||||
|
Budget time.Duration
|
||||||
|
Check func(ctx context.Context) error
|
||||||
|
}
|
||||||
|
|
||||||
// ServingChecker 覆盖无外部硬依赖的占位或 stub gRPC 服务 readiness。
|
// ServingChecker 覆盖无外部硬依赖的占位或 stub gRPC 服务 readiness。
|
||||||
type ServingChecker struct {
|
type ServingChecker struct {
|
||||||
serviceName string
|
serviceName string
|
||||||
serving atomic.Bool
|
dependencies []Dependency
|
||||||
draining atomic.Bool
|
serving atomic.Bool
|
||||||
|
draining atomic.Bool
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewServingChecker 创建只依赖 gRPC listener 和 draining 状态的 checker。
|
// NewServingChecker 创建只依赖 gRPC listener 和 draining 状态的 checker。
|
||||||
func NewServingChecker(serviceName string) *ServingChecker {
|
func NewServingChecker(serviceName string, dependencies ...Dependency) *ServingChecker {
|
||||||
return &ServingChecker{serviceName: serviceName}
|
return &ServingChecker{serviceName: serviceName, dependencies: dependencies}
|
||||||
}
|
}
|
||||||
|
|
||||||
// ServiceName 返回标准 gRPC health 使用的服务边界名。
|
// ServiceName 返回标准 gRPC health 使用的服务边界名。
|
||||||
@ -37,10 +49,35 @@ func (c *ServingChecker) MarkDraining() {
|
|||||||
c.draining.Store(true)
|
c.draining.Store(true)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Ready 返回无数据库类硬依赖服务的最小 readiness。
|
// Ready 返回 listener、draining 和外部硬依赖的聚合 readiness。
|
||||||
func (c *ServingChecker) Ready(_ context.Context) []Check {
|
func (c *ServingChecker) Ready(ctx context.Context) []Check {
|
||||||
return []Check{
|
checks := []Check{
|
||||||
Bool("grpc_listener", c.serving.Load(), "gRPC listener is not serving"),
|
Bool("grpc_listener", c.serving.Load(), "gRPC listener is not serving"),
|
||||||
Bool("draining", !c.draining.Load(), "node is draining"),
|
Bool("draining", !c.draining.Load(), "node is draining"),
|
||||||
}
|
}
|
||||||
|
for _, dependency := range c.dependencies {
|
||||||
|
checks = append(checks, dependency.Ready(ctx))
|
||||||
|
}
|
||||||
|
return checks
|
||||||
|
}
|
||||||
|
|
||||||
|
// Ready executes a dependency check with a short bounded budget.
|
||||||
|
func (d Dependency) Ready(ctx context.Context) Check {
|
||||||
|
name := strings.TrimSpace(d.Name)
|
||||||
|
if name == "" {
|
||||||
|
name = "dependency"
|
||||||
|
}
|
||||||
|
if d.Check == nil {
|
||||||
|
return Fail(name, "dependency check is not configured")
|
||||||
|
}
|
||||||
|
budget := d.Budget
|
||||||
|
if budget <= 0 {
|
||||||
|
budget = defaultDependencyBudget
|
||||||
|
}
|
||||||
|
checkCtx, cancel := context.WithTimeout(ctx, budget)
|
||||||
|
defer cancel()
|
||||||
|
if err := d.Check(checkCtx); err != nil {
|
||||||
|
return Fail(name, err.Error())
|
||||||
|
}
|
||||||
|
return OK(name)
|
||||||
}
|
}
|
||||||
|
|||||||
34
pkg/grpcshutdown/graceful.go
Normal file
34
pkg/grpcshutdown/graceful.go
Normal file
@ -0,0 +1,34 @@
|
|||||||
|
// Package grpcshutdown centralizes gRPC draining semantics for all services.
|
||||||
|
package grpcshutdown
|
||||||
|
|
||||||
|
import (
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"google.golang.org/grpc"
|
||||||
|
)
|
||||||
|
|
||||||
|
const defaultTimeout = 15 * time.Second
|
||||||
|
|
||||||
|
// GracefulStop waits for in-flight RPCs to finish, then falls back to Stop so a
|
||||||
|
// wedged client stream cannot block deploy shutdown forever.
|
||||||
|
func GracefulStop(server *grpc.Server, timeout time.Duration) {
|
||||||
|
if server == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if timeout <= 0 {
|
||||||
|
timeout = defaultTimeout
|
||||||
|
}
|
||||||
|
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() {
|
||||||
|
server.GracefulStop()
|
||||||
|
close(done)
|
||||||
|
}()
|
||||||
|
|
||||||
|
select {
|
||||||
|
case <-done:
|
||||||
|
case <-time.After(timeout):
|
||||||
|
server.Stop()
|
||||||
|
<-done
|
||||||
|
}
|
||||||
|
}
|
||||||
134
pkg/healthhttp/server.go
Normal file
134
pkg/healthhttp/server.go
Normal file
@ -0,0 +1,134 @@
|
|||||||
|
// Package healthhttp exposes process-local HTTP health endpoints for services
|
||||||
|
// whose primary protocol is not HTTP.
|
||||||
|
package healthhttp
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"hyapp/pkg/grpchealth"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
statusOK = "ok"
|
||||||
|
statusFail = "fail"
|
||||||
|
)
|
||||||
|
|
||||||
|
// LiveChecker allows services with richer runtime liveness checks, such as
|
||||||
|
// room-service, to reuse their existing process probes.
|
||||||
|
type LiveChecker interface {
|
||||||
|
Live() []grpchealth.Check
|
||||||
|
}
|
||||||
|
|
||||||
|
// Response is intentionally close to gateway health responses so CLB probes and
|
||||||
|
// deploy scripts can consume one stable shape across all services.
|
||||||
|
type Response struct {
|
||||||
|
Status string `json:"status"`
|
||||||
|
Service string `json:"service"`
|
||||||
|
NodeID string `json:"node_id"`
|
||||||
|
Checks []grpchealth.Check `json:"checks"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Server owns the separate HTTP listener used by CLB health checks for gRPC
|
||||||
|
// processes. It does not replace gRPC health; it makes health visible to CLB.
|
||||||
|
type Server struct {
|
||||||
|
nodeID string
|
||||||
|
checker grpchealth.Checker
|
||||||
|
server *http.Server
|
||||||
|
listener net.Listener
|
||||||
|
once sync.Once
|
||||||
|
}
|
||||||
|
|
||||||
|
// New creates a health HTTP server. Empty addr disables the HTTP health port so
|
||||||
|
// unit tests and special one-off tools can keep a single listener.
|
||||||
|
func New(addr string, nodeID string, checker grpchealth.Checker) (*Server, error) {
|
||||||
|
addr = strings.TrimSpace(addr)
|
||||||
|
if addr == "" {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
if checker == nil {
|
||||||
|
return nil, errors.New("health checker is required")
|
||||||
|
}
|
||||||
|
|
||||||
|
listener, err := net.Listen("tcp", addr)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
s := &Server{
|
||||||
|
nodeID: strings.TrimSpace(nodeID),
|
||||||
|
checker: checker,
|
||||||
|
listener: listener,
|
||||||
|
}
|
||||||
|
mux := http.NewServeMux()
|
||||||
|
mux.HandleFunc("/healthz/live", s.live)
|
||||||
|
mux.HandleFunc("/healthz/ready", s.ready)
|
||||||
|
s.server = &http.Server{
|
||||||
|
Handler: mux,
|
||||||
|
ReadHeaderTimeout: 2 * time.Second,
|
||||||
|
}
|
||||||
|
|
||||||
|
return s, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run serves health endpoints until Close is called or the listener fails.
|
||||||
|
func (s *Server) Run() error {
|
||||||
|
if s == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
err := s.server.Serve(s.listener)
|
||||||
|
if errors.Is(err, http.ErrServerClosed) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close stops the health endpoint. Readiness should already be failing before
|
||||||
|
// this is called because the owning service marks its checker draining first.
|
||||||
|
func (s *Server) Close(ctx context.Context) error {
|
||||||
|
if s == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var err error
|
||||||
|
s.once.Do(func() {
|
||||||
|
err = s.server.Shutdown(ctx)
|
||||||
|
})
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) live(writer http.ResponseWriter, request *http.Request) {
|
||||||
|
checks := []grpchealth.Check{grpchealth.OK("process")}
|
||||||
|
if live, ok := s.checker.(LiveChecker); ok {
|
||||||
|
// Services with process-local invariants should expose them on live, but
|
||||||
|
// live still avoids slow external dependency probes.
|
||||||
|
checks = live.Live()
|
||||||
|
}
|
||||||
|
s.write(writer, checks)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) ready(writer http.ResponseWriter, request *http.Request) {
|
||||||
|
s.write(writer, s.checker.Ready(request.Context()))
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) write(writer http.ResponseWriter, checks []grpchealth.Check) {
|
||||||
|
status := statusFail
|
||||||
|
code := http.StatusServiceUnavailable
|
||||||
|
if grpchealth.Healthy(checks) {
|
||||||
|
status = statusOK
|
||||||
|
code = http.StatusOK
|
||||||
|
}
|
||||||
|
writer.Header().Set("Content-Type", "application/json")
|
||||||
|
writer.WriteHeader(code)
|
||||||
|
_ = json.NewEncoder(writer).Encode(Response{
|
||||||
|
Status: status,
|
||||||
|
Service: s.checker.ServiceName(),
|
||||||
|
NodeID: s.nodeID,
|
||||||
|
Checks: checks,
|
||||||
|
})
|
||||||
|
}
|
||||||
73
pkg/healthhttp/server_test.go
Normal file
73
pkg/healthhttp/server_test.go
Normal file
@ -0,0 +1,73 @@
|
|||||||
|
package healthhttp
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"hyapp/pkg/grpchealth"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestHTTPReadyReflectsCheckerState(t *testing.T) {
|
||||||
|
checker := grpchealth.NewServingChecker("test-service")
|
||||||
|
server, err := New("127.0.0.1:0", "test-node", checker)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("New failed: %v", err)
|
||||||
|
}
|
||||||
|
go func() {
|
||||||
|
_ = server.Run()
|
||||||
|
}()
|
||||||
|
t.Cleanup(func() {
|
||||||
|
_ = server.Close(t.Context())
|
||||||
|
})
|
||||||
|
|
||||||
|
baseURL := "http://" + server.listener.Addr().String()
|
||||||
|
resp, err := http.Get(baseURL + "/healthz/ready")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GET not-ready failed: %v", err)
|
||||||
|
}
|
||||||
|
if resp.StatusCode != http.StatusServiceUnavailable {
|
||||||
|
t.Fatalf("not-ready status mismatch: got %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
_ = resp.Body.Close()
|
||||||
|
|
||||||
|
checker.MarkServing()
|
||||||
|
resp, err = http.Get(baseURL + "/healthz/ready")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("GET ready failed: %v", err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
t.Fatalf("ready status mismatch: got %d", resp.StatusCode)
|
||||||
|
}
|
||||||
|
var body Response
|
||||||
|
if err := json.NewDecoder(resp.Body).Decode(&body); err != nil {
|
||||||
|
t.Fatalf("decode response failed: %v", err)
|
||||||
|
}
|
||||||
|
if body.Service != "test-service" || body.NodeID != "test-node" || body.Status != "ok" {
|
||||||
|
t.Fatalf("unexpected response: %+v", body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewDisabledWhenAddrEmpty(t *testing.T) {
|
||||||
|
server, err := New("", "node", grpchealth.NewServingChecker("svc"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("New empty addr failed: %v", err)
|
||||||
|
}
|
||||||
|
if server != nil {
|
||||||
|
t.Fatalf("empty addr should disable health HTTP server")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestNewFailsWhenPortAlreadyBound(t *testing.T) {
|
||||||
|
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("listen failed: %v", err)
|
||||||
|
}
|
||||||
|
defer listener.Close()
|
||||||
|
|
||||||
|
if _, err := New(listener.Addr().String(), "node", grpchealth.NewServingChecker("svc")); err == nil {
|
||||||
|
t.Fatalf("New should fail on occupied health port")
|
||||||
|
}
|
||||||
|
}
|
||||||
@ -30,7 +30,7 @@ COPY services/activity-service/configs/config.docker.yaml /app/config.yaml
|
|||||||
|
|
||||||
USER appuser
|
USER appuser
|
||||||
|
|
||||||
EXPOSE 13006
|
EXPOSE 13006 13106
|
||||||
|
|
||||||
ENTRYPOINT ["/app/server"]
|
ENTRYPOINT ["/app/server"]
|
||||||
CMD ["-config", "/app/config.yaml"]
|
CMD ["-config", "/app/config.yaml"]
|
||||||
|
|||||||
@ -8,6 +8,7 @@ log:
|
|||||||
include_response_body: false
|
include_response_body: false
|
||||||
max_payload_bytes: 2048
|
max_payload_bytes: 2048
|
||||||
grpc_addr: ":13006"
|
grpc_addr: ":13006"
|
||||||
|
health_http_addr: ":13106"
|
||||||
mysql_dsn: "hyapp:hyapp@tcp(mysql:3306)/hyapp_activity?parseTime=true&charset=utf8mb4&loc=UTC"
|
mysql_dsn: "hyapp:hyapp@tcp(mysql:3306)/hyapp_activity?parseTime=true&charset=utf8mb4&loc=UTC"
|
||||||
user_service_addr: "user-service:13005"
|
user_service_addr: "user-service:13005"
|
||||||
wallet_service_addr: "wallet-service:13004"
|
wallet_service_addr: "wallet-service:13004"
|
||||||
|
|||||||
@ -8,6 +8,7 @@ log:
|
|||||||
include_response_body: false
|
include_response_body: false
|
||||||
max_payload_bytes: 2048
|
max_payload_bytes: 2048
|
||||||
grpc_addr: ":13006"
|
grpc_addr: ":13006"
|
||||||
|
health_http_addr: ":13106"
|
||||||
mysql_dsn: "USER:PASSWORD@tcp(TENCENT_CDB_HOST:3306)/hyapp_activity?parseTime=true&charset=utf8mb4&loc=UTC&tls=false"
|
mysql_dsn: "USER:PASSWORD@tcp(TENCENT_CDB_HOST:3306)/hyapp_activity?parseTime=true&charset=utf8mb4&loc=UTC&tls=false"
|
||||||
user_service_addr: "user-service.internal:13005"
|
user_service_addr: "user-service.internal:13005"
|
||||||
wallet_service_addr: "wallet-service.internal:13004"
|
wallet_service_addr: "wallet-service.internal:13004"
|
||||||
|
|||||||
@ -8,6 +8,7 @@ log:
|
|||||||
include_response_body: false
|
include_response_body: false
|
||||||
max_payload_bytes: 2048
|
max_payload_bytes: 2048
|
||||||
grpc_addr: ":13006"
|
grpc_addr: ":13006"
|
||||||
|
health_http_addr: ":13106"
|
||||||
mysql_dsn: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_activity?parseTime=true&charset=utf8mb4&loc=UTC"
|
mysql_dsn: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_activity?parseTime=true&charset=utf8mb4&loc=UTC"
|
||||||
user_service_addr: "127.0.0.1:13005"
|
user_service_addr: "127.0.0.1:13005"
|
||||||
wallet_service_addr: "127.0.0.1:13004"
|
wallet_service_addr: "127.0.0.1:13004"
|
||||||
|
|||||||
@ -13,6 +13,8 @@ import (
|
|||||||
activityv1 "hyapp.local/api/proto/activity/v1"
|
activityv1 "hyapp.local/api/proto/activity/v1"
|
||||||
walletv1 "hyapp.local/api/proto/wallet/v1"
|
walletv1 "hyapp.local/api/proto/wallet/v1"
|
||||||
"hyapp/pkg/grpchealth"
|
"hyapp/pkg/grpchealth"
|
||||||
|
"hyapp/pkg/grpcshutdown"
|
||||||
|
"hyapp/pkg/healthhttp"
|
||||||
"hyapp/pkg/logx"
|
"hyapp/pkg/logx"
|
||||||
"hyapp/pkg/tencentim"
|
"hyapp/pkg/tencentim"
|
||||||
"hyapp/services/activity-service/internal/client"
|
"hyapp/services/activity-service/internal/client"
|
||||||
@ -30,6 +32,7 @@ type App struct {
|
|||||||
server *grpc.Server
|
server *grpc.Server
|
||||||
listener net.Listener
|
listener net.Listener
|
||||||
health *grpchealth.ServingChecker
|
health *grpchealth.ServingChecker
|
||||||
|
healthHTTP *healthhttp.Server
|
||||||
mysqlRepo *mysqlstorage.Repository
|
mysqlRepo *mysqlstorage.Repository
|
||||||
broadcast *broadcastservice.Service
|
broadcast *broadcastservice.Service
|
||||||
broadcastWorkerEnabled bool
|
broadcastWorkerEnabled bool
|
||||||
@ -111,14 +114,26 @@ func New(cfg config.Config) (*App, error) {
|
|||||||
broadcastServer := grpcserver.NewBroadcastServer(broadcastSvc)
|
broadcastServer := grpcserver.NewBroadcastServer(broadcastSvc)
|
||||||
activityv1.RegisterBroadcastServiceServer(server, broadcastServer)
|
activityv1.RegisterBroadcastServiceServer(server, broadcastServer)
|
||||||
activityv1.RegisterRoomEventConsumerServiceServer(server, broadcastServer)
|
activityv1.RegisterRoomEventConsumerServiceServer(server, broadcastServer)
|
||||||
health := grpchealth.NewServingChecker("activity-service")
|
health := grpchealth.NewServingChecker("activity-service", grpchealth.Dependency{
|
||||||
|
Name: "mysql",
|
||||||
|
Check: repository.Ping,
|
||||||
|
})
|
||||||
healthgrpc.RegisterHealthServer(server, grpchealth.NewServer(health))
|
healthgrpc.RegisterHealthServer(server, grpchealth.NewServer(health))
|
||||||
|
healthHTTP, err := healthhttp.New(cfg.HealthHTTPAddr, cfg.NodeID, health)
|
||||||
|
if err != nil {
|
||||||
|
_ = walletConn.Close()
|
||||||
|
_ = userConn.Close()
|
||||||
|
_ = listener.Close()
|
||||||
|
_ = repository.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
workerCtx, workerStop := context.WithCancel(context.Background())
|
workerCtx, workerStop := context.WithCancel(context.Background())
|
||||||
|
|
||||||
return &App{
|
return &App{
|
||||||
server: server,
|
server: server,
|
||||||
listener: listener,
|
listener: listener,
|
||||||
health: health,
|
health: health,
|
||||||
|
healthHTTP: healthHTTP,
|
||||||
mysqlRepo: repository,
|
mysqlRepo: repository,
|
||||||
broadcast: broadcastSvc,
|
broadcast: broadcastSvc,
|
||||||
broadcastWorkerEnabled: cfg.Broadcast.Enabled,
|
broadcastWorkerEnabled: cfg.Broadcast.Enabled,
|
||||||
@ -131,6 +146,7 @@ func New(cfg config.Config) (*App, error) {
|
|||||||
|
|
||||||
// Run 启动 gRPC 服务。
|
// Run 启动 gRPC 服务。
|
||||||
func (a *App) Run() error {
|
func (a *App) Run() error {
|
||||||
|
a.runHealthHTTP()
|
||||||
a.health.MarkServing()
|
a.health.MarkServing()
|
||||||
if a.broadcastWorkerEnabled && a.broadcast != nil && a.workerCtx != nil {
|
if a.broadcastWorkerEnabled && a.broadcast != nil && a.workerCtx != nil {
|
||||||
a.workerWG.Go(func() {
|
a.workerWG.Go(func() {
|
||||||
@ -158,7 +174,8 @@ func (a *App) Close() {
|
|||||||
a.workerStop()
|
a.workerStop()
|
||||||
}
|
}
|
||||||
a.workerWG.Wait()
|
a.workerWG.Wait()
|
||||||
a.server.GracefulStop()
|
grpcshutdown.GracefulStop(a.server, 15*time.Second)
|
||||||
|
a.closeHealthHTTP()
|
||||||
if a.userConn != nil {
|
if a.userConn != nil {
|
||||||
_ = a.userConn.Close()
|
_ = a.userConn.Close()
|
||||||
}
|
}
|
||||||
@ -171,3 +188,23 @@ func (a *App) Close() {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (a *App) runHealthHTTP() {
|
||||||
|
if a.healthHTTP == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
go func() {
|
||||||
|
if err := a.healthHTTP.Run(); err != nil {
|
||||||
|
logx.Error(context.Background(), "health_http_run_failed", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (a *App) closeHealthHTTP() {
|
||||||
|
if a.healthHTTP == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
_ = a.healthHTTP.Close(ctx)
|
||||||
|
}
|
||||||
|
|||||||
@ -16,6 +16,8 @@ type Config struct {
|
|||||||
NodeID string `yaml:"node_id"`
|
NodeID string `yaml:"node_id"`
|
||||||
Environment string `yaml:"environment"`
|
Environment string `yaml:"environment"`
|
||||||
GRPCAddr string `yaml:"grpc_addr"`
|
GRPCAddr string `yaml:"grpc_addr"`
|
||||||
|
// HealthHTTPAddr 是给 CLB/发布脚本探活使用的 HTTP 监听地址,不承载业务流量。
|
||||||
|
HealthHTTPAddr string `yaml:"health_http_addr"`
|
||||||
// MySQLDSN 是活动状态、事件消费幂等和 outbox 的必需存储连接串。
|
// MySQLDSN 是活动状态、事件消费幂等和 outbox 的必需存储连接串。
|
||||||
MySQLDSN string `yaml:"mysql_dsn"`
|
MySQLDSN string `yaml:"mysql_dsn"`
|
||||||
// UserServiceAddr 是 fanout worker 读取目标用户游标的内部 gRPC 地址。
|
// UserServiceAddr 是 fanout worker 读取目标用户游标的内部 gRPC 地址。
|
||||||
@ -91,6 +93,7 @@ func Default() Config {
|
|||||||
NodeID: "activity-local",
|
NodeID: "activity-local",
|
||||||
Environment: "local",
|
Environment: "local",
|
||||||
GRPCAddr: ":13006",
|
GRPCAddr: ":13006",
|
||||||
|
HealthHTTPAddr: ":13106",
|
||||||
MySQLDSN: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_activity?parseTime=true&charset=utf8mb4&loc=UTC",
|
MySQLDSN: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_activity?parseTime=true&charset=utf8mb4&loc=UTC",
|
||||||
UserServiceAddr: "127.0.0.1:13005",
|
UserServiceAddr: "127.0.0.1:13005",
|
||||||
WalletServiceAddr: "127.0.0.1:13004",
|
WalletServiceAddr: "127.0.0.1:13004",
|
||||||
@ -155,6 +158,10 @@ func Load(path string) (Config, error) {
|
|||||||
if cfg.Environment == "" {
|
if cfg.Environment == "" {
|
||||||
cfg.Environment = "local"
|
cfg.Environment = "local"
|
||||||
}
|
}
|
||||||
|
cfg.HealthHTTPAddr = strings.TrimSpace(cfg.HealthHTTPAddr)
|
||||||
|
if cfg.HealthHTTPAddr == "" {
|
||||||
|
cfg.HealthHTTPAddr = ":13106"
|
||||||
|
}
|
||||||
cfg.TencentIM.AdminIdentifier = strings.TrimSpace(cfg.TencentIM.AdminIdentifier)
|
cfg.TencentIM.AdminIdentifier = strings.TrimSpace(cfg.TencentIM.AdminIdentifier)
|
||||||
if cfg.TencentIM.AdminIdentifier == "" {
|
if cfg.TencentIM.AdminIdentifier == "" {
|
||||||
cfg.TencentIM.AdminIdentifier = "administrator"
|
cfg.TencentIM.AdminIdentifier = "administrator"
|
||||||
|
|||||||
@ -30,7 +30,7 @@ COPY services/cron-service/configs/config.docker.yaml /app/config.yaml
|
|||||||
|
|
||||||
USER appuser
|
USER appuser
|
||||||
|
|
||||||
EXPOSE 13007
|
EXPOSE 13007 13107
|
||||||
|
|
||||||
ENTRYPOINT ["/app/server"]
|
ENTRYPOINT ["/app/server"]
|
||||||
CMD ["-config", "/app/config.yaml"]
|
CMD ["-config", "/app/config.yaml"]
|
||||||
|
|||||||
@ -8,6 +8,7 @@ log:
|
|||||||
include_response_body: false
|
include_response_body: false
|
||||||
max_payload_bytes: 2048
|
max_payload_bytes: 2048
|
||||||
grpc_addr: ":13007"
|
grpc_addr: ":13007"
|
||||||
|
health_http_addr: ":13107"
|
||||||
mysql_dsn: "hyapp:hyapp@tcp(mysql:3306)/hyapp_cron?parseTime=true&charset=utf8mb4&loc=UTC"
|
mysql_dsn: "hyapp:hyapp@tcp(mysql:3306)/hyapp_cron?parseTime=true&charset=utf8mb4&loc=UTC"
|
||||||
user_service_addr: "user-service:13005"
|
user_service_addr: "user-service:13005"
|
||||||
activity_service_addr: "activity-service:13006"
|
activity_service_addr: "activity-service:13006"
|
||||||
|
|||||||
@ -8,6 +8,7 @@ log:
|
|||||||
include_response_body: false
|
include_response_body: false
|
||||||
max_payload_bytes: 2048
|
max_payload_bytes: 2048
|
||||||
grpc_addr: ":13007"
|
grpc_addr: ":13007"
|
||||||
|
health_http_addr: ":13107"
|
||||||
mysql_dsn: "USER:PASSWORD@tcp(TENCENT_CDB_HOST:3306)/hyapp_cron?parseTime=true&charset=utf8mb4&loc=UTC&tls=false"
|
mysql_dsn: "USER:PASSWORD@tcp(TENCENT_CDB_HOST:3306)/hyapp_cron?parseTime=true&charset=utf8mb4&loc=UTC&tls=false"
|
||||||
user_service_addr: "user-service.internal:13005"
|
user_service_addr: "user-service.internal:13005"
|
||||||
activity_service_addr: "activity-service.internal:13006"
|
activity_service_addr: "activity-service.internal:13006"
|
||||||
|
|||||||
@ -8,6 +8,7 @@ log:
|
|||||||
include_response_body: false
|
include_response_body: false
|
||||||
max_payload_bytes: 2048
|
max_payload_bytes: 2048
|
||||||
grpc_addr: ":13007"
|
grpc_addr: ":13007"
|
||||||
|
health_http_addr: ":13107"
|
||||||
mysql_dsn: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_cron?parseTime=true&charset=utf8mb4&loc=UTC"
|
mysql_dsn: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_cron?parseTime=true&charset=utf8mb4&loc=UTC"
|
||||||
user_service_addr: "127.0.0.1:13005"
|
user_service_addr: "127.0.0.1:13005"
|
||||||
activity_service_addr: "127.0.0.1:13006"
|
activity_service_addr: "127.0.0.1:13006"
|
||||||
|
|||||||
@ -14,6 +14,8 @@ import (
|
|||||||
activityv1 "hyapp.local/api/proto/activity/v1"
|
activityv1 "hyapp.local/api/proto/activity/v1"
|
||||||
userv1 "hyapp.local/api/proto/user/v1"
|
userv1 "hyapp.local/api/proto/user/v1"
|
||||||
"hyapp/pkg/grpchealth"
|
"hyapp/pkg/grpchealth"
|
||||||
|
"hyapp/pkg/grpcshutdown"
|
||||||
|
"hyapp/pkg/healthhttp"
|
||||||
"hyapp/pkg/logx"
|
"hyapp/pkg/logx"
|
||||||
"hyapp/services/cron-service/internal/config"
|
"hyapp/services/cron-service/internal/config"
|
||||||
"hyapp/services/cron-service/internal/integration"
|
"hyapp/services/cron-service/internal/integration"
|
||||||
@ -26,6 +28,7 @@ type App struct {
|
|||||||
server *grpc.Server
|
server *grpc.Server
|
||||||
listener net.Listener
|
listener net.Listener
|
||||||
health *grpchealth.ServingChecker
|
health *grpchealth.ServingChecker
|
||||||
|
healthHTTP *healthhttp.Server
|
||||||
repository *mysqlstorage.Repository
|
repository *mysqlstorage.Repository
|
||||||
scheduler *scheduler.Scheduler
|
scheduler *scheduler.Scheduler
|
||||||
userConn *grpc.ClientConn
|
userConn *grpc.ClientConn
|
||||||
@ -67,8 +70,19 @@ func New(cfg config.Config) (*App, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
server := grpc.NewServer(grpc.UnaryInterceptor(logx.UnaryServerInterceptor("cron-service")))
|
server := grpc.NewServer(grpc.UnaryInterceptor(logx.UnaryServerInterceptor("cron-service")))
|
||||||
health := grpchealth.NewServingChecker("cron-service")
|
health := grpchealth.NewServingChecker("cron-service", grpchealth.Dependency{
|
||||||
|
Name: "mysql",
|
||||||
|
Check: repository.Ping,
|
||||||
|
})
|
||||||
healthgrpc.RegisterHealthServer(server, grpchealth.NewServer(health))
|
healthgrpc.RegisterHealthServer(server, grpchealth.NewServer(health))
|
||||||
|
healthHTTP, err := healthhttp.New(cfg.HealthHTTPAddr, cfg.NodeID, health)
|
||||||
|
if err != nil {
|
||||||
|
_ = activityConn.Close()
|
||||||
|
_ = userConn.Close()
|
||||||
|
_ = listener.Close()
|
||||||
|
_ = repository.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
workerCtx, workerCancel := context.WithCancel(context.Background())
|
workerCtx, workerCancel := context.WithCancel(context.Background())
|
||||||
userCron := integration.NewUserCronClient(userv1.NewUserCronServiceClient(userConn))
|
userCron := integration.NewUserCronClient(userv1.NewUserCronServiceClient(userConn))
|
||||||
@ -84,6 +98,7 @@ func New(cfg config.Config) (*App, error) {
|
|||||||
server: server,
|
server: server,
|
||||||
listener: listener,
|
listener: listener,
|
||||||
health: health,
|
health: health,
|
||||||
|
healthHTTP: healthHTTP,
|
||||||
repository: repository,
|
repository: repository,
|
||||||
scheduler: taskScheduler,
|
scheduler: taskScheduler,
|
||||||
userConn: userConn,
|
userConn: userConn,
|
||||||
@ -95,6 +110,7 @@ func New(cfg config.Config) (*App, error) {
|
|||||||
|
|
||||||
// Run starts scheduler loops and gRPC health serving.
|
// Run starts scheduler loops and gRPC health serving.
|
||||||
func (a *App) Run() error {
|
func (a *App) Run() error {
|
||||||
|
a.runHealthHTTP()
|
||||||
a.health.MarkServing()
|
a.health.MarkServing()
|
||||||
defer func() {
|
defer func() {
|
||||||
if a.workerCancel != nil {
|
if a.workerCancel != nil {
|
||||||
@ -124,7 +140,8 @@ func (a *App) Close() {
|
|||||||
a.workerCancel()
|
a.workerCancel()
|
||||||
}
|
}
|
||||||
a.workerWG.Wait()
|
a.workerWG.Wait()
|
||||||
a.server.GracefulStop()
|
grpcshutdown.GracefulStop(a.server, 15*time.Second)
|
||||||
|
a.closeHealthHTTP()
|
||||||
if a.repository != nil {
|
if a.repository != nil {
|
||||||
_ = a.repository.Close()
|
_ = a.repository.Close()
|
||||||
}
|
}
|
||||||
@ -136,3 +153,23 @@ func (a *App) Close() {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (a *App) runHealthHTTP() {
|
||||||
|
if a.healthHTTP == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
go func() {
|
||||||
|
if err := a.healthHTTP.Run(); err != nil {
|
||||||
|
logx.Error(context.Background(), "health_http_run_failed", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (a *App) closeHealthHTTP() {
|
||||||
|
if a.healthHTTP == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
_ = a.healthHTTP.Close(ctx)
|
||||||
|
}
|
||||||
|
|||||||
@ -21,6 +21,8 @@ type Config struct {
|
|||||||
Environment string `yaml:"environment"`
|
Environment string `yaml:"environment"`
|
||||||
// GRPCAddr exposes gRPC health and future internal management APIs.
|
// GRPCAddr exposes gRPC health and future internal management APIs.
|
||||||
GRPCAddr string `yaml:"grpc_addr"`
|
GRPCAddr string `yaml:"grpc_addr"`
|
||||||
|
// HealthHTTPAddr exposes HTTP health for CLB and deploy scripts only.
|
||||||
|
HealthHTTPAddr string `yaml:"health_http_addr"`
|
||||||
// MySQLDSN stores cron run history and schedule-level leases only.
|
// MySQLDSN stores cron run history and schedule-level leases only.
|
||||||
MySQLDSN string `yaml:"mysql_dsn"`
|
MySQLDSN string `yaml:"mysql_dsn"`
|
||||||
// UserServiceAddr is the owner endpoint for user/auth/host batch tasks.
|
// UserServiceAddr is the owner endpoint for user/auth/host batch tasks.
|
||||||
@ -62,6 +64,7 @@ func Default() Config {
|
|||||||
NodeID: "cron-local",
|
NodeID: "cron-local",
|
||||||
Environment: "local",
|
Environment: "local",
|
||||||
GRPCAddr: ":13007",
|
GRPCAddr: ":13007",
|
||||||
|
HealthHTTPAddr: ":13107",
|
||||||
MySQLDSN: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_cron?parseTime=true&charset=utf8mb4&loc=UTC",
|
MySQLDSN: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_cron?parseTime=true&charset=utf8mb4&loc=UTC",
|
||||||
UserServiceAddr: "127.0.0.1:13005",
|
UserServiceAddr: "127.0.0.1:13005",
|
||||||
ActivityServiceAddr: "127.0.0.1:13006",
|
ActivityServiceAddr: "127.0.0.1:13006",
|
||||||
@ -105,6 +108,10 @@ func normalize(cfg *Config) {
|
|||||||
if cfg.GRPCAddr == "" {
|
if cfg.GRPCAddr == "" {
|
||||||
cfg.GRPCAddr = ":13007"
|
cfg.GRPCAddr = ":13007"
|
||||||
}
|
}
|
||||||
|
cfg.HealthHTTPAddr = strings.TrimSpace(cfg.HealthHTTPAddr)
|
||||||
|
if cfg.HealthHTTPAddr == "" {
|
||||||
|
cfg.HealthHTTPAddr = ":13107"
|
||||||
|
}
|
||||||
cfg.Tasks = normalizeTasks(cfg.Tasks)
|
cfg.Tasks = normalizeTasks(cfg.Tasks)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@ -20,5 +20,5 @@ COPY --from=builder /out/server /app/server
|
|||||||
COPY --from=builder /out/grpc-health-probe /app/grpc-health-probe
|
COPY --from=builder /out/grpc-health-probe /app/grpc-health-probe
|
||||||
COPY services/game-service/configs/config.docker.yaml /app/config.yaml
|
COPY services/game-service/configs/config.docker.yaml /app/config.yaml
|
||||||
USER appuser
|
USER appuser
|
||||||
EXPOSE 13008
|
EXPOSE 13008 13108
|
||||||
ENTRYPOINT ["/app/server", "-config", "/app/config.yaml"]
|
ENTRYPOINT ["/app/server", "-config", "/app/config.yaml"]
|
||||||
|
|||||||
@ -2,6 +2,7 @@ service_name: game-service
|
|||||||
node_id: game-docker
|
node_id: game-docker
|
||||||
environment: local
|
environment: local
|
||||||
grpc_addr: ":13008"
|
grpc_addr: ":13008"
|
||||||
|
health_http_addr: ":13108"
|
||||||
mysql_dsn: "hyapp:hyapp@tcp(mysql:3306)/hyapp_game?parseTime=true&charset=utf8mb4&loc=UTC&multiStatements=true"
|
mysql_dsn: "hyapp:hyapp@tcp(mysql:3306)/hyapp_game?parseTime=true&charset=utf8mb4&loc=UTC&multiStatements=true"
|
||||||
wallet_service_addr: "wallet-service:13004"
|
wallet_service_addr: "wallet-service:13004"
|
||||||
user_service_addr: "user-service:13005"
|
user_service_addr: "user-service:13005"
|
||||||
|
|||||||
@ -2,6 +2,7 @@ service_name: game-service
|
|||||||
node_id: game-prod-1
|
node_id: game-prod-1
|
||||||
environment: prod
|
environment: prod
|
||||||
grpc_addr: ":13008"
|
grpc_addr: ":13008"
|
||||||
|
health_http_addr: ":13108"
|
||||||
mysql_dsn: "${GAME_MYSQL_DSN}"
|
mysql_dsn: "${GAME_MYSQL_DSN}"
|
||||||
wallet_service_addr: "wallet-service:13004"
|
wallet_service_addr: "wallet-service:13004"
|
||||||
user_service_addr: "user-service:13005"
|
user_service_addr: "user-service:13005"
|
||||||
|
|||||||
@ -2,6 +2,7 @@ service_name: game-service
|
|||||||
node_id: game-local
|
node_id: game-local
|
||||||
environment: local
|
environment: local
|
||||||
grpc_addr: ":13008"
|
grpc_addr: ":13008"
|
||||||
|
health_http_addr: ":13108"
|
||||||
mysql_dsn: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_game?parseTime=true&charset=utf8mb4&loc=UTC&multiStatements=true"
|
mysql_dsn: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_game?parseTime=true&charset=utf8mb4&loc=UTC&multiStatements=true"
|
||||||
wallet_service_addr: "127.0.0.1:13004"
|
wallet_service_addr: "127.0.0.1:13004"
|
||||||
user_service_addr: "127.0.0.1:13005"
|
user_service_addr: "127.0.0.1:13005"
|
||||||
|
|||||||
@ -12,6 +12,8 @@ import (
|
|||||||
healthgrpc "google.golang.org/grpc/health/grpc_health_v1"
|
healthgrpc "google.golang.org/grpc/health/grpc_health_v1"
|
||||||
gamev1 "hyapp.local/api/proto/game/v1"
|
gamev1 "hyapp.local/api/proto/game/v1"
|
||||||
"hyapp/pkg/grpchealth"
|
"hyapp/pkg/grpchealth"
|
||||||
|
"hyapp/pkg/grpcshutdown"
|
||||||
|
"hyapp/pkg/healthhttp"
|
||||||
"hyapp/pkg/logx"
|
"hyapp/pkg/logx"
|
||||||
"hyapp/services/game-service/internal/client"
|
"hyapp/services/game-service/internal/client"
|
||||||
"hyapp/services/game-service/internal/config"
|
"hyapp/services/game-service/internal/config"
|
||||||
@ -25,6 +27,7 @@ type App struct {
|
|||||||
server *grpc.Server
|
server *grpc.Server
|
||||||
listener net.Listener
|
listener net.Listener
|
||||||
health *grpchealth.ServingChecker
|
health *grpchealth.ServingChecker
|
||||||
|
healthHTTP *healthhttp.Server
|
||||||
repo *mysqlstorage.Repository
|
repo *mysqlstorage.Repository
|
||||||
walletConn *grpc.ClientConn
|
walletConn *grpc.ClientConn
|
||||||
userConn *grpc.ClientConn
|
userConn *grpc.ClientConn
|
||||||
@ -64,13 +67,25 @@ func New(cfg config.Config) (*App, error) {
|
|||||||
gamev1.RegisterGameAppServiceServer(server, transport)
|
gamev1.RegisterGameAppServiceServer(server, transport)
|
||||||
gamev1.RegisterGameCallbackServiceServer(server, transport)
|
gamev1.RegisterGameCallbackServiceServer(server, transport)
|
||||||
gamev1.RegisterGameAdminServiceServer(server, transport)
|
gamev1.RegisterGameAdminServiceServer(server, transport)
|
||||||
health := grpchealth.NewServingChecker("game-service")
|
health := grpchealth.NewServingChecker("game-service", grpchealth.Dependency{
|
||||||
|
Name: "mysql",
|
||||||
|
Check: repo.Ping,
|
||||||
|
})
|
||||||
healthgrpc.RegisterHealthServer(server, grpchealth.NewServer(health))
|
healthgrpc.RegisterHealthServer(server, grpchealth.NewServer(health))
|
||||||
|
healthHTTP, err := healthhttp.New(cfg.HealthHTTPAddr, cfg.NodeID, health)
|
||||||
|
if err != nil {
|
||||||
|
_ = listener.Close()
|
||||||
|
_ = userConn.Close()
|
||||||
|
_ = walletConn.Close()
|
||||||
|
_ = repo.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
return &App{server: server, listener: listener, health: health, repo: repo, walletConn: walletConn, userConn: userConn}, nil
|
return &App{server: server, listener: listener, health: health, healthHTTP: healthHTTP, repo: repo, walletConn: walletConn, userConn: userConn}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (a *App) Run() error {
|
func (a *App) Run() error {
|
||||||
|
a.runHealthHTTP()
|
||||||
a.health.MarkServing()
|
a.health.MarkServing()
|
||||||
err := a.server.Serve(a.listener)
|
err := a.server.Serve(a.listener)
|
||||||
a.health.MarkStopped()
|
a.health.MarkStopped()
|
||||||
@ -83,7 +98,8 @@ func (a *App) Run() error {
|
|||||||
func (a *App) Close() {
|
func (a *App) Close() {
|
||||||
a.closeOnce.Do(func() {
|
a.closeOnce.Do(func() {
|
||||||
a.health.MarkDraining()
|
a.health.MarkDraining()
|
||||||
a.server.GracefulStop()
|
grpcshutdown.GracefulStop(a.server, 15*time.Second)
|
||||||
|
a.closeHealthHTTP()
|
||||||
if a.walletConn != nil {
|
if a.walletConn != nil {
|
||||||
_ = a.walletConn.Close()
|
_ = a.walletConn.Close()
|
||||||
}
|
}
|
||||||
@ -95,3 +111,23 @@ func (a *App) Close() {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (a *App) runHealthHTTP() {
|
||||||
|
if a.healthHTTP == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
go func() {
|
||||||
|
if err := a.healthHTTP.Run(); err != nil {
|
||||||
|
logx.Error(context.Background(), "health_http_run_failed", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (a *App) closeHealthHTTP() {
|
||||||
|
if a.healthHTTP == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
_ = a.healthHTTP.Close(ctx)
|
||||||
|
}
|
||||||
|
|||||||
@ -11,10 +11,12 @@ import (
|
|||||||
|
|
||||||
// Config 描述 game-service 启动所需的最小依赖。
|
// Config 描述 game-service 启动所需的最小依赖。
|
||||||
type Config struct {
|
type Config struct {
|
||||||
ServiceName string `yaml:"service_name"`
|
ServiceName string `yaml:"service_name"`
|
||||||
NodeID string `yaml:"node_id"`
|
NodeID string `yaml:"node_id"`
|
||||||
Environment string `yaml:"environment"`
|
Environment string `yaml:"environment"`
|
||||||
GRPCAddr string `yaml:"grpc_addr"`
|
GRPCAddr string `yaml:"grpc_addr"`
|
||||||
|
// HealthHTTPAddr 是给 CLB/发布脚本探活使用的 HTTP 监听地址,不承载业务流量。
|
||||||
|
HealthHTTPAddr string `yaml:"health_http_addr"`
|
||||||
MySQLDSN string `yaml:"mysql_dsn"`
|
MySQLDSN string `yaml:"mysql_dsn"`
|
||||||
WalletServiceAddr string `yaml:"wallet_service_addr"`
|
WalletServiceAddr string `yaml:"wallet_service_addr"`
|
||||||
UserServiceAddr string `yaml:"user_service_addr"`
|
UserServiceAddr string `yaml:"user_service_addr"`
|
||||||
@ -29,6 +31,7 @@ func Default() Config {
|
|||||||
NodeID: "game-local",
|
NodeID: "game-local",
|
||||||
Environment: "local",
|
Environment: "local",
|
||||||
GRPCAddr: ":13008",
|
GRPCAddr: ":13008",
|
||||||
|
HealthHTTPAddr: ":13108",
|
||||||
MySQLDSN: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_game?parseTime=true&charset=utf8mb4&loc=UTC&multiStatements=true",
|
MySQLDSN: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_game?parseTime=true&charset=utf8mb4&loc=UTC&multiStatements=true",
|
||||||
WalletServiceAddr: "127.0.0.1:13004",
|
WalletServiceAddr: "127.0.0.1:13004",
|
||||||
UserServiceAddr: "127.0.0.1:13005",
|
UserServiceAddr: "127.0.0.1:13005",
|
||||||
@ -73,6 +76,10 @@ func (cfg *Config) Normalize() error {
|
|||||||
if cfg.GRPCAddr == "" {
|
if cfg.GRPCAddr == "" {
|
||||||
cfg.GRPCAddr = ":13008"
|
cfg.GRPCAddr = ":13008"
|
||||||
}
|
}
|
||||||
|
cfg.HealthHTTPAddr = strings.TrimSpace(cfg.HealthHTTPAddr)
|
||||||
|
if cfg.HealthHTTPAddr == "" {
|
||||||
|
cfg.HealthHTTPAddr = ":13108"
|
||||||
|
}
|
||||||
cfg.MySQLDSN = strings.TrimSpace(cfg.MySQLDSN)
|
cfg.MySQLDSN = strings.TrimSpace(cfg.MySQLDSN)
|
||||||
if cfg.MySQLDSN == "" {
|
if cfg.MySQLDSN == "" {
|
||||||
return fmt.Errorf("mysql_dsn is required")
|
return fmt.Errorf("mysql_dsn is required")
|
||||||
|
|||||||
@ -14,6 +14,17 @@ user_service_addr: "user-service:13005"
|
|||||||
wallet_service_addr: "wallet-service:13004"
|
wallet_service_addr: "wallet-service:13004"
|
||||||
activity_service_addr: "activity-service:13006"
|
activity_service_addr: "activity-service:13006"
|
||||||
game_service_addr: "game-service:13008"
|
game_service_addr: "game-service:13008"
|
||||||
|
grpc_client:
|
||||||
|
# Docker 本地也使用生产同形态的 deadline、keepalive 和窄重试策略。
|
||||||
|
default_timeout: "5s"
|
||||||
|
connect_timeout: "2s"
|
||||||
|
keepalive_time: "30s"
|
||||||
|
keepalive_timeout: "10s"
|
||||||
|
retry_max_attempts: 2
|
||||||
|
retry_initial_backoff: "100ms"
|
||||||
|
retry_max_backoff: "500ms"
|
||||||
|
retry_backoff_multiplier: 2
|
||||||
|
retryable_status_codes: ["UNAVAILABLE"]
|
||||||
app_config:
|
app_config:
|
||||||
# H5 地址由后台 APP配置/H5配置 写入 hyapp_admin.admin_app_configs,gateway 只读下发给 App。
|
# H5 地址由后台 APP配置/H5配置 写入 hyapp_admin.admin_app_configs,gateway 只读下发给 App。
|
||||||
mysql_dsn: "hyapp:hyapp@tcp(mysql:3306)/hyapp_admin?parseTime=true&charset=utf8mb4&loc=UTC"
|
mysql_dsn: "hyapp:hyapp@tcp(mysql:3306)/hyapp_admin?parseTime=true&charset=utf8mb4&loc=UTC"
|
||||||
|
|||||||
@ -13,6 +13,18 @@ room_service_addr: "room-service.internal:13001"
|
|||||||
user_service_addr: "user-service.internal:13005"
|
user_service_addr: "user-service.internal:13005"
|
||||||
wallet_service_addr: "wallet-service.internal:13004"
|
wallet_service_addr: "wallet-service.internal:13004"
|
||||||
activity_service_addr: "activity-service.internal:13006"
|
activity_service_addr: "activity-service.internal:13006"
|
||||||
|
game_service_addr: "game-service.internal:13008"
|
||||||
|
grpc_client:
|
||||||
|
# 线上通过内网 CLB/DNS 访问内部服务;重试只覆盖瞬时 UNAVAILABLE,避免放大业务副作用。
|
||||||
|
default_timeout: "5s"
|
||||||
|
connect_timeout: "2s"
|
||||||
|
keepalive_time: "30s"
|
||||||
|
keepalive_timeout: "10s"
|
||||||
|
retry_max_attempts: 2
|
||||||
|
retry_initial_backoff: "100ms"
|
||||||
|
retry_max_backoff: "500ms"
|
||||||
|
retry_backoff_multiplier: 2
|
||||||
|
retryable_status_codes: ["UNAVAILABLE"]
|
||||||
app_config:
|
app_config:
|
||||||
# H5 地址由后台 APP配置/H5配置 写入 hyapp_admin.admin_app_configs,gateway 只读下发给 App。
|
# H5 地址由后台 APP配置/H5配置 写入 hyapp_admin.admin_app_configs,gateway 只读下发给 App。
|
||||||
mysql_dsn: "TENCENT_MYSQL_USER:TENCENT_MYSQL_PASSWORD@tcp(TENCENT_MYSQL_HOST:3306)/hyapp_admin?parseTime=true&charset=utf8mb4&loc=UTC"
|
mysql_dsn: "TENCENT_MYSQL_USER:TENCENT_MYSQL_PASSWORD@tcp(TENCENT_MYSQL_HOST:3306)/hyapp_admin?parseTime=true&charset=utf8mb4&loc=UTC"
|
||||||
|
|||||||
@ -14,6 +14,17 @@ user_service_addr: "127.0.0.1:13005"
|
|||||||
wallet_service_addr: "127.0.0.1:13004"
|
wallet_service_addr: "127.0.0.1:13004"
|
||||||
activity_service_addr: "127.0.0.1:13006"
|
activity_service_addr: "127.0.0.1:13006"
|
||||||
game_service_addr: "127.0.0.1:13008"
|
game_service_addr: "127.0.0.1:13008"
|
||||||
|
grpc_client:
|
||||||
|
# gateway 内部 gRPC 默认 deadline 包含一次 UNAVAILABLE 重试;业务长调用应在 handler 上下文显式收紧或放宽。
|
||||||
|
default_timeout: "5s"
|
||||||
|
connect_timeout: "2s"
|
||||||
|
keepalive_time: "30s"
|
||||||
|
keepalive_timeout: "10s"
|
||||||
|
retry_max_attempts: 2
|
||||||
|
retry_initial_backoff: "100ms"
|
||||||
|
retry_max_backoff: "500ms"
|
||||||
|
retry_backoff_multiplier: 2
|
||||||
|
retryable_status_codes: ["UNAVAILABLE"]
|
||||||
app_config:
|
app_config:
|
||||||
# H5 地址由后台 APP配置/H5配置 写入 hyapp_admin.admin_app_configs,gateway 只读下发给 App。
|
# H5 地址由后台 APP配置/H5配置 写入 hyapp_admin.admin_app_configs,gateway 只读下发给 App。
|
||||||
mysql_dsn: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_admin?parseTime=true&charset=utf8mb4&loc=UTC"
|
mysql_dsn: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_admin?parseTime=true&charset=utf8mb4&loc=UTC"
|
||||||
|
|||||||
@ -12,7 +12,7 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"google.golang.org/grpc"
|
"google.golang.org/grpc"
|
||||||
"google.golang.org/grpc/credentials/insecure"
|
"hyapp/pkg/grpcclient"
|
||||||
"hyapp/pkg/tencentcos"
|
"hyapp/pkg/tencentcos"
|
||||||
"hyapp/services/gateway-service/internal/appconfig"
|
"hyapp/services/gateway-service/internal/appconfig"
|
||||||
"hyapp/services/gateway-service/internal/auth"
|
"hyapp/services/gateway-service/internal/auth"
|
||||||
@ -43,31 +43,32 @@ func New(cfg config.Config) (*App, error) {
|
|||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
roomConn, err := grpc.Dial(cfg.RoomServiceAddr, grpc.WithTransportCredentials(insecure.NewCredentials()))
|
grpcConfig := cfg.GRPCClient.Runtime()
|
||||||
|
roomConn, err := grpcclient.Dial(cfg.RoomServiceAddr, grpcConfig)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
userConn, err := grpc.Dial(cfg.UserServiceAddr, grpc.WithTransportCredentials(insecure.NewCredentials()))
|
userConn, err := grpcclient.Dial(cfg.UserServiceAddr, grpcConfig)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
_ = roomConn.Close()
|
_ = roomConn.Close()
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|
||||||
walletConn, err := grpc.Dial(cfg.WalletServiceAddr, grpc.WithTransportCredentials(insecure.NewCredentials()))
|
walletConn, err := grpcclient.Dial(cfg.WalletServiceAddr, grpcConfig)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
_ = roomConn.Close()
|
_ = roomConn.Close()
|
||||||
_ = userConn.Close()
|
_ = userConn.Close()
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
activityConn, err := grpc.Dial(cfg.ActivityServiceAddr, grpc.WithTransportCredentials(insecure.NewCredentials()))
|
activityConn, err := grpcclient.Dial(cfg.ActivityServiceAddr, grpcConfig)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
_ = roomConn.Close()
|
_ = roomConn.Close()
|
||||||
_ = userConn.Close()
|
_ = userConn.Close()
|
||||||
_ = walletConn.Close()
|
_ = walletConn.Close()
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
gameConn, err := grpc.Dial(cfg.GameServiceAddr, grpc.WithTransportCredentials(insecure.NewCredentials()))
|
gameConn, err := grpcclient.Dial(cfg.GameServiceAddr, grpcConfig)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
_ = roomConn.Close()
|
_ = roomConn.Close()
|
||||||
_ = userConn.Close()
|
_ = userConn.Close()
|
||||||
@ -203,7 +204,12 @@ func New(cfg config.Config) (*App, error) {
|
|||||||
}
|
}
|
||||||
redisClose = joinClose(redisClose, loginRiskRedisClose)
|
redisClose = joinClose(redisClose, loginRiskRedisClose)
|
||||||
verifier := auth.NewVerifier(cfg.JWTSecret)
|
verifier := auth.NewVerifier(cfg.JWTSecret)
|
||||||
healthState := healthcheck.NewState(nodeID(), cfg.JWTSecret, roomConn)
|
healthState := healthcheck.NewState(nodeID(), cfg.JWTSecret, roomConn,
|
||||||
|
healthcheck.GRPCDependency{Name: "user_grpc", Conn: userConn},
|
||||||
|
healthcheck.GRPCDependency{Name: "wallet_grpc", Conn: walletConn},
|
||||||
|
healthcheck.GRPCDependency{Name: "activity_grpc", Conn: activityConn},
|
||||||
|
healthcheck.GRPCDependency{Name: "game_grpc", Conn: gameConn},
|
||||||
|
)
|
||||||
|
|
||||||
mux := http.NewServeMux()
|
mux := http.NewServeMux()
|
||||||
healthHandler := healthcheck.NewHTTPHandler(healthState)
|
healthHandler := healthcheck.NewHTTPHandler(healthState)
|
||||||
@ -328,6 +334,14 @@ func (a *App) Close() error {
|
|||||||
var err error
|
var err error
|
||||||
a.closeOnce.Do(func() {
|
a.closeOnce.Do(func() {
|
||||||
a.health.MarkDraining()
|
a.health.MarkDraining()
|
||||||
|
shutdownCtx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
// HTTP Shutdown drains accepted requests before client connections close,
|
||||||
|
// so in-flight handlers can finish their internal gRPC calls.
|
||||||
|
if shutdownErr := a.server.Shutdown(shutdownCtx); shutdownErr != nil {
|
||||||
|
// 超过 drain 预算后强制关闭,避免发布脚本被异常长连接卡住。
|
||||||
|
err = errors.Join(err, shutdownErr, a.server.Close())
|
||||||
|
}
|
||||||
if a.roomConn != nil {
|
if a.roomConn != nil {
|
||||||
err = errors.Join(err, a.roomConn.Close())
|
err = errors.Join(err, a.roomConn.Close())
|
||||||
}
|
}
|
||||||
@ -349,8 +363,6 @@ func (a *App) Close() error {
|
|||||||
if a.redisClose != nil {
|
if a.redisClose != nil {
|
||||||
err = errors.Join(err, a.redisClose())
|
err = errors.Join(err, a.redisClose())
|
||||||
}
|
}
|
||||||
|
|
||||||
err = errors.Join(err, a.server.Close())
|
|
||||||
})
|
})
|
||||||
|
|
||||||
return err
|
return err
|
||||||
|
|||||||
@ -6,6 +6,7 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"hyapp/pkg/configx"
|
"hyapp/pkg/configx"
|
||||||
|
"hyapp/pkg/grpcclient"
|
||||||
"hyapp/pkg/logx"
|
"hyapp/pkg/logx"
|
||||||
"hyapp/pkg/tencentrtc"
|
"hyapp/pkg/tencentrtc"
|
||||||
)
|
)
|
||||||
@ -22,6 +23,7 @@ type Config struct {
|
|||||||
WalletServiceAddr string `yaml:"wallet_service_addr"`
|
WalletServiceAddr string `yaml:"wallet_service_addr"`
|
||||||
ActivityServiceAddr string `yaml:"activity_service_addr"`
|
ActivityServiceAddr string `yaml:"activity_service_addr"`
|
||||||
GameServiceAddr string `yaml:"game_service_addr"`
|
GameServiceAddr string `yaml:"game_service_addr"`
|
||||||
|
GRPCClient GRPCClientConfig `yaml:"grpc_client"`
|
||||||
AuthRateLimit AuthRateLimitConfig `yaml:"auth_rate_limit"`
|
AuthRateLimit AuthRateLimitConfig `yaml:"auth_rate_limit"`
|
||||||
LoginRisk LoginRiskConfig `yaml:"login_risk"`
|
LoginRisk LoginRiskConfig `yaml:"login_risk"`
|
||||||
AppConfig AppConfigConfig `yaml:"app_config"`
|
AppConfig AppConfigConfig `yaml:"app_config"`
|
||||||
@ -31,6 +33,43 @@ type Config struct {
|
|||||||
Log logx.Config `yaml:"log"`
|
Log logx.Config `yaml:"log"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// GRPCClientConfig 控制 gateway 访问内部服务的 gRPC transport 策略。
|
||||||
|
type GRPCClientConfig struct {
|
||||||
|
// DefaultTimeout 是没有显式 request deadline 时的 RPC 总预算,包含重试耗时。
|
||||||
|
DefaultTimeout time.Duration `yaml:"default_timeout"`
|
||||||
|
// ConnectTimeout 是单次建连最小预算,避免内网目标不可达时长期卡住。
|
||||||
|
ConnectTimeout time.Duration `yaml:"connect_timeout"`
|
||||||
|
// KeepaliveTime 是空闲连接探活周期,帮助 CLB/NAT 后的半开连接尽早发现。
|
||||||
|
KeepaliveTime time.Duration `yaml:"keepalive_time"`
|
||||||
|
// KeepaliveTimeout 是 keepalive ack 等待时间。
|
||||||
|
KeepaliveTimeout time.Duration `yaml:"keepalive_timeout"`
|
||||||
|
// RetryMaxAttempts 是包含首次调用在内的最大尝试次数;小于 2 时关闭显式重试。
|
||||||
|
RetryMaxAttempts int `yaml:"retry_max_attempts"`
|
||||||
|
// RetryInitialBackoff 是第一次重试前等待时间。
|
||||||
|
RetryInitialBackoff time.Duration `yaml:"retry_initial_backoff"`
|
||||||
|
// RetryMaxBackoff 是重试退避上限。
|
||||||
|
RetryMaxBackoff time.Duration `yaml:"retry_max_backoff"`
|
||||||
|
// RetryBackoffMultiplier 控制退避增长倍率。
|
||||||
|
RetryBackoffMultiplier float64 `yaml:"retry_backoff_multiplier"`
|
||||||
|
// RetryableStatusCodes 限定可重试状态,生产默认只允许 UNAVAILABLE。
|
||||||
|
RetryableStatusCodes []string `yaml:"retryable_status_codes"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Runtime 转换为共享 gRPC client 配置,避免 app 层重复维护默认值。
|
||||||
|
func (c GRPCClientConfig) Runtime() grpcclient.Config {
|
||||||
|
return grpcclient.Normalize(grpcclient.Config{
|
||||||
|
DefaultTimeout: c.DefaultTimeout,
|
||||||
|
ConnectTimeout: c.ConnectTimeout,
|
||||||
|
KeepaliveTime: c.KeepaliveTime,
|
||||||
|
KeepaliveTimeout: c.KeepaliveTimeout,
|
||||||
|
RetryMaxAttempts: c.RetryMaxAttempts,
|
||||||
|
RetryInitialBackoff: c.RetryInitialBackoff,
|
||||||
|
RetryMaxBackoff: c.RetryMaxBackoff,
|
||||||
|
RetryBackoffMultiplier: c.RetryBackoffMultiplier,
|
||||||
|
RetryableStatusCodes: c.RetryableStatusCodes,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
// LoginRiskConfig 控制 gateway 登录入口快拦和 Redis 风控缓存读取。
|
// LoginRiskConfig 控制 gateway 登录入口快拦和 Redis 风控缓存读取。
|
||||||
type LoginRiskConfig struct {
|
type LoginRiskConfig struct {
|
||||||
Enabled bool `yaml:"enabled"`
|
Enabled bool `yaml:"enabled"`
|
||||||
@ -147,6 +186,17 @@ func Default() Config {
|
|||||||
WalletServiceAddr: "127.0.0.1:13004",
|
WalletServiceAddr: "127.0.0.1:13004",
|
||||||
ActivityServiceAddr: "127.0.0.1:13006",
|
ActivityServiceAddr: "127.0.0.1:13006",
|
||||||
GameServiceAddr: "127.0.0.1:13008",
|
GameServiceAddr: "127.0.0.1:13008",
|
||||||
|
GRPCClient: GRPCClientConfig{
|
||||||
|
DefaultTimeout: 5 * time.Second,
|
||||||
|
ConnectTimeout: 2 * time.Second,
|
||||||
|
KeepaliveTime: 30 * time.Second,
|
||||||
|
KeepaliveTimeout: 10 * time.Second,
|
||||||
|
RetryMaxAttempts: 2,
|
||||||
|
RetryInitialBackoff: 100 * time.Millisecond,
|
||||||
|
RetryMaxBackoff: 500 * time.Millisecond,
|
||||||
|
RetryBackoffMultiplier: 2,
|
||||||
|
RetryableStatusCodes: []string{"UNAVAILABLE"},
|
||||||
|
},
|
||||||
AppConfig: AppConfigConfig{
|
AppConfig: AppConfigConfig{
|
||||||
MySQLDSN: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_admin?parseTime=true&charset=utf8mb4&loc=UTC",
|
MySQLDSN: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_admin?parseTime=true&charset=utf8mb4&loc=UTC",
|
||||||
},
|
},
|
||||||
@ -234,6 +284,18 @@ func (cfg *Config) Normalize() error {
|
|||||||
if cfg.GameServiceAddr == "" {
|
if cfg.GameServiceAddr == "" {
|
||||||
cfg.GameServiceAddr = "127.0.0.1:13008"
|
cfg.GameServiceAddr = "127.0.0.1:13008"
|
||||||
}
|
}
|
||||||
|
runtimeGRPC := cfg.GRPCClient.Runtime()
|
||||||
|
cfg.GRPCClient = GRPCClientConfig{
|
||||||
|
DefaultTimeout: runtimeGRPC.DefaultTimeout,
|
||||||
|
ConnectTimeout: runtimeGRPC.ConnectTimeout,
|
||||||
|
KeepaliveTime: runtimeGRPC.KeepaliveTime,
|
||||||
|
KeepaliveTimeout: runtimeGRPC.KeepaliveTimeout,
|
||||||
|
RetryMaxAttempts: runtimeGRPC.RetryMaxAttempts,
|
||||||
|
RetryInitialBackoff: runtimeGRPC.RetryInitialBackoff,
|
||||||
|
RetryMaxBackoff: runtimeGRPC.RetryMaxBackoff,
|
||||||
|
RetryBackoffMultiplier: runtimeGRPC.RetryBackoffMultiplier,
|
||||||
|
RetryableStatusCodes: runtimeGRPC.RetryableStatusCodes,
|
||||||
|
}
|
||||||
cfg.AppConfig.MySQLDSN = strings.TrimSpace(cfg.AppConfig.MySQLDSN)
|
cfg.AppConfig.MySQLDSN = strings.TrimSpace(cfg.AppConfig.MySQLDSN)
|
||||||
cfg.LoginRisk.RedisAddr = strings.TrimSpace(cfg.LoginRisk.RedisAddr)
|
cfg.LoginRisk.RedisAddr = strings.TrimSpace(cfg.LoginRisk.RedisAddr)
|
||||||
cfg.LoginRisk.KeyPrefix = strings.TrimSpace(cfg.LoginRisk.KeyPrefix)
|
cfg.LoginRisk.KeyPrefix = strings.TrimSpace(cfg.LoginRisk.KeyPrefix)
|
||||||
|
|||||||
@ -22,6 +22,9 @@ func TestLoad(t *testing.T) {
|
|||||||
if cfg.RoomServiceAddr != "127.0.0.1:13001" {
|
if cfg.RoomServiceAddr != "127.0.0.1:13001" {
|
||||||
t.Fatalf("unexpected room addr: %s", cfg.RoomServiceAddr)
|
t.Fatalf("unexpected room addr: %s", cfg.RoomServiceAddr)
|
||||||
}
|
}
|
||||||
|
if cfg.GRPCClient.DefaultTimeout != 5*time.Second || cfg.GRPCClient.RetryMaxAttempts != 2 || len(cfg.GRPCClient.RetryableStatusCodes) != 1 || cfg.GRPCClient.RetryableStatusCodes[0] != "UNAVAILABLE" {
|
||||||
|
t.Fatalf("unexpected grpc client policy: %+v", cfg.GRPCClient)
|
||||||
|
}
|
||||||
if cfg.AppConfig.MySQLDSN == "" || !strings.Contains(cfg.AppConfig.MySQLDSN, "hyapp_admin") {
|
if cfg.AppConfig.MySQLDSN == "" || !strings.Contains(cfg.AppConfig.MySQLDSN, "hyapp_admin") {
|
||||||
t.Fatalf("gateway app config mysql dsn must point to admin config db: %+v", cfg.AppConfig)
|
t.Fatalf("gateway app config mysql dsn must point to admin config db: %+v", cfg.AppConfig)
|
||||||
}
|
}
|
||||||
|
|||||||
@ -15,8 +15,8 @@ import (
|
|||||||
const (
|
const (
|
||||||
serviceName = "gateway-service"
|
serviceName = "gateway-service"
|
||||||
|
|
||||||
readyBudget = 500 * time.Millisecond
|
readyBudget = 2 * time.Second
|
||||||
roomGRPCBudget = 300 * time.Millisecond
|
grpcCheckBudget = 500 * time.Millisecond
|
||||||
componentStatus = "ok"
|
componentStatus = "ok"
|
||||||
failStatus = "fail"
|
failStatus = "fail"
|
||||||
)
|
)
|
||||||
@ -28,22 +28,32 @@ type Check struct {
|
|||||||
Message string `json:"message,omitempty"`
|
Message string `json:"message,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// GRPCDependency describes one internal service connection required before the
|
||||||
|
// gateway should receive new external traffic.
|
||||||
|
type GRPCDependency struct {
|
||||||
|
Name string
|
||||||
|
Conn *grpc.ClientConn
|
||||||
|
}
|
||||||
|
|
||||||
// State 记录 gateway 接新流量所需的进程状态和 room-service 连接状态。
|
// State 记录 gateway 接新流量所需的进程状态和 room-service 连接状态。
|
||||||
type State struct {
|
type State struct {
|
||||||
nodeID string
|
nodeID string
|
||||||
jwtSecret string
|
jwtSecret string
|
||||||
room *grpc.ClientConn
|
dependencies []GRPCDependency
|
||||||
|
|
||||||
httpServing atomic.Bool
|
httpServing atomic.Bool
|
||||||
draining atomic.Bool
|
draining atomic.Bool
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewState 创建 gateway 健康状态所有者。
|
// NewState 创建 gateway 健康状态所有者。
|
||||||
func NewState(nodeID string, jwtSecret string, room *grpc.ClientConn) *State {
|
func NewState(nodeID string, jwtSecret string, room *grpc.ClientConn, dependencies ...GRPCDependency) *State {
|
||||||
|
all := make([]GRPCDependency, 0, 1+len(dependencies))
|
||||||
|
all = append(all, GRPCDependency{Name: "room_grpc", Conn: room})
|
||||||
|
all = append(all, dependencies...)
|
||||||
return &State{
|
return &State{
|
||||||
nodeID: nodeID,
|
nodeID: nodeID,
|
||||||
jwtSecret: jwtSecret,
|
jwtSecret: jwtSecret,
|
||||||
room: room,
|
dependencies: all,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -90,13 +100,7 @@ func (s *State) Ready(ctx context.Context) []Check {
|
|||||||
boolCheck("draining", !s.draining.Load(), "node is draining"),
|
boolCheck("draining", !s.draining.Load(), "node is draining"),
|
||||||
}
|
}
|
||||||
|
|
||||||
roomCtx, roomCancel := context.WithTimeout(ctx, roomGRPCBudget)
|
checks = append(checks, s.grpcChecks(ctx)...)
|
||||||
defer roomCancel()
|
|
||||||
if err := waitGRPCReady(roomCtx, s.room); err != nil {
|
|
||||||
checks = append(checks, failCheck("room_grpc", err.Error()))
|
|
||||||
} else {
|
|
||||||
checks = append(checks, okCheck("room_grpc"))
|
|
||||||
}
|
|
||||||
|
|
||||||
return checks
|
return checks
|
||||||
}
|
}
|
||||||
@ -112,6 +116,36 @@ func Healthy(checks []Check) bool {
|
|||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (s *State) grpcChecks(ctx context.Context) []Check {
|
||||||
|
results := make([]Check, len(s.dependencies))
|
||||||
|
done := make(chan struct{})
|
||||||
|
|
||||||
|
for index, dependency := range s.dependencies {
|
||||||
|
index := index
|
||||||
|
dependency := dependency
|
||||||
|
go func() {
|
||||||
|
name := strings.TrimSpace(dependency.Name)
|
||||||
|
if name == "" {
|
||||||
|
name = "grpc_dependency"
|
||||||
|
}
|
||||||
|
|
||||||
|
checkCtx, cancel := context.WithTimeout(ctx, grpcCheckBudget)
|
||||||
|
defer cancel()
|
||||||
|
if err := waitGRPCReady(checkCtx, dependency.Conn); err != nil {
|
||||||
|
results[index] = failCheck(name, err.Error())
|
||||||
|
} else {
|
||||||
|
results[index] = okCheck(name)
|
||||||
|
}
|
||||||
|
done <- struct{}{}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
|
||||||
|
for range s.dependencies {
|
||||||
|
<-done
|
||||||
|
}
|
||||||
|
return results
|
||||||
|
}
|
||||||
|
|
||||||
func waitGRPCReady(ctx context.Context, conn *grpc.ClientConn) error {
|
func waitGRPCReady(ctx context.Context, conn *grpc.ClientConn) error {
|
||||||
if conn == nil {
|
if conn == nil {
|
||||||
return errors.New("room gRPC connection is not initialized")
|
return errors.New("room gRPC connection is not initialized")
|
||||||
|
|||||||
@ -30,7 +30,7 @@ COPY services/room-service/configs/config.docker.yaml /app/config.yaml
|
|||||||
|
|
||||||
USER appuser
|
USER appuser
|
||||||
|
|
||||||
EXPOSE 13001
|
EXPOSE 13001 13101
|
||||||
|
|
||||||
ENTRYPOINT ["/app/server"]
|
ENTRYPOINT ["/app/server"]
|
||||||
CMD ["-config", "/app/config.yaml"]
|
CMD ["-config", "/app/config.yaml"]
|
||||||
|
|||||||
@ -8,6 +8,11 @@ log:
|
|||||||
include_response_body: false
|
include_response_body: false
|
||||||
max_payload_bytes: 2048
|
max_payload_bytes: 2048
|
||||||
grpc_addr: ":13001"
|
grpc_addr: ":13001"
|
||||||
|
advertise_addr: "room-service:13001"
|
||||||
|
health_http_addr: ":13101"
|
||||||
|
node_registry_ttl: "30s"
|
||||||
|
node_registry_heartbeat_interval: "10s"
|
||||||
|
owner_forward_timeout: "2s"
|
||||||
lease_ttl: "10s"
|
lease_ttl: "10s"
|
||||||
rank_limit: 20
|
rank_limit: 20
|
||||||
snapshot_every_n: 10
|
snapshot_every_n: 10
|
||||||
|
|||||||
@ -8,6 +8,11 @@ log:
|
|||||||
include_response_body: false
|
include_response_body: false
|
||||||
max_payload_bytes: 2048
|
max_payload_bytes: 2048
|
||||||
grpc_addr: ":13001"
|
grpc_addr: ":13001"
|
||||||
|
advertise_addr: "ROOM_NODE_PRIVATE_IP_OR_DNS:13001"
|
||||||
|
health_http_addr: ":13101"
|
||||||
|
node_registry_ttl: "30s"
|
||||||
|
node_registry_heartbeat_interval: "10s"
|
||||||
|
owner_forward_timeout: "2s"
|
||||||
lease_ttl: "10s"
|
lease_ttl: "10s"
|
||||||
rank_limit: 20
|
rank_limit: 20
|
||||||
snapshot_every_n: 10
|
snapshot_every_n: 10
|
||||||
|
|||||||
@ -11,6 +11,11 @@ lease_ttl: "10s"
|
|||||||
rank_limit: 20
|
rank_limit: 20
|
||||||
snapshot_every_n: 10
|
snapshot_every_n: 10
|
||||||
grpc_addr: ":13001"
|
grpc_addr: ":13001"
|
||||||
|
advertise_addr: "127.0.0.1:13001"
|
||||||
|
health_http_addr: ":13101"
|
||||||
|
node_registry_ttl: "30s"
|
||||||
|
node_registry_heartbeat_interval: "10s"
|
||||||
|
owner_forward_timeout: "2s"
|
||||||
wallet_service_addr: "127.0.0.1:13004"
|
wallet_service_addr: "127.0.0.1:13004"
|
||||||
activity_service_addr: "127.0.0.1:13006"
|
activity_service_addr: "127.0.0.1:13006"
|
||||||
tencent_im:
|
tencent_im:
|
||||||
|
|||||||
@ -4,6 +4,7 @@ package app
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"errors"
|
"errors"
|
||||||
|
"log/slog"
|
||||||
"net"
|
"net"
|
||||||
"sync"
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
@ -14,6 +15,8 @@ import (
|
|||||||
activityv1 "hyapp.local/api/proto/activity/v1"
|
activityv1 "hyapp.local/api/proto/activity/v1"
|
||||||
roomv1 "hyapp.local/api/proto/room/v1"
|
roomv1 "hyapp.local/api/proto/room/v1"
|
||||||
"hyapp/pkg/grpchealth"
|
"hyapp/pkg/grpchealth"
|
||||||
|
"hyapp/pkg/grpcshutdown"
|
||||||
|
"hyapp/pkg/healthhttp"
|
||||||
"hyapp/pkg/logx"
|
"hyapp/pkg/logx"
|
||||||
"hyapp/pkg/tencentim"
|
"hyapp/pkg/tencentim"
|
||||||
"hyapp/services/room-service/internal/config"
|
"hyapp/services/room-service/internal/config"
|
||||||
@ -43,8 +46,12 @@ type App struct {
|
|||||||
repository *mysqlstorage.Repository
|
repository *mysqlstorage.Repository
|
||||||
// redisClose 关闭 Redis client,Redis directory 本身不拥有额外生命周期。
|
// redisClose 关闭 Redis client,Redis directory 本身不拥有额外生命周期。
|
||||||
redisClose func() error
|
redisClose func() error
|
||||||
|
// nodeRegistry 保存当前节点可被 owner 转发直连的实例地址。
|
||||||
|
nodeRegistry router.NodeRegistry
|
||||||
// health 汇总 gRPC、runtime、MySQL 和 Redis lease 探测状态。
|
// health 汇总 gRPC、runtime、MySQL 和 Redis lease 探测状态。
|
||||||
health *healthcheck.State
|
health *healthcheck.State
|
||||||
|
// healthHTTP 给 CLB 和发布脚本提供 HTTP readiness,不承载业务请求。
|
||||||
|
healthHTTP *healthhttp.Server
|
||||||
// workerCtx 统一控制本节点后台 worker,关闭时必须先停止 worker 再释放 MySQL/Redis。
|
// workerCtx 统一控制本节点后台 worker,关闭时必须先停止 worker 再释放 MySQL/Redis。
|
||||||
workerCtx context.Context
|
workerCtx context.Context
|
||||||
// workerCancel 停止 presence stale 和 outbox 补偿 worker。
|
// workerCancel 停止 presence stale 和 outbox 补偿 worker。
|
||||||
@ -135,16 +142,26 @@ func New(cfg config.Config) (*App, error) {
|
|||||||
}, directory, repository, integration.NewGRPCWalletClient(walletConn), roomPublisher, outboxPublisher)
|
}, directory, repository, integration.NewGRPCWalletClient(walletConn), roomPublisher, outboxPublisher)
|
||||||
|
|
||||||
server := grpc.NewServer(grpc.UnaryInterceptor(logx.UnaryServerInterceptor("room-service")))
|
server := grpc.NewServer(grpc.UnaryInterceptor(logx.UnaryServerInterceptor("room-service")))
|
||||||
|
roomServer := grpcserver.NewServer(svc, grpcserver.WithOwnerForwarding(cfg.NodeID, directory, directory, cfg.OwnerForwardTimeout))
|
||||||
// 命令服务处理房间状态变更,守卫服务给腾讯云 IM 回调或 gateway 做发言和进群校验。
|
// 命令服务处理房间状态变更,守卫服务给腾讯云 IM 回调或 gateway 做发言和进群校验。
|
||||||
roomv1.RegisterRoomCommandServiceServer(server, grpcserver.NewServer(svc))
|
roomv1.RegisterRoomCommandServiceServer(server, roomServer)
|
||||||
roomv1.RegisterRoomGuardServiceServer(server, grpcserver.NewServer(svc))
|
roomv1.RegisterRoomGuardServiceServer(server, roomServer)
|
||||||
roomv1.RegisterRoomQueryServiceServer(server, grpcserver.NewServer(svc))
|
roomv1.RegisterRoomQueryServiceServer(server, roomServer)
|
||||||
healthState := healthcheck.NewState(cfg.NodeID, svc, repository, directory)
|
healthState := healthcheck.NewState(cfg.NodeID, svc, repository, directory)
|
||||||
healthgrpc.RegisterHealthServer(server, grpchealth.NewServer(healthState))
|
healthgrpc.RegisterHealthServer(server, grpchealth.NewServer(healthState))
|
||||||
|
healthHTTP, err := healthhttp.New(cfg.HealthHTTPAddr, cfg.NodeID, healthState)
|
||||||
|
if err != nil {
|
||||||
|
_ = activityConn.Close()
|
||||||
|
_ = walletConn.Close()
|
||||||
|
_ = repository.Close()
|
||||||
|
_ = redisClient.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
// listener 在 New 阶段创建,可以把端口占用作为启动失败暴露。
|
// listener 在 New 阶段创建,可以把端口占用作为启动失败暴露。
|
||||||
listener, err := net.Listen("tcp", cfg.GRPCAddr)
|
listener, err := net.Listen("tcp", cfg.GRPCAddr)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
_ = healthHTTP.Close(context.Background())
|
||||||
_ = activityConn.Close()
|
_ = activityConn.Close()
|
||||||
_ = walletConn.Close()
|
_ = walletConn.Close()
|
||||||
_ = repository.Close()
|
_ = repository.Close()
|
||||||
@ -162,7 +179,9 @@ func New(cfg config.Config) (*App, error) {
|
|||||||
activityConn: activityConn,
|
activityConn: activityConn,
|
||||||
repository: repository,
|
repository: repository,
|
||||||
redisClose: redisClient.Close,
|
redisClose: redisClient.Close,
|
||||||
|
nodeRegistry: directory,
|
||||||
health: healthState,
|
health: healthState,
|
||||||
|
healthHTTP: healthHTTP,
|
||||||
workerCtx: workerCtx,
|
workerCtx: workerCtx,
|
||||||
workerCancel: workerCancel,
|
workerCancel: workerCancel,
|
||||||
}, nil
|
}, nil
|
||||||
@ -170,14 +189,22 @@ func New(cfg config.Config) (*App, error) {
|
|||||||
|
|
||||||
// Run 启动 gRPC 服务。
|
// Run 启动 gRPC 服务。
|
||||||
func (a *App) Run() error {
|
func (a *App) Run() error {
|
||||||
|
a.runHealthHTTP()
|
||||||
// health 先标记 serving,再进入 Serve,ready 才能在监听开始后通过。
|
// health 先标记 serving,再进入 Serve,ready 才能在监听开始后通过。
|
||||||
a.health.MarkGRPCServing()
|
a.health.MarkGRPCServing()
|
||||||
|
if err := a.registerRoomNode(context.Background()); err != nil {
|
||||||
|
a.health.MarkGRPCStopped()
|
||||||
|
return err
|
||||||
|
}
|
||||||
defer func() {
|
defer func() {
|
||||||
if a.workerCancel != nil {
|
if a.workerCancel != nil {
|
||||||
a.workerCancel()
|
a.workerCancel()
|
||||||
}
|
}
|
||||||
a.workerWG.Wait()
|
a.workerWG.Wait()
|
||||||
}()
|
}()
|
||||||
|
a.workerWG.Go(func() {
|
||||||
|
a.runNodeRegistryHeartbeat(a.workerCtx)
|
||||||
|
})
|
||||||
if a.cfg.PresenceStaleScanInterval > 0 {
|
if a.cfg.PresenceStaleScanInterval > 0 {
|
||||||
// presence worker 只清理本节点已装载 Cell,命令仍走 Room Cell 持久化链路。
|
// presence worker 只清理本节点已装载 Cell,命令仍走 Room Cell 持久化链路。
|
||||||
a.workerWG.Go(func() {
|
a.workerWG.Go(func() {
|
||||||
@ -228,7 +255,15 @@ func (a *App) Close() {
|
|||||||
}
|
}
|
||||||
a.workerWG.Wait()
|
a.workerWG.Wait()
|
||||||
// GracefulStop 让已进入的 gRPC 请求自然结束。
|
// GracefulStop 让已进入的 gRPC 请求自然结束。
|
||||||
a.grpcServer.GracefulStop()
|
grpcshutdown.GracefulStop(a.grpcServer, 15*time.Second)
|
||||||
|
if a.service != nil {
|
||||||
|
// gRPC drain 完成后释放本节点持有的 room route,降低滚动发布时等待 Redis TTL 的窗口。
|
||||||
|
releaseCtx, releaseCancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||||
|
a.service.ReleaseLoadedRoomLeases(releaseCtx)
|
||||||
|
releaseCancel()
|
||||||
|
}
|
||||||
|
a.unregisterRoomNode()
|
||||||
|
a.closeHealthHTTP()
|
||||||
if a.walletConn != nil {
|
if a.walletConn != nil {
|
||||||
_ = a.walletConn.Close()
|
_ = a.walletConn.Close()
|
||||||
}
|
}
|
||||||
@ -245,3 +280,74 @@ func (a *App) Close() {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (a *App) registerRoomNode(ctx context.Context) error {
|
||||||
|
if a.nodeRegistry == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if _, ok := ctx.Deadline(); !ok {
|
||||||
|
var cancel context.CancelFunc
|
||||||
|
ctx, cancel = context.WithTimeout(ctx, 2*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
}
|
||||||
|
return a.nodeRegistry.RegisterNode(ctx, router.NodeRegistration{
|
||||||
|
NodeID: a.cfg.NodeID,
|
||||||
|
GRPCAddr: a.cfg.AdvertiseAddr,
|
||||||
|
}, a.cfg.NodeRegistryTTL)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (a *App) runNodeRegistryHeartbeat(ctx context.Context) {
|
||||||
|
if a.nodeRegistry == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ticker := time.NewTicker(a.cfg.NodeRegistryHeartbeatInterval)
|
||||||
|
defer ticker.Stop()
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-ticker.C:
|
||||||
|
if err := a.registerRoomNode(ctx); err != nil {
|
||||||
|
logx.Warn(ctx, "room_node_register_failed",
|
||||||
|
slog.String("node_id", a.cfg.NodeID),
|
||||||
|
slog.String("advertise_addr", a.cfg.AdvertiseAddr),
|
||||||
|
slog.String("error", err.Error()),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (a *App) unregisterRoomNode() {
|
||||||
|
if a.nodeRegistry == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
if err := a.nodeRegistry.UnregisterNode(ctx, a.cfg.NodeID); err != nil {
|
||||||
|
logx.Warn(ctx, "room_node_unregister_failed",
|
||||||
|
slog.String("node_id", a.cfg.NodeID),
|
||||||
|
slog.String("error", err.Error()),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (a *App) runHealthHTTP() {
|
||||||
|
if a.healthHTTP == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
go func() {
|
||||||
|
if err := a.healthHTTP.Run(); err != nil {
|
||||||
|
logx.Error(context.Background(), "health_http_run_failed", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (a *App) closeHealthHTTP() {
|
||||||
|
if a.healthHTTP == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
_ = a.healthHTTP.Close(ctx)
|
||||||
|
}
|
||||||
|
|||||||
@ -3,6 +3,7 @@ package config
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"net"
|
||||||
"net/http"
|
"net/http"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
@ -22,6 +23,16 @@ type Config struct {
|
|||||||
NodeID string `yaml:"node_id"`
|
NodeID string `yaml:"node_id"`
|
||||||
// GRPCAddr 是内部 gRPC 监听地址,gateway-service 通过它访问 room-service。
|
// GRPCAddr 是内部 gRPC 监听地址,gateway-service 通过它访问 room-service。
|
||||||
GRPCAddr string `yaml:"grpc_addr"`
|
GRPCAddr string `yaml:"grpc_addr"`
|
||||||
|
// AdvertiseAddr 是其他 room-service 实例转发到本节点时使用的私网地址,不能写负载均衡地址。
|
||||||
|
AdvertiseAddr string `yaml:"advertise_addr"`
|
||||||
|
// HealthHTTPAddr 是给 CLB/发布脚本探活使用的 HTTP 监听地址,不承载业务流量。
|
||||||
|
HealthHTTPAddr string `yaml:"health_http_addr"`
|
||||||
|
// NodeRegistryTTL 是 node_id -> advertise_addr 注册的有效期。
|
||||||
|
NodeRegistryTTL time.Duration `yaml:"node_registry_ttl"`
|
||||||
|
// NodeRegistryHeartbeatInterval 是刷新节点注册的周期。
|
||||||
|
NodeRegistryHeartbeatInterval time.Duration `yaml:"node_registry_heartbeat_interval"`
|
||||||
|
// OwnerForwardTimeout 是非 owner 节点转发到当前 owner 的单次 RPC 超时。
|
||||||
|
OwnerForwardTimeout time.Duration `yaml:"owner_forward_timeout"`
|
||||||
// LeaseTTL 是 room owner 租约有效期,节点故障后必须等它过期才能接管。
|
// LeaseTTL 是 room owner 租约有效期,节点故障后必须等它过期才能接管。
|
||||||
LeaseTTL time.Duration `yaml:"lease_ttl"`
|
LeaseTTL time.Duration `yaml:"lease_ttl"`
|
||||||
// RankLimit 是房间本地礼物榜 top N。
|
// RankLimit 是房间本地礼物榜 top N。
|
||||||
@ -129,6 +140,10 @@ func Default() Config {
|
|||||||
Environment: "local",
|
Environment: "local",
|
||||||
NodeID: "room-node-local",
|
NodeID: "room-node-local",
|
||||||
GRPCAddr: ":13001",
|
GRPCAddr: ":13001",
|
||||||
|
AdvertiseAddr: "127.0.0.1:13001",
|
||||||
|
HealthHTTPAddr: ":13101",
|
||||||
|
NodeRegistryTTL: 30 * time.Second,
|
||||||
|
OwnerForwardTimeout: 2 * time.Second,
|
||||||
LeaseTTL: 10 * time.Second,
|
LeaseTTL: 10 * time.Second,
|
||||||
RankLimit: 20,
|
RankLimit: 20,
|
||||||
SnapshotEveryN: 10,
|
SnapshotEveryN: 10,
|
||||||
@ -190,6 +205,33 @@ func Normalize(cfg Config) (Config, error) {
|
|||||||
if cfg.NodeID == "" {
|
if cfg.NodeID == "" {
|
||||||
cfg.NodeID = cfg.ServiceName
|
cfg.NodeID = cfg.ServiceName
|
||||||
}
|
}
|
||||||
|
cfg.GRPCAddr = strings.TrimSpace(cfg.GRPCAddr)
|
||||||
|
if cfg.GRPCAddr == "" {
|
||||||
|
cfg.GRPCAddr = ":13001"
|
||||||
|
}
|
||||||
|
cfg.AdvertiseAddr = strings.TrimSpace(cfg.AdvertiseAddr)
|
||||||
|
if cfg.AdvertiseAddr == "" {
|
||||||
|
cfg.AdvertiseAddr = defaultAdvertiseAddr(cfg.GRPCAddr)
|
||||||
|
}
|
||||||
|
cfg.HealthHTTPAddr = strings.TrimSpace(cfg.HealthHTTPAddr)
|
||||||
|
if cfg.HealthHTTPAddr == "" {
|
||||||
|
cfg.HealthHTTPAddr = ":13101"
|
||||||
|
}
|
||||||
|
if cfg.NodeRegistryTTL <= 0 {
|
||||||
|
cfg.NodeRegistryTTL = 30 * time.Second
|
||||||
|
}
|
||||||
|
if cfg.NodeRegistryHeartbeatInterval <= 0 {
|
||||||
|
cfg.NodeRegistryHeartbeatInterval = cfg.NodeRegistryTTL / 3
|
||||||
|
}
|
||||||
|
if cfg.NodeRegistryHeartbeatInterval <= 0 {
|
||||||
|
cfg.NodeRegistryHeartbeatInterval = 10 * time.Second
|
||||||
|
}
|
||||||
|
if cfg.NodeRegistryHeartbeatInterval >= cfg.NodeRegistryTTL {
|
||||||
|
cfg.NodeRegistryHeartbeatInterval = cfg.NodeRegistryTTL / 3
|
||||||
|
}
|
||||||
|
if cfg.OwnerForwardTimeout <= 0 {
|
||||||
|
cfg.OwnerForwardTimeout = 2 * time.Second
|
||||||
|
}
|
||||||
cfg.WalletServiceAddr = strings.TrimSpace(cfg.WalletServiceAddr)
|
cfg.WalletServiceAddr = strings.TrimSpace(cfg.WalletServiceAddr)
|
||||||
if cfg.WalletServiceAddr == "" {
|
if cfg.WalletServiceAddr == "" {
|
||||||
cfg.WalletServiceAddr = "127.0.0.1:13004"
|
cfg.WalletServiceAddr = "127.0.0.1:13004"
|
||||||
@ -207,6 +249,19 @@ func Normalize(cfg Config) (Config, error) {
|
|||||||
return cfg, nil
|
return cfg, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func defaultAdvertiseAddr(grpcAddr string) string {
|
||||||
|
host, port, err := net.SplitHostPort(grpcAddr)
|
||||||
|
if err != nil {
|
||||||
|
// 非 host:port 形态保持原值,由启动拨测暴露错误。
|
||||||
|
return grpcAddr
|
||||||
|
}
|
||||||
|
if host == "" || host == "0.0.0.0" || host == "::" {
|
||||||
|
// 本地默认值只能服务单机开发;多机部署必须显式配置真实私网 IP 或主机名。
|
||||||
|
host = "127.0.0.1"
|
||||||
|
}
|
||||||
|
return net.JoinHostPort(host, port)
|
||||||
|
}
|
||||||
|
|
||||||
func normalizeOutboxWorkerConfig(cfg OutboxWorkerConfig) (OutboxWorkerConfig, error) {
|
func normalizeOutboxWorkerConfig(cfg OutboxWorkerConfig) (OutboxWorkerConfig, error) {
|
||||||
// Outbox worker 的所有时间和批量参数都必须有安全下限,避免配置缺省导致 busy loop。
|
// Outbox worker 的所有时间和批量参数都必须有安全下限,避免配置缺省导致 busy loop。
|
||||||
defaults := defaultOutboxWorkerConfig()
|
defaults := defaultOutboxWorkerConfig()
|
||||||
|
|||||||
@ -14,10 +14,16 @@ func TestLoad(t *testing.T) {
|
|||||||
t.Fatalf("Load failed: %v", err)
|
t.Fatalf("Load failed: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
// room-service 默认只暴露 gRPC,端口必须保持 13xxx 约束。
|
// room-service 业务只走 gRPC,HTTP health 仅服务 CLB 探活,端口必须保持 13xxx 约束。
|
||||||
if cfg.GRPCAddr != ":13001" {
|
if cfg.GRPCAddr != ":13001" {
|
||||||
t.Fatalf("unexpected grpc addr: %s", cfg.GRPCAddr)
|
t.Fatalf("unexpected grpc addr: %s", cfg.GRPCAddr)
|
||||||
}
|
}
|
||||||
|
if cfg.HealthHTTPAddr != ":13101" {
|
||||||
|
t.Fatalf("unexpected health http addr: %s", cfg.HealthHTTPAddr)
|
||||||
|
}
|
||||||
|
if cfg.AdvertiseAddr != "127.0.0.1:13001" || cfg.NodeRegistryTTL != 30*time.Second || cfg.NodeRegistryHeartbeatInterval != 10*time.Second || cfg.OwnerForwardTimeout != 2*time.Second {
|
||||||
|
t.Fatalf("unexpected owner routing config: advertise=%s ttl=%s heartbeat=%s timeout=%s", cfg.AdvertiseAddr, cfg.NodeRegistryTTL, cfg.NodeRegistryHeartbeatInterval, cfg.OwnerForwardTimeout)
|
||||||
|
}
|
||||||
|
|
||||||
// lease TTL 是故障接管抖动窗口的基础参数。
|
// lease TTL 是故障接管抖动窗口的基础参数。
|
||||||
if cfg.LeaseTTL != 10*time.Second {
|
if cfg.LeaseTTL != 10*time.Second {
|
||||||
@ -55,6 +61,9 @@ func TestLoadTencentExample(t *testing.T) {
|
|||||||
if cfg.TencentIM.RequestTimeout != 5*time.Second {
|
if cfg.TencentIM.RequestTimeout != 5*time.Second {
|
||||||
t.Fatalf("unexpected request timeout: %s", cfg.TencentIM.RequestTimeout)
|
t.Fatalf("unexpected request timeout: %s", cfg.TencentIM.RequestTimeout)
|
||||||
}
|
}
|
||||||
|
if cfg.AdvertiseAddr == "" || cfg.AdvertiseAddr == "room-service.internal:13001" {
|
||||||
|
t.Fatalf("tencent example must use per-instance advertise_addr, got %q", cfg.AdvertiseAddr)
|
||||||
|
}
|
||||||
if !cfg.OutboxWorker.Enabled || cfg.OutboxWorker.RetryStrategy != "exponential_backoff" || cfg.OutboxWorker.MaxRetryCount != 10 {
|
if !cfg.OutboxWorker.Enabled || cfg.OutboxWorker.RetryStrategy != "exponential_backoff" || cfg.OutboxWorker.MaxRetryCount != 10 {
|
||||||
t.Fatalf("tencent example must configure outbox worker: %+v", cfg.OutboxWorker)
|
t.Fatalf("tencent example must configure outbox worker: %+v", cfg.OutboxWorker)
|
||||||
}
|
}
|
||||||
|
|||||||
@ -53,6 +53,12 @@ func runtimeRoomKey(appCode string, roomID string) string {
|
|||||||
return appCode + "\x00" + roomID
|
return appCode + "\x00" + roomID
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RuntimeRoomKey 返回 Redis owner lease 使用的运行时路由 key。
|
||||||
|
// transport 层 owner 转发必须和领域层 EnsureOwner 使用同一个 key,否则会出现同一房间多 owner。
|
||||||
|
func RuntimeRoomKey(appCode string, roomID string) string {
|
||||||
|
return runtimeRoomKey(appCode, roomID)
|
||||||
|
}
|
||||||
|
|
||||||
func runtimeRoomKeyFromContext(ctx context.Context, roomID string) string {
|
func runtimeRoomKeyFromContext(ctx context.Context, roomID string) string {
|
||||||
return runtimeRoomKey(appcode.FromContext(ctx), roomID)
|
return runtimeRoomKey(appcode.FromContext(ctx), roomID)
|
||||||
}
|
}
|
||||||
|
|||||||
35
services/room-service/internal/room/service/lease_release.go
Normal file
35
services/room-service/internal/room/service/lease_release.go
Normal file
@ -0,0 +1,35 @@
|
|||||||
|
package service
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"log/slog"
|
||||||
|
|
||||||
|
"hyapp/pkg/logx"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ReleaseLoadedRoomLeases 释放当前节点仍持有的已加载房间 lease。
|
||||||
|
// 该方法只应在 gRPC 已停止接新请求并完成 in-flight drain 后调用;提前释放会破坏单房间单 owner。
|
||||||
|
func (s *Service) ReleaseLoadedRoomLeases(ctx context.Context) {
|
||||||
|
if s == nil || s.directory == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for _, roomRef := range s.loadedRoomRefs() {
|
||||||
|
released, err := s.directory.ReleaseOwner(ctx, runtimeRoomKey(roomRef.AppCode, roomRef.RoomID), s.nodeID)
|
||||||
|
if err != nil {
|
||||||
|
logx.Warn(ctx, "room_lease_release_failed",
|
||||||
|
slog.String("node_id", s.nodeID),
|
||||||
|
slog.String("app_code", roomRef.AppCode),
|
||||||
|
slog.String("room_id", roomRef.RoomID),
|
||||||
|
slog.String("error", err.Error()),
|
||||||
|
)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if released {
|
||||||
|
logx.Info(ctx, "room_lease_released",
|
||||||
|
slog.String("node_id", s.nodeID),
|
||||||
|
slog.String("app_code", roomRef.AppCode),
|
||||||
|
slog.String("room_id", roomRef.RoomID),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
File diff suppressed because it is too large
Load Diff
@ -21,6 +21,24 @@ type Lease struct {
|
|||||||
ExpiresAt time.Time
|
ExpiresAt time.Time
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// NodeRegistration 记录一个 room-service 实例可被其他实例直连的内网地址。
|
||||||
|
type NodeRegistration struct {
|
||||||
|
// NodeID 必须和 Redis lease 中的 owner node_id 完全一致。
|
||||||
|
NodeID string
|
||||||
|
// GRPCAddr 是实例私网 gRPC 地址,必须指向单个实例,不能写 room-service 负载均衡地址。
|
||||||
|
GRPCAddr string
|
||||||
|
// UpdatedAt 是最近一次心跳写入时间,便于排查节点注册是否卡住。
|
||||||
|
UpdatedAt time.Time
|
||||||
|
// ExpiresAt 是节点注册过期时间;过期节点不能作为 owner 转发目标。
|
||||||
|
ExpiresAt time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// ValidAt 判断节点注册在指定时间是否仍可作为转发目标。
|
||||||
|
func (n NodeRegistration) ValidAt(now time.Time) bool {
|
||||||
|
// 没有地址或过期时间的注册不能被使用,避免把请求转发到空地址或旧实例。
|
||||||
|
return n.NodeID != "" && n.GRPCAddr != "" && !n.ExpiresAt.IsZero() && now.Before(n.ExpiresAt)
|
||||||
|
}
|
||||||
|
|
||||||
// ValidAt 判断租约在指定时刻是否仍有效。
|
// ValidAt 判断租约在指定时刻是否仍有效。
|
||||||
func (l Lease) ValidAt(now time.Time) bool {
|
func (l Lease) ValidAt(now time.Time) bool {
|
||||||
// 零值过期时间永远无效,避免空 lease 被误认为可继续持有。
|
// 零值过期时间永远无效,避免空 lease 被误认为可继续持有。
|
||||||
@ -35,22 +53,37 @@ type Directory interface {
|
|||||||
EnsureOwner(ctx context.Context, roomID string, nodeID string, now time.Time, ttl time.Duration) (Lease, error)
|
EnsureOwner(ctx context.Context, roomID string, nodeID string, now time.Time, ttl time.Duration) (Lease, error)
|
||||||
// VerifyOwner 校验当前节点和 lease token 仍然持有执行权,不续租也不接管。
|
// VerifyOwner 校验当前节点和 lease token 仍然持有执行权,不续租也不接管。
|
||||||
VerifyOwner(ctx context.Context, roomID string, nodeID string, leaseToken string, now time.Time) (bool, error)
|
VerifyOwner(ctx context.Context, roomID string, nodeID string, leaseToken string, now time.Time) (bool, error)
|
||||||
|
// ReleaseOwner 在节点安全下线后释放自己仍持有的 owner lease,让新节点无需等待 TTL 才能恢复。
|
||||||
|
ReleaseOwner(ctx context.Context, roomID string, nodeID string) (bool, error)
|
||||||
// ForceExpire 只用于测试模拟租约过期。
|
// ForceExpire 只用于测试模拟租约过期。
|
||||||
ForceExpire(roomID string)
|
ForceExpire(roomID string)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// NodeRegistry 维护 room-service node_id 到实例私网 gRPC 地址的短租约。
|
||||||
|
type NodeRegistry interface {
|
||||||
|
// RegisterNode 写入或续约当前节点地址;ttl 过期后其他节点不能继续转发到该实例。
|
||||||
|
RegisterNode(ctx context.Context, node NodeRegistration, ttl time.Duration) error
|
||||||
|
// LookupNode 查询一个 owner node_id 的实例地址。
|
||||||
|
LookupNode(ctx context.Context, nodeID string) (NodeRegistration, bool, error)
|
||||||
|
// UnregisterNode 在实例安全退出后删除注册,避免转发到已停止进程。
|
||||||
|
UnregisterNode(ctx context.Context, nodeID string) error
|
||||||
|
}
|
||||||
|
|
||||||
// MemoryDirectory 是首版测试与本地运行使用的内存目录实现。
|
// MemoryDirectory 是首版测试与本地运行使用的内存目录实现。
|
||||||
type MemoryDirectory struct {
|
type MemoryDirectory struct {
|
||||||
// mu 保护 leases,测试里可能用多个服务实例模拟接管。
|
// mu 保护 leases,测试里可能用多个服务实例模拟接管。
|
||||||
mu sync.Mutex
|
mu sync.Mutex
|
||||||
// leases 保存 room_id -> lease。
|
// leases 保存 room_id -> lease。
|
||||||
leases map[string]Lease
|
leases map[string]Lease
|
||||||
|
// nodes 保存 node_id -> 注册地址,供 transport 层 owner 转发测试复用。
|
||||||
|
nodes map[string]NodeRegistration
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewMemoryDirectory 初始化空的内存目录。
|
// NewMemoryDirectory 初始化空的内存目录。
|
||||||
func NewMemoryDirectory() *MemoryDirectory {
|
func NewMemoryDirectory() *MemoryDirectory {
|
||||||
return &MemoryDirectory{
|
return &MemoryDirectory{
|
||||||
leases: make(map[string]Lease),
|
leases: make(map[string]Lease),
|
||||||
|
nodes: make(map[string]NodeRegistration),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -112,6 +145,20 @@ func (d *MemoryDirectory) VerifyOwner(_ context.Context, roomID string, nodeID s
|
|||||||
return current.NodeID == nodeID && current.LeaseToken == leaseToken && current.ValidAt(now), nil
|
return current.NodeID == nodeID && current.LeaseToken == leaseToken && current.ValidAt(now), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ReleaseOwner 删除当前节点持有的内存 lease;其他节点持有时保持不变。
|
||||||
|
func (d *MemoryDirectory) ReleaseOwner(_ context.Context, roomID string, nodeID string) (bool, error) {
|
||||||
|
d.mu.Lock()
|
||||||
|
defer d.mu.Unlock()
|
||||||
|
|
||||||
|
current, ok := d.leases[roomID]
|
||||||
|
if !ok || current.NodeID != nodeID {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
delete(d.leases, roomID)
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
|
||||||
// ForceExpire 只用于测试时模拟租约自然过期。
|
// ForceExpire 只用于测试时模拟租约自然过期。
|
||||||
func (d *MemoryDirectory) ForceExpire(roomID string) {
|
func (d *MemoryDirectory) ForceExpire(roomID string) {
|
||||||
d.mu.Lock()
|
d.mu.Lock()
|
||||||
@ -126,3 +173,40 @@ func (d *MemoryDirectory) ForceExpire(roomID string) {
|
|||||||
lease.ExpiresAt = time.Unix(0, 0)
|
lease.ExpiresAt = time.Unix(0, 0)
|
||||||
d.leases[roomID] = lease
|
d.leases[roomID] = lease
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RegisterNode 写入测试用节点地址。
|
||||||
|
func (d *MemoryDirectory) RegisterNode(_ context.Context, node NodeRegistration, ttl time.Duration) error {
|
||||||
|
d.mu.Lock()
|
||||||
|
defer d.mu.Unlock()
|
||||||
|
|
||||||
|
now := time.Now().UTC()
|
||||||
|
if ttl <= 0 {
|
||||||
|
ttl = 30 * time.Second
|
||||||
|
}
|
||||||
|
node.UpdatedAt = now
|
||||||
|
node.ExpiresAt = now.Add(ttl)
|
||||||
|
d.nodes[node.NodeID] = node
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// LookupNode 查询测试用节点地址,过期注册按不存在处理。
|
||||||
|
func (d *MemoryDirectory) LookupNode(_ context.Context, nodeID string) (NodeRegistration, bool, error) {
|
||||||
|
d.mu.Lock()
|
||||||
|
defer d.mu.Unlock()
|
||||||
|
|
||||||
|
node, ok := d.nodes[nodeID]
|
||||||
|
if !ok || !node.ValidAt(time.Now().UTC()) {
|
||||||
|
return NodeRegistration{}, false, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return node, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// UnregisterNode 删除测试用节点地址。
|
||||||
|
func (d *MemoryDirectory) UnregisterNode(_ context.Context, nodeID string) error {
|
||||||
|
d.mu.Lock()
|
||||||
|
defer d.mu.Unlock()
|
||||||
|
|
||||||
|
delete(d.nodes, nodeID)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|||||||
@ -4,7 +4,9 @@ package router
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
"strconv"
|
"strconv"
|
||||||
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/redis/go-redis/v9"
|
"github.com/redis/go-redis/v9"
|
||||||
@ -14,6 +16,9 @@ import (
|
|||||||
// roomRouteKeyPrefix 是生产房间 owner lease 的 Redis key 前缀。
|
// roomRouteKeyPrefix 是生产房间 owner lease 的 Redis key 前缀。
|
||||||
const roomRouteKeyPrefix = "room:route:"
|
const roomRouteKeyPrefix = "room:route:"
|
||||||
|
|
||||||
|
// roomNodeKeyPrefix 保存 room-service 节点的实例直连地址,owner 转发只读取该短租约。
|
||||||
|
const roomNodeKeyPrefix = "room:node:"
|
||||||
|
|
||||||
// healthLeaseKey 是 ready 探针专用 key,避免探针污染真实房间路由。
|
// healthLeaseKey 是 ready 探针专用 key,避免探针污染真实房间路由。
|
||||||
const healthLeaseKey = "room:healthcheck:lease"
|
const healthLeaseKey = "room:healthcheck:lease"
|
||||||
|
|
||||||
@ -70,6 +75,25 @@ end
|
|||||||
return 0
|
return 0
|
||||||
`
|
`
|
||||||
|
|
||||||
|
const releaseOwnerScript = `
|
||||||
|
local current = redis.call("GET", KEYS[1])
|
||||||
|
if not current then
|
||||||
|
return 0
|
||||||
|
end
|
||||||
|
|
||||||
|
local ok, decoded = pcall(cjson.decode, current)
|
||||||
|
if not ok then
|
||||||
|
return 0
|
||||||
|
end
|
||||||
|
|
||||||
|
if decoded["node_id"] == ARGV[1] then
|
||||||
|
redis.call("DEL", KEYS[1])
|
||||||
|
return 1
|
||||||
|
end
|
||||||
|
|
||||||
|
return 0
|
||||||
|
`
|
||||||
|
|
||||||
// RedisDirectory 用 Redis 保存 room_id 到执行节点的带 TTL 租约。
|
// RedisDirectory 用 Redis 保存 room_id 到执行节点的带 TTL 租约。
|
||||||
type RedisDirectory struct {
|
type RedisDirectory struct {
|
||||||
// client 是 go-redis 客户端,连接生命周期由 app.App 关闭。
|
// client 是 go-redis 客户端,连接生命周期由 app.App 关闭。
|
||||||
@ -84,6 +108,14 @@ type redisLeaseValue struct {
|
|||||||
ExpiresAt int64 `json:"expires_at_ms"`
|
ExpiresAt int64 `json:"expires_at_ms"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// redisNodeValue 是 Redis 中的节点注册 JSON,字段保持可读以便 redis-cli 排查。
|
||||||
|
type redisNodeValue struct {
|
||||||
|
NodeID string `json:"node_id"`
|
||||||
|
GRPCAddr string `json:"grpc_addr"`
|
||||||
|
UpdatedAt int64 `json:"updated_at_ms"`
|
||||||
|
ExpiresAt int64 `json:"expires_at_ms"`
|
||||||
|
}
|
||||||
|
|
||||||
// NewRedisDirectory 创建 Redis 目录实现。
|
// NewRedisDirectory 创建 Redis 目录实现。
|
||||||
func NewRedisDirectory(client *redis.Client) *RedisDirectory {
|
func NewRedisDirectory(client *redis.Client) *RedisDirectory {
|
||||||
return &RedisDirectory{client: client}
|
return &RedisDirectory{client: client}
|
||||||
@ -222,17 +254,80 @@ func (d *RedisDirectory) VerifyOwner(ctx context.Context, roomID string, nodeID
|
|||||||
return value == 1, nil
|
return value == 1, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ReleaseOwner 在节点安全下线后删除自己持有的 route lease。
|
||||||
|
func (d *RedisDirectory) ReleaseOwner(ctx context.Context, roomID string, nodeID string) (bool, error) {
|
||||||
|
value, err := d.client.Eval(ctx, releaseOwnerScript, []string{routeKey(roomID)}, nodeID).Int()
|
||||||
|
if err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
|
||||||
|
return value == 1, nil
|
||||||
|
}
|
||||||
|
|
||||||
// ForceExpire 只服务测试,生产启动路径不会调用。
|
// ForceExpire 只服务测试,生产启动路径不会调用。
|
||||||
func (d *RedisDirectory) ForceExpire(roomID string) {
|
func (d *RedisDirectory) ForceExpire(roomID string) {
|
||||||
// Redis 实现直接删除 key 来模拟 lease 过期,避免等待真实 TTL。
|
// Redis 实现直接删除 key 来模拟 lease 过期,避免等待真实 TTL。
|
||||||
_ = d.client.Del(context.Background(), routeKey(roomID)).Err()
|
_ = d.client.Del(context.Background(), routeKey(roomID)).Err()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RegisterNode 写入当前 room-service 实例地址;TTL 过期后 owner 转发不会再使用该节点。
|
||||||
|
func (d *RedisDirectory) RegisterNode(ctx context.Context, node NodeRegistration, ttl time.Duration) error {
|
||||||
|
if ttl <= 0 {
|
||||||
|
ttl = 30 * time.Second
|
||||||
|
}
|
||||||
|
node.NodeID = strings.TrimSpace(node.NodeID)
|
||||||
|
node.GRPCAddr = strings.TrimSpace(node.GRPCAddr)
|
||||||
|
if node.NodeID == "" || node.GRPCAddr == "" {
|
||||||
|
return fmt.Errorf("node_id and grpc_addr are required")
|
||||||
|
}
|
||||||
|
now := time.Now().UTC()
|
||||||
|
node.UpdatedAt = now
|
||||||
|
node.ExpiresAt = now.Add(ttl)
|
||||||
|
payload, err := encodeNode(node)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
return d.client.Set(ctx, nodeKey(node.NodeID), payload, ttl).Err()
|
||||||
|
}
|
||||||
|
|
||||||
|
// LookupNode 查询 owner node_id 对应的实例地址。
|
||||||
|
func (d *RedisDirectory) LookupNode(ctx context.Context, nodeID string) (NodeRegistration, bool, error) {
|
||||||
|
value, err := d.client.Get(ctx, nodeKey(nodeID)).Result()
|
||||||
|
if err != nil {
|
||||||
|
if err == redis.Nil {
|
||||||
|
return NodeRegistration{}, false, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return NodeRegistration{}, false, err
|
||||||
|
}
|
||||||
|
|
||||||
|
node, err := decodeNode(value)
|
||||||
|
if err != nil {
|
||||||
|
return NodeRegistration{}, false, err
|
||||||
|
}
|
||||||
|
if !node.ValidAt(time.Now().UTC()) {
|
||||||
|
return NodeRegistration{}, false, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return node, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// UnregisterNode 删除节点注册;下线节点停止接受 owner 转发前必须调用。
|
||||||
|
func (d *RedisDirectory) UnregisterNode(ctx context.Context, nodeID string) error {
|
||||||
|
return d.client.Del(ctx, nodeKey(nodeID)).Err()
|
||||||
|
}
|
||||||
|
|
||||||
func routeKey(roomID string) string {
|
func routeKey(roomID string) string {
|
||||||
// 路由 key 只由房间 ID 决定,所有节点必须计算出同一个 Redis key。
|
// 路由 key 只由房间 ID 决定,所有节点必须计算出同一个 Redis key。
|
||||||
return roomRouteKeyPrefix + roomID
|
return roomRouteKeyPrefix + roomID
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func nodeKey(nodeID string) string {
|
||||||
|
// 节点 key 使用 node_id,不包含 app_code;一个实例可以服务多个 App 的房间。
|
||||||
|
return roomNodeKeyPrefix + strings.TrimSpace(nodeID)
|
||||||
|
}
|
||||||
|
|
||||||
func encodeLease(lease Lease) (string, error) {
|
func encodeLease(lease Lease) (string, error) {
|
||||||
// 使用 JSON 而不是二进制,方便线上通过 redis-cli 直接排查 owner。
|
// 使用 JSON 而不是二进制,方便线上通过 redis-cli 直接排查 owner。
|
||||||
payload, err := json.Marshal(redisLeaseValue{
|
payload, err := json.Marshal(redisLeaseValue{
|
||||||
@ -248,6 +343,20 @@ func encodeLease(lease Lease) (string, error) {
|
|||||||
return string(payload), nil
|
return string(payload), nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func encodeNode(node NodeRegistration) (string, error) {
|
||||||
|
payload, err := json.Marshal(redisNodeValue{
|
||||||
|
NodeID: strings.TrimSpace(node.NodeID),
|
||||||
|
GRPCAddr: strings.TrimSpace(node.GRPCAddr),
|
||||||
|
UpdatedAt: node.UpdatedAt.UnixMilli(),
|
||||||
|
ExpiresAt: node.ExpiresAt.UnixMilli(),
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
|
||||||
|
return string(payload), nil
|
||||||
|
}
|
||||||
|
|
||||||
func decodeLease(value string) (Lease, error) {
|
func decodeLease(value string) (Lease, error) {
|
||||||
var payload redisLeaseValue
|
var payload redisLeaseValue
|
||||||
if err := json.Unmarshal([]byte(value), &payload); err != nil {
|
if err := json.Unmarshal([]byte(value), &payload); err != nil {
|
||||||
@ -263,3 +372,17 @@ func decodeLease(value string) (Lease, error) {
|
|||||||
ExpiresAt: time.UnixMilli(payload.ExpiresAt),
|
ExpiresAt: time.UnixMilli(payload.ExpiresAt),
|
||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func decodeNode(value string) (NodeRegistration, error) {
|
||||||
|
var payload redisNodeValue
|
||||||
|
if err := json.Unmarshal([]byte(value), &payload); err != nil {
|
||||||
|
return NodeRegistration{}, err
|
||||||
|
}
|
||||||
|
|
||||||
|
return NodeRegistration{
|
||||||
|
NodeID: strings.TrimSpace(payload.NodeID),
|
||||||
|
GRPCAddr: strings.TrimSpace(payload.GRPCAddr),
|
||||||
|
UpdatedAt: time.UnixMilli(payload.UpdatedAt),
|
||||||
|
ExpiresAt: time.UnixMilli(payload.ExpiresAt),
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
|
|||||||
172
services/room-service/internal/transport/grpc/owner_forwarder.go
Normal file
172
services/room-service/internal/transport/grpc/owner_forwarder.go
Normal file
@ -0,0 +1,172 @@
|
|||||||
|
package grpc
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"hyapp/pkg/xerr"
|
||||||
|
roomservice "hyapp/services/room-service/internal/room/service"
|
||||||
|
"hyapp/services/room-service/internal/router"
|
||||||
|
|
||||||
|
"google.golang.org/grpc"
|
||||||
|
"google.golang.org/grpc/credentials/insecure"
|
||||||
|
"google.golang.org/grpc/metadata"
|
||||||
|
roomv1 "hyapp.local/api/proto/room/v1"
|
||||||
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
// ownerForwardDepthHeader 防止错误注册到负载均衡地址后出现递归转发。
|
||||||
|
ownerForwardDepthHeader = "x-hyapp-room-owner-forward-depth"
|
||||||
|
// maxOwnerForwardDepth 允许一次中转;超过后让本节点走本地逻辑并返回真实 owner 冲突。
|
||||||
|
maxOwnerForwardDepth = 1
|
||||||
|
// defaultOwnerForwardTimeout 限制转发占用调用方请求预算的最长时间。
|
||||||
|
defaultOwnerForwardTimeout = 2 * time.Second
|
||||||
|
)
|
||||||
|
|
||||||
|
type ownerForwarder struct {
|
||||||
|
// nodeID 是当前 room-service 实例 ID,用于判断自己是否就是 owner。
|
||||||
|
nodeID string
|
||||||
|
// directory 保存 room_id -> owner node_id 的短租约。
|
||||||
|
directory router.Directory
|
||||||
|
// registry 保存 owner node_id -> 实例 gRPC 地址的短租约。
|
||||||
|
registry router.NodeRegistry
|
||||||
|
// timeout 是单次 owner 转发的额外 deadline 上限。
|
||||||
|
timeout time.Duration
|
||||||
|
|
||||||
|
mu sync.Mutex
|
||||||
|
conns map[string]forwardConn
|
||||||
|
}
|
||||||
|
|
||||||
|
type forwardConn struct {
|
||||||
|
addr string
|
||||||
|
conn *grpc.ClientConn
|
||||||
|
}
|
||||||
|
|
||||||
|
func newOwnerForwarder(nodeID string, directory router.Directory, registry router.NodeRegistry, timeout time.Duration) *ownerForwarder {
|
||||||
|
if timeout <= 0 {
|
||||||
|
timeout = defaultOwnerForwardTimeout
|
||||||
|
}
|
||||||
|
return &ownerForwarder{
|
||||||
|
nodeID: strings.TrimSpace(nodeID),
|
||||||
|
directory: directory,
|
||||||
|
registry: registry,
|
||||||
|
timeout: timeout,
|
||||||
|
conns: make(map[string]forwardConn),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func forwardCommandToOwner[T any](f *ownerForwarder, ctx context.Context, appCode string, roomID string, call func(context.Context, roomv1.RoomCommandServiceClient) (T, error)) (T, bool, error) {
|
||||||
|
return forwardToOwner(ctx, f, appCode, roomID, func(callCtx context.Context, conn *grpc.ClientConn) (T, error) {
|
||||||
|
return call(callCtx, roomv1.NewRoomCommandServiceClient(conn))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func forwardGuardToOwner[T any](f *ownerForwarder, ctx context.Context, appCode string, roomID string, call func(context.Context, roomv1.RoomGuardServiceClient) (T, error)) (T, bool, error) {
|
||||||
|
return forwardToOwner(ctx, f, appCode, roomID, func(callCtx context.Context, conn *grpc.ClientConn) (T, error) {
|
||||||
|
return call(callCtx, roomv1.NewRoomGuardServiceClient(conn))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func forwardQueryToOwner[T any](f *ownerForwarder, ctx context.Context, appCode string, roomID string, call func(context.Context, roomv1.RoomQueryServiceClient) (T, error)) (T, bool, error) {
|
||||||
|
return forwardToOwner(ctx, f, appCode, roomID, func(callCtx context.Context, conn *grpc.ClientConn) (T, error) {
|
||||||
|
return call(callCtx, roomv1.NewRoomQueryServiceClient(conn))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func forwardToOwner[T any](ctx context.Context, f *ownerForwarder, appCode string, roomID string, call func(context.Context, *grpc.ClientConn) (T, error)) (T, bool, error) {
|
||||||
|
var zero T
|
||||||
|
if f == nil || f.directory == nil || f.registry == nil || strings.TrimSpace(roomID) == "" {
|
||||||
|
return zero, false, nil
|
||||||
|
}
|
||||||
|
if ownerForwardDepth(ctx) >= maxOwnerForwardDepth {
|
||||||
|
// 已经中转过的请求继续走本地逻辑,避免注册错误造成递归 RPC。
|
||||||
|
return zero, false, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
now := time.Now().UTC()
|
||||||
|
lease, exists, err := f.directory.Lookup(ctx, roomservice.RuntimeRoomKey(appCode, roomID))
|
||||||
|
if err != nil {
|
||||||
|
return zero, true, xerr.ToGRPCError(xerr.New(xerr.Unavailable, "room owner lookup failed"))
|
||||||
|
}
|
||||||
|
if !exists || !lease.ValidAt(now) || lease.NodeID == f.nodeID {
|
||||||
|
return zero, false, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
node, exists, err := f.registry.LookupNode(ctx, lease.NodeID)
|
||||||
|
if err != nil {
|
||||||
|
return zero, true, xerr.ToGRPCError(xerr.New(xerr.Unavailable, "room owner node lookup failed"))
|
||||||
|
}
|
||||||
|
if !exists || !node.ValidAt(now) {
|
||||||
|
return zero, true, xerr.ToGRPCError(xerr.New(xerr.Unavailable, "room owner node is not registered"))
|
||||||
|
}
|
||||||
|
|
||||||
|
conn, err := f.connForNode(ctx, node)
|
||||||
|
if err != nil {
|
||||||
|
return zero, true, xerr.ToGRPCError(xerr.New(xerr.Unavailable, fmt.Sprintf("room owner node dial failed: %v", err)))
|
||||||
|
}
|
||||||
|
|
||||||
|
callCtx, cancel := context.WithTimeout(ctx, f.timeout)
|
||||||
|
defer cancel()
|
||||||
|
callCtx = metadata.AppendToOutgoingContext(callCtx, ownerForwardDepthHeader, strconv.Itoa(ownerForwardDepth(ctx)+1))
|
||||||
|
result, err := call(callCtx, conn)
|
||||||
|
return result, true, err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *ownerForwarder) connForNode(_ context.Context, node router.NodeRegistration) (*grpc.ClientConn, error) {
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
|
||||||
|
if cached, ok := f.conns[node.NodeID]; ok && cached.addr == node.GRPCAddr {
|
||||||
|
return cached.conn, nil
|
||||||
|
}
|
||||||
|
if cached, ok := f.conns[node.NodeID]; ok && cached.conn != nil {
|
||||||
|
// 节点地址变化通常来自实例重建;关闭旧连接,避免继续向旧进程转发。
|
||||||
|
_ = cached.conn.Close()
|
||||||
|
}
|
||||||
|
|
||||||
|
conn, err := grpc.Dial(node.GRPCAddr, grpc.WithTransportCredentials(insecure.NewCredentials()))
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
f.conns[node.NodeID] = forwardConn{addr: node.GRPCAddr, conn: conn}
|
||||||
|
return conn, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *ownerForwarder) close() {
|
||||||
|
if f == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
f.mu.Lock()
|
||||||
|
defer f.mu.Unlock()
|
||||||
|
|
||||||
|
for nodeID, cached := range f.conns {
|
||||||
|
if cached.conn != nil {
|
||||||
|
_ = cached.conn.Close()
|
||||||
|
}
|
||||||
|
delete(f.conns, nodeID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func ownerForwardDepth(ctx context.Context) int {
|
||||||
|
var values []string
|
||||||
|
if incoming, ok := metadata.FromIncomingContext(ctx); ok {
|
||||||
|
values = incoming.Get(ownerForwardDepthHeader)
|
||||||
|
}
|
||||||
|
if len(values) == 0 {
|
||||||
|
if outgoing, ok := metadata.FromOutgoingContext(ctx); ok {
|
||||||
|
values = outgoing.Get(ownerForwardDepthHeader)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(values) == 0 {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
depth, err := strconv.Atoi(strings.TrimSpace(values[len(values)-1]))
|
||||||
|
if err != nil || depth < 0 {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
return depth
|
||||||
|
}
|
||||||
@ -3,11 +3,13 @@ package grpc
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"time"
|
||||||
|
|
||||||
roomv1 "hyapp.local/api/proto/room/v1"
|
roomv1 "hyapp.local/api/proto/room/v1"
|
||||||
"hyapp/pkg/appcode"
|
"hyapp/pkg/appcode"
|
||||||
"hyapp/pkg/xerr"
|
"hyapp/pkg/xerr"
|
||||||
roomservice "hyapp/services/room-service/internal/room/service"
|
roomservice "hyapp/services/room-service/internal/room/service"
|
||||||
|
"hyapp/services/room-service/internal/router"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Server 把 room-service 领域实现挂到 gRPC 接口上。
|
// Server 把 room-service 领域实现挂到 gRPC 接口上。
|
||||||
@ -21,11 +23,29 @@ type Server struct {
|
|||||||
|
|
||||||
// svc 是领域服务入口,gRPC 层不保存任何房间状态。
|
// svc 是领域服务入口,gRPC 层不保存任何房间状态。
|
||||||
svc *roomservice.Service
|
svc *roomservice.Service
|
||||||
|
// ownerForwarder 在请求命中非 owner 节点时把房间命令转到当前 owner 实例。
|
||||||
|
ownerForwarder *ownerForwarder
|
||||||
|
}
|
||||||
|
|
||||||
|
// Option 调整 room-service gRPC 边界行为。
|
||||||
|
type Option func(*Server)
|
||||||
|
|
||||||
|
// WithOwnerForwarding 启用 room_id -> owner node -> instance addr 转发。
|
||||||
|
func WithOwnerForwarding(nodeID string, directory router.Directory, registry router.NodeRegistry, timeout time.Duration) Option {
|
||||||
|
return func(s *Server) {
|
||||||
|
s.ownerForwarder = newOwnerForwarder(nodeID, directory, registry, timeout)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewServer 创建一个可注册到 gRPC 的 room server。
|
// NewServer 创建一个可注册到 gRPC 的 room server。
|
||||||
func NewServer(svc *roomservice.Service) *Server {
|
func NewServer(svc *roomservice.Service, opts ...Option) *Server {
|
||||||
return &Server{svc: svc}
|
server := &Server{svc: svc}
|
||||||
|
for _, opt := range opts {
|
||||||
|
if opt != nil {
|
||||||
|
opt(server)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return server
|
||||||
}
|
}
|
||||||
|
|
||||||
// mapServiceResult 把领域错误固定转换为 gRPC status + ErrorInfo.reason。
|
// mapServiceResult 把领域错误固定转换为 gRPC status + ErrorInfo.reason。
|
||||||
@ -44,6 +64,11 @@ func mapServiceResult[T any](result T, err error) (T, error) {
|
|||||||
func (s *Server) CreateRoom(ctx context.Context, req *roomv1.CreateRoomRequest) (*roomv1.CreateRoomResponse, error) {
|
func (s *Server) CreateRoom(ctx context.Context, req *roomv1.CreateRoomRequest) (*roomv1.CreateRoomResponse, error) {
|
||||||
// 创建房间会获取 Redis lease、写 MySQL meta/command/outbox/snapshot,并安装本地 Cell。
|
// 创建房间会获取 Redis lease、写 MySQL meta/command/outbox/snapshot,并安装本地 Cell。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.CreateRoomResponse, error) {
|
||||||
|
return client.CreateRoom(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.CreateRoom(ctx, req))
|
return mapServiceResult(s.svc.CreateRoom(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -51,6 +76,11 @@ func (s *Server) CreateRoom(ctx context.Context, req *roomv1.CreateRoomRequest)
|
|||||||
func (s *Server) UpdateRoomProfile(ctx context.Context, req *roomv1.UpdateRoomProfileRequest) (*roomv1.UpdateRoomProfileResponse, error) {
|
func (s *Server) UpdateRoomProfile(ctx context.Context, req *roomv1.UpdateRoomProfileRequest) (*roomv1.UpdateRoomProfileResponse, error) {
|
||||||
// 房间资料和座位数变更必须经 Room Cell 串行提交,gRPC 层不直接修改快照。
|
// 房间资料和座位数变更必须经 Room Cell 串行提交,gRPC 层不直接修改快照。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.UpdateRoomProfileResponse, error) {
|
||||||
|
return client.UpdateRoomProfile(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.UpdateRoomProfile(ctx, req))
|
return mapServiceResult(s.svc.UpdateRoomProfile(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -58,6 +88,11 @@ func (s *Server) UpdateRoomProfile(ctx context.Context, req *roomv1.UpdateRoomPr
|
|||||||
func (s *Server) JoinRoom(ctx context.Context, req *roomv1.JoinRoomRequest) (*roomv1.JoinRoomResponse, error) {
|
func (s *Server) JoinRoom(ctx context.Context, req *roomv1.JoinRoomRequest) (*roomv1.JoinRoomResponse, error) {
|
||||||
// JoinRoom 只维护 room-service presence,真实消息连接由腾讯云 IM SDK 处理。
|
// JoinRoom 只维护 room-service presence,真实消息连接由腾讯云 IM SDK 处理。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.JoinRoomResponse, error) {
|
||||||
|
return client.JoinRoom(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.JoinRoom(ctx, req))
|
return mapServiceResult(s.svc.JoinRoom(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -65,6 +100,11 @@ func (s *Server) JoinRoom(ctx context.Context, req *roomv1.JoinRoomRequest) (*ro
|
|||||||
func (s *Server) RoomHeartbeat(ctx context.Context, req *roomv1.RoomHeartbeatRequest) (*roomv1.RoomHeartbeatResponse, error) {
|
func (s *Server) RoomHeartbeat(ctx context.Context, req *roomv1.RoomHeartbeatRequest) (*roomv1.RoomHeartbeatResponse, error) {
|
||||||
// 心跳只刷新已有业务 presence,不能隐式创建进房状态。
|
// 心跳只刷新已有业务 presence,不能隐式创建进房状态。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.RoomHeartbeatResponse, error) {
|
||||||
|
return client.RoomHeartbeat(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.RoomHeartbeat(ctx, req))
|
return mapServiceResult(s.svc.RoomHeartbeat(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -72,6 +112,11 @@ func (s *Server) RoomHeartbeat(ctx context.Context, req *roomv1.RoomHeartbeatReq
|
|||||||
func (s *Server) LeaveRoom(ctx context.Context, req *roomv1.LeaveRoomRequest) (*roomv1.LeaveRoomResponse, error) {
|
func (s *Server) LeaveRoom(ctx context.Context, req *roomv1.LeaveRoomRequest) (*roomv1.LeaveRoomResponse, error) {
|
||||||
// LeaveRoom 不直接操作腾讯云 IM 连接,只修改房间业务态。
|
// LeaveRoom 不直接操作腾讯云 IM 连接,只修改房间业务态。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.LeaveRoomResponse, error) {
|
||||||
|
return client.LeaveRoom(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.LeaveRoom(ctx, req))
|
return mapServiceResult(s.svc.LeaveRoom(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -79,6 +124,11 @@ func (s *Server) LeaveRoom(ctx context.Context, req *roomv1.LeaveRoomRequest) (*
|
|||||||
func (s *Server) CloseRoom(ctx context.Context, req *roomv1.CloseRoomRequest) (*roomv1.CloseRoomResponse, error) {
|
func (s *Server) CloseRoom(ctx context.Context, req *roomv1.CloseRoomRequest) (*roomv1.CloseRoomResponse, error) {
|
||||||
// CloseRoom 统一关闭房间生命周期、清理 presence/麦位并写出系统事件。
|
// CloseRoom 统一关闭房间生命周期、清理 presence/麦位并写出系统事件。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.CloseRoomResponse, error) {
|
||||||
|
return client.CloseRoom(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.CloseRoom(ctx, req))
|
return mapServiceResult(s.svc.CloseRoom(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -86,6 +136,11 @@ func (s *Server) CloseRoom(ctx context.Context, req *roomv1.CloseRoomRequest) (*
|
|||||||
func (s *Server) MicUp(ctx context.Context, req *roomv1.MicUpRequest) (*roomv1.MicUpResponse, error) {
|
func (s *Server) MicUp(ctx context.Context, req *roomv1.MicUpRequest) (*roomv1.MicUpResponse, error) {
|
||||||
// 麦位修改由 Room Cell 串行执行,gRPC 层不做状态判断。
|
// 麦位修改由 Room Cell 串行执行,gRPC 层不做状态判断。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.MicUpResponse, error) {
|
||||||
|
return client.MicUp(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.MicUp(ctx, req))
|
return mapServiceResult(s.svc.MicUp(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -93,6 +148,11 @@ func (s *Server) MicUp(ctx context.Context, req *roomv1.MicUpRequest) (*roomv1.M
|
|||||||
func (s *Server) MicDown(ctx context.Context, req *roomv1.MicDownRequest) (*roomv1.MicDownResponse, error) {
|
func (s *Server) MicDown(ctx context.Context, req *roomv1.MicDownRequest) (*roomv1.MicDownResponse, error) {
|
||||||
// 下麦结果会进入 command log 和 outbox,失败不产生部分状态。
|
// 下麦结果会进入 command log 和 outbox,失败不产生部分状态。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.MicDownResponse, error) {
|
||||||
|
return client.MicDown(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.MicDown(ctx, req))
|
return mapServiceResult(s.svc.MicDown(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -100,6 +160,11 @@ func (s *Server) MicDown(ctx context.Context, req *roomv1.MicDownRequest) (*room
|
|||||||
func (s *Server) ChangeMicSeat(ctx context.Context, req *roomv1.ChangeMicSeatRequest) (*roomv1.ChangeMicSeatResponse, error) {
|
func (s *Server) ChangeMicSeat(ctx context.Context, req *roomv1.ChangeMicSeatRequest) (*roomv1.ChangeMicSeatResponse, error) {
|
||||||
// 换位在领域层原子修改源麦位和目标麦位。
|
// 换位在领域层原子修改源麦位和目标麦位。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.ChangeMicSeatResponse, error) {
|
||||||
|
return client.ChangeMicSeat(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.ChangeMicSeat(ctx, req))
|
return mapServiceResult(s.svc.ChangeMicSeat(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -107,6 +172,11 @@ func (s *Server) ChangeMicSeat(ctx context.Context, req *roomv1.ChangeMicSeatReq
|
|||||||
func (s *Server) ConfirmMicPublishing(ctx context.Context, req *roomv1.ConfirmMicPublishingRequest) (*roomv1.ConfirmMicPublishingResponse, error) {
|
func (s *Server) ConfirmMicPublishing(ctx context.Context, req *roomv1.ConfirmMicPublishingRequest) (*roomv1.ConfirmMicPublishingResponse, error) {
|
||||||
// 发流确认必须进入 Room Cell 串行状态机,避免旧 RTC 事件覆盖新 mic_session。
|
// 发流确认必须进入 Room Cell 串行状态机,避免旧 RTC 事件覆盖新 mic_session。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.ConfirmMicPublishingResponse, error) {
|
||||||
|
return client.ConfirmMicPublishing(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.ConfirmMicPublishing(ctx, req))
|
return mapServiceResult(s.svc.ConfirmMicPublishing(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -114,6 +184,11 @@ func (s *Server) ConfirmMicPublishing(ctx context.Context, req *roomv1.ConfirmMi
|
|||||||
func (s *Server) SetMicMute(ctx context.Context, req *roomv1.SetMicMuteRequest) (*roomv1.SetMicMuteResponse, error) {
|
func (s *Server) SetMicMute(ctx context.Context, req *roomv1.SetMicMuteRequest) (*roomv1.SetMicMuteResponse, error) {
|
||||||
// 服务端可见静音态必须经 Room Cell 提交,确保其他用户 UI 和系统消息一致。
|
// 服务端可见静音态必须经 Room Cell 提交,确保其他用户 UI 和系统消息一致。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.SetMicMuteResponse, error) {
|
||||||
|
return client.SetMicMute(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.SetMicMute(ctx, req))
|
return mapServiceResult(s.svc.SetMicMute(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -121,6 +196,11 @@ func (s *Server) SetMicMute(ctx context.Context, req *roomv1.SetMicMuteRequest)
|
|||||||
func (s *Server) ApplyRTCEvent(ctx context.Context, req *roomv1.ApplyRTCEventRequest) (*roomv1.ApplyRTCEventResponse, error) {
|
func (s *Server) ApplyRTCEvent(ctx context.Context, req *roomv1.ApplyRTCEventRequest) (*roomv1.ApplyRTCEventResponse, error) {
|
||||||
// 腾讯 RTC 回调已在 gateway 验签;房间状态变化仍必须由 Room Cell 串行提交。
|
// 腾讯 RTC 回调已在 gateway 验签;房间状态变化仍必须由 Room Cell 串行提交。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.ApplyRTCEventResponse, error) {
|
||||||
|
return client.ApplyRTCEvent(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.ApplyRTCEvent(ctx, req))
|
return mapServiceResult(s.svc.ApplyRTCEvent(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -128,6 +208,11 @@ func (s *Server) ApplyRTCEvent(ctx context.Context, req *roomv1.ApplyRTCEventReq
|
|||||||
func (s *Server) SetMicSeatLock(ctx context.Context, req *roomv1.SetMicSeatLockRequest) (*roomv1.SetMicSeatLockResponse, error) {
|
func (s *Server) SetMicSeatLock(ctx context.Context, req *roomv1.SetMicSeatLockRequest) (*roomv1.SetMicSeatLockResponse, error) {
|
||||||
// 锁麦权限和目标麦位状态由 Room Cell 判定。
|
// 锁麦权限和目标麦位状态由 Room Cell 判定。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.SetMicSeatLockResponse, error) {
|
||||||
|
return client.SetMicSeatLock(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.SetMicSeatLock(ctx, req))
|
return mapServiceResult(s.svc.SetMicSeatLock(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -135,6 +220,11 @@ func (s *Server) SetMicSeatLock(ctx context.Context, req *roomv1.SetMicSeatLockR
|
|||||||
func (s *Server) SetChatEnabled(ctx context.Context, req *roomv1.SetChatEnabledRequest) (*roomv1.SetChatEnabledResponse, error) {
|
func (s *Server) SetChatEnabled(ctx context.Context, req *roomv1.SetChatEnabledRequest) (*roomv1.SetChatEnabledResponse, error) {
|
||||||
// 公屏开关是 room-service 发言守卫的权威状态。
|
// 公屏开关是 room-service 发言守卫的权威状态。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.SetChatEnabledResponse, error) {
|
||||||
|
return client.SetChatEnabled(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.SetChatEnabled(ctx, req))
|
return mapServiceResult(s.svc.SetChatEnabled(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -142,6 +232,11 @@ func (s *Server) SetChatEnabled(ctx context.Context, req *roomv1.SetChatEnabledR
|
|||||||
func (s *Server) SetRoomAdmin(ctx context.Context, req *roomv1.SetRoomAdminRequest) (*roomv1.SetRoomAdminResponse, error) {
|
func (s *Server) SetRoomAdmin(ctx context.Context, req *roomv1.SetRoomAdminRequest) (*roomv1.SetRoomAdminResponse, error) {
|
||||||
// 管理员集合只能由 room-service 权限矩阵修改。
|
// 管理员集合只能由 room-service 权限矩阵修改。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.SetRoomAdminResponse, error) {
|
||||||
|
return client.SetRoomAdmin(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.SetRoomAdmin(ctx, req))
|
return mapServiceResult(s.svc.SetRoomAdmin(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -149,6 +244,11 @@ func (s *Server) SetRoomAdmin(ctx context.Context, req *roomv1.SetRoomAdminReque
|
|||||||
func (s *Server) TransferRoomHost(ctx context.Context, req *roomv1.TransferRoomHostRequest) (*roomv1.TransferRoomHostResponse, error) {
|
func (s *Server) TransferRoomHost(ctx context.Context, req *roomv1.TransferRoomHostRequest) (*roomv1.TransferRoomHostResponse, error) {
|
||||||
// 主持人转移由 Room Cell 串行提交,并进入 command log/outbox。
|
// 主持人转移由 Room Cell 串行提交,并进入 command log/outbox。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.TransferRoomHostResponse, error) {
|
||||||
|
return client.TransferRoomHost(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.TransferRoomHost(ctx, req))
|
return mapServiceResult(s.svc.TransferRoomHost(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -156,6 +256,11 @@ func (s *Server) TransferRoomHost(ctx context.Context, req *roomv1.TransferRoomH
|
|||||||
func (s *Server) MuteUser(ctx context.Context, req *roomv1.MuteUserRequest) (*roomv1.MuteUserResponse, error) {
|
func (s *Server) MuteUser(ctx context.Context, req *roomv1.MuteUserRequest) (*roomv1.MuteUserResponse, error) {
|
||||||
// 禁言状态是腾讯云 IM 发言回调的权威来源。
|
// 禁言状态是腾讯云 IM 发言回调的权威来源。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.MuteUserResponse, error) {
|
||||||
|
return client.MuteUser(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.MuteUser(ctx, req))
|
return mapServiceResult(s.svc.MuteUser(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -163,6 +268,11 @@ func (s *Server) MuteUser(ctx context.Context, req *roomv1.MuteUserRequest) (*ro
|
|||||||
func (s *Server) KickUser(ctx context.Context, req *roomv1.KickUserRequest) (*roomv1.KickUserResponse, error) {
|
func (s *Server) KickUser(ctx context.Context, req *roomv1.KickUserRequest) (*roomv1.KickUserResponse, error) {
|
||||||
// 踢人会修改 presence、ban 集合和麦位占用。
|
// 踢人会修改 presence、ban 集合和麦位占用。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.KickUserResponse, error) {
|
||||||
|
return client.KickUser(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.KickUser(ctx, req))
|
return mapServiceResult(s.svc.KickUser(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -170,6 +280,11 @@ func (s *Server) KickUser(ctx context.Context, req *roomv1.KickUserRequest) (*ro
|
|||||||
func (s *Server) UnbanUser(ctx context.Context, req *roomv1.UnbanUserRequest) (*roomv1.UnbanUserResponse, error) {
|
func (s *Server) UnbanUser(ctx context.Context, req *roomv1.UnbanUserRequest) (*roomv1.UnbanUserResponse, error) {
|
||||||
// 解封只解除 ban,不恢复 presence、管理员身份或麦位。
|
// 解封只解除 ban,不恢复 presence、管理员身份或麦位。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.UnbanUserResponse, error) {
|
||||||
|
return client.UnbanUser(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.UnbanUser(ctx, req))
|
return mapServiceResult(s.svc.UnbanUser(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -177,6 +292,11 @@ func (s *Server) UnbanUser(ctx context.Context, req *roomv1.UnbanUserRequest) (*
|
|||||||
func (s *Server) SendGift(ctx context.Context, req *roomv1.SendGiftRequest) (*roomv1.SendGiftResponse, error) {
|
func (s *Server) SendGift(ctx context.Context, req *roomv1.SendGiftRequest) (*roomv1.SendGiftResponse, error) {
|
||||||
// SendGift 先同步扣费,成功后才更新房间热度和本地榜。
|
// SendGift 先同步扣费,成功后才更新房间热度和本地榜。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardCommand(s, ctx, req.GetMeta(), func(callCtx context.Context, client roomv1.RoomCommandServiceClient) (*roomv1.SendGiftResponse, error) {
|
||||||
|
return client.SendGift(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.SendGift(ctx, req))
|
return mapServiceResult(s.svc.SendGift(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -184,6 +304,11 @@ func (s *Server) SendGift(ctx context.Context, req *roomv1.SendGiftRequest) (*ro
|
|||||||
func (s *Server) CheckSpeakPermission(ctx context.Context, req *roomv1.CheckSpeakPermissionRequest) (*roomv1.CheckSpeakPermissionResponse, error) {
|
func (s *Server) CheckSpeakPermission(ctx context.Context, req *roomv1.CheckSpeakPermissionRequest) (*roomv1.CheckSpeakPermissionResponse, error) {
|
||||||
// 外部 IM 发公屏前调用该守卫,避免复制禁言和房间状态规则。
|
// 外部 IM 发公屏前调用该守卫,避免复制禁言和房间状态规则。
|
||||||
ctx = appcode.WithContext(ctx, req.GetAppCode())
|
ctx = appcode.WithContext(ctx, req.GetAppCode())
|
||||||
|
if resp, forwarded, err := forwardGuard(s, ctx, req.GetAppCode(), req.GetRoomId(), func(callCtx context.Context, client roomv1.RoomGuardServiceClient) (*roomv1.CheckSpeakPermissionResponse, error) {
|
||||||
|
return client.CheckSpeakPermission(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.CheckSpeakPermission(ctx, req))
|
return mapServiceResult(s.svc.CheckSpeakPermission(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -191,6 +316,11 @@ func (s *Server) CheckSpeakPermission(ctx context.Context, req *roomv1.CheckSpea
|
|||||||
func (s *Server) VerifyRoomPresence(ctx context.Context, req *roomv1.VerifyRoomPresenceRequest) (*roomv1.VerifyRoomPresenceResponse, error) {
|
func (s *Server) VerifyRoomPresence(ctx context.Context, req *roomv1.VerifyRoomPresenceRequest) (*roomv1.VerifyRoomPresenceResponse, error) {
|
||||||
// 外部 IM 进群前调用该守卫,防止未进房或被踢用户收消息。
|
// 外部 IM 进群前调用该守卫,防止未进房或被踢用户收消息。
|
||||||
ctx = appcode.WithContext(ctx, req.GetAppCode())
|
ctx = appcode.WithContext(ctx, req.GetAppCode())
|
||||||
|
if resp, forwarded, err := forwardGuard(s, ctx, req.GetAppCode(), req.GetRoomId(), func(callCtx context.Context, client roomv1.RoomGuardServiceClient) (*roomv1.VerifyRoomPresenceResponse, error) {
|
||||||
|
return client.VerifyRoomPresence(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.VerifyRoomPresence(ctx, req))
|
return mapServiceResult(s.svc.VerifyRoomPresence(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -226,6 +356,11 @@ func (s *Server) GetCurrentRoom(ctx context.Context, req *roomv1.GetCurrentRoomR
|
|||||||
func (s *Server) GetRoomSnapshot(ctx context.Context, req *roomv1.GetRoomSnapshotRequest) (*roomv1.GetRoomSnapshotResponse, error) {
|
func (s *Server) GetRoomSnapshot(ctx context.Context, req *roomv1.GetRoomSnapshotRequest) (*roomv1.GetRoomSnapshotResponse, error) {
|
||||||
// 快照查询不刷新 presence,不替代 heartbeat,也不能让未进房用户读取完整房间态。
|
// 快照查询不刷新 presence,不替代 heartbeat,也不能让未进房用户读取完整房间态。
|
||||||
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
ctx = contextWithMetaApp(ctx, req.GetMeta())
|
||||||
|
if resp, forwarded, err := forwardQuery(s, ctx, req.GetMeta().GetAppCode(), req.GetRoomId(), func(callCtx context.Context, client roomv1.RoomQueryServiceClient) (*roomv1.GetRoomSnapshotResponse, error) {
|
||||||
|
return client.GetRoomSnapshot(callCtx, req)
|
||||||
|
}); forwarded {
|
||||||
|
return resp, err
|
||||||
|
}
|
||||||
return mapServiceResult(s.svc.GetRoomSnapshot(ctx, req))
|
return mapServiceResult(s.svc.GetRoomSnapshot(ctx, req))
|
||||||
}
|
}
|
||||||
|
|
||||||
@ -242,3 +377,27 @@ func contextWithMetaApp(ctx context.Context, meta *roomv1.RequestMeta) context.C
|
|||||||
}
|
}
|
||||||
return appcode.WithContext(ctx, meta.GetAppCode())
|
return appcode.WithContext(ctx, meta.GetAppCode())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func forwardCommand[T any](s *Server, ctx context.Context, meta *roomv1.RequestMeta, call func(context.Context, roomv1.RoomCommandServiceClient) (T, error)) (T, bool, error) {
|
||||||
|
var zero T
|
||||||
|
if s == nil || s.ownerForwarder == nil || meta == nil {
|
||||||
|
return zero, false, nil
|
||||||
|
}
|
||||||
|
return forwardCommandToOwner(s.ownerForwarder, ctx, meta.GetAppCode(), meta.GetRoomId(), call)
|
||||||
|
}
|
||||||
|
|
||||||
|
func forwardGuard[T any](s *Server, ctx context.Context, appCode string, roomID string, call func(context.Context, roomv1.RoomGuardServiceClient) (T, error)) (T, bool, error) {
|
||||||
|
var zero T
|
||||||
|
if s == nil || s.ownerForwarder == nil {
|
||||||
|
return zero, false, nil
|
||||||
|
}
|
||||||
|
return forwardGuardToOwner(s.ownerForwarder, ctx, appCode, roomID, call)
|
||||||
|
}
|
||||||
|
|
||||||
|
func forwardQuery[T any](s *Server, ctx context.Context, appCode string, roomID string, call func(context.Context, roomv1.RoomQueryServiceClient) (T, error)) (T, bool, error) {
|
||||||
|
var zero T
|
||||||
|
if s == nil || s.ownerForwarder == nil {
|
||||||
|
return zero, false, nil
|
||||||
|
}
|
||||||
|
return forwardQueryToOwner(s.ownerForwarder, ctx, appCode, roomID, call)
|
||||||
|
}
|
||||||
|
|||||||
@ -2,18 +2,23 @@ package grpc
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"net"
|
||||||
|
"os"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
roomv1 "hyapp.local/api/proto/room/v1"
|
roomv1 "hyapp.local/api/proto/room/v1"
|
||||||
walletv1 "hyapp.local/api/proto/wallet/v1"
|
walletv1 "hyapp.local/api/proto/wallet/v1"
|
||||||
|
"hyapp/pkg/appcode"
|
||||||
"hyapp/pkg/xerr"
|
"hyapp/pkg/xerr"
|
||||||
"hyapp/services/room-service/internal/integration"
|
"hyapp/services/room-service/internal/integration"
|
||||||
roomservice "hyapp/services/room-service/internal/room/service"
|
roomservice "hyapp/services/room-service/internal/room/service"
|
||||||
"hyapp/services/room-service/internal/router"
|
"hyapp/services/room-service/internal/router"
|
||||||
"hyapp/services/room-service/internal/testutil/mysqltest"
|
"hyapp/services/room-service/internal/testutil/mysqltest"
|
||||||
|
|
||||||
|
grpcpkg "google.golang.org/grpc"
|
||||||
"google.golang.org/grpc/codes"
|
"google.golang.org/grpc/codes"
|
||||||
|
"google.golang.org/grpc/credentials/insecure"
|
||||||
grpcstatus "google.golang.org/grpc/status"
|
grpcstatus "google.golang.org/grpc/status"
|
||||||
)
|
)
|
||||||
|
|
||||||
@ -61,3 +66,172 @@ func TestCreateRoomMapsDomainErrorToGRPCReason(t *testing.T) {
|
|||||||
t.Fatalf("message mismatch: got %q", message)
|
t.Fatalf("message mismatch: got %q", message)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestOwnerForwardingUsesRedisRouteAndNodeRegistry(t *testing.T) {
|
||||||
|
redisAddr := os.Getenv("ROOM_SERVICE_REDIS_TEST_ADDR")
|
||||||
|
if redisAddr == "" {
|
||||||
|
t.Skip("set ROOM_SERVICE_REDIS_TEST_ADDR to run Redis-backed owner forwarding test")
|
||||||
|
}
|
||||||
|
ctx := context.Background()
|
||||||
|
redisClient, err := router.NewRedisClient(ctx, redisAddr, os.Getenv("ROOM_SERVICE_REDIS_TEST_PASSWORD"), 0)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("open redis directory failed: %v", err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { _ = redisClient.Close() })
|
||||||
|
directory := router.NewRedisDirectory(redisClient)
|
||||||
|
repository := mysqltest.NewRepository(t)
|
||||||
|
|
||||||
|
addrA, stopA := startForwardingRoomServer(t, "node-forward-a", directory, repository)
|
||||||
|
defer stopA()
|
||||||
|
addrB, stopB := startForwardingRoomServer(t, "node-forward-b", directory, repository)
|
||||||
|
defer stopB()
|
||||||
|
|
||||||
|
if err := directory.RegisterNode(ctx, router.NodeRegistration{NodeID: "node-forward-a", GRPCAddr: addrA}, time.Minute); err != nil {
|
||||||
|
t.Fatalf("register node a failed: %v", err)
|
||||||
|
}
|
||||||
|
if err := directory.RegisterNode(ctx, router.NodeRegistration{NodeID: "node-forward-b", GRPCAddr: addrB}, time.Minute); err != nil {
|
||||||
|
t.Fatalf("register node b failed: %v", err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() {
|
||||||
|
_ = directory.UnregisterNode(context.Background(), "node-forward-a")
|
||||||
|
_ = directory.UnregisterNode(context.Background(), "node-forward-b")
|
||||||
|
})
|
||||||
|
|
||||||
|
connA, err := grpcpkg.Dial(addrA, grpcpkg.WithTransportCredentials(insecure.NewCredentials()))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("dial node a failed: %v", err)
|
||||||
|
}
|
||||||
|
defer connA.Close()
|
||||||
|
connB, err := grpcpkg.Dial(addrB, grpcpkg.WithTransportCredentials(insecure.NewCredentials()))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("dial node b failed: %v", err)
|
||||||
|
}
|
||||||
|
defer connB.Close()
|
||||||
|
|
||||||
|
roomID := "room-forward-test"
|
||||||
|
directory.ForceExpire(roomservice.RuntimeRoomKey(appcode.Default, roomID))
|
||||||
|
t.Cleanup(func() {
|
||||||
|
directory.ForceExpire(roomservice.RuntimeRoomKey(appcode.Default, roomID))
|
||||||
|
})
|
||||||
|
createResp, err := roomv1.NewRoomCommandServiceClient(connA).CreateRoom(ctx, &roomv1.CreateRoomRequest{
|
||||||
|
Meta: &roomv1.RequestMeta{
|
||||||
|
RequestId: "req-forward-create",
|
||||||
|
CommandId: "cmd-forward-create",
|
||||||
|
ActorUserId: 1001,
|
||||||
|
RoomId: roomID,
|
||||||
|
SentAtMs: time.Now().UTC().UnixMilli(),
|
||||||
|
AppCode: appcode.Default,
|
||||||
|
},
|
||||||
|
SeatCount: 10,
|
||||||
|
Mode: "voice",
|
||||||
|
RoomName: "Forward Room",
|
||||||
|
RoomShortId: "forward-short",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("create room on node a failed: %v", err)
|
||||||
|
}
|
||||||
|
if createResp.GetRoom().GetRoomId() != roomID {
|
||||||
|
t.Fatalf("create room id mismatch: %+v", createResp.GetRoom())
|
||||||
|
}
|
||||||
|
|
||||||
|
lease, exists, err := directory.Lookup(ctx, roomservice.RuntimeRoomKey(appcode.Default, roomID))
|
||||||
|
if err != nil || !exists {
|
||||||
|
t.Fatalf("lookup owner lease failed: exists=%v err=%v", exists, err)
|
||||||
|
}
|
||||||
|
if lease.NodeID != "node-forward-a" {
|
||||||
|
t.Fatalf("room should be owned by node a before forwarding, got %+v", lease)
|
||||||
|
}
|
||||||
|
|
||||||
|
joinResp, err := roomv1.NewRoomCommandServiceClient(connB).JoinRoom(ctx, &roomv1.JoinRoomRequest{
|
||||||
|
Meta: &roomv1.RequestMeta{
|
||||||
|
RequestId: "req-forward-join",
|
||||||
|
CommandId: "cmd-forward-join",
|
||||||
|
ActorUserId: 1002,
|
||||||
|
RoomId: roomID,
|
||||||
|
SentAtMs: time.Now().UTC().UnixMilli(),
|
||||||
|
AppCode: appcode.Default,
|
||||||
|
},
|
||||||
|
Role: "audience",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("join room through node b should forward to owner: %v", err)
|
||||||
|
}
|
||||||
|
if findUser(joinResp.GetRoom(), 1002) == nil {
|
||||||
|
t.Fatalf("forwarded join did not update owner room snapshot: %+v", joinResp.GetRoom().GetOnlineUsers())
|
||||||
|
}
|
||||||
|
|
||||||
|
guardResp, err := roomv1.NewRoomGuardServiceClient(connB).CheckSpeakPermission(ctx, &roomv1.CheckSpeakPermissionRequest{
|
||||||
|
RoomId: roomID,
|
||||||
|
UserId: 1002,
|
||||||
|
AppCode: appcode.Default,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("guard through node b should forward to owner: %v", err)
|
||||||
|
}
|
||||||
|
if !guardResp.GetAllowed() {
|
||||||
|
t.Fatalf("forwarded guard should allow joined user, got %+v", guardResp)
|
||||||
|
}
|
||||||
|
|
||||||
|
snapshotResp, err := roomv1.NewRoomQueryServiceClient(connB).GetRoomSnapshot(ctx, &roomv1.GetRoomSnapshotRequest{
|
||||||
|
Meta: &roomv1.RequestMeta{
|
||||||
|
RequestId: "req-forward-snapshot",
|
||||||
|
ActorUserId: 1002,
|
||||||
|
RoomId: roomID,
|
||||||
|
AppCode: appcode.Default,
|
||||||
|
},
|
||||||
|
RoomId: roomID,
|
||||||
|
ViewerUserId: 1002,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("snapshot through node b should forward to owner: %v", err)
|
||||||
|
}
|
||||||
|
if snapshotResp.GetRoom().GetVersion() != joinResp.GetRoom().GetVersion() {
|
||||||
|
t.Fatalf("forwarded snapshot version mismatch: got %d want %d", snapshotResp.GetRoom().GetVersion(), joinResp.GetRoom().GetVersion())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func startForwardingRoomServer(t *testing.T, nodeID string, directory *router.RedisDirectory, repository roomservice.Repository) (string, func()) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
svc := roomservice.New(roomservice.Config{
|
||||||
|
NodeID: nodeID,
|
||||||
|
LeaseTTL: 10 * time.Second,
|
||||||
|
RankLimit: 20,
|
||||||
|
SnapshotEveryN: 1,
|
||||||
|
}, directory, repository, fakeWallet{}, integration.NewNoopRoomEventPublisher(), integration.NewNoopOutboxPublisher())
|
||||||
|
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("listen forwarding test server failed: %v", err)
|
||||||
|
}
|
||||||
|
grpcServer := grpcpkg.NewServer()
|
||||||
|
roomServer := NewServer(svc, WithOwnerForwarding(nodeID, directory, directory, time.Second))
|
||||||
|
roomv1.RegisterRoomCommandServiceServer(grpcServer, roomServer)
|
||||||
|
roomv1.RegisterRoomGuardServiceServer(grpcServer, roomServer)
|
||||||
|
roomv1.RegisterRoomQueryServiceServer(grpcServer, roomServer)
|
||||||
|
errCh := make(chan error, 1)
|
||||||
|
go func() {
|
||||||
|
errCh <- grpcServer.Serve(listener)
|
||||||
|
}()
|
||||||
|
stop := func() {
|
||||||
|
roomServer.ownerForwarder.close()
|
||||||
|
grpcServer.GracefulStop()
|
||||||
|
select {
|
||||||
|
case err := <-errCh:
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("forwarding test server stopped with error: %v", err)
|
||||||
|
}
|
||||||
|
case <-time.After(2 * time.Second):
|
||||||
|
t.Fatal("forwarding test server did not stop")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return listener.Addr().String(), stop
|
||||||
|
}
|
||||||
|
|
||||||
|
func findUser(snapshot *roomv1.RoomSnapshot, userID int64) *roomv1.RoomUser {
|
||||||
|
for _, user := range snapshot.GetOnlineUsers() {
|
||||||
|
if user.GetUserId() == userID {
|
||||||
|
return user
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|||||||
@ -30,7 +30,7 @@ COPY services/user-service/configs/config.docker.yaml /app/config.yaml
|
|||||||
|
|
||||||
USER appuser
|
USER appuser
|
||||||
|
|
||||||
EXPOSE 13005
|
EXPOSE 13005 13105
|
||||||
|
|
||||||
ENTRYPOINT ["/app/server"]
|
ENTRYPOINT ["/app/server"]
|
||||||
CMD ["-config", "/app/config.yaml"]
|
CMD ["-config", "/app/config.yaml"]
|
||||||
|
|||||||
@ -8,6 +8,7 @@ log:
|
|||||||
include_response_body: false
|
include_response_body: false
|
||||||
max_payload_bytes: 2048
|
max_payload_bytes: 2048
|
||||||
grpc_addr: ":13005"
|
grpc_addr: ":13005"
|
||||||
|
health_http_addr: ":13105"
|
||||||
mysql_dsn: "hyapp:hyapp@tcp(mysql:3306)/hyapp_user?parseTime=true&charset=utf8mb4&loc=UTC"
|
mysql_dsn: "hyapp:hyapp@tcp(mysql:3306)/hyapp_user?parseTime=true&charset=utf8mb4&loc=UTC"
|
||||||
mysql_auto_migrate: false
|
mysql_auto_migrate: false
|
||||||
id_generator:
|
id_generator:
|
||||||
|
|||||||
@ -8,6 +8,7 @@ log:
|
|||||||
include_response_body: false
|
include_response_body: false
|
||||||
max_payload_bytes: 2048
|
max_payload_bytes: 2048
|
||||||
grpc_addr: ":13005"
|
grpc_addr: ":13005"
|
||||||
|
health_http_addr: ":13105"
|
||||||
mysql_dsn: "USER:PASSWORD@tcp(TENCENT_CDB_HOST:3306)/hyapp_user?parseTime=true&charset=utf8mb4&loc=UTC&tls=false"
|
mysql_dsn: "USER:PASSWORD@tcp(TENCENT_CDB_HOST:3306)/hyapp_user?parseTime=true&charset=utf8mb4&loc=UTC&tls=false"
|
||||||
mysql_auto_migrate: false
|
mysql_auto_migrate: false
|
||||||
id_generator:
|
id_generator:
|
||||||
|
|||||||
@ -8,6 +8,7 @@ log:
|
|||||||
include_response_body: false
|
include_response_body: false
|
||||||
max_payload_bytes: 2048
|
max_payload_bytes: 2048
|
||||||
grpc_addr: ":13005"
|
grpc_addr: ":13005"
|
||||||
|
health_http_addr: ":13105"
|
||||||
mysql_dsn: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_user?parseTime=true&charset=utf8mb4&loc=UTC"
|
mysql_dsn: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_user?parseTime=true&charset=utf8mb4&loc=UTC"
|
||||||
mysql_auto_migrate: false
|
mysql_auto_migrate: false
|
||||||
id_generator:
|
id_generator:
|
||||||
|
|||||||
@ -11,6 +11,8 @@ import (
|
|||||||
|
|
||||||
userv1 "hyapp.local/api/proto/user/v1"
|
userv1 "hyapp.local/api/proto/user/v1"
|
||||||
"hyapp/pkg/grpchealth"
|
"hyapp/pkg/grpchealth"
|
||||||
|
"hyapp/pkg/grpcshutdown"
|
||||||
|
"hyapp/pkg/healthhttp"
|
||||||
"hyapp/pkg/idgen"
|
"hyapp/pkg/idgen"
|
||||||
"hyapp/pkg/logx"
|
"hyapp/pkg/logx"
|
||||||
"hyapp/services/user-service/internal/config"
|
"hyapp/services/user-service/internal/config"
|
||||||
@ -36,6 +38,8 @@ type App struct {
|
|||||||
listener net.Listener
|
listener net.Listener
|
||||||
// health 是标准 gRPC health 的 serving/draining 状态来源。
|
// health 是标准 gRPC health 的 serving/draining 状态来源。
|
||||||
health *grpchealth.ServingChecker
|
health *grpchealth.ServingChecker
|
||||||
|
// healthHTTP 给 CLB 和发布脚本提供 HTTP readiness,不承载业务请求。
|
||||||
|
healthHTTP *healthhttp.Server
|
||||||
// mysqlRepo 持有用户主数据、认证身份、session 和短号事务的唯一运行时存储。
|
// mysqlRepo 持有用户主数据、认证身份、session 和短号事务的唯一运行时存储。
|
||||||
mysqlRepo *mysqlstorage.Repository
|
mysqlRepo *mysqlstorage.Repository
|
||||||
// walletDB 只用于读取 wallet_outbox 充值事实,实际账务状态仍由 wallet-service owner。
|
// walletDB 只用于读取 wallet_outbox 充值事实,实际账务状态仍由 wallet-service owner。
|
||||||
@ -226,15 +230,47 @@ func New(cfg config.Config) (*App, error) {
|
|||||||
userv1.RegisterUserHostAdminServiceServer(server, userServer)
|
userv1.RegisterUserHostAdminServiceServer(server, userServer)
|
||||||
userv1.RegisterUserCronServiceServer(server, userServer)
|
userv1.RegisterUserCronServiceServer(server, userServer)
|
||||||
|
|
||||||
health := grpchealth.NewServingChecker("user-service")
|
healthDependencies := []grpchealth.Dependency{{
|
||||||
|
Name: "mysql",
|
||||||
|
Check: mysqlRepo.Ping,
|
||||||
|
}}
|
||||||
|
if walletDB != nil {
|
||||||
|
healthDependencies = append(healthDependencies, grpchealth.Dependency{
|
||||||
|
Name: "wallet_mysql",
|
||||||
|
Check: walletDB.PingContext,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
if roomDB != nil {
|
||||||
|
healthDependencies = append(healthDependencies, grpchealth.Dependency{
|
||||||
|
Name: "room_mysql",
|
||||||
|
Check: roomDB.PingContext,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
health := grpchealth.NewServingChecker("user-service", healthDependencies...)
|
||||||
// user-service 目前使用简单 serving checker,存储探测由后续专用 health 可扩展。
|
// user-service 目前使用简单 serving checker,存储探测由后续专用 health 可扩展。
|
||||||
healthgrpc.RegisterHealthServer(server, grpchealth.NewServer(health))
|
healthgrpc.RegisterHealthServer(server, grpchealth.NewServer(health))
|
||||||
|
healthHTTP, err := healthhttp.New(cfg.HealthHTTPAddr, cfg.NodeID, health)
|
||||||
|
if err != nil {
|
||||||
|
if roomDB != nil {
|
||||||
|
_ = roomDB.Close()
|
||||||
|
}
|
||||||
|
if walletDB != nil {
|
||||||
|
_ = walletDB.Close()
|
||||||
|
}
|
||||||
|
if loginRiskRedisClose != nil {
|
||||||
|
_ = loginRiskRedisClose()
|
||||||
|
}
|
||||||
|
_ = listener.Close()
|
||||||
|
_ = mysqlRepo.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
workerCtx, workerCancel := context.WithCancel(context.Background())
|
workerCtx, workerCancel := context.WithCancel(context.Background())
|
||||||
|
|
||||||
return &App{
|
return &App{
|
||||||
server: server,
|
server: server,
|
||||||
listener: listener,
|
listener: listener,
|
||||||
health: health,
|
health: health,
|
||||||
|
healthHTTP: healthHTTP,
|
||||||
mysqlRepo: mysqlRepo,
|
mysqlRepo: mysqlRepo,
|
||||||
walletDB: walletDB,
|
walletDB: walletDB,
|
||||||
roomDB: roomDB,
|
roomDB: roomDB,
|
||||||
@ -251,6 +287,7 @@ func New(cfg config.Config) (*App, error) {
|
|||||||
|
|
||||||
// Run 启动 gRPC 服务。
|
// Run 启动 gRPC 服务。
|
||||||
func (a *App) Run() error {
|
func (a *App) Run() error {
|
||||||
|
a.runHealthHTTP()
|
||||||
// 只有 listener 进入 Serve 后才标记 serving,避免健康检查提前放行。
|
// 只有 listener 进入 Serve 后才标记 serving,避免健康检查提前放行。
|
||||||
a.health.MarkServing()
|
a.health.MarkServing()
|
||||||
defer func() {
|
defer func() {
|
||||||
@ -297,7 +334,8 @@ func (a *App) Close() {
|
|||||||
}
|
}
|
||||||
a.workerWG.Wait()
|
a.workerWG.Wait()
|
||||||
// GracefulStop 等待已进入的 RPC 结束,避免 token/session 写入被硬切。
|
// GracefulStop 等待已进入的 RPC 结束,避免 token/session 写入被硬切。
|
||||||
a.server.GracefulStop()
|
grpcshutdown.GracefulStop(a.server, 15*time.Second)
|
||||||
|
a.closeHealthHTTP()
|
||||||
if a.mysqlRepo != nil {
|
if a.mysqlRepo != nil {
|
||||||
// MySQL 连接池最后关闭,保证 GracefulStop 期间 repository 仍可用。
|
// MySQL 连接池最后关闭,保证 GracefulStop 期间 repository 仍可用。
|
||||||
_ = a.mysqlRepo.Close()
|
_ = a.mysqlRepo.Close()
|
||||||
@ -313,3 +351,23 @@ func (a *App) Close() {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (a *App) runHealthHTTP() {
|
||||||
|
if a.healthHTTP == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
go func() {
|
||||||
|
if err := a.healthHTTP.Run(); err != nil {
|
||||||
|
logx.Error(context.Background(), "health_http_run_failed", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (a *App) closeHealthHTTP() {
|
||||||
|
if a.healthHTTP == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
_ = a.healthHTTP.Close(ctx)
|
||||||
|
}
|
||||||
|
|||||||
@ -18,6 +18,8 @@ type Config struct {
|
|||||||
Environment string `yaml:"environment"`
|
Environment string `yaml:"environment"`
|
||||||
// GRPCAddr 是 user-service 内部 gRPC 监听地址。
|
// GRPCAddr 是 user-service 内部 gRPC 监听地址。
|
||||||
GRPCAddr string `yaml:"grpc_addr"`
|
GRPCAddr string `yaml:"grpc_addr"`
|
||||||
|
// HealthHTTPAddr 是给 CLB/发布脚本探活使用的 HTTP 监听地址,不承载业务流量。
|
||||||
|
HealthHTTPAddr string `yaml:"health_http_addr"`
|
||||||
// MySQLDSN 是用户主数据、认证身份、session 和短号事务的必需存储连接串。
|
// MySQLDSN 是用户主数据、认证身份、session 和短号事务的必需存储连接串。
|
||||||
MySQLDSN string `yaml:"mysql_dsn"`
|
MySQLDSN string `yaml:"mysql_dsn"`
|
||||||
// MySQLAutoMigrate 保留给显式本地迁移;线上 schema 由发布流程或 initdb 管理。
|
// MySQLAutoMigrate 保留给显式本地迁移;线上 schema 由发布流程或 initdb 管理。
|
||||||
@ -132,6 +134,7 @@ func Default() Config {
|
|||||||
NodeID: "user-local",
|
NodeID: "user-local",
|
||||||
Environment: "local",
|
Environment: "local",
|
||||||
GRPCAddr: ":13005",
|
GRPCAddr: ":13005",
|
||||||
|
HealthHTTPAddr: ":13105",
|
||||||
MySQLDSN: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_user?parseTime=true&charset=utf8mb4&loc=UTC",
|
MySQLDSN: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_user?parseTime=true&charset=utf8mb4&loc=UTC",
|
||||||
MySQLAutoMigrate: false,
|
MySQLAutoMigrate: false,
|
||||||
IDGenerator: IDGeneratorConfig{
|
IDGenerator: IDGeneratorConfig{
|
||||||
@ -209,6 +212,10 @@ func Load(path string) (Config, error) {
|
|||||||
if cfg.Environment == "" {
|
if cfg.Environment == "" {
|
||||||
cfg.Environment = "local"
|
cfg.Environment = "local"
|
||||||
}
|
}
|
||||||
|
cfg.HealthHTTPAddr = strings.TrimSpace(cfg.HealthHTTPAddr)
|
||||||
|
if cfg.HealthHTTPAddr == "" {
|
||||||
|
cfg.HealthHTTPAddr = ":13105"
|
||||||
|
}
|
||||||
cfg.LoginRisk.RedisAddr = strings.TrimSpace(cfg.LoginRisk.RedisAddr)
|
cfg.LoginRisk.RedisAddr = strings.TrimSpace(cfg.LoginRisk.RedisAddr)
|
||||||
if cfg.LoginRisk.BlockedTTLSec <= 0 {
|
if cfg.LoginRisk.BlockedTTLSec <= 0 {
|
||||||
cfg.LoginRisk.BlockedTTLSec = 2592000
|
cfg.LoginRisk.BlockedTTLSec = 2592000
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@ -6,11 +6,12 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
mysqldriver "github.com/go-sql-driver/mysql"
|
|
||||||
"hyapp/pkg/appcode"
|
"hyapp/pkg/appcode"
|
||||||
"hyapp/pkg/xerr"
|
"hyapp/pkg/xerr"
|
||||||
userdomain "hyapp/services/user-service/internal/domain/user"
|
userdomain "hyapp/services/user-service/internal/domain/user"
|
||||||
"hyapp/services/user-service/internal/storage/mysql/shared"
|
"hyapp/services/user-service/internal/storage/mysql/shared"
|
||||||
|
|
||||||
|
mysqldriver "github.com/go-sql-driver/mysql"
|
||||||
)
|
)
|
||||||
|
|
||||||
func (r *Repository) ResolveEnabledCountryByCode(ctx context.Context, countryCode string) (userdomain.Country, bool, error) {
|
func (r *Repository) ResolveEnabledCountryByCode(ctx context.Context, countryCode string) (userdomain.Country, bool, error) {
|
||||||
@ -20,7 +21,8 @@ func (r *Repository) ResolveEnabledCountryByCode(ctx context.Context, countryCod
|
|||||||
// ListRegistrationCountries 返回注册页可选国家,只包含 enabled 的国家。
|
// ListRegistrationCountries 返回注册页可选国家,只包含 enabled 的国家。
|
||||||
|
|
||||||
func (r *Repository) ListRegistrationCountries(ctx context.Context) ([]userdomain.Country, error) {
|
func (r *Repository) ListRegistrationCountries(ctx context.Context) ([]userdomain.Country, error) {
|
||||||
return listCountries(ctx, r.db, userdomain.CountryFilter{Enabled: new(true)})
|
enabled := true
|
||||||
|
return listCountries(ctx, r.db, userdomain.CountryFilter{Enabled: &enabled})
|
||||||
}
|
}
|
||||||
|
|
||||||
// ResolveActiveRegionByCountry 按国家码读取当前 active 区域映射;无显式映射时由 regions.go 返回 GLOBAL。
|
// ResolveActiveRegionByCountry 按国家码读取当前 active 区域映射;无显式映射时由 regions.go 返回 GLOBAL。
|
||||||
@ -130,11 +132,6 @@ func scanCountry(scanner interface{ Scan(dest ...any) error }) (userdomain.Count
|
|||||||
return country, err
|
return country, err
|
||||||
}
|
}
|
||||||
|
|
||||||
//go:fix inline
|
|
||||||
func boolPtr(value bool) *bool {
|
|
||||||
return new(value)
|
|
||||||
}
|
|
||||||
|
|
||||||
func canonicalCountryCodes(ctx context.Context, q queryer, countries []string) ([]string, error) {
|
func canonicalCountryCodes(ctx context.Context, q queryer, countries []string) ([]string, error) {
|
||||||
seen := make(map[string]struct{}, len(countries))
|
seen := make(map[string]struct{}, len(countries))
|
||||||
result := make([]string, 0, len(countries))
|
result := make([]string, 0, len(countries))
|
||||||
|
|||||||
@ -30,7 +30,7 @@ COPY services/wallet-service/configs/config.docker.yaml /app/config.yaml
|
|||||||
|
|
||||||
USER appuser
|
USER appuser
|
||||||
|
|
||||||
EXPOSE 13004
|
EXPOSE 13004 13104
|
||||||
|
|
||||||
ENTRYPOINT ["/app/server"]
|
ENTRYPOINT ["/app/server"]
|
||||||
CMD ["-config", "/app/config.yaml"]
|
CMD ["-config", "/app/config.yaml"]
|
||||||
|
|||||||
@ -8,5 +8,6 @@ log:
|
|||||||
include_response_body: false
|
include_response_body: false
|
||||||
max_payload_bytes: 2048
|
max_payload_bytes: 2048
|
||||||
grpc_addr: ":13004"
|
grpc_addr: ":13004"
|
||||||
|
health_http_addr: ":13104"
|
||||||
mysql_dsn: "hyapp:hyapp@tcp(mysql:3306)/hyapp_wallet?parseTime=true&charset=utf8mb4&loc=UTC"
|
mysql_dsn: "hyapp:hyapp@tcp(mysql:3306)/hyapp_wallet?parseTime=true&charset=utf8mb4&loc=UTC"
|
||||||
mysql_auto_migrate: false
|
mysql_auto_migrate: false
|
||||||
|
|||||||
@ -8,5 +8,6 @@ log:
|
|||||||
include_response_body: false
|
include_response_body: false
|
||||||
max_payload_bytes: 2048
|
max_payload_bytes: 2048
|
||||||
grpc_addr: ":13004"
|
grpc_addr: ":13004"
|
||||||
|
health_http_addr: ":13104"
|
||||||
mysql_dsn: "USER:PASSWORD@tcp(TENCENT_CDB_HOST:3306)/hyapp_wallet?parseTime=true&charset=utf8mb4&loc=UTC&tls=false"
|
mysql_dsn: "USER:PASSWORD@tcp(TENCENT_CDB_HOST:3306)/hyapp_wallet?parseTime=true&charset=utf8mb4&loc=UTC&tls=false"
|
||||||
mysql_auto_migrate: false
|
mysql_auto_migrate: false
|
||||||
|
|||||||
@ -8,5 +8,6 @@ log:
|
|||||||
include_response_body: false
|
include_response_body: false
|
||||||
max_payload_bytes: 2048
|
max_payload_bytes: 2048
|
||||||
grpc_addr: ":13004"
|
grpc_addr: ":13004"
|
||||||
|
health_http_addr: ":13104"
|
||||||
mysql_dsn: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_wallet?parseTime=true&charset=utf8mb4&loc=UTC"
|
mysql_dsn: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_wallet?parseTime=true&charset=utf8mb4&loc=UTC"
|
||||||
mysql_auto_migrate: false
|
mysql_auto_migrate: false
|
||||||
|
|||||||
@ -11,6 +11,8 @@ import (
|
|||||||
healthgrpc "google.golang.org/grpc/health/grpc_health_v1"
|
healthgrpc "google.golang.org/grpc/health/grpc_health_v1"
|
||||||
walletv1 "hyapp.local/api/proto/wallet/v1"
|
walletv1 "hyapp.local/api/proto/wallet/v1"
|
||||||
"hyapp/pkg/grpchealth"
|
"hyapp/pkg/grpchealth"
|
||||||
|
"hyapp/pkg/grpcshutdown"
|
||||||
|
"hyapp/pkg/healthhttp"
|
||||||
"hyapp/pkg/logx"
|
"hyapp/pkg/logx"
|
||||||
"hyapp/services/wallet-service/internal/config"
|
"hyapp/services/wallet-service/internal/config"
|
||||||
walletservice "hyapp/services/wallet-service/internal/service/wallet"
|
walletservice "hyapp/services/wallet-service/internal/service/wallet"
|
||||||
@ -20,11 +22,12 @@ import (
|
|||||||
|
|
||||||
// App 装配 wallet-service gRPC 入口。
|
// App 装配 wallet-service gRPC 入口。
|
||||||
type App struct {
|
type App struct {
|
||||||
server *grpc.Server
|
server *grpc.Server
|
||||||
listener net.Listener
|
listener net.Listener
|
||||||
health *grpchealth.ServingChecker
|
health *grpchealth.ServingChecker
|
||||||
mysqlRepo *mysqlstorage.Repository
|
healthHTTP *healthhttp.Server
|
||||||
closeOnce sync.Once
|
mysqlRepo *mysqlstorage.Repository
|
||||||
|
closeOnce sync.Once
|
||||||
}
|
}
|
||||||
|
|
||||||
// New 初始化 wallet-service。
|
// New 初始化 wallet-service。
|
||||||
@ -47,19 +50,30 @@ func New(cfg config.Config) (*App, error) {
|
|||||||
server := grpc.NewServer(grpc.UnaryInterceptor(logx.UnaryServerInterceptor("wallet-service")))
|
server := grpc.NewServer(grpc.UnaryInterceptor(logx.UnaryServerInterceptor("wallet-service")))
|
||||||
svc := walletservice.New(repository)
|
svc := walletservice.New(repository)
|
||||||
walletv1.RegisterWalletServiceServer(server, grpcserver.NewServer(svc))
|
walletv1.RegisterWalletServiceServer(server, grpcserver.NewServer(svc))
|
||||||
health := grpchealth.NewServingChecker("wallet-service")
|
health := grpchealth.NewServingChecker("wallet-service", grpchealth.Dependency{
|
||||||
|
Name: "mysql",
|
||||||
|
Check: repository.Ping,
|
||||||
|
})
|
||||||
healthgrpc.RegisterHealthServer(server, grpchealth.NewServer(health))
|
healthgrpc.RegisterHealthServer(server, grpchealth.NewServer(health))
|
||||||
|
healthHTTP, err := healthhttp.New(cfg.HealthHTTPAddr, cfg.NodeID, health)
|
||||||
|
if err != nil {
|
||||||
|
_ = listener.Close()
|
||||||
|
_ = repository.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
return &App{
|
return &App{
|
||||||
server: server,
|
server: server,
|
||||||
listener: listener,
|
listener: listener,
|
||||||
health: health,
|
health: health,
|
||||||
mysqlRepo: repository,
|
healthHTTP: healthHTTP,
|
||||||
|
mysqlRepo: repository,
|
||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// Run 启动 gRPC 服务。
|
// Run 启动 gRPC 服务。
|
||||||
func (a *App) Run() error {
|
func (a *App) Run() error {
|
||||||
|
a.runHealthHTTP()
|
||||||
a.health.MarkServing()
|
a.health.MarkServing()
|
||||||
err := a.server.Serve(a.listener)
|
err := a.server.Serve(a.listener)
|
||||||
a.health.MarkStopped()
|
a.health.MarkStopped()
|
||||||
@ -74,10 +88,31 @@ func (a *App) Run() error {
|
|||||||
func (a *App) Close() {
|
func (a *App) Close() {
|
||||||
a.closeOnce.Do(func() {
|
a.closeOnce.Do(func() {
|
||||||
a.health.MarkDraining()
|
a.health.MarkDraining()
|
||||||
a.server.GracefulStop()
|
grpcshutdown.GracefulStop(a.server, 15*time.Second)
|
||||||
|
a.closeHealthHTTP()
|
||||||
if a.mysqlRepo != nil {
|
if a.mysqlRepo != nil {
|
||||||
// MySQL 连接池最后关闭,避免正在 drain 的扣费请求丢失提交能力。
|
// MySQL 连接池最后关闭,避免正在 drain 的扣费请求丢失提交能力。
|
||||||
_ = a.mysqlRepo.Close()
|
_ = a.mysqlRepo.Close()
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (a *App) runHealthHTTP() {
|
||||||
|
if a.healthHTTP == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
go func() {
|
||||||
|
if err := a.healthHTTP.Run(); err != nil {
|
||||||
|
logx.Error(context.Background(), "health_http_run_failed", err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (a *App) closeHealthHTTP() {
|
||||||
|
if a.healthHTTP == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
_ = a.healthHTTP.Close(ctx)
|
||||||
|
}
|
||||||
|
|||||||
@ -13,6 +13,8 @@ type Config struct {
|
|||||||
NodeID string `yaml:"node_id"`
|
NodeID string `yaml:"node_id"`
|
||||||
Environment string `yaml:"environment"`
|
Environment string `yaml:"environment"`
|
||||||
GRPCAddr string `yaml:"grpc_addr"`
|
GRPCAddr string `yaml:"grpc_addr"`
|
||||||
|
// HealthHTTPAddr 是给 CLB/发布脚本探活使用的 HTTP 监听地址,不承载业务流量。
|
||||||
|
HealthHTTPAddr string `yaml:"health_http_addr"`
|
||||||
// MySQLDSN 是账户、交易、分录和 outbox 的必需存储连接串。
|
// MySQLDSN 是账户、交易、分录和 outbox 的必需存储连接串。
|
||||||
MySQLDSN string `yaml:"mysql_dsn"`
|
MySQLDSN string `yaml:"mysql_dsn"`
|
||||||
// MySQLAutoMigrate 保留给显式本地迁移;线上 schema 由发布流程或 initdb 管理。
|
// MySQLAutoMigrate 保留给显式本地迁移;线上 schema 由发布流程或 initdb 管理。
|
||||||
@ -27,6 +29,7 @@ func Default() Config {
|
|||||||
NodeID: "wallet-local",
|
NodeID: "wallet-local",
|
||||||
Environment: "local",
|
Environment: "local",
|
||||||
GRPCAddr: ":13004",
|
GRPCAddr: ":13004",
|
||||||
|
HealthHTTPAddr: ":13104",
|
||||||
MySQLDSN: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_wallet?parseTime=true&charset=utf8mb4&loc=UTC",
|
MySQLDSN: "hyapp:hyapp@tcp(127.0.0.1:23306)/hyapp_wallet?parseTime=true&charset=utf8mb4&loc=UTC",
|
||||||
MySQLAutoMigrate: false,
|
MySQLAutoMigrate: false,
|
||||||
Log: logx.Config{
|
Log: logx.Config{
|
||||||
@ -60,6 +63,10 @@ func Load(path string) (Config, error) {
|
|||||||
if cfg.Environment == "" {
|
if cfg.Environment == "" {
|
||||||
cfg.Environment = "local"
|
cfg.Environment = "local"
|
||||||
}
|
}
|
||||||
|
cfg.HealthHTTPAddr = strings.TrimSpace(cfg.HealthHTTPAddr)
|
||||||
|
if cfg.HealthHTTPAddr == "" {
|
||||||
|
cfg.HealthHTTPAddr = ":13104"
|
||||||
|
}
|
||||||
|
|
||||||
return cfg, nil
|
return cfg, nil
|
||||||
}
|
}
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user