diff --git a/AGENTS.md b/AGENTS.md index 449a817..b2ee441 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -24,3 +24,27 @@ 2. 禁止用静态假数据兜底展示微服务运行态。 3. 配置 YAML 只在列表右侧三点和详情配置区展示。 4. 读取 YAML 时使用 `includeConfig=1`,密钥字段保持脱敏。 + +## 测试 server / testbox + +1. 测试 server 指 HyApp 后端测试机 testbox:`43.165.195.39` / `10.11.0.2`,实例 `lhins-q0m38zc6`,区域 `ap-jakarta`。 +2. 测试机根目录固定为 `/opt/hyapp-server-test`,代码目录为 `/opt/hyapp-server-test/source`。 +3. 测试机部署不走生产镜像仓库,不摘 CLB;当前模式是测试机拉取远端 `test` 分支。 +4. 自动更新由 `hyapp-server-test-deploy.timer` 每分钟触发,部署脚本为 `/opt/hyapp-server-test/bin/deploy.sh`。 +5. 只有 Compose 服务全部 healthy 后,才认为部署成功并写入 `/opt/hyapp-server-test/.deployed_commit`。 +6. 测试服务页面必须通过 `GET /deploy/api/hyapp/testbox/status` 读取真实状态,禁止静态假数据兜底。 +7. 测试服务操作必须走固定 API 和 `deploy/tencent_tat/hyapp_testbox_config.py`,禁止在页面开放任意 shell。 +8. 服务操作只允许 `update/start/stop/restart`,timer 操作只允许 `start/stop/restart/status`。 +9. 测试机配置读取默认脱敏;除非服务端显式设置 `HYAPP_TESTBOX_CONFIG_ALLOW_RAW=1` 且请求带 `raw=1`,否则不得返回原始密钥。 +10. 保存配置前必须备份旧文件;包含 `***` 脱敏占位符的配置禁止保存。 +11. 测试库 migration 只允许列出或执行测试机仓库 `source/server/admin/migrations/*.sql`,禁止页面手写 SQL。 + +## 测试 server 日志 debug + +1. Debug 前先用 `GET /deploy/api/hyapp/testbox/status` 或 `python3 deploy/tencent_tat/hyapp_testbox_config.py status --env-file /opt/hy-app-monitor/.env` 确认真实状态。 +2. 页面或 API 异常时,先查 deploy-platform 自身:`journalctl -u hyapp-deploy-platform -n 200 --no-pager` 和 `docker logs --tail=200 hyapp-deploy-platform`。 +3. 测试机自动部署异常时,查 `hyapp-server-test-deploy.timer` 和 `hyapp-server-test-deploy.service` 的 `journalctl`。 +4. 业务服务异常时,在测试机 `/opt/hyapp-server-test/source` 下查 Docker Compose 项目 `hyapp-test` 的 `ps`、`logs`、`inspect`。 +5. 如果 `/opt/hyapp-server-test/docker-compose.testbox.yml` 不存在,Compose 命令只能使用 `source/docker-compose.yml`。 +6. Debug 过程可以通过 SSH 或腾讯云 TAT 执行只读命令;不要新增生产 SQL、Mongo 查询入口,不要把 MySQL、Redis、MQ 当服务器 SSH。 +7. 修复后必须验证:刷新测试服务页、调用 status API,或直接运行 `hyapp_testbox_config.py status`,确认 commit、timer、服务 health 与预期一致。 diff --git a/README.md b/README.md index c53deed..f4fb1d7 100644 --- a/README.md +++ b/README.md @@ -57,6 +57,81 @@ make up - 列表:`GET /deploy/api/hyapp/discover?remote=1` - 详情 YAML:`GET /deploy/api/hyapp/discover?remote=1&includeConfig=1` -- 数据来源:TAT 读取 systemd、Docker、镜像、资源、日志和配置 +- 链路:Tencent TAT -> deploy 机器 `43.164.75.199` -> SSH 业务机器 +- 数据来源:SSH 读取 systemd、Docker、镜像、资源、日志和配置 - YAML 入口:列表右侧三点、详情配置区 - 禁止:静态假数据兜底展示运行态 + +## 测试 Server + +这里的测试 server 指 HyApp 后端测试机 testbox,不是生产业务机。 + +- 测试机:`43.165.195.39` / `10.11.0.2` +- CVM 实例:`lhins-q0m38zc6` +- 区域:`ap-jakarta` +- 根目录:`/opt/hyapp-server-test` +- 代码目录:`/opt/hyapp-server-test/source` +- 部署脚本:`/opt/hyapp-server-test/bin/deploy.sh` +- 自动更新 timer:`hyapp-server-test-deploy.timer` +- 部署完成标记:`/opt/hyapp-server-test/.deployed_commit` +- 后台前端:`http://1.14.164.2:7001/` + +当前测试机不走生产镜像仓库,也不摘 CLB。部署模式是测试机自己拉取远端 `test` 分支,`hyapp-server-test-deploy.timer` 每分钟触发一次部署检查;部署成功且 Compose 服务全部 healthy 后,才写入 `.deployed_commit`。 + +平台里的“测试服务”页面通过 deploy-platform API 调用 `deploy/tencent_tat/hyapp_testbox_config.py`,再通过 Tencent TAT 到测试机执行固定动作。页面不开放任意 shell: + +- `GET /deploy/api/hyapp/testbox/status`:读取 test 分支、远端 test commit、已部署 commit、timer 和服务状态。 +- `GET /deploy/api/hyapp/testbox/configs`:读取测试机服务 YAML,默认脱敏密钥字段。 +- `POST /deploy/api/hyapp/testbox/service`:固定服务操作 `update/start/stop/restart`。 +- `POST /deploy/api/hyapp/testbox/timer`:固定 timer 操作 `start/stop/restart/status`。 +- `POST /deploy/api/hyapp/testbox/config`:写入单个服务配置,先备份旧文件,可选保存后重启。 +- `GET /deploy/api/hyapp/testbox/migrations`:只列出 `source/server/admin/migrations/*.sql`。 +- `POST /deploy/api/hyapp/testbox/migrations/apply`:只执行测试机仓库里的 migration 文件。 + +本地或 deploy-platform 机器上确认状态: + +```bash +curl -fsS "http://127.0.0.1:29200/deploy/api/hyapp/testbox/status?services=gateway-service,room-service,admin-server" +``` + +直接走脚本确认状态: + +```bash +python3 deploy/tencent_tat/hyapp_testbox_config.py status \ + --env-file /opt/hy-app-monitor/.env \ + --services gateway-service,room-service,admin-server +``` + +## 测试 Server 日志 Debug + +先看平台状态,再看测试机日志。 + +1. 在“测试服务”页面刷新,确认 `test 分支`、`已部署 commit`、`远端 commit`、`timerState`、服务 `state/health/restartCount`。 +2. 如果页面或 API 报错,先看 deploy-platform 自身日志: + +```bash +journalctl -u hyapp-deploy-platform -n 200 --no-pager +docker logs --tail=200 -f hyapp-deploy-platform +``` + +3. 如果 TAT 到测试机成功但服务异常,登录测试机或通过腾讯云 TAT 执行同等只读命令: + +```bash +ssh root@43.165.195.39 + +systemctl status hyapp-server-test-deploy.timer --no-pager +journalctl -u hyapp-server-test-deploy.timer -n 100 --no-pager +journalctl -u hyapp-server-test-deploy.service -n 200 --no-pager + +cd /opt/hyapp-server-test/source +docker compose -p hyapp-test -f docker-compose.yml -f /opt/hyapp-server-test/docker-compose.testbox.yml ps +docker compose -p hyapp-test -f docker-compose.yml -f /opt/hyapp-server-test/docker-compose.testbox.yml logs --tail=200 -f gateway-service +docker inspect gateway-service + +git -C /opt/hyapp-server-test/source status +git -C /opt/hyapp-server-test/source rev-parse HEAD +git -C /opt/hyapp-server-test/source ls-remote origin refs/heads/test +cat /opt/hyapp-server-test/.deployed_commit +``` + +如果 `/opt/hyapp-server-test/docker-compose.testbox.yml` 不存在,去掉第二个 `-f`。Debug 时优先查具体异常服务的 Compose 日志,例如 `room-service`、`wallet-service`、`admin-server`。配置问题先通过页面查看脱敏 YAML;需要保存配置时必须补齐真实密钥,包含 `***` 的配置禁止保存。 diff --git a/deploy/README.md b/deploy/README.md index ef90d8a..10a7379 100644 --- a/deploy/README.md +++ b/deploy/README.md @@ -5,8 +5,10 @@ 目录职责: - `server.mjs`:同源运行 `dist` 静态文件和 `/deploy/api/*` 运维查询接口。 -- `standalone/`:目标服务器 systemd + Docker 安装模板。 -- `tencent-tat/`:发布机通过腾讯云 TAT 调度远端重启和镜像更新,包含当前平台自身发布脚本和 HyApp 微服务发布脚本。 +- `mysql/initdb/`:从 hyapp-server 迁出的共享 MySQL 初始化 SQL;hyapp-server 本地 compose 只引用这里,不再保留根部署目录。 +- `rocketmq/`:从 hyapp-server 迁出的本地 RocketMQ broker 配置。 +- `standalone/`:当前平台和 HyApp 后端服务的目标服务器 systemd + Docker 安装模板。 +- `tencent_tat/`:发布机通过腾讯云 TAT 调度远端重启和镜像更新,包含当前平台自身发布脚本和 HyApp 微服务发布脚本。 最小部署链路: @@ -14,8 +16,8 @@ 2. 执行 `npm run deploy:serve`,一个 Node 进程同时运行 `dist` 静态站点和 `deploy` API。 3. 发布机构建并推送 `deploy-platform` 镜像。 4. 目标机安装 `hyapp-deploy-platform.service` 和 `/etc/hyapp/deploy-platform/docker.env`。 -5. 发布机执行 `deploy/tencent-tat/deploy_platform_tat_deploy.py deploy` 发布当前平台。 -6. 平台服务端调用 `deploy/tencent-tat/hyapp_tat_deploy.py` 查询或发布 HyApp 后端服务。 +5. 发布机执行 `deploy/tencent_tat/deploy_platform_tat_deploy.py deploy` 发布当前平台。 +6. 平台服务端调用 `deploy/tencent_tat/hyapp_tat_deploy.py` 查询或发布 HyApp 后端服务。 本地一键运行: @@ -42,7 +44,7 @@ npm run deploy:serve - `GET /deploy/api/health`:只返回凭据是否存在,不返回密钥明文。 - `GET /deploy/api/hyapp/inventory`:读取 HyApp 生产 inventory。 - `GET /deploy/api/hyapp/plan?services=gateway-service,room-service`:生成发布计划,不执行远端动作。 -- `GET /deploy/api/hyapp/discover?services=user-service&remote=1`:通过 Tencent TAT 读取真实实例、systemd、Docker、镜像、资源、端口和 CLB 权重。 +- `GET /deploy/api/hyapp/discover?services=user-service&remote=1`:通过 Tencent TAT 连接 deploy 机器 `43.164.75.199`,再由 deploy 机器 SSH 到业务机器读取真实实例、systemd、Docker、镜像、资源、端口和 CLB 权重。 - `GET /deploy/api/hyapp/discover?services=user-service&remote=1&includeConfig=1`:读取详情 YAML,密钥字段脱敏。 - `GET /deploy/api/hyapp/resources?remote=1`:数据资源状态。链路:TAT -> deploy;业务服务器配置 SSH 失败后 TAT 兜底。 - `POST /deploy/api/hyapp/deploy`:真实发布,必须带 `confirmed=true`;测试时使用 `dryRun=true`。 @@ -57,4 +59,6 @@ npm run deploy:serve 详细命令见: - `deploy/standalone/README.md` -- `deploy/tencent-tat/README.md` +- `deploy/tencent_tat/README.md` + +HyApp 后端仓库不再负责生产部署编排。新增或调整生产部署脚本、inventory、systemd、Docker env、托管资源边界和公共 initdb/rocketmq 配置时,都在本仓库维护。 diff --git a/deploy/__init__.py b/deploy/__init__.py new file mode 100644 index 0000000..b5b1475 --- /dev/null +++ b/deploy/__init__.py @@ -0,0 +1 @@ +# Package marker for deployment helper tests. diff --git a/deploy/mysql/initdb/005_utf8mb4_chinese_support.sql b/deploy/mysql/initdb/005_utf8mb4_chinese_support.sql new file mode 100644 index 0000000..f2d5753 --- /dev/null +++ b/deploy/mysql/initdb/005_utf8mb4_chinese_support.sql @@ -0,0 +1,45 @@ +SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci; + + +-- 使用 utf8mb4 连接字符集,保证中文注释、中文名称和 emoji 在导入时不被降级。 +-- 本地 Docker 入口脚本会先创建 MYSQL_DATABASE,服务 DDL 随后创建各业务库。 +-- 这里统一修复新旧本地卷的默认字符集,保证中文文本、中文 COMMENT 和 emoji 不被旧默认字符集截断。 +ALTER DATABASE hyapp_room CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER DATABASE hyapp_user CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER DATABASE hyapp_wallet CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER DATABASE hyapp_activity CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; + +USE hyapp_room; +ALTER TABLE rooms CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE room_list_entries CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE room_snapshots CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE room_command_log CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE room_outbox CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; + +USE hyapp_user; +ALTER TABLE users CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE countries CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE regions CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE region_countries CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE region_change_logs CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE user_region_rebuild_tasks CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE user_display_user_ids CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE pretty_display_user_id_leases CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE display_user_id_change_logs CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE user_country_change_logs CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE password_accounts CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE third_party_identities CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE auth_sessions CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE login_audit CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; + +USE hyapp_wallet; +ALTER TABLE wallet_accounts CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE wallet_transactions CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE wallet_entries CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE wallet_outbox CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE wallet_gift_prices CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; + +USE hyapp_activity; +ALTER TABLE activities CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE activity_event_consumption CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; +ALTER TABLE activity_outbox CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; diff --git a/deploy/mysql/initdb/006_admin_database.sql b/deploy/mysql/initdb/006_admin_database.sql new file mode 100644 index 0000000..d6d5b92 --- /dev/null +++ b/deploy/mysql/initdb/006_admin_database.sql @@ -0,0 +1,5 @@ +SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci; + + +-- 使用 utf8mb4 连接字符集,保证中文注释、中文名称和 emoji 在导入时不被降级。 +CREATE DATABASE IF NOT EXISTS hyapp_admin DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; diff --git a/deploy/mysql/initdb/007_resource_group_wallet_asset_items.sql b/deploy/mysql/initdb/007_resource_group_wallet_asset_items.sql new file mode 100644 index 0000000..77b1300 --- /dev/null +++ b/deploy/mysql/initdb/007_resource_group_wallet_asset_items.sql @@ -0,0 +1,57 @@ +SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci; + +-- 使用 utf8mb4 连接字符集,保证中文注释、中文名称和 emoji 在导入时不被降级。 + +USE hyapp_wallet; + +DROP PROCEDURE IF EXISTS migrate_resource_group_wallet_asset_items; + +DELIMITER $$ + +CREATE PROCEDURE migrate_resource_group_wallet_asset_items() +BEGIN + IF NOT EXISTS ( + SELECT 1 FROM information_schema.COLUMNS + WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'resource_group_items' AND COLUMN_NAME = 'item_type' + ) THEN + ALTER TABLE resource_group_items + ADD COLUMN item_type VARCHAR(32) NOT NULL DEFAULT 'resource' COMMENT '明细类型' AFTER group_id; + END IF; + + IF NOT EXISTS ( + SELECT 1 FROM information_schema.COLUMNS + WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'resource_group_items' AND COLUMN_NAME = 'wallet_asset_type' + ) THEN + ALTER TABLE resource_group_items + ADD COLUMN wallet_asset_type VARCHAR(32) NOT NULL DEFAULT '' COMMENT '钱包资产类型' AFTER resource_id; + END IF; + + IF NOT EXISTS ( + SELECT 1 FROM information_schema.COLUMNS + WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'resource_group_items' AND COLUMN_NAME = 'wallet_asset_amount' + ) THEN + ALTER TABLE resource_group_items + ADD COLUMN wallet_asset_amount BIGINT NOT NULL DEFAULT 0 COMMENT '钱包资产数量' AFTER wallet_asset_type; + END IF; + + IF EXISTS ( + SELECT 1 FROM information_schema.STATISTICS + WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'resource_group_items' AND INDEX_NAME = 'uk_resource_group_items_resource' + ) THEN + ALTER TABLE resource_group_items DROP INDEX uk_resource_group_items_resource; + END IF; + + IF NOT EXISTS ( + SELECT 1 FROM information_schema.STATISTICS + WHERE TABLE_SCHEMA = DATABASE() AND TABLE_NAME = 'resource_group_items' AND INDEX_NAME = 'uk_resource_group_items_member' + ) THEN + ALTER TABLE resource_group_items + ADD UNIQUE KEY uk_resource_group_items_member (app_code, group_id, item_type, resource_id, wallet_asset_type); + END IF; +END$$ + +DELIMITER ; + +CALL migrate_resource_group_wallet_asset_items(); + +DROP PROCEDURE migrate_resource_group_wallet_asset_items; diff --git a/deploy/mysql/initdb/008_remove_taiwan_country.sql b/deploy/mysql/initdb/008_remove_taiwan_country.sql new file mode 100644 index 0000000..6f5c9eb --- /dev/null +++ b/deploy/mysql/initdb/008_remove_taiwan_country.sql @@ -0,0 +1,14 @@ +SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci; + +-- 使用 utf8mb4 连接字符集,保证中文注释、中文名称和 emoji 在导入时不被降级。 + +USE hyapp_user; + +-- 对已经执行过旧国家种子的本地库做幂等修正,保证国家主数据与当前产品范围一致。 +DELETE FROM region_countries +WHERE app_code = 'lalu' + AND country_code = 'TW'; + +DELETE FROM countries +WHERE app_code = 'lalu' + AND country_code = 'TW'; diff --git a/deploy/mysql/initdb/999_local_grants.sql b/deploy/mysql/initdb/999_local_grants.sql new file mode 100644 index 0000000..1af9b24 --- /dev/null +++ b/deploy/mysql/initdb/999_local_grants.sql @@ -0,0 +1,17 @@ +SET NAMES utf8mb4 COLLATE utf8mb4_unicode_ci; + + +-- 使用 utf8mb4 连接字符集,保证中文注释、中文名称和 emoji 在导入时不被降级。 +CREATE USER IF NOT EXISTS 'hyapp'@'%' IDENTIFIED BY 'hyapp'; + +GRANT ALL PRIVILEGES ON hyapp_room.* TO 'hyapp'@'%'; +GRANT ALL PRIVILEGES ON hyapp_user.* TO 'hyapp'@'%'; +GRANT ALL PRIVILEGES ON hyapp_wallet.* TO 'hyapp'@'%'; +GRANT ALL PRIVILEGES ON hyapp_activity.* TO 'hyapp'@'%'; +GRANT ALL PRIVILEGES ON hyapp_admin.* TO 'hyapp'@'%'; +GRANT ALL PRIVILEGES ON hyapp_cron.* TO 'hyapp'@'%'; +GRANT ALL PRIVILEGES ON hyapp_game.* TO 'hyapp'@'%'; +GRANT ALL PRIVILEGES ON hyapp_notice.* TO 'hyapp'@'%'; +GRANT ALL PRIVILEGES ON hyapp_statistics.* TO 'hyapp'@'%'; + +FLUSH PRIVILEGES; diff --git a/deploy/rocketmq/broker.conf b/deploy/rocketmq/broker.conf new file mode 100644 index 0000000..e6cf93e --- /dev/null +++ b/deploy/rocketmq/broker.conf @@ -0,0 +1,9 @@ +brokerClusterName = hyapp-local +brokerName = broker-a +brokerId = 0 +deleteWhen = 04 +fileReservedTime = 48 +brokerRole = ASYNC_MASTER +flushDiskType = ASYNC_FLUSH +autoCreateTopicEnable = true +listenPort = 10911 diff --git a/deploy/server.mjs b/deploy/server.mjs index 4da3797..e9ae55c 100644 --- a/deploy/server.mjs +++ b/deploy/server.mjs @@ -9,7 +9,7 @@ const deployDir = join(rootDir, "deploy"); const distDir = resolve(process.env.DEPLOY_DIST_DIR || join(rootDir, "dist")); const apiPort = Number(process.env.DEPLOY_API_PORT || process.env.PORT || 29200); const hyappInventoryPath = resolve( - process.env.HYAPP_DEPLOY_INVENTORY || join(deployDir, "tencent-tat", "hyapp-services.inventory.prod.example.json"), + process.env.HYAPP_DEPLOY_INVENTORY || join(deployDir, "tencent_tat", "hyapp-services.inventory.prod.example.json"), ); const hyappTestbox = { instanceId: process.env.HYAPP_TESTBOX_INSTANCE_ID || "lhins-q0m38zc6", @@ -20,7 +20,7 @@ const hyappTestboxConfigServices = process.env.HYAPP_TESTBOX_CONFIG_SERVICES || "gateway-service,room-service,wallet-service,user-service,activity-service,cron-service,game-service,notice-service,admin-server"; const platformInventoryPath = resolve( - process.env.DEPLOY_PLATFORM_INVENTORY || join(deployDir, "tencent-tat", "inventory.prod.example.json"), + process.env.DEPLOY_PLATFORM_INVENTORY || join(deployDir, "tencent_tat", "inventory.prod.example.json"), ); const pythonBin = process.env.PYTHON_BIN || "python3"; const tencentEnvFile = process.env.TENCENT_ENV_FILE || ""; @@ -190,7 +190,7 @@ function runDeployScript(scriptName, args) { return new Promise((resolvePromise) => { const child = execFile( pythonBin, - [join(deployDir, "tencent-tat", scriptName), ...args], + [join(deployDir, "tencent_tat", scriptName), ...args], { cwd: rootDir, env: deployEnv, @@ -393,6 +393,65 @@ async function handleApi(request, response, url) { return; } + if (url.pathname === "/deploy/api/hyapp/config") { + if (request.method !== "POST") { + sendError(response, 405, "POST is required"); + return; + } + + const body = await readRequestJson(request); + requireConfirmed(body); + + const services = parseServices(body.service || body.services, ""); + const hosts = parseHosts(body.hosts); + const content = String(body.content ?? ""); + + if (services.length !== 1) { + throw new Error("exactly one service is required"); + } + + if (!content.trim()) { + throw new Error("config content is required"); + } + + if (content.length > 80_000) { + throw new Error("config content is too large"); + } + + if (content.includes("***")) { + throw new Error("redacted config cannot be saved; replace masked secret fields before saving"); + } + + const args = [ + "write-config", + ...buildEnvArgs(), + "--inventory", + hyappInventoryPath, + "--services", + services.join(","), + "--content-base64", + Buffer.from(content, "utf-8").toString("base64"), + ]; + + if (hosts.length > 0) { + args.push("--hosts", hosts.join(",")); + } + + if (body.restart === true) { + args.push("--restart"); + } + + if (body.dryRun === true) { + args.push("--dry-run"); + } else { + args.push("--yes"); + } + + const result = await runDeployScript("hyapp_tat_deploy.py", args); + sendJson(response, result.ok ? 200 : 500, result.payload); + return; + } + if (url.pathname === "/deploy/api/hyapp/discover") { const services = parseServices(url.searchParams.get("services"), "gateway-service,room-service,user-service"); const executeRemote = url.searchParams.get("remote") === "1"; diff --git a/deploy/standalone/README.md b/deploy/standalone/README.md index 7931125..bc5916a 100644 --- a/deploy/standalone/README.md +++ b/deploy/standalone/README.md @@ -1,8 +1,8 @@ -# Deploy Platform Standalone Deployment +# Standalone Deployment Templates -本目录用于把当前 `deploy-platform` 前端项目单独部署到 CVM。运行形态是 systemd 托管 Docker 容器,容器内通过 `node deploy/server.mjs` 同源托管 Vite 构建后的静态文件和 `/deploy/api/*` 运维查询接口。 +本目录集中维护生产 CVM 的 systemd + Docker 模板。`hyapp-server` 不再保留根部署目录;HyApp 后端服务和当前 `deploy-platform` 自身的部署模板都在这里维护。 -## Runtime +## Deploy Platform Runtime - 镜像:`deploy-platform` - 容器:`hyapp-deploy-platform` @@ -28,7 +28,7 @@ - `GET /deploy/api/hyapp/testbox/migrations` - `POST /deploy/api/hyapp/testbox/migrations/apply` -`resources?remote=1`:TAT -> deploy;业务服务器配置 SSH 失败后 TAT 兜底;不开放 SQL/Mongo 查询入口。`deploy` 和 `restart` 是真实操作接口,必须在 JSON body 中显式传入 `confirmed: true`。联调和验收使用 `dryRun: true`。 +`discover?remote=1`:TAT -> deploy `43.164.75.199` -> SSH 业务机器。`resources?remote=1`:TAT -> deploy;业务服务器配置 SSH 失败后 TAT 兜底;不开放 SQL/Mongo 查询入口。`deploy` 和 `restart` 是真实操作接口,必须在 JSON body 中显式传入 `confirmed: true`。联调和验收使用 `dryRun: true`。 ## Build Image @@ -58,3 +58,33 @@ curl -fsS http://127.0.0.1:29200/deploy/api/health ``` `docker.env` 只保存镜像、容器名和停止等待时间。腾讯云凭据不写入镜像;需要执行远端 discover 时,通过容器环境变量或 `TENCENT_ENV_FILE` 挂载注入。 + +## HyApp Service Templates + +从 `hyapp-server/deploy/standalone` 迁入的模板: + +- `systemd/hyapp-*-service.service`:后端 Go 服务 systemd unit。 +- `systemd/hyapp-admin-server.service`、`systemd/hyapp-admin-platform.service`:后台服务和后台前端 unit。 +- `docker/*-service.env.example`、`docker/admin-*.env.example`:每个服务一份 Docker 运行参数示例。 + +这些模板只管理业务容器,不安装或重启 MySQL、Redis、MQ。生产 MySQL、Redis、MQ 继续使用腾讯云托管资源,服务配置里的 endpoint 和密钥由发布系统渲染到目标机 `/etc/hyapp//config.yaml`。 + +以 `gateway-service` 为例: + +```bash +sudo mkdir -p /etc/hyapp/gateway-service +sudo cp deploy/standalone/docker/gateway-service.env.example /etc/hyapp/gateway-service/docker.env +sudo cp /opt/deploy/sources/hyapp-server/services/gateway-service/configs/config.tencent.example.yaml /etc/hyapp/gateway-service/config.yaml +sudo vi /etc/hyapp/gateway-service/docker.env +sudo vi /etc/hyapp/gateway-service/config.yaml + +sudo cp deploy/standalone/systemd/hyapp-gateway-service.service /etc/systemd/system/ +sudo systemctl daemon-reload +sudo systemctl enable --now hyapp-gateway-service +sudo systemctl status hyapp-gateway-service +sudo docker inspect --format '{{json .State.Health}}' hyapp-gateway-service +``` + +模板默认使用 `--network=host`,容器直接监听项目约定的 `13xxx` 端口。`room-service` 多实例必须配置唯一 `node_id`,且 `advertise_addr` 必须是实例私网地址,不能写内网 CLB 地址。 + +HyApp 后端滚动发布通过 `deploy/tencent_tat/hyapp_tat_deploy.py` 执行,inventory 使用 `deploy/tencent_tat/hyapp-services.inventory.prod.example.json` 作为模板。 diff --git a/deploy/standalone/docker/activity-service.env.example b/deploy/standalone/docker/activity-service.env.example new file mode 100644 index 0000000..ff6e626 --- /dev/null +++ b/deploy/standalone/docker/activity-service.env.example @@ -0,0 +1,6 @@ +# Image is immutable per release; publish scripts pull this tag before restarting systemd. +IMAGE=10.2.1.3:18082/hyapp/activity-service:REPLACE_WITH_RELEASE_TAG +CONTAINER_NAME=hyapp-activity-service +CONFIG_PATH=/etc/hyapp/activity-service/config.yaml +STOP_TIMEOUT_SEC=60 + diff --git a/deploy/standalone/docker/admin-platform.env.example b/deploy/standalone/docker/admin-platform.env.example new file mode 100644 index 0000000..e95acbb --- /dev/null +++ b/deploy/standalone/docker/admin-platform.env.example @@ -0,0 +1,3 @@ +IMAGE=10.2.1.3:18082/hyapp/admin-platform:REPLACE_TAG +CONTAINER_NAME=hyapp-admin-platform +STOP_TIMEOUT_SEC=15 diff --git a/deploy/standalone/docker/admin-server.env.example b/deploy/standalone/docker/admin-server.env.example new file mode 100644 index 0000000..cea565c --- /dev/null +++ b/deploy/standalone/docker/admin-server.env.example @@ -0,0 +1,4 @@ +IMAGE=10.2.1.3:18082/hyapp/admin-server:REPLACE_TAG +CONTAINER_NAME=hyapp-admin-server +CONFIG_PATH=/etc/hyapp/admin-server/config.yaml +STOP_TIMEOUT_SEC=30 diff --git a/deploy/standalone/docker/cron-service.env.example b/deploy/standalone/docker/cron-service.env.example new file mode 100644 index 0000000..bf3b277 --- /dev/null +++ b/deploy/standalone/docker/cron-service.env.example @@ -0,0 +1,6 @@ +# cron-service already uses MySQL task leases, but keep one replica until every task is verified under multi-node scheduling. +IMAGE=10.2.1.3:18082/hyapp/cron-service:REPLACE_WITH_RELEASE_TAG +CONTAINER_NAME=hyapp-cron-service +CONFIG_PATH=/etc/hyapp/cron-service/config.yaml +STOP_TIMEOUT_SEC=90 + diff --git a/deploy/standalone/docker/game-service.env.example b/deploy/standalone/docker/game-service.env.example new file mode 100644 index 0000000..f6ad9ab --- /dev/null +++ b/deploy/standalone/docker/game-service.env.example @@ -0,0 +1,5 @@ +IMAGE=10.2.1.3:18082/hyapp/game-service:REPLACE_WITH_RELEASE_TAG +CONTAINER_NAME=hyapp-game-service +CONFIG_PATH=/etc/hyapp/game-service/config.yaml +STOP_TIMEOUT_SEC=60 + diff --git a/deploy/standalone/docker/gateway-service.env.example b/deploy/standalone/docker/gateway-service.env.example new file mode 100644 index 0000000..9ce0cc7 --- /dev/null +++ b/deploy/standalone/docker/gateway-service.env.example @@ -0,0 +1,6 @@ +# gateway instances are stateless HTTP entries and should be attached behind the public CLB only after /healthz/ready passes. +IMAGE=10.2.1.3:18082/hyapp/gateway-service:REPLACE_WITH_RELEASE_TAG +CONTAINER_NAME=hyapp-gateway-service +CONFIG_PATH=/etc/hyapp/gateway-service/config.yaml +STOP_TIMEOUT_SEC=60 + diff --git a/deploy/standalone/docker/notice-service.env.example b/deploy/standalone/docker/notice-service.env.example new file mode 100644 index 0000000..ee8b434 --- /dev/null +++ b/deploy/standalone/docker/notice-service.env.example @@ -0,0 +1,5 @@ +# Image is immutable per release; publish scripts pull this tag before restarting systemd. +IMAGE=10.2.1.3:18082/hyapp/notice-service:REPLACE_WITH_RELEASE_TAG +CONTAINER_NAME=hyapp-notice-service +CONFIG_PATH=/etc/hyapp/notice-service/config.yaml +STOP_TIMEOUT_SEC=60 diff --git a/deploy/standalone/docker/room-service.env.example b/deploy/standalone/docker/room-service.env.example new file mode 100644 index 0000000..27d1e32 --- /dev/null +++ b/deploy/standalone/docker/room-service.env.example @@ -0,0 +1,6 @@ +# Keep room-service single-active until owner routing or non-owner forwarding is implemented. +IMAGE=10.2.1.3:18082/hyapp/room-service:REPLACE_WITH_RELEASE_TAG +CONTAINER_NAME=hyapp-room-service +CONFIG_PATH=/etc/hyapp/room-service/config.yaml +STOP_TIMEOUT_SEC=90 + diff --git a/deploy/standalone/docker/statistics-service.env.example b/deploy/standalone/docker/statistics-service.env.example new file mode 100644 index 0000000..9c94427 --- /dev/null +++ b/deploy/standalone/docker/statistics-service.env.example @@ -0,0 +1,4 @@ +IMAGE=10.2.1.3:18082/hyapp/statistics-service:RELEASE_TAG +CONTAINER_NAME=hyapp-statistics-service +CONFIG_PATH=/etc/hyapp/statistics-service/config.yaml +STOP_TIMEOUT_SEC=60 diff --git a/deploy/standalone/docker/user-service.env.example b/deploy/standalone/docker/user-service.env.example new file mode 100644 index 0000000..c720972 --- /dev/null +++ b/deploy/standalone/docker/user-service.env.example @@ -0,0 +1,6 @@ +# id_generator.node_id in config.yaml must be unique for every active user-service instance. +IMAGE=10.2.1.3:18082/hyapp/user-service:REPLACE_WITH_RELEASE_TAG +CONTAINER_NAME=hyapp-user-service +CONFIG_PATH=/etc/hyapp/user-service/config.yaml +STOP_TIMEOUT_SEC=60 + diff --git a/deploy/standalone/docker/wallet-service.env.example b/deploy/standalone/docker/wallet-service.env.example new file mode 100644 index 0000000..44f1ef3 --- /dev/null +++ b/deploy/standalone/docker/wallet-service.env.example @@ -0,0 +1,5 @@ +IMAGE=10.2.1.3:18082/hyapp/wallet-service:REPLACE_WITH_RELEASE_TAG +CONTAINER_NAME=hyapp-wallet-service +CONFIG_PATH=/etc/hyapp/wallet-service/config.yaml +STOP_TIMEOUT_SEC=60 + diff --git a/deploy/standalone/systemd/hyapp-activity-service.service b/deploy/standalone/systemd/hyapp-activity-service.service new file mode 100644 index 0000000..3e2761d --- /dev/null +++ b/deploy/standalone/systemd/hyapp-activity-service.service @@ -0,0 +1,24 @@ +[Unit] +Description=Hyapp activity-service container +Requires=docker.service +After=network-online.target docker.service +Wants=network-online.target +StartLimitIntervalSec=120 +StartLimitBurst=3 + +[Service] +Type=simple +EnvironmentFile=/etc/hyapp/activity-service/docker.env +ExecStartPre=-/usr/bin/env docker rm -f ${CONTAINER_NAME} +ExecStart=/usr/bin/env docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --health-cmd="/app/grpc-health-probe -addr=127.0.0.1:13006 -service=activity-service" --health-interval=5s --health-timeout=3s --health-retries=6 --health-start-period=10s ${IMAGE} +ExecStop=/usr/bin/env docker stop --time=${STOP_TIMEOUT_SEC} ${CONTAINER_NAME} +Restart=on-failure +RestartSec=5s +TimeoutStartSec=60 +TimeoutStopSec=90 +KillMode=none +LimitNOFILE=1048576 + +[Install] +WantedBy=multi-user.target + diff --git a/deploy/standalone/systemd/hyapp-admin-platform.service b/deploy/standalone/systemd/hyapp-admin-platform.service new file mode 100644 index 0000000..cd3e9ca --- /dev/null +++ b/deploy/standalone/systemd/hyapp-admin-platform.service @@ -0,0 +1,17 @@ +[Unit] +Description=HYApp admin platform frontend +After=docker.service network-online.target hyapp-admin-server.service +Wants=network-online.target +Requires=docker.service + +[Service] +Type=simple +EnvironmentFile=/etc/hyapp/admin-platform/docker.env +ExecStartPre=-/usr/bin/env docker rm -f ${CONTAINER_NAME} +ExecStart=/usr/bin/env docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 ${IMAGE} +ExecStop=/usr/bin/env docker stop -t ${STOP_TIMEOUT_SEC} ${CONTAINER_NAME} +Restart=always +RestartSec=3 + +[Install] +WantedBy=multi-user.target diff --git a/deploy/standalone/systemd/hyapp-admin-server.service b/deploy/standalone/systemd/hyapp-admin-server.service new file mode 100644 index 0000000..80d7e78 --- /dev/null +++ b/deploy/standalone/systemd/hyapp-admin-server.service @@ -0,0 +1,17 @@ +[Unit] +Description=HYApp admin-server +After=docker.service network-online.target +Wants=network-online.target +Requires=docker.service + +[Service] +Type=simple +EnvironmentFile=/etc/hyapp/admin-server/docker.env +ExecStartPre=-/usr/bin/env docker rm -f ${CONTAINER_NAME} +ExecStart=/usr/bin/env docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --mount=type=bind,src=/var/lib/hyapp/admin-server/storage,dst=/app/storage ${IMAGE} +ExecStop=/usr/bin/env docker stop -t ${STOP_TIMEOUT_SEC} ${CONTAINER_NAME} +Restart=always +RestartSec=3 + +[Install] +WantedBy=multi-user.target diff --git a/deploy/standalone/systemd/hyapp-cron-service.service b/deploy/standalone/systemd/hyapp-cron-service.service new file mode 100644 index 0000000..c2fa3bf --- /dev/null +++ b/deploy/standalone/systemd/hyapp-cron-service.service @@ -0,0 +1,24 @@ +[Unit] +Description=Hyapp cron-service container +Requires=docker.service +After=network-online.target docker.service +Wants=network-online.target +StartLimitIntervalSec=120 +StartLimitBurst=3 + +[Service] +Type=simple +EnvironmentFile=/etc/hyapp/cron-service/docker.env +ExecStartPre=-/usr/bin/env docker rm -f ${CONTAINER_NAME} +ExecStart=/usr/bin/env docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --health-cmd="/app/grpc-health-probe -addr=127.0.0.1:13007 -service=cron-service" --health-interval=5s --health-timeout=3s --health-retries=6 --health-start-period=10s ${IMAGE} +ExecStop=/usr/bin/env docker stop --time=${STOP_TIMEOUT_SEC} ${CONTAINER_NAME} +Restart=on-failure +RestartSec=5s +TimeoutStartSec=60 +TimeoutStopSec=120 +KillMode=none +LimitNOFILE=1048576 + +[Install] +WantedBy=multi-user.target + diff --git a/deploy/standalone/systemd/hyapp-game-service.service b/deploy/standalone/systemd/hyapp-game-service.service new file mode 100644 index 0000000..37f0f8c --- /dev/null +++ b/deploy/standalone/systemd/hyapp-game-service.service @@ -0,0 +1,24 @@ +[Unit] +Description=Hyapp game-service container +Requires=docker.service +After=network-online.target docker.service +Wants=network-online.target +StartLimitIntervalSec=120 +StartLimitBurst=3 + +[Service] +Type=simple +EnvironmentFile=/etc/hyapp/game-service/docker.env +ExecStartPre=-/usr/bin/env docker rm -f ${CONTAINER_NAME} +ExecStart=/usr/bin/env docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --health-cmd="/app/grpc-health-probe -addr=127.0.0.1:13008 -service=game-service" --health-interval=5s --health-timeout=3s --health-retries=6 --health-start-period=10s ${IMAGE} +ExecStop=/usr/bin/env docker stop --time=${STOP_TIMEOUT_SEC} ${CONTAINER_NAME} +Restart=on-failure +RestartSec=5s +TimeoutStartSec=60 +TimeoutStopSec=90 +KillMode=none +LimitNOFILE=1048576 + +[Install] +WantedBy=multi-user.target + diff --git a/deploy/standalone/systemd/hyapp-gateway-service.service b/deploy/standalone/systemd/hyapp-gateway-service.service new file mode 100644 index 0000000..499fd7a --- /dev/null +++ b/deploy/standalone/systemd/hyapp-gateway-service.service @@ -0,0 +1,24 @@ +[Unit] +Description=Hyapp gateway-service container +Requires=docker.service +After=network-online.target docker.service +Wants=network-online.target +StartLimitIntervalSec=120 +StartLimitBurst=3 + +[Service] +Type=simple +EnvironmentFile=/etc/hyapp/gateway-service/docker.env +ExecStartPre=-/usr/bin/env docker rm -f ${CONTAINER_NAME} +ExecStart=/usr/bin/env docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --health-cmd="wget -q -O - http://127.0.0.1:13000/healthz/ready >/dev/null" --health-interval=5s --health-timeout=3s --health-retries=6 --health-start-period=10s ${IMAGE} +ExecStop=/usr/bin/env docker stop --time=${STOP_TIMEOUT_SEC} ${CONTAINER_NAME} +Restart=on-failure +RestartSec=5s +TimeoutStartSec=60 +TimeoutStopSec=90 +KillMode=none +LimitNOFILE=1048576 + +[Install] +WantedBy=multi-user.target + diff --git a/deploy/standalone/systemd/hyapp-notice-service.service b/deploy/standalone/systemd/hyapp-notice-service.service new file mode 100644 index 0000000..bab6fcd --- /dev/null +++ b/deploy/standalone/systemd/hyapp-notice-service.service @@ -0,0 +1,23 @@ +[Unit] +Description=Hyapp notice-service container +Requires=docker.service +After=network-online.target docker.service +Wants=network-online.target +StartLimitIntervalSec=120 +StartLimitBurst=3 + +[Service] +Type=simple +EnvironmentFile=/etc/hyapp/notice-service/docker.env +ExecStartPre=-/usr/bin/env docker rm -f ${CONTAINER_NAME} +ExecStart=/usr/bin/env docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --health-cmd="/app/grpc-health-probe -addr=127.0.0.1:13009 -service=notice-service" --health-interval=5s --health-timeout=3s --health-retries=6 --health-start-period=10s ${IMAGE} +ExecStop=/usr/bin/env docker stop --time=${STOP_TIMEOUT_SEC} ${CONTAINER_NAME} +Restart=on-failure +RestartSec=5s +TimeoutStartSec=60 +TimeoutStopSec=90 +KillMode=none +LimitNOFILE=1048576 + +[Install] +WantedBy=multi-user.target diff --git a/deploy/standalone/systemd/hyapp-room-service.service b/deploy/standalone/systemd/hyapp-room-service.service new file mode 100644 index 0000000..08abcb7 --- /dev/null +++ b/deploy/standalone/systemd/hyapp-room-service.service @@ -0,0 +1,24 @@ +[Unit] +Description=Hyapp room-service container +Requires=docker.service +After=network-online.target docker.service +Wants=network-online.target +StartLimitIntervalSec=120 +StartLimitBurst=3 + +[Service] +Type=simple +EnvironmentFile=/etc/hyapp/room-service/docker.env +ExecStartPre=-/usr/bin/env docker rm -f ${CONTAINER_NAME} +ExecStart=/usr/bin/env docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --health-cmd="/app/grpc-health-probe -addr=127.0.0.1:13001 -service=room-service" --health-interval=5s --health-timeout=3s --health-retries=6 --health-start-period=10s ${IMAGE} +ExecStop=/usr/bin/env docker stop --time=${STOP_TIMEOUT_SEC} ${CONTAINER_NAME} +Restart=on-failure +RestartSec=5s +TimeoutStartSec=60 +TimeoutStopSec=120 +KillMode=none +LimitNOFILE=1048576 + +[Install] +WantedBy=multi-user.target + diff --git a/deploy/standalone/systemd/hyapp-statistics-service.service b/deploy/standalone/systemd/hyapp-statistics-service.service new file mode 100644 index 0000000..0be1a37 --- /dev/null +++ b/deploy/standalone/systemd/hyapp-statistics-service.service @@ -0,0 +1,17 @@ +[Unit] +Description=HYApp statistics-service +After=docker.service network-online.target +Wants=network-online.target +Requires=docker.service + +[Service] +Type=simple +EnvironmentFile=/etc/hyapp/statistics-service/docker.env +ExecStartPre=-/usr/bin/env docker rm -f ${CONTAINER_NAME} +ExecStart=/usr/bin/env docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly ${IMAGE} +ExecStop=/usr/bin/env docker stop -t ${STOP_TIMEOUT_SEC} ${CONTAINER_NAME} +Restart=always +RestartSec=3 + +[Install] +WantedBy=multi-user.target diff --git a/deploy/standalone/systemd/hyapp-user-service.service b/deploy/standalone/systemd/hyapp-user-service.service new file mode 100644 index 0000000..1670e1f --- /dev/null +++ b/deploy/standalone/systemd/hyapp-user-service.service @@ -0,0 +1,24 @@ +[Unit] +Description=Hyapp user-service container +Requires=docker.service +After=network-online.target docker.service +Wants=network-online.target +StartLimitIntervalSec=120 +StartLimitBurst=3 + +[Service] +Type=simple +EnvironmentFile=/etc/hyapp/user-service/docker.env +ExecStartPre=-/usr/bin/env docker rm -f ${CONTAINER_NAME} +ExecStart=/usr/bin/env docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --health-cmd="/app/grpc-health-probe -addr=127.0.0.1:13005 -service=user-service" --health-interval=5s --health-timeout=3s --health-retries=6 --health-start-period=10s ${IMAGE} +ExecStop=/usr/bin/env docker stop --time=${STOP_TIMEOUT_SEC} ${CONTAINER_NAME} +Restart=on-failure +RestartSec=5s +TimeoutStartSec=60 +TimeoutStopSec=90 +KillMode=none +LimitNOFILE=1048576 + +[Install] +WantedBy=multi-user.target + diff --git a/deploy/standalone/systemd/hyapp-wallet-service.service b/deploy/standalone/systemd/hyapp-wallet-service.service new file mode 100644 index 0000000..381c86d --- /dev/null +++ b/deploy/standalone/systemd/hyapp-wallet-service.service @@ -0,0 +1,23 @@ +[Unit] +Description=Hyapp wallet-service container +Requires=docker.service +After=network-online.target docker.service +Wants=network-online.target +StartLimitIntervalSec=120 +StartLimitBurst=3 + +[Service] +Type=simple +EnvironmentFile=/etc/hyapp/wallet-service/docker.env +ExecStartPre=-/usr/bin/env docker rm -f ${CONTAINER_NAME} +ExecStart=/usr/bin/env docker run --name=${CONTAINER_NAME} --rm --network=host --init --pull=never --stop-signal=SIGTERM --stop-timeout=${STOP_TIMEOUT_SEC} --env=TZ=UTC --log-driver=json-file --log-opt=max-size=100m --log-opt=max-file=5 --mount=type=bind,src=${CONFIG_PATH},dst=/app/config.yaml,readonly --mount=type=bind,src=/etc/hyapp/secrets,dst=/etc/hyapp/secrets,readonly --health-cmd="/app/grpc-health-probe -addr=127.0.0.1:13004 -service=wallet-service" --health-interval=5s --health-timeout=3s --health-retries=6 --health-start-period=10s ${IMAGE} +ExecStop=/usr/bin/env docker stop --time=${STOP_TIMEOUT_SEC} ${CONTAINER_NAME} +Restart=on-failure +RestartSec=5s +TimeoutStartSec=60 +TimeoutStopSec=90 +KillMode=none +LimitNOFILE=1048576 + +[Install] +WantedBy=multi-user.target diff --git a/deploy/tencent-tat/README.md b/deploy/tencent_tat/README.md similarity index 63% rename from deploy/tencent-tat/README.md rename to deploy/tencent_tat/README.md index ebe9661..fdaf8fc 100644 --- a/deploy/tencent-tat/README.md +++ b/deploy/tencent_tat/README.md @@ -3,8 +3,9 @@ 本目录包含两类 TAT 发布入口: - `deploy_platform_tat_deploy.py`:发布当前 `deploy-platform` 前端项目。 -- `hyapp_tat_deploy.py`:发布 HyApp 后端 Go 微服务,读取真实 systemd、Docker、镜像、资源、日志和 YAML。 +- `hyapp_tat_deploy.py`:发布 HyApp 后端 Go 微服务;运行态发现使用 TAT 连接 deploy 机器 `43.164.75.199`,再由 deploy 机器 SSH 读取真实 systemd、Docker、镜像、资源、日志和 YAML。 - `hyapp_data_resources.py`:数据资源状态。TAT -> deploy;业务服务器配置 SSH 失败后 TAT 兜底。 +- `validate_rendered_configs.py`:校验 HyApp 服务 YAML 字段契约,服务源码默认读取相邻 `../hyapp-server`,也可以用 `--hyapp-root` 指定。 两者使用同一套 TAT/CLB 滚动部署模型:可选 CLB 摘流,TAT 远端更新 `docker.env` 镜像并重启 systemd unit,健康恢复后恢复权重。 @@ -38,8 +39,8 @@ export TENCENTCLOUD_REGION="me-saudi-arabia" ## Inventory ```bash -cp deploy/tencent-tat/inventory.prod.example.json deploy/tencent-tat/deploy-platform.inventory.prod.json -vi deploy/tencent-tat/deploy-platform.inventory.prod.json +cp deploy/tencent_tat/inventory.prod.example.json deploy/tencent_tat/deploy-platform.inventory.prod.json +vi deploy/tencent_tat/deploy-platform.inventory.prod.json ``` 必须替换: @@ -53,26 +54,26 @@ vi deploy/tencent-tat/deploy-platform.inventory.prod.json 只看计划,不执行远端动作: ```bash -python3 deploy/tencent-tat/deploy_platform_tat_deploy.py plan \ - --inventory deploy/tencent-tat/deploy-platform.inventory.prod.json \ +python3 deploy/tencent_tat/deploy_platform_tat_deploy.py plan \ + --inventory deploy/tencent_tat/deploy-platform.inventory.prod.json \ --services deploy-platform ``` 发现目标机当前状态: ```bash -python3 deploy/tencent-tat/deploy_platform_tat_deploy.py discover \ +python3 deploy/tencent_tat/deploy_platform_tat_deploy.py discover \ --env-file /opt/hy-app-monitor/.env \ - --inventory deploy/tencent-tat/deploy-platform.inventory.prod.json \ + --inventory deploy/tencent_tat/deploy-platform.inventory.prod.json \ --services deploy-platform ``` 滚动更新镜像: ```bash -python3 deploy/tencent-tat/deploy_platform_tat_deploy.py deploy \ +python3 deploy/tencent_tat/deploy_platform_tat_deploy.py deploy \ --env-file /opt/hy-app-monitor/.env \ - --inventory deploy/tencent-tat/deploy-platform.inventory.prod.json \ + --inventory deploy/tencent_tat/deploy-platform.inventory.prod.json \ --services deploy-platform \ --tag 20260514-1800 \ --yes @@ -81,9 +82,9 @@ python3 deploy/tencent-tat/deploy_platform_tat_deploy.py deploy \ 只重启,不改镜像: ```bash -python3 deploy/tencent-tat/deploy_platform_tat_deploy.py restart \ +python3 deploy/tencent_tat/deploy_platform_tat_deploy.py restart \ --env-file /opt/hy-app-monitor/.env \ - --inventory deploy/tencent-tat/deploy-platform.inventory.prod.json \ + --inventory deploy/tencent_tat/deploy-platform.inventory.prod.json \ --services deploy-platform \ --yes ``` @@ -97,34 +98,54 @@ python3 deploy/tencent-tat/deploy_platform_tat_deploy.py restart \ 平台后续接服务端执行器时,直接调用当前仓库内的 HyApp 微服务发布脚本: ```bash -python3 deploy/tencent-tat/hyapp_tat_deploy.py plan \ - --inventory deploy/tencent-tat/hyapp-services.inventory.prod.example.json \ +python3 deploy/tencent_tat/hyapp_tat_deploy.py plan \ + --inventory deploy/tencent_tat/hyapp-services.inventory.prod.example.json \ --services gateway-service,room-service,user-service ``` 读取真实运行态和配置 YAML: ```bash -python3 deploy/tencent-tat/hyapp_tat_deploy.py discover \ +python3 deploy/tencent_tat/hyapp_tat_deploy.py discover \ --env-file /opt/hy-app-monitor/.env \ - --inventory deploy/tencent-tat/hyapp-services.inventory.prod.json \ + --inventory deploy/tencent_tat/hyapp-services.inventory.prod.json \ --services gateway-service \ --include-config ``` +`discover` 的远端链路固定为 TAT -> deploy 机器 -> SSH 业务机器,TAT 入口来自 inventory 的 `deploy_server.instance_id` / `deploy_server.public_ip`。 +业务机 SSH 默认使用 `root` 和 deploy 机默认 SSH key;如需指定账号或 key,可在对应 `hosts.` 上设置 `ssh_user` / `ssh_identity_file`,或通过 `DEPLOY_SERVICE_SSH_USER` / `DEPLOY_SERVICE_SSH_KEY` 注入。 + 生产使用时复制成真实 inventory: ```bash -cp deploy/tencent-tat/hyapp-services.inventory.prod.example.json deploy/tencent-tat/hyapp-services.inventory.prod.json -vi deploy/tencent-tat/hyapp-services.inventory.prod.json +cp deploy/tencent_tat/hyapp-services.inventory.prod.example.json deploy/tencent_tat/hyapp-services.inventory.prod.json +vi deploy/tencent_tat/hyapp-services.inventory.prod.json +``` + +部署前校验服务配置字段契约: + +```bash +python3 deploy/tencent_tat/validate_rendered_configs.py \ + --hyapp-root /opt/deploy/sources/hyapp-server \ + --inventory deploy/tencent_tat/hyapp-services.inventory.prod.json +``` + +如果已经渲染了生产最终配置,再校验目标 YAML 是否缺字段: + +```bash +python3 deploy/tencent_tat/validate_rendered_configs.py \ + --hyapp-root /opt/deploy/sources/hyapp-server \ + --inventory deploy/tencent_tat/hyapp-services.inventory.prod.json \ + --rendered-dir /tmp/hyapp-rendered-prod ``` 数据资源状态只读查询: ```bash -python3 deploy/tencent-tat/hyapp_data_resources.py resources \ +python3 deploy/tencent_tat/hyapp_data_resources.py resources \ --env-file /opt/hy-app-monitor/.env \ - --inventory deploy/tencent-tat/hyapp-services.inventory.prod.json + --inventory deploy/tencent_tat/hyapp-services.inventory.prod.json ``` 规则: @@ -149,7 +170,7 @@ python3 deploy/tencent-tat/hyapp_data_resources.py resources \ 测试机 Go 服务 YAML 通过只读 TAT 接口读取: ```bash -python3 deploy/tencent-tat/hyapp_testbox_config.py configs \ +python3 deploy/tencent_tat/hyapp_testbox_config.py configs \ --env-file /opt/hy-app-monitor/.env \ --services gateway-service,room-service,admin-server ``` diff --git a/deploy/tencent_tat/__init__.py b/deploy/tencent_tat/__init__.py new file mode 100644 index 0000000..e2ecf81 --- /dev/null +++ b/deploy/tencent_tat/__init__.py @@ -0,0 +1 @@ +# Package marker for Tencent TAT deployment helper tests. diff --git a/deploy/tencent-tat/deploy_platform_tat_deploy.py b/deploy/tencent_tat/deploy_platform_tat_deploy.py similarity index 99% rename from deploy/tencent-tat/deploy_platform_tat_deploy.py rename to deploy/tencent_tat/deploy_platform_tat_deploy.py index 026cafa..b025993 100644 --- a/deploy/tencent-tat/deploy_platform_tat_deploy.py +++ b/deploy/tencent_tat/deploy_platform_tat_deploy.py @@ -655,7 +655,7 @@ def execute_plan(inventory: dict[str, Any], plan: list[dict[str, Any]], *, dry_r def parse_args(argv: list[str]) -> argparse.Namespace: parser = argparse.ArgumentParser(description="Deploy deploy-platform through Tencent TAT with optional CLB drain.") parser.add_argument("action", choices=["plan", "discover", "restart", "deploy"]) - parser.add_argument("--inventory", default="deploy/tencent-tat/inventory.prod.example.json") + parser.add_argument("--inventory", default="deploy/tencent_tat/inventory.prod.example.json") parser.add_argument("--env-file", default="", help="Optional .env file with Tencent Cloud credentials.") parser.add_argument("--services", required=True, help="Comma separated service names.") parser.add_argument("--hosts", default="", help="Optional comma separated host names for one-node maintenance.") diff --git a/deploy/tencent-tat/hyapp-services.inventory.prod.example.json b/deploy/tencent_tat/hyapp-services.inventory.prod.example.json similarity index 87% rename from deploy/tencent-tat/hyapp-services.inventory.prod.example.json rename to deploy/tencent_tat/hyapp-services.inventory.prod.example.json index 3e237ed..853caff 100644 --- a/deploy/tencent-tat/hyapp-services.inventory.prod.example.json +++ b/deploy/tencent_tat/hyapp-services.inventory.prod.example.json @@ -8,9 +8,9 @@ "service_health_timeout_seconds": 150, "stabilize_seconds": 5, "managed_dependencies": { - "mysql": "TencentDB MySQL", - "redis": "TencentDB Redis", - "mq": "Tencent Cloud MQ" + "mysql": "TencentDB for MySQL private endpoint; this deployment never starts local MySQL.", + "redis": "TencentDB for Redis private endpoint; room owner route and login/risk cache use this managed Redis.", + "mq": "Tencent Cloud managed MQ; consumers only receive connection/topic config and are not deployed as local infra." }, "data_resources": { "mysql": { @@ -237,6 +237,25 @@ ], "drain_seconds": 5 } + }, + "statistics-service": { + "unit": "hyapp-statistics-service", + "container": "hyapp-statistics-service", + "env_file": "/etc/hyapp/statistics-service/docker.env", + "image_template": "${REGISTRY}/statistics-service:${TAG}", + "target_port": 13010, + "hosts": [ + "new-app-1", + "new-app-2" + ], + "clb": { + "enabled": true, + "load_balancer_id": "lb-epvnr4o0", + "listener_ids": [ + "lbl-a0myhr6q" + ], + "drain_seconds": 5 + } } } } diff --git a/deploy/tencent-tat/hyapp_data_resources.py b/deploy/tencent_tat/hyapp_data_resources.py similarity index 99% rename from deploy/tencent-tat/hyapp_data_resources.py rename to deploy/tencent_tat/hyapp_data_resources.py index 7be2905..12cefa7 100644 --- a/deploy/tencent-tat/hyapp_data_resources.py +++ b/deploy/tencent_tat/hyapp_data_resources.py @@ -894,7 +894,7 @@ def build_parser() -> argparse.ArgumentParser: parser.add_argument("action", choices=("resources",)) parser.add_argument("--dry-run", action="store_true", help="Print resources configured in inventory without calling Tencent APIs.") parser.add_argument("--env-file", default="", help="Optional .env file with Tencent Cloud credentials.") - parser.add_argument("--inventory", default="deploy/tencent-tat/hyapp-services.inventory.prod.example.json") + parser.add_argument("--inventory", default="deploy/tencent_tat/hyapp-services.inventory.prod.example.json") parser.add_argument("--max-bytes", type=int, default=24_000) parser.add_argument("--poll-seconds", type=int, default=2) parser.add_argument("--timeout-seconds", type=int, default=180) diff --git a/deploy/tencent-tat/hyapp_tat_deploy.py b/deploy/tencent_tat/hyapp_tat_deploy.py similarity index 71% rename from deploy/tencent-tat/hyapp_tat_deploy.py rename to deploy/tencent_tat/hyapp_tat_deploy.py index cc21bd4..ecf1926 100644 --- a/deploy/tencent-tat/hyapp_tat_deploy.py +++ b/deploy/tencent_tat/hyapp_tat_deploy.py @@ -8,7 +8,7 @@ import os import shlex import sys import time -from concurrent.futures import ThreadPoolExecutor, as_completed +import zlib from pathlib import Path from typing import Any @@ -41,6 +41,10 @@ def load_inventory(path: str) -> dict[str, Any]: def validate_inventory(inventory: dict[str, Any]) -> None: # 发布脚本只认明确的 host/service 映射,避免误操作同机已有 Java 项目。 + if not isinstance(inventory.get("deploy_server"), dict): + raise RuntimeError("inventory.deploy_server is required") + if not str(inventory["deploy_server"].get("instance_id") or "").strip(): + raise RuntimeError("inventory.deploy_server.instance_id is required") if not isinstance(inventory.get("hosts"), dict) or not inventory["hosts"]: raise RuntimeError("inventory.hosts is required") if not isinstance(inventory.get("services"), dict) or not inventory["services"]: @@ -509,6 +513,112 @@ def remote_restart_script(service: dict[str, Any], *, image: str, health_timeout return "\n".join(lines) + "\n" +def remote_write_config_script(service: dict[str, Any], *, content: str, restart: bool, health_timeout_seconds: int) -> str: + payload = { + "configPath": service.get("config_path") or "", + "container": service.get("container") or "", + "content": content, + "healthTimeout": int(health_timeout_seconds), + "restart": bool(restart), + "unit": service.get("unit") or "", + } + blob = base64.b64encode(zlib.compress(json.dumps(payload).encode("utf-8"))).decode("ascii") + return f"""set -euo pipefail +python3 - <<'PY' +import base64 +import datetime +import json +import os +import shutil +import stat +import subprocess +import tempfile +import time +import zlib + +payload = json.loads(zlib.decompress(base64.b64decode({blob!r})).decode("utf-8")) +config_path = str(payload.get("configPath") or "") +content = str(payload.get("content") or "") +unit = str(payload.get("unit") or "") +container = str(payload.get("container") or "") +restart = bool(payload.get("restart")) +health_timeout = int(payload.get("healthTimeout") or 150) + +if not config_path.startswith("/"): + raise RuntimeError("absolute configPath is required") +if not content.strip(): + raise RuntimeError("config content is required") +if "***" in content: + raise RuntimeError("redacted config cannot be saved") +if restart and not unit: + raise RuntimeError("unit is required for restart") + +directory = os.path.dirname(config_path) +os.makedirs(directory, exist_ok=True) +exists = os.path.exists(config_path) +backup_path = "" +old_stat = None +if exists: + old_stat = os.stat(config_path) + timestamp = datetime.datetime.utcnow().strftime("%Y%m%d%H%M%S") + backup_path = config_path + ".bak." + timestamp + shutil.copy2(config_path, backup_path) + +fd, tmp_path = tempfile.mkstemp(prefix=".config.", dir=directory, text=True) +try: + with os.fdopen(fd, "w", encoding="utf-8") as handle: + handle.write(content) + if old_stat is not None: + os.chmod(tmp_path, stat.S_IMODE(old_stat.st_mode)) + os.chown(tmp_path, old_stat.st_uid, old_stat.st_gid) + else: + os.chmod(tmp_path, 0o640) + os.replace(tmp_path, config_path) +finally: + if os.path.exists(tmp_path): + os.unlink(tmp_path) + +restarted = False +service_state = "" +container_state = "" +if restart: + subprocess.check_output(["systemctl", "restart", unit], text=True, stderr=subprocess.STDOUT, timeout=60) + restarted = True + deadline = time.time() + health_timeout + while time.time() < deadline: + service_state = subprocess.run(["systemctl", "is-active", unit], text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT).stdout.strip() + if service_state == "active": + break + time.sleep(2) + if service_state != "active": + status = subprocess.run(["systemctl", "status", unit, "--no-pager"], text=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT).stdout + raise RuntimeError("service did not become active: " + status[-1600:]) + if container: + while time.time() < deadline: + container_state = subprocess.run( + ["docker", "inspect", "--format", "{{if .State.Health}}{{.State.Health.Status}}{{else}}{{.State.Status}}{{end}}", container], + text=True, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + ).stdout.strip() + if container_state in ("healthy", "running"): + break + time.sleep(2) + if container_state not in ("healthy", "running"): + raise RuntimeError("container did not become healthy: " + container_state) + +print(json.dumps({ + "backupPath": backup_path, + "configPath": config_path, + "containerState": container_state, + "ok": True, + "restarted": restarted, + "serviceState": service_state, +}, ensure_ascii=False)) +PY +""" + + def remote_discover_script(*, host_services: list[dict[str, Any]], include_config: bool, max_config_bytes: int) -> str: # 只读取 inventory 明确声明的 hyapp unit/env/container/config,不扫描或修改 Java 项目。 payload = { @@ -736,6 +846,156 @@ PY return script.replace("__PAYLOAD__", repr(blob)) +def target_payload(inventory: dict[str, Any]) -> dict[str, Any]: + deploy_server = inventory["deploy_server"] + return { + "instanceId": str(deploy_server.get("instance_id") or ""), + "name": str(deploy_server.get("name") or "deploy"), + "privateIp": str(deploy_server.get("private_ip") or ""), + "publicIp": str(deploy_server.get("public_ip") or ""), + } + + +def remote_discover_via_deploy_script( + *, + inventory: dict[str, Any], + seen_hosts: list[str], + services_by_host: dict[str, list[dict[str, Any]]], + include_config: bool, + max_config_bytes: int, +) -> str: + hosts = [] + for host_name in seen_hosts: + host = inventory["hosts"][host_name] + hosts.append( + { + "host": host_name, + "instance_id": str(host.get("instance_id") or ""), + "private_ip": str(host.get("private_ip") or ""), + "script": remote_discover_script( + host_services=services_by_host[host_name], + include_config=include_config, + max_config_bytes=max_config_bytes, + ), + "ssh_identity_file": str(host.get("ssh_identity_file") or os.environ.get("DEPLOY_SERVICE_SSH_KEY") or ""), + "ssh_user": str(host.get("ssh_user") or os.environ.get("DEPLOY_SERVICE_SSH_USER") or "root"), + } + ) + payload = { + "hosts": hosts, + "sshConnectTimeout": int(inventory.get("discover_ssh_connect_timeout_seconds") or 5), + "sshTimeout": int(inventory.get("discover_ssh_timeout_seconds") or 45), + } + blob = base64.b64encode(zlib.compress(json.dumps(payload).encode("utf-8"))).decode("ascii") + return f"""set -euo pipefail +python3 - <<'PY' +import base64 +import json +import subprocess +import zlib +from concurrent.futures import ThreadPoolExecutor, as_completed + +payload = json.loads(zlib.decompress(base64.b64decode({blob!r})).decode("utf-8")) +hosts = payload.get("hosts") or [] +ssh_timeout = int(payload.get("sshTimeout") or 45) +connect_timeout = int(payload.get("sshConnectTimeout") or 5) + +ssh_options = [ + "-o", "BatchMode=yes", + "-o", "ConnectTimeout=" + str(connect_timeout), + "-o", "ServerAliveInterval=5", + "-o", "ServerAliveCountMax=1", + "-o", "StrictHostKeyChecking=accept-new", +] + +def discover_host(host): + host_name = str(host.get("host") or "") + private_ip = str(host.get("private_ip") or "") + identity_file = str(host.get("ssh_identity_file") or "") + ssh_user = str(host.get("ssh_user") or "root") + if not private_ip: + return {{ + "error": "private_ip missing", + "host": host_name, + "instance_id": str(host.get("instance_id") or ""), + "private_ip": "", + "services": [], + "status": "ERROR", + "transport": "ssh", + }} + try: + ssh_command = ["ssh", *ssh_options] + if identity_file: + ssh_command.extend(["-i", identity_file]) + ssh_command.extend([ssh_user + "@" + private_ip, "bash", "-s"]) + completed = subprocess.run( + ssh_command, + input=str(host.get("script") or ""), + text=True, + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + timeout=ssh_timeout, + check=False, + ) + except Exception as exc: + return {{ + "error": str(exc), + "host": host_name, + "instance_id": str(host.get("instance_id") or ""), + "private_ip": private_ip, + "services": [], + "status": "ERROR", + "transport": "ssh", + }} + output = completed.stdout or "" + if completed.returncode != 0: + return {{ + "error": output[-3000:], + "host": host_name, + "instance_id": str(host.get("instance_id") or ""), + "private_ip": private_ip, + "services": [], + "status": "ERROR", + "transport": "ssh", + }} + try: + decoded = json.loads(output or "{{}}") + except Exception as exc: + return {{ + "error": "invalid ssh json: " + str(exc), + "host": host_name, + "instance_id": str(host.get("instance_id") or ""), + "output": output[-3000:], + "private_ip": private_ip, + "services": [], + "status": "ERROR", + "transport": "ssh", + }} + return {{ + "host": host_name, + "hostname": decoded.get("hostname") or "", + "instance_id": str(host.get("instance_id") or ""), + "private_ip": private_ip, + "services": decoded.get("services") or [], + "status": "SUCCESS" if decoded.get("ok") else "ERROR", + "transport": "ssh", + }} + +host_results = {{}} +max_workers = max(1, min(len(hosts), 6)) +with ThreadPoolExecutor(max_workers=max_workers) as executor: + futures = {{executor.submit(discover_host, host): str(host.get("host") or "") for host in hosts}} + for future in as_completed(futures): + host_results[futures[future]] = future.result() + +print(json.dumps({{ + "hosts": [host_results[str(host.get("host") or "")] for host in hosts], + "ok": True, +}}, ensure_ascii=False)) +PY +""" + + def execute_discovery( inventory: dict[str, Any], plan: list[dict[str, Any]], @@ -745,60 +1005,36 @@ def execute_discovery( max_config_bytes: int, ) -> dict[str, Any]: if dry_run: - return {"ok": True, "dry_run": True, "steps": plan} + return {"ok": True, "dry_run": True, "steps": plan, "target": target_payload(inventory)} clb_client = build_clb_client(inventory) seen_hosts = list(dict.fromkeys(step["host"] for step in plan)) - by_host = {name: next(step for step in plan if step["host"] == name) for name in seen_hosts} services_by_host = {name: [step for step in plan if step["host"] == name] for name in seen_hosts} - - def discover_host(host_name: str, seed: dict[str, Any]) -> dict[str, Any]: - try: - tat_client = build_tat_client(inventory) - result = run_tat_shell( - tat_client, - instance_id=seed["instance_id"], - script=remote_discover_script( - host_services=services_by_host[host_name], - include_config=include_config, - max_config_bytes=max_config_bytes, - ), - command_name=f"hyapp-discover-{host_name}", - timeout_seconds=int(inventory.get("tat_timeout_seconds") or 900), - poll_seconds=int(inventory.get("tat_poll_seconds") or 2), - ) - output = str(result.get("output") or "") - try: - payload = json.loads(output or "{}") - except json.JSONDecodeError: - payload = {} - services = list(payload.get("services") or []) - return { - "host": host_name, - "instance_id": seed["instance_id"], - "private_ip": seed["private_ip"], - "status": result["status"], - "output": "" if services else output[-5000:], - "services": services, - } - except Exception as exc: # noqa: BLE001 - return { - "host": host_name, - "instance_id": seed["instance_id"], - "private_ip": seed["private_ip"], - "status": "ERROR", - "error": str(exc), - "services": [], - } - - host_results: dict[str, dict[str, Any]] = {} - max_workers = max(1, min(len(by_host), int(inventory.get("discover_parallelism") or 6))) - with ThreadPoolExecutor(max_workers=max_workers) as executor: - future_to_host = {executor.submit(discover_host, host_name, seed): host_name for host_name, seed in by_host.items()} - for future in as_completed(future_to_host): - host_name = future_to_host[future] - host_results[host_name] = future.result() - - hosts = [host_results[name] for name in seen_hosts] + deploy_server = inventory["deploy_server"] + deploy_instance_id = str(deploy_server.get("instance_id") or "").strip() + tat_client = build_tat_client(inventory) + result = run_tat_shell( + tat_client, + instance_id=deploy_instance_id, + script=remote_discover_via_deploy_script( + inventory=inventory, + seen_hosts=seen_hosts, + services_by_host=services_by_host, + include_config=include_config, + max_config_bytes=max_config_bytes, + ), + command_name="hyapp-discover-via-deploy", + timeout_seconds=int(inventory.get("tat_timeout_seconds") or 900), + poll_seconds=int(inventory.get("tat_poll_seconds") or 2), + ) + tat = {"instanceId": deploy_instance_id, "status": result["status"], "target": target_payload(inventory)} + if result["status"] != "SUCCESS": + return {"error": "TAT command failed", "hosts": [], "ok": False, "output": result.get("output") or "", "tat": tat} + output = str(result.get("output") or "") + try: + payload = json.loads(output or "{}") + except json.JSONDecodeError: + return {"error": "invalid TAT json output", "hosts": [], "ok": False, "output": output, "tat": tat} + hosts = list(payload.get("hosts") or []) clb_bindings: list[dict[str, Any]] = [] for step in plan: @@ -824,7 +1060,7 @@ def execute_discovery( "error": str(exc), } ) - return {"ok": True, "hosts": hosts, "clb": clb_bindings} + return {"ok": True, "hosts": hosts, "clb": clb_bindings, "target": target_payload(inventory), "tat": tat} def execute_plan(inventory: dict[str, Any], plan: list[dict[str, Any]], *, dry_run: bool, assume_yes: bool) -> dict[str, Any]: @@ -887,10 +1123,95 @@ def execute_plan(inventory: dict[str, Any], plan: list[dict[str, Any]], *, dry_r return {"ok": True, "steps": results} +def execute_write_config( + inventory: dict[str, Any], + plan: list[dict[str, Any]], + *, + content: str, + dry_run: bool, + restart: bool, + assume_yes: bool, +) -> dict[str, Any]: + if dry_run: + return { + "contentBytes": len(content.encode("utf-8")), + "dry_run": True, + "ok": True, + "restart": restart, + "steps": plan, + } + if not assume_yes: + raise RuntimeError("real execution requires --yes") + if not content.strip(): + raise RuntimeError("--content-base64 is required") + if "***" in content: + raise RuntimeError("redacted config cannot be saved; replace masked secret fields before saving") + + tat_client = build_tat_client(inventory) + clb_client = build_clb_client(inventory) if restart and any(step["clb_enabled"] for step in plan) else None + results: list[dict[str, Any]] = [] + for step in plan: + started = time.time() + record = dict(step) + snapshot: dict[str, Any] | None = None + try: + if restart and step["clb_enabled"]: + snapshot = prepare_clb_snapshot(clb_client, step) + set_clb_snapshot_weight(clb_client, inventory, snapshot, 0) + if step["drain_seconds"] > 0: + time.sleep(step["drain_seconds"]) + record["clb_drained"] = True + + service_cfg = inventory["services"][step["service"]] + script = remote_write_config_script( + service_cfg, + content=content, + restart=restart, + health_timeout_seconds=int(inventory.get("service_health_timeout_seconds") or 150), + ) + remote = run_tat_shell( + tat_client, + instance_id=step["instance_id"], + script=script, + command_name=f"hyapp-config-{step['service']}-{step['host']}", + timeout_seconds=int(inventory.get("tat_timeout_seconds") or 900), + poll_seconds=int(inventory.get("tat_poll_seconds") or 2), + ) + record["remote_status"] = remote["status"] + record["remote_output"] = str(remote.get("output") or "")[-2000:] + if remote["status"] != "SUCCESS": + raise RuntimeError(f"TAT command failed: {remote['status']}") + try: + remote_payload = json.loads(str(remote.get("output") or "{}")) + except json.JSONDecodeError: + remote_payload = {} + record["backup_path"] = remote_payload.get("backupPath") or "" + record["config_path"] = remote_payload.get("configPath") or record.get("config_path") or "" + record["restarted"] = bool(remote_payload.get("restarted")) + + if snapshot is not None: + set_clb_snapshot_weight(clb_client, inventory, snapshot, None) + record["clb_restored"] = True + + stabilize = int(inventory.get("stabilize_seconds") or 0) + if restart and stabilize > 0: + time.sleep(stabilize) + record["status"] = "success" + record["duration_seconds"] = round(time.time() - started, 2) + results.append(record) + except Exception as exc: # noqa: BLE001 + record["status"] = "failed" + record["error"] = str(exc) + record["duration_seconds"] = round(time.time() - started, 2) + results.append(record) + return {"error": str(exc), "ok": False, "steps": results} + return {"ok": True, "restart": restart, "steps": results} + + def parse_args(argv: list[str]) -> argparse.Namespace: parser = argparse.ArgumentParser(description="Deploy hyapp services through Tencent TAT with optional CLB drain.") - parser.add_argument("action", choices=["plan", "discover", "restart", "deploy"]) - parser.add_argument("--inventory", default="deploy/tencent-tat/hyapp-services.inventory.prod.example.json") + parser.add_argument("action", choices=["plan", "discover", "restart", "deploy", "write-config"]) + parser.add_argument("--inventory", default="deploy/tencent_tat/hyapp-services.inventory.prod.example.json") parser.add_argument("--env-file", default="", help="Optional .env file with Tencent Cloud credentials.") parser.add_argument("--services", required=True, help="Comma separated service names.") parser.add_argument("--hosts", default="", help="Optional comma separated host names for one-node maintenance.") @@ -900,6 +1221,8 @@ def parse_args(argv: list[str]) -> argparse.Namespace: parser.add_argument("--dry-run", action="store_true", help="Print the execution plan without calling Tencent APIs.") parser.add_argument("--include-config", action="store_true", help="Include redacted service config YAML in discover output.") parser.add_argument("--max-config-bytes", type=int, default=12_000, help="Maximum bytes to read from each config file during discover.") + parser.add_argument("--content-base64", default="", help="Base64 encoded config content for write-config.") + parser.add_argument("--restart", action="store_true", help="Restart the service after write-config.") parser.add_argument("--yes", action="store_true", help="Required for real restart/deploy execution.") return parser.parse_args(argv) @@ -934,6 +1257,21 @@ def main(argv: list[str]) -> int: ) print(json.dumps(result, ensure_ascii=False, indent=2)) return 0 if result.get("ok") else 1 + if args.action == "write-config": + try: + content = base64.b64decode(args.content_base64).decode("utf-8") if args.content_base64 else "" + except Exception as exc: + raise RuntimeError("--content-base64 must be valid utf-8 base64") from exc + result = execute_write_config( + inventory, + plan, + content=content, + dry_run=args.dry_run, + restart=args.restart, + assume_yes=args.yes, + ) + print(json.dumps(result, ensure_ascii=False, indent=2)) + return 0 if result.get("ok") else 1 result = execute_plan(inventory, plan, dry_run=args.dry_run, assume_yes=args.yes) print(json.dumps(result, ensure_ascii=False, indent=2)) return 0 if result.get("ok") else 1 diff --git a/deploy/tencent-tat/hyapp_testbox_config.py b/deploy/tencent_tat/hyapp_testbox_config.py similarity index 95% rename from deploy/tencent-tat/hyapp_testbox_config.py rename to deploy/tencent_tat/hyapp_testbox_config.py index 2ddcc95..b0f0852 100755 --- a/deploy/tencent-tat/hyapp_testbox_config.py +++ b/deploy/tencent_tat/hyapp_testbox_config.py @@ -345,23 +345,7 @@ def inspect_container(name): service_states = [] for service in services: - if service == "admin-server": - active = command_output(["systemctl", "is-active", "hyapp-admin-server.service"], timeout=5) - service_states.append({{ - "composeService": "hyapp-admin-server.service", - "configPath": service_path(service), - "exists": active not in ("inactive", "unknown"), - "health": "systemd", - "image": "systemd", - "name": "hyapp-admin-server.service", - "restartCount": 0, - "service": service, - "startedAt": command_output(["systemctl", "show", "hyapp-admin-server.service", "--property=ActiveEnterTimestamp", "--value"], timeout=5), - "state": active, - "status": "healthy" if active == "active" else ("unknown" if active.startswith("Command") else "critical"), - }}) - else: - service_states.append(inspect_container(service)) + service_states.append(inspect_container(service)) remote_test_raw = command_output(["git", "-C", source, "ls-remote", "origin", "refs/heads/test"], timeout=15) @@ -421,8 +405,7 @@ def compose_args(): args.extend(["-f", override]) return args -go_services = [service for service in services if service != "admin-server"] -has_admin = "admin-server" in services +go_services = services commands = [] ok = True @@ -440,8 +423,6 @@ if operation == "update": record(run(["git", "pull", "--ff-only", "origin", "test"], cwd=source, timeout=180)) if ok and go_services: record(run(compose_args() + ["up", "-d", "--build"] + go_services, cwd=source, timeout=1200)) - if ok and has_admin: - record(run(["systemctl", "restart", "hyapp-admin-server.service"], timeout=180)) if ok and not services: record(run([os.path.join(root, "bin", "deploy.sh")], timeout=1800)) elif operation in ("restart", "start", "stop"): @@ -449,8 +430,6 @@ elif operation in ("restart", "start", "stop"): compose_operation = "up" if operation == "start" else operation extra = ["-d"] + go_services if operation == "start" else go_services record(run(compose_args() + [compose_operation] + extra, cwd=source, timeout=420)) - if has_admin: - record(run(["systemctl", operation, "hyapp-admin-server.service"], timeout=180)) else: ok = False commands.append({{"args": [], "code": 1, "output": f"unsupported operation: {{operation}}"}}) @@ -562,10 +541,7 @@ else: commands = [] if restart: - if service == "admin-server": - commands.append(run(["systemctl", "restart", "hyapp-admin-server.service"], timeout=180)) - else: - commands.append(run(compose_args() + ["restart", service], cwd=source, timeout=300)) + commands.append(run(compose_args() + ["restart", service], cwd=source, timeout=300)) print(json.dumps({{ "backupPath": backup, diff --git a/deploy/tencent-tat/inventory.prod.example.json b/deploy/tencent_tat/inventory.prod.example.json similarity index 100% rename from deploy/tencent-tat/inventory.prod.example.json rename to deploy/tencent_tat/inventory.prod.example.json diff --git a/deploy/tencent_tat/tests/test_hyapp_tat_deploy.py b/deploy/tencent_tat/tests/test_hyapp_tat_deploy.py new file mode 100644 index 0000000..bbe7609 --- /dev/null +++ b/deploy/tencent_tat/tests/test_hyapp_tat_deploy.py @@ -0,0 +1,116 @@ +from __future__ import annotations + +import json +import sys +import tempfile +import unittest +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parents[3])) + +from deploy.tencent_tat import hyapp_tat_deploy as deploy + + +class TatDeployPlanTests(unittest.TestCase): + def load_inventory(self) -> dict: + return deploy.load_inventory("deploy/tencent_tat/hyapp-services.inventory.prod.example.json") + + def test_inventory_contains_current_cvm_topology(self) -> None: + inventory = self.load_inventory() + self.assertEqual(inventory["hosts"]["gateway-1"]["instance_id"], "ins-aeyaoj40") + self.assertEqual(inventory["hosts"]["gateway-2"]["private_ip"], "10.2.2.17") + self.assertEqual(inventory["services"]["wallet-service"]["hosts"], ["pay-1", "pay-2"]) + self.assertEqual(inventory["services"]["game-service"]["target_port"], 13008) + self.assertEqual(inventory["services"]["notice-service"]["hosts"], ["new-app-activity-1", "new-app-activity-2"]) + self.assertEqual(inventory["services"]["notice-service"]["target_port"], 13009) + self.assertEqual(inventory["services"]["statistics-service"]["target_port"], 13010) + self.assertIn("TencentDB for MySQL", inventory["managed_dependencies"]["mysql"]) + + def test_plan_is_service_first_and_keeps_clb_enabled(self) -> None: + inventory = self.load_inventory() + plan = deploy.build_plan( + inventory, + action="deploy", + service_names=["gateway-service", "wallet-service"], + raw_hosts="", + tag="20260513", + overrides={}, + no_clb=False, + ) + self.assertEqual( + [(step["service"], step["host"]) for step in plan], + [ + ("gateway-service", "gateway-1"), + ("gateway-service", "gateway-2"), + ("wallet-service", "pay-1"), + ("wallet-service", "pay-2"), + ], + ) + self.assertTrue(all(step["clb_enabled"] for step in plan)) + self.assertEqual(plan[0]["image"], "10.2.1.3:18082/hyapp/gateway-service:20260513") + + def test_host_filter_must_belong_to_service(self) -> None: + inventory = self.load_inventory() + with self.assertRaisesRegex(RuntimeError, "not assigned"): + deploy.build_plan( + inventory, + action="restart", + service_names=["gateway-service"], + raw_hosts="pay-1", + tag="", + overrides={}, + no_clb=False, + ) + + def test_remote_deploy_script_only_touches_hyapp_unit_and_env(self) -> None: + inventory = self.load_inventory() + service = inventory["services"]["gateway-service"] + script = deploy.remote_restart_script(service, image="registry/gateway:tag", health_timeout_seconds=30) + self.assertIn("systemctl restart \"$UNIT\"", script) + self.assertIn("ENV_FILE=/etc/hyapp/gateway-service/docker.env", script) + self.assertIn("docker pull \"$IMAGE\"", script) + self.assertNotIn("docker compose", script) + self.assertNotIn("java", script.lower()) + + def test_discover_plan_does_not_require_image_tag(self) -> None: + inventory = self.load_inventory() + plan = deploy.build_plan( + inventory, + action="discover", + service_names=["gateway-service"], + raw_hosts="", + tag="", + overrides={}, + no_clb=False, + ) + self.assertEqual([step["action"] for step in plan], ["discover", "discover"]) + self.assertEqual([step["image"] for step in plan], ["", ""]) + + def test_real_clb_ids_disable_auto_discovery_but_placeholders_enable_it(self) -> None: + inventory = self.load_inventory() + step = deploy.build_plan( + inventory, + action="restart", + service_names=["gateway-service"], + raw_hosts="gateway-1", + tag="", + overrides={}, + no_clb=False, + )[0] + self.assertFalse(deploy.should_auto_discover_clb(step)) + + step["clb_load_balancer_id"] = "lb-REPLACE_PUBLIC_GATEWAY" + self.assertTrue(deploy.should_auto_discover_clb(step)) + + def test_validate_inventory_rejects_unknown_service_host(self) -> None: + inventory = self.load_inventory() + inventory["services"]["gateway-service"]["hosts"] = ["missing-host"] + with tempfile.TemporaryDirectory() as tmp: + path = Path(tmp) / "inventory.json" + path.write_text(json.dumps(inventory), encoding="utf-8") + with self.assertRaisesRegex(RuntimeError, "unknown host"): + deploy.load_inventory(str(path)) + + +if __name__ == "__main__": + unittest.main() diff --git a/deploy/tencent_tat/validate_rendered_configs.py b/deploy/tencent_tat/validate_rendered_configs.py new file mode 100644 index 0000000..6001289 --- /dev/null +++ b/deploy/tencent_tat/validate_rendered_configs.py @@ -0,0 +1,136 @@ +#!/usr/bin/env python3 +from __future__ import annotations + +import argparse +import json +import os +import sys +from pathlib import Path +from typing import Any + +import yaml + + +CONFIG_FILENAMES = ("config.yaml", "config.docker.yaml", "config.tencent.example.yaml") + + +def default_hyapp_root() -> Path: + env_root = os.environ.get("HYAPP_SERVER_ROOT") or os.environ.get("HYAPP_ROOT") or "" + if env_root: + return Path(env_root).expanduser().resolve() + return (Path.cwd().parent / "hyapp-server").resolve() + + +def load_yaml(path: Path) -> dict[str, Any]: + if not path.exists(): + raise RuntimeError(f"config file missing: {path}") + data = yaml.safe_load(path.read_text(encoding="utf-8")) or {} + if not isinstance(data, dict): + raise RuntimeError(f"config file must be a mapping: {path}") + return data + + +def load_inventory(path: Path) -> dict[str, Any]: + if not path.exists(): + raise RuntimeError(f"inventory missing: {path}") + data = json.loads(path.read_text(encoding="utf-8")) + if not isinstance(data.get("services"), dict): + raise RuntimeError("inventory.services is required") + return data + + +def mapping_paths(value: Any, prefix: str = "") -> set[str]: + # 只比较 YAML object key,不比较值;密钥和环境差异不会被打印或误判。 + if not isinstance(value, dict): + return set() + paths: set[str] = set() + for key, child in value.items(): + current = f"{prefix}.{key}" if prefix else str(key) + paths.add(current) + paths.update(mapping_paths(child, current)) + return paths + + +def selected_service_names(inventory: dict[str, Any], raw_services: str) -> list[str]: + names = [item.strip() for item in raw_services.split(",") if item.strip()] + if not names: + return sorted(str(name) for name in inventory["services"].keys()) + unknown = [name for name in names if name not in inventory["services"]] + if unknown: + raise RuntimeError("unknown services: " + ", ".join(unknown)) + return list(dict.fromkeys(names)) + + +def selected_hosts(service: dict[str, Any], raw_hosts: str) -> list[str]: + hosts = [str(item) for item in service.get("hosts") or []] + requested = [item.strip() for item in raw_hosts.split(",") if item.strip()] + if not requested: + return hosts + invalid = [host for host in requested if host not in hosts] + if invalid: + raise RuntimeError("selected hosts are not assigned to this service: " + ", ".join(invalid)) + return requested + + +def validate_service_templates(hyapp_root: Path, service_name: str) -> list[str]: + errors: list[str] = [] + config_dir = hyapp_root / "services" / service_name / "configs" + configs = {name: load_yaml(config_dir / name) for name in CONFIG_FILENAMES} + contract_paths = mapping_paths(configs["config.tencent.example.yaml"]) + for filename in ("config.yaml", "config.docker.yaml"): + missing = sorted(mapping_paths(configs[filename]) - contract_paths) + if missing: + errors.append( + f"{service_name}: {filename} contains keys missing from config.tencent.example.yaml: " + + ", ".join(missing) + ) + return errors + + +def validate_rendered_config(hyapp_root: Path, service_name: str, host_name: str, rendered_dir: Path) -> list[str]: + contract = load_yaml(hyapp_root / "services" / service_name / "configs" / "config.tencent.example.yaml") + rendered_path = rendered_dir / host_name / service_name / "config.yaml" + rendered = load_yaml(rendered_path) + missing = sorted(mapping_paths(contract) - mapping_paths(rendered)) + if not missing: + return [] + return [ + f"{service_name}@{host_name}: rendered config missing keys required by config.tencent.example.yaml: " + + ", ".join(missing) + ] + + +def main(argv: list[str]) -> int: + parser = argparse.ArgumentParser(description="Validate service config contracts before TAT deployment.") + parser.add_argument("--inventory", default="deploy/tencent_tat/hyapp-services.inventory.prod.example.json") + parser.add_argument("--rendered-dir", default="", help="Directory with //config.yaml files.") + parser.add_argument("--services", default="", help="Comma separated service names. Defaults to all inventory services.") + parser.add_argument("--hosts", default="", help="Optional comma separated hosts assigned to the selected service(s).") + parser.add_argument("--hyapp-root", default="", help="Path to hyapp-server source tree. Defaults to HYAPP_SERVER_ROOT or ../hyapp-server.") + args = parser.parse_args(argv) + + try: + inventory = load_inventory(Path(args.inventory)) + hyapp_root = Path(args.hyapp_root).expanduser().resolve() if args.hyapp_root else default_hyapp_root() + if not (hyapp_root / "services").is_dir(): + raise RuntimeError(f"hyapp services directory missing: {hyapp_root / 'services'}") + rendered_dir = Path(args.rendered_dir).expanduser().resolve() if args.rendered_dir else None + errors: list[str] = [] + for service_name in selected_service_names(inventory, args.services): + errors.extend(validate_service_templates(hyapp_root, service_name)) + if rendered_dir is None: + continue + for host_name in selected_hosts(inventory["services"][service_name], args.hosts): + errors.extend(validate_rendered_config(hyapp_root, service_name, host_name, rendered_dir)) + if errors: + print(json.dumps({"ok": False, "errors": errors}, ensure_ascii=False, indent=2), file=sys.stderr) + return 1 + print(json.dumps({"ok": True}, ensure_ascii=False)) + return 0 + except Exception as exc: # noqa: BLE001 + print(json.dumps({"ok": False, "error": str(exc)}, ensure_ascii=False), file=sys.stderr) + return 1 + + +if __name__ == "__main__": + raise SystemExit(main(sys.argv[1:])) diff --git a/src/app/router.tsx b/src/app/router.tsx index 84047d7..e2ab06b 100644 --- a/src/app/router.tsx +++ b/src/app/router.tsx @@ -1,25 +1,15 @@ import { createBrowserRouter, Navigate } from "react-router-dom"; import { App } from "./App"; -import { OverviewPage } from "../pages/overview"; -import { ReleasesPage } from "../pages/releases"; -import { ResourcesPage } from "../pages/resources"; -import { ServiceDetailPage } from "../pages/service-detail"; -import { ServicesPage } from "../pages/services"; -import { TestServicesPage } from "../pages/test-services"; +import { ConfigCenterPage } from "../pages/config-center"; export const router = createBrowserRouter([ { path: "/", element: , children: [ - { index: true, element: }, - { path: "overview", element: }, - { path: "services", element: }, - { path: "services/:serviceId", element: }, - { path: "test-services", element: }, - { path: "releases", element: }, - { path: "resources", element: }, - { path: "*", element: }, + { index: true, element: }, + { path: "config-center", element: }, + { path: "*", element: }, ], }, ]); diff --git a/src/entities/deployment-plan/api/getDeployApiInventory.ts b/src/entities/deployment-plan/api/getDeployApiInventory.ts index 04740c5..5c792a3 100644 --- a/src/entities/deployment-plan/api/getDeployApiInventory.ts +++ b/src/entities/deployment-plan/api/getDeployApiInventory.ts @@ -118,6 +118,8 @@ export type DeployApiDiscoverResponse = { private_ip: string; services?: Array<{ configPath?: string; + configTruncated?: boolean; + configYaml?: string; container?: string; containerStatus?: string; cpuPercent?: number | null; @@ -139,11 +141,17 @@ export type DeployApiDiscoverResponse = { ok: boolean; }; -export async function getDeployApiDiscover(services: string[], remote: boolean): Promise { +export async function getDeployApiDiscover(services: string[], remote: boolean, includeConfig = false): Promise { const params = new URLSearchParams({ + includeConfig: includeConfig ? "1" : "0", remote: remote ? "1" : "0", services: services.join(","), }); + + if (includeConfig) { + params.set("maxConfigBytes", "50000"); + } + const response = await fetch(`/deploy/api/hyapp/discover?${params.toString()}`, { headers: { accept: "application/json", @@ -153,6 +161,56 @@ export async function getDeployApiDiscover(services: string[], remote: boolean): return readJsonResponse(response); } +export type DeployApiActionStep = { + backup_path?: string; + clb_drained?: boolean; + clb_restored?: boolean; + config_path?: string; + duration_seconds?: number; + error?: string; + host: string; + private_ip?: string; + remote_output?: string; + remote_status?: string; + restarted?: boolean; + service: string; + status?: "success" | "failed"; +}; + +export type DeployApiConfigWriteResponse = { + ok: boolean; + restart?: boolean; + steps: DeployApiActionStep[]; +}; + +export function saveDeployApiServiceConfig(params: { + content: string; + hosts?: string[]; + restart: boolean; + service: string; +}): Promise { + return postJson("/deploy/api/hyapp/config", { + confirmed: true, + content: params.content, + hosts: params.hosts ?? [], + restart: params.restart, + service: params.service, + }); +} + +export type DeployApiRestartResponse = { + ok: boolean; + steps: DeployApiActionStep[]; +}; + +export function restartDeployApiService(params: { hosts?: string[]; service: string }): Promise { + return postJson("/deploy/api/hyapp/restart", { + confirmed: true, + hosts: params.hosts ?? [], + services: [params.service], + }); +} + export type DeployApiTestboxConfigResponse = { deployedCommit?: string; files: Array<{ diff --git a/src/entities/deployment-plan/index.ts b/src/entities/deployment-plan/index.ts index 7bbbb64..3e4454a 100644 --- a/src/entities/deployment-plan/index.ts +++ b/src/entities/deployment-plan/index.ts @@ -8,14 +8,20 @@ export { } from "./api/useDeployApiQueries"; export { applyDeployApiTestboxMigrations, + getDeployApiDiscover, + restartDeployApiService, runDeployApiTestboxServiceAction, runDeployApiTestboxTimerAction, + saveDeployApiServiceConfig, saveDeployApiTestboxConfig, } from "./api/getDeployApiInventory"; export type { + DeployApiActionStep, + DeployApiConfigWriteResponse, DeployApiDiscoverResponse, DeployApiInventoryResponse, DeployApiPlanResponse, + DeployApiRestartResponse, DeployApiTestboxCommand, DeployApiTestboxConfigResponse, DeployApiTestboxConfigWriteResponse, diff --git a/src/entities/service/api/deployApi.ts b/src/entities/service/api/deployApi.ts index 028e715..74a8523 100644 --- a/src/entities/service/api/deployApi.ts +++ b/src/entities/service/api/deployApi.ts @@ -13,6 +13,8 @@ export type DeployInventoryService = { export type DeployInventoryHost = { instance_id?: string; private_ip?: string; + ssh_identity_file?: string; + ssh_user?: string; }; export type DeployInventoryResponse = { diff --git a/src/features/release-service/ui/ReleaseDrawer.tsx b/src/features/release-service/ui/ReleaseDrawer.tsx index 490d468..51f06f5 100644 --- a/src/features/release-service/ui/ReleaseDrawer.tsx +++ b/src/features/release-service/ui/ReleaseDrawer.tsx @@ -60,9 +60,9 @@ export function ReleaseDrawer({ clusterName, environment, onClose, service, user const canSubmit = targetVersion !== "" && targetVersion !== service.version && hasExecutableTarget && (!isProd || confirmed); const hasWarning = prechecks.some((item) => item.result === "warning"); const deployCommand = [ - "python3 deploy/tencent-tat/hyapp_tat_deploy.py deploy", + "python3 deploy/tencent_tat/hyapp_tat_deploy.py deploy", " --env-file /opt/hy-app-monitor/.env", - " --inventory deploy/tencent-tat/hyapp-services.inventory.prod.json", + " --inventory deploy/tencent_tat/hyapp-services.inventory.prod.json", ` --services ${service.name}`, ` --tag ${targetVersion || ""}`, isProd ? " --yes" : " --dry-run", diff --git a/src/pages/config-center/ConfigCenterPage.module.css b/src/pages/config-center/ConfigCenterPage.module.css new file mode 100644 index 0000000..8b9ffe3 --- /dev/null +++ b/src/pages/config-center/ConfigCenterPage.module.css @@ -0,0 +1,344 @@ +.root { + height: 100%; + min-height: 0; +} + +.headerActions { + display: flex; + align-items: center; + gap: var(--space-2); +} + +.banner { + display: flex; + align-items: center; + gap: var(--space-2); + min-height: 36px; + padding: 0 var(--space-4); + border: 1px solid rgba(229, 72, 77, 0.2); + border-radius: 8px; + color: var(--color-danger); + background: var(--color-danger-bg); +} + +.success { + border-color: rgba(24, 169, 87, 0.22); + color: var(--color-success); + background: var(--color-success-bg); +} + +.workspace { + display: grid; + flex: 1; + min-height: 0; + grid-template-columns: 300px minmax(0, 1fr) 280px; + gap: var(--space-4); +} + +.servicePanel, +.editorPanel, +.instancePanel { + min-height: 0; + border: 1px solid rgba(255, 255, 255, 0.72); + border-radius: 8px; + background: + linear-gradient(135deg, rgba(255, 255, 255, 0.66), rgba(245, 250, 255, 0.38)), + rgba(255, 255, 255, 0.36); + box-shadow: var(--shadow-glass-sm); + backdrop-filter: blur(24px) saturate(165%); + -webkit-backdrop-filter: blur(24px) saturate(165%); +} + +.servicePanel, +.instancePanel { + display: flex; + flex-direction: column; + padding: var(--space-3); +} + +.search { + display: flex; + height: 36px; + align-items: center; + gap: var(--space-2); + padding: 0 var(--space-3); + border: 1px solid var(--color-border-soft); + border-radius: 8px; + color: var(--color-text-tertiary); + background: rgba(255, 255, 255, 0.54); +} + +.search input { + width: 100%; + min-width: 0; + border: 0; + color: var(--color-text-primary); + background: transparent; + outline: 0; +} + +.serviceList, +.instanceList { + display: flex; + min-height: 0; + flex: 1; + flex-direction: column; + gap: var(--space-2); + overflow-y: auto; + padding-top: var(--space-3); +} + +.serviceItem { + display: grid; + grid-template-columns: minmax(0, 1fr) auto; + gap: var(--space-3); + min-height: 68px; + align-items: center; + padding: var(--space-3); + border: 1px solid transparent; + border-radius: 8px; + color: var(--color-text-primary); + text-align: left; + background: transparent; + transition: + background var(--duration-fast) var(--ease-standard), + border-color var(--duration-fast) var(--ease-standard), + box-shadow var(--duration-fast) var(--ease-standard); +} + +.serviceItem:hover, +.serviceItemActive { + border-color: rgba(47, 125, 246, 0.18); + background: rgba(255, 255, 255, 0.64); + box-shadow: var(--shadow-liquid-control); +} + +.serviceItem strong, +.serviceItem small { + display: block; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.serviceItem strong { + font-size: 13px; + line-height: 18px; +} + +.serviceItem small { + margin-top: 5px; + color: var(--color-text-tertiary); + font-size: 12px; + line-height: 16px; +} + +.serviceItem em { + min-width: 46px; + padding: 4px 8px; + border-radius: 999px; + color: var(--color-brand); + font-size: 12px; + font-style: normal; + font-weight: 800; + text-align: center; + background: var(--color-brand-soft); +} + +.editorPanel { + display: flex; + flex-direction: column; + overflow: hidden; +} + +.editorTop { + display: flex; + min-height: 64px; + align-items: center; + justify-content: space-between; + gap: var(--space-3); + padding: var(--space-3) var(--space-4); + border-bottom: 1px solid var(--color-border-soft); +} + +.serviceTitle { + display: flex; + min-width: 0; + align-items: center; + gap: var(--space-2); +} + +.serviceTitle > svg { + flex: 0 0 auto; + color: var(--color-brand); +} + +.serviceTitle strong, +.serviceTitle span { + display: block; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.serviceTitle strong { + font-size: 15px; + line-height: 20px; +} + +.serviceTitle span { + color: var(--color-text-tertiary); + font-size: 12px; + line-height: 17px; +} + +.actionGroup { + display: flex; + flex: 0 0 auto; + align-items: center; + gap: var(--space-2); +} + +.restartToggle { + display: inline-flex; + align-items: center; + gap: 7px; + color: var(--color-text-secondary); + font-size: 12px; + font-weight: 700; + white-space: nowrap; +} + +.hostStrip { + display: flex; + flex: 0 0 auto; + gap: var(--space-2); + overflow-x: auto; + padding: var(--space-3) var(--space-4); + border-bottom: 1px solid var(--color-border-soft); +} + +.hostChip { + height: 30px; + padding: 0 12px; + border: 1px solid var(--color-border-soft); + border-radius: 999px; + color: var(--color-text-secondary); + font-size: 12px; + font-weight: 800; + white-space: nowrap; + background: rgba(255, 255, 255, 0.42); +} + +.hostChipActive { + color: #fff; + border-color: rgba(47, 125, 246, 0.2); + background: var(--color-brand); +} + +.editor { + flex: 1; + min-height: 0; + width: 100%; + resize: none; + border: 0; + padding: var(--space-4); + color: #182536; + background: rgba(255, 255, 255, 0.54); + font-family: "SFMono-Regular", Consolas, "Liberation Mono", monospace; + font-size: 13px; + line-height: 20px; + outline: 0; + tab-size: 2; +} + +.editor:disabled { + color: var(--color-text-tertiary); +} + +.warning { + padding: 9px var(--space-4); + border-top: 1px solid rgba(245, 165, 36, 0.22); + color: #8a5a00; + font-size: 12px; + font-weight: 700; + background: var(--color-warning-bg); +} + +.panelTitle { + display: flex; + height: 36px; + align-items: center; + gap: var(--space-2); + color: var(--color-text-primary); +} + +.panelTitle svg { + color: var(--color-brand); +} + +.instanceItem { + position: relative; + display: grid; + gap: var(--space-3); + padding: var(--space-3); + border: 1px solid var(--color-border-soft); + border-radius: 8px; + background: rgba(255, 255, 255, 0.5); +} + +.instanceItem > div:first-child strong, +.instanceItem > div:first-child span { + display: block; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.instanceItem > div:first-child span { + margin-top: 3px; + color: var(--color-text-tertiary); + font-size: 12px; +} + +.instanceItem dl { + display: grid; + grid-template-columns: repeat(3, minmax(0, 1fr)); + gap: var(--space-2); + margin: 0; +} + +.instanceItem dt, +.instanceItem dd { + margin: 0; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.instanceItem dt { + color: var(--color-text-tertiary); + font-size: 11px; + line-height: 15px; +} + +.instanceItem dd { + color: var(--color-text-primary); + font-size: 12px; + font-weight: 800; + line-height: 16px; +} + +.instanceItem > button { + position: absolute; + top: var(--space-3); + right: var(--space-3); +} + +.empty { + display: grid; + min-height: 120px; + place-items: center; + color: var(--color-text-tertiary); + font-weight: 700; +} diff --git a/src/pages/config-center/ConfigCenterPage.tsx b/src/pages/config-center/ConfigCenterPage.tsx new file mode 100644 index 0000000..5f8dc90 --- /dev/null +++ b/src/pages/config-center/ConfigCenterPage.tsx @@ -0,0 +1,311 @@ +import { useEffect, useMemo, useState } from "react"; +import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query"; +import { AlertTriangle, CheckCircle2, FileText, RefreshCw, RotateCcw, Save, Search, Server } from "lucide-react"; +import clsx from "clsx"; +import { + getDeployApiDiscover, + restartDeployApiService, + saveDeployApiServiceConfig, + useDeployApiInventoryQuery, + type DeployApiDiscoverResponse, +} from "../../entities/deployment-plan"; +import { queryKeys } from "../../shared/api/queryKeys"; +import { Button } from "../../shared/ui/Button"; +import { IconButton } from "../../shared/ui/IconButton"; +import { StatusBadge } from "../../shared/ui/StatusBadge"; +import styles from "./ConfigCenterPage.module.css"; + +type RuntimeService = NonNullable[number]["services"]>[number] & { + host: string; + privateIp: string; +}; + +function runtimeServices(discover: DeployApiDiscoverResponse | undefined): RuntimeService[] { + return (discover?.hosts ?? []).flatMap((host) => + (host.services ?? []).map((service) => ({ + ...service, + host: host.host, + privateIp: host.private_ip, + })), + ); +} + +function statusText(status: string | undefined) { + const labels: Record = { + critical: "异常", + healthy: "正常", + running: "启动中", + unknown: "未知", + warning: "告警", + }; + + return labels[status ?? ""] ?? "未知"; +} + +function firstAvailableConfig(runtimes: RuntimeService[]) { + return runtimes.find((runtime) => runtime.configYaml)?.configYaml ?? ""; +} + +export function ConfigCenterPage() { + const queryClient = useQueryClient(); + const inventoryQuery = useDeployApiInventoryQuery(); + const serviceNames = useMemo(() => Object.keys(inventoryQuery.data?.services ?? {}), [inventoryQuery.data?.services]); + const [filter, setFilter] = useState(""); + const [selectedService, setSelectedService] = useState(""); + const [selectedHost, setSelectedHost] = useState("all"); + const [draft, setDraft] = useState(""); + const [restartAfterSave, setRestartAfterSave] = useState(false); + + const discoverQuery = useQuery({ + enabled: serviceNames.length > 0, + queryFn: () => getDeployApiDiscover(serviceNames, true, true), + queryKey: queryKeys.configCenterDiscover(serviceNames), + retry: 0, + }); + + const runtimes = useMemo(() => runtimeServices(discoverQuery.data), [discoverQuery.data]); + const filteredServices = useMemo( + () => serviceNames.filter((service) => service.toLowerCase().includes(filter.trim().toLowerCase())), + [filter, serviceNames], + ); + + useEffect(() => { + const firstService = serviceNames[0]; + + if (!selectedService && firstService) { + setSelectedService(firstService); + } + }, [selectedService, serviceNames]); + + const selectedRuntimes = useMemo( + () => runtimes.filter((runtime) => runtime.name === selectedService), + [runtimes, selectedService], + ); + const selectedRuntime = + selectedHost === "all" + ? selectedRuntimes.find((runtime) => runtime.configYaml) ?? selectedRuntimes[0] + : selectedRuntimes.find((runtime) => runtime.host === selectedHost); + const activeConfig = selectedHost === "all" ? firstAvailableConfig(selectedRuntimes) : selectedRuntime?.configYaml ?? ""; + const selectedInventory = selectedService ? inventoryQuery.data?.services?.[selectedService] : undefined; + const configPath = selectedRuntime?.configPath ?? selectedInventory?.config_path ?? ""; + const dirty = draft !== activeConfig; + const hasRedactedValue = draft.includes("***"); + const canOperate = Boolean(selectedService) && !inventoryQuery.isLoading && !discoverQuery.isLoading; + + useEffect(() => { + setSelectedHost("all"); + }, [selectedService]); + + useEffect(() => { + setDraft(activeConfig); + }, [activeConfig, selectedHost, selectedService]); + + const invalidateConfig = () => { + void queryClient.invalidateQueries({ queryKey: queryKeys.configCenterDiscover(serviceNames) }); + void queryClient.invalidateQueries({ queryKey: queryKeys.deployApiDiscover(serviceNames) }); + }; + + const saveMutation = useMutation({ + mutationFn: () => + saveDeployApiServiceConfig({ + content: draft, + hosts: selectedHost === "all" ? [] : [selectedHost], + restart: restartAfterSave, + service: selectedService, + }), + onSuccess: invalidateConfig, + }); + + const restartMutation = useMutation({ + mutationFn: (hosts: string[]) => + restartDeployApiService({ + hosts, + service: selectedService, + }), + onSuccess: invalidateConfig, + }); + + const isBusy = saveMutation.isPending || restartMutation.isPending; + const error = inventoryQuery.error ?? discoverQuery.error ?? saveMutation.error ?? restartMutation.error; + const successText = saveMutation.isSuccess + ? restartAfterSave + ? "配置已保存并重启" + : "配置已保存" + : restartMutation.isSuccess + ? "服务已重启" + : ""; + + return ( +
+
+
+

配置中心

+
+
+ +
+
+ + {error ? ( +
+ + {error instanceof Error ? error.message : "操作失败"} +
+ ) : null} + + {successText ? ( +
+ + {successText} +
+ ) : null} + +
+ + +
+
+
+ +
+ {selectedService || "未选择服务"} + {configPath || "未采集配置路径"} +
+
+
+ + + +
+
+ +
+ + {selectedRuntimes.map((runtime) => ( + + ))} +
+ +