更新功能,增加compose部署
This commit is contained in:
parent
e700149a99
commit
34b976b098
14
.env.example
14
.env.example
@ -8,12 +8,26 @@ PROBE_MAX_WORKERS=12
|
|||||||
HOST_METRIC_CACHE_TTL_SECONDS=25
|
HOST_METRIC_CACHE_TTL_SECONDS=25
|
||||||
TAT_METRIC_TIMEOUT_SECONDS=30
|
TAT_METRIC_TIMEOUT_SECONDS=30
|
||||||
TAT_CPU_SAMPLE_SECONDS=1
|
TAT_CPU_SAMPLE_SECONDS=1
|
||||||
|
ROLLING_RESTART_TIMEOUT_SECONDS=180
|
||||||
|
ROLLING_RESTART_POLL_SECONDS=2
|
||||||
|
ROLLING_RESTART_STABILIZE_SECONDS=2
|
||||||
|
SERVICE_BUILD_TIMEOUT_SECONDS=3600
|
||||||
|
SERVICE_PULL_TIMEOUT_SECONDS=600
|
||||||
NACOS_BASE_URL=http://10.2.1.17:8848
|
NACOS_BASE_URL=http://10.2.1.17:8848
|
||||||
NACOS_NAMESPACE_ID=f7a8594e-090c-4830-a3ad-b5c0f9afcc07
|
NACOS_NAMESPACE_ID=f7a8594e-090c-4830-a3ad-b5c0f9afcc07
|
||||||
NACOS_NAMESPACE_NAME=RedCircleServiceProd
|
NACOS_NAMESPACE_NAME=RedCircleServiceProd
|
||||||
NACOS_DISCOVERY_GROUP=DEFAULT_GROUP
|
NACOS_DISCOVERY_GROUP=DEFAULT_GROUP
|
||||||
NACOS_DISCOVERY_CLUSTER_NAME=DEFAULT
|
NACOS_DISCOVERY_CLUSTER_NAME=DEFAULT
|
||||||
NACOS_CACHE_TTL_SECONDS=15
|
NACOS_CACHE_TTL_SECONDS=15
|
||||||
|
HARBOR_REGISTRY=10.2.1.3:18082
|
||||||
|
HARBOR_PROJECT=likei
|
||||||
|
HARBOR_USERNAME=
|
||||||
|
HARBOR_PASSWORD=
|
||||||
|
BUILD_DOCKER_PLATFORM=linux/amd64
|
||||||
|
UPDATE_DEFAULT_JAVA_GIT_REF=main
|
||||||
|
UPDATE_DEFAULT_GO_GIT_REF=main
|
||||||
|
JAVA_REPO_ROOT=/opt/deploy/sources/chatapp3-java
|
||||||
|
GOLANG_REPO_ROOT=/opt/deploy/sources/chatapp3-golang
|
||||||
MONGO_URI=
|
MONGO_URI=
|
||||||
MONGO_DB_NAME=tarab_all
|
MONGO_DB_NAME=tarab_all
|
||||||
MONGO_INSTANCE_ID=ins-o9mmob66
|
MONGO_INSTANCE_ID=ins-o9mmob66
|
||||||
|
|||||||
49
README.md
49
README.md
@ -39,6 +39,19 @@ TENCENT_SECRET_ID=***
|
|||||||
TENCENT_SECRET_KEY=***
|
TENCENT_SECRET_KEY=***
|
||||||
```
|
```
|
||||||
|
|
||||||
|
如果你要直接在页面里做“构建镜像 -> 推 Harbor -> 更新服务”,还需要补这几项:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
HARBOR_REGISTRY=10.2.1.3:18082
|
||||||
|
HARBOR_PROJECT=likei
|
||||||
|
HARBOR_USERNAME=***
|
||||||
|
HARBOR_PASSWORD=***
|
||||||
|
JAVA_REPO_ROOT=/opt/deploy/sources/chatapp3-java
|
||||||
|
GOLANG_REPO_ROOT=/opt/deploy/sources/chatapp3-golang
|
||||||
|
UPDATE_DEFAULT_JAVA_GIT_REF=main
|
||||||
|
UPDATE_DEFAULT_GO_GIT_REF=main
|
||||||
|
```
|
||||||
|
|
||||||
可以直接复制模板:
|
可以直接复制模板:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@ -101,6 +114,41 @@ python3 ops/deploy_via_tat.py
|
|||||||
5. 安装并启用 `systemd`
|
5. 安装并启用 `systemd`
|
||||||
6. 重启服务并检查 `2026` 监听
|
6. 重启服务并检查 `2026` 监听
|
||||||
|
|
||||||
|
## 维护完整 Compose
|
||||||
|
|
||||||
|
默认只刷新本地完整模板:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 ops/sync_runtime_compose.py
|
||||||
|
```
|
||||||
|
|
||||||
|
如果要把完整模板同步到远端业务主机:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 ops/sync_runtime_compose.py --sync-remote
|
||||||
|
```
|
||||||
|
|
||||||
|
模板文件会写到本地 `run/compose_templates/`。
|
||||||
|
滚动重启在真正执行 `docker compose up -d --force-recreate --no-deps <service>` 之前,也会先把对应宿主机的完整 `docker-compose.yml` 同步到远端,避免线上 compose 被裁剪成子集。
|
||||||
|
|
||||||
|
## 服务更新
|
||||||
|
|
||||||
|
页面里的“服务更新”会直接在当前机器上执行这条链路:
|
||||||
|
|
||||||
|
1. 从 `JAVA_REPO_ROOT` / `GOLANG_REPO_ROOT` 指向的业务仓按 `git ref` 创建临时 `git worktree`
|
||||||
|
2. 本地构建镜像并推送到 Harbor
|
||||||
|
3. 取回 Harbor `digest`
|
||||||
|
4. 用新 `digest` 更新本地完整 compose 模板
|
||||||
|
5. 把完整 compose 下发到目标宿主机
|
||||||
|
6. 先在目标机 `docker pull` 新镜像
|
||||||
|
7. 最后只对你选中的服务执行 `docker compose up -d --force-recreate --no-deps <service>`
|
||||||
|
|
||||||
|
说明:
|
||||||
|
|
||||||
|
- `other` 现在也按 Harbor 业务镜像更新,不再继续依赖本地 `./other/service.jar` 挂载
|
||||||
|
- 页面里的“滚动重启”仍然保留,适合只想让配置生效、不涉及代码构建的场景
|
||||||
|
- 这条链路默认假设 `hy-app-monitor` 就运行在打包机 / deploy 机上,并且本机已经能访问业务仓、Docker 和 Harbor
|
||||||
|
|
||||||
## 页面内容
|
## 页面内容
|
||||||
|
|
||||||
- 主机:`CPU / 内存 / 磁盘 / 进程数`,带阈值颜色分级
|
- 主机:`CPU / 内存 / 磁盘 / 进程数`,带阈值颜色分级
|
||||||
@ -108,6 +156,7 @@ python3 ops/deploy_via_tat.py
|
|||||||
- Nacos:命名空间、服务、注册实例、健康状态
|
- Nacos:命名空间、服务、注册实例、健康状态
|
||||||
- Nacos 配置:读取当前 namespace 下的配置列表,支持格式校验、在线编辑保存
|
- Nacos 配置:读取当前 namespace 下的配置列表,支持格式校验、在线编辑保存
|
||||||
- Golang 配置:读取 app 节点当前运行中的 `service.env`,展示 `chatapp3-golang` 实际消费的环境变量并支持保存
|
- Golang 配置:读取 app 节点当前运行中的 `service.env`,展示 `chatapp3-golang` 实际消费的环境变量并支持保存
|
||||||
|
- 服务更新:本地构建选中服务、推送 Harbor、更新完整 compose,并只重建选中的服务
|
||||||
- 滚动重启:可按服务选择 Java / Golang 节点,逐台重启并等待健康恢复
|
- 滚动重启:可按服务选择 Java / Golang 节点,逐台重启并等待健康恢复
|
||||||
- MongoDB:连接、内存、网络、操作计数、复制集、数据库统计
|
- MongoDB:连接、内存、网络、操作计数、复制集、数据库统计
|
||||||
- 自动刷新:前端可切换刷新间隔,并保存在浏览器本地
|
- 自动刷新:前端可切换刷新间隔,并保存在浏览器本地
|
||||||
|
|||||||
547
monitor/compose_templates.py
Normal file
547
monitor/compose_templates.py
Normal file
@ -0,0 +1,547 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
# compose 模板生成需要解析 docker inspect 的 JSON。
|
||||||
|
import json
|
||||||
|
# 本地模板目录和远端运行目录都走现有配置。
|
||||||
|
from pathlib import Path
|
||||||
|
# 远端命令和文件路径都需要安全转义。
|
||||||
|
import shlex
|
||||||
|
# 一些字段要做正则判断。
|
||||||
|
import re
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from .config import ROOT, RUNTIME_BASE_DIR, load_hosts, utc_now
|
||||||
|
from .runtime_config import read_remote_text, write_remote_text
|
||||||
|
from .tencent_tat import run_shell_command
|
||||||
|
|
||||||
|
|
||||||
|
# 本地完整 compose 模板统一放到 run 目录,避免把运行期产物写进 Git。
|
||||||
|
COMPOSE_TEMPLATE_DIR = ROOT / "run" / "compose_templates"
|
||||||
|
|
||||||
|
def managed_compose_hosts() -> list[dict[str, Any]]:
|
||||||
|
# 只维护当前 targets.json 里真正承载业务服务的宿主机。
|
||||||
|
return [host for host in load_hosts() if host.get("services")]
|
||||||
|
|
||||||
|
|
||||||
|
def managed_compose_host_names() -> list[str]:
|
||||||
|
# 页面和运维脚本都更适合拿主机名列表。
|
||||||
|
return [host["host"] for host in managed_compose_hosts()]
|
||||||
|
|
||||||
|
|
||||||
|
def managed_host_map() -> dict[str, dict[str, Any]]:
|
||||||
|
# 方便按主机名快速定位实例 ID 和服务清单。
|
||||||
|
return {host["host"]: host for host in managed_compose_hosts()}
|
||||||
|
|
||||||
|
|
||||||
|
def compose_template_path(host_name: str) -> Path:
|
||||||
|
# 每台机器一份本地模板文件。
|
||||||
|
return COMPOSE_TEMPLATE_DIR / f"{host_name}.yml"
|
||||||
|
|
||||||
|
|
||||||
|
def compose_remote_path(host_name: str) -> str:
|
||||||
|
# 远端 compose 路径沿用现网目录结构。
|
||||||
|
return f"{RUNTIME_BASE_DIR.rstrip('/')}/{host_name}/docker-compose.yml"
|
||||||
|
|
||||||
|
|
||||||
|
def container_name(host_name: str, service_name: str) -> str:
|
||||||
|
# 当前线上容器命名规则固定。
|
||||||
|
return f"likei-{host_name}-{service_name}"
|
||||||
|
|
||||||
|
|
||||||
|
def docker_inspect_payload(host: dict[str, Any], service_names: list[str]) -> list[dict[str, Any]]:
|
||||||
|
# 这里只抓 compose 需要的关键字段,避免完整 inspect JSON 被 TAT 输出截断。
|
||||||
|
container_names = [container_name(host["host"], service_name) for service_name in service_names]
|
||||||
|
if not container_names:
|
||||||
|
raise RuntimeError(f"no compose services configured for host: {host['host']}")
|
||||||
|
|
||||||
|
quoted_names = " ".join(shlex.quote(name) for name in container_names)
|
||||||
|
python_code = """
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
|
||||||
|
payload = json.load(sys.stdin)
|
||||||
|
for item in payload:
|
||||||
|
config = item.get("Config") or {}
|
||||||
|
host_config = item.get("HostConfig") or {}
|
||||||
|
print(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"Name": item.get("Name"),
|
||||||
|
"Config": {
|
||||||
|
"Image": config.get("Image"),
|
||||||
|
"StopSignal": config.get("StopSignal"),
|
||||||
|
"WorkingDir": config.get("WorkingDir"),
|
||||||
|
"Entrypoint": config.get("Entrypoint"),
|
||||||
|
"Cmd": config.get("Cmd"),
|
||||||
|
"Healthcheck": config.get("Healthcheck"),
|
||||||
|
},
|
||||||
|
"HostConfig": {
|
||||||
|
"NetworkMode": host_config.get("NetworkMode"),
|
||||||
|
"RestartPolicy": {"Name": (host_config.get("RestartPolicy") or {}).get("Name")},
|
||||||
|
"StopTimeout": host_config.get("StopTimeout"),
|
||||||
|
},
|
||||||
|
"Mounts": item.get("Mounts") or [],
|
||||||
|
},
|
||||||
|
ensure_ascii=False,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
""".strip()
|
||||||
|
script = f"""set -eu
|
||||||
|
docker inspect {quoted_names} | python3 -c {shlex.quote(python_code)}
|
||||||
|
"""
|
||||||
|
result_map = run_shell_command(
|
||||||
|
[host["instanceId"]],
|
||||||
|
script,
|
||||||
|
command_name=f"inspect-compose-{host['host']}",
|
||||||
|
timeout_seconds=90,
|
||||||
|
)
|
||||||
|
result = result_map[host["instanceId"]]
|
||||||
|
if result.get("status") != "SUCCESS":
|
||||||
|
raise RuntimeError(f"docker inspect failed on {host['host']}: {result.get('status')}")
|
||||||
|
|
||||||
|
payload: list[dict[str, Any]] = []
|
||||||
|
for raw_line in str(result.get("output") or "").splitlines():
|
||||||
|
line = raw_line.strip()
|
||||||
|
if not line:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
payload.append(json.loads(line))
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
raise RuntimeError(f"invalid inspect field payload on {host['host']}") from exc
|
||||||
|
|
||||||
|
if not payload:
|
||||||
|
raise RuntimeError(f"empty docker inspect payload on {host['host']}")
|
||||||
|
return payload
|
||||||
|
|
||||||
|
|
||||||
|
def ns_to_duration_text(raw_value: int | None) -> str | None:
|
||||||
|
# docker inspect 的健康检查时间单位是纳秒,compose 更适合秒级文本。
|
||||||
|
if not raw_value:
|
||||||
|
return None
|
||||||
|
if raw_value % 1_000_000_000 == 0:
|
||||||
|
return f"{raw_value // 1_000_000_000}s"
|
||||||
|
if raw_value % 1_000_000 == 0:
|
||||||
|
return f"{raw_value // 1_000_000}ms"
|
||||||
|
return str(raw_value)
|
||||||
|
|
||||||
|
|
||||||
|
def stop_grace_period_text(raw_value: int | None) -> str | None:
|
||||||
|
# docker inspect 的 StopTimeout 单位本来就是秒。
|
||||||
|
if raw_value is None or raw_value <= 0:
|
||||||
|
return None
|
||||||
|
return f"{int(raw_value)}s"
|
||||||
|
|
||||||
|
|
||||||
|
def image_is_pullable(image_ref: str) -> bool:
|
||||||
|
# 纯十六进制镜像 ID 不能当成远端可拉取镜像。
|
||||||
|
text = str(image_ref or "").strip()
|
||||||
|
if not text:
|
||||||
|
return False
|
||||||
|
return re.fullmatch(r"[0-9a-f]{12,64}", text) is None
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_command(raw_value: Any) -> list[str] | str | None:
|
||||||
|
# docker inspect 的 Cmd / Entrypoint 可能是数组、空值或字符串。
|
||||||
|
if raw_value is None:
|
||||||
|
return None
|
||||||
|
if isinstance(raw_value, list):
|
||||||
|
return [str(item) for item in raw_value]
|
||||||
|
if isinstance(raw_value, str):
|
||||||
|
return raw_value
|
||||||
|
return [str(item) for item in raw_value]
|
||||||
|
|
||||||
|
|
||||||
|
def healthcheck_block(config: dict[str, Any]) -> dict[str, Any] | None:
|
||||||
|
# 把 inspect 里的健康检查结构翻回 compose 可接受格式。
|
||||||
|
healthcheck = config.get("Healthcheck") or {}
|
||||||
|
test = healthcheck.get("Test")
|
||||||
|
if not test:
|
||||||
|
return None
|
||||||
|
|
||||||
|
block: dict[str, Any] = {"test": [str(item) for item in test]}
|
||||||
|
interval = ns_to_duration_text(int(healthcheck.get("Interval") or 0))
|
||||||
|
timeout = ns_to_duration_text(int(healthcheck.get("Timeout") or 0))
|
||||||
|
start_period = ns_to_duration_text(int(healthcheck.get("StartPeriod") or 0))
|
||||||
|
retries = int(healthcheck.get("Retries") or 0)
|
||||||
|
if interval:
|
||||||
|
block["interval"] = interval
|
||||||
|
if timeout:
|
||||||
|
block["timeout"] = timeout
|
||||||
|
if start_period:
|
||||||
|
block["start_period"] = start_period
|
||||||
|
if retries > 0:
|
||||||
|
block["retries"] = retries
|
||||||
|
return block
|
||||||
|
|
||||||
|
|
||||||
|
def render_compose_yaml(service_map: dict[str, dict[str, Any]]) -> str:
|
||||||
|
# 这里按需导入 PyYAML,和现有项目依赖策略保持一致。
|
||||||
|
try:
|
||||||
|
import yaml
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
raise RuntimeError(f"yaml renderer unavailable: {exc}") from exc
|
||||||
|
|
||||||
|
payload = {"services": service_map}
|
||||||
|
return yaml.safe_dump(payload, allow_unicode=True, sort_keys=False).rstrip() + "\n"
|
||||||
|
|
||||||
|
|
||||||
|
def parse_compose_yaml(content: str) -> dict[str, Any]:
|
||||||
|
# 服务更新需要读取本地模板并直接改 image 字段。
|
||||||
|
try:
|
||||||
|
import yaml
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
raise RuntimeError(f"yaml parser unavailable: {exc}") from exc
|
||||||
|
|
||||||
|
payload = yaml.safe_load(content) or {}
|
||||||
|
if not isinstance(payload, dict):
|
||||||
|
raise RuntimeError("invalid compose yaml payload")
|
||||||
|
|
||||||
|
services = payload.get("services") or {}
|
||||||
|
if not isinstance(services, dict):
|
||||||
|
raise RuntimeError("invalid compose yaml services block")
|
||||||
|
return {"services": {str(name): dict(value or {}) for name, value in services.items()}}
|
||||||
|
|
||||||
|
|
||||||
|
def render_volume_specs(host_name: str, mounts: list[dict[str, Any]]) -> list[str]:
|
||||||
|
# 把 docker inspect 的挂载信息翻回 compose volumes。
|
||||||
|
host_dir = f"{RUNTIME_BASE_DIR.rstrip('/')}/{host_name}/"
|
||||||
|
specs: list[str] = []
|
||||||
|
for mount in mounts:
|
||||||
|
mount_type = str(mount.get("Type") or "").strip()
|
||||||
|
source = str(mount.get("Source") or "").strip()
|
||||||
|
destination = str(mount.get("Destination") or "").strip()
|
||||||
|
if mount_type != "bind" or not source or not destination:
|
||||||
|
continue
|
||||||
|
|
||||||
|
rendered_source = source
|
||||||
|
if source.startswith(host_dir):
|
||||||
|
rendered_source = f"./{source[len(host_dir):]}"
|
||||||
|
|
||||||
|
volume_spec = f"{rendered_source}:{destination}"
|
||||||
|
if not bool(mount.get("RW")):
|
||||||
|
volume_spec += ":ro"
|
||||||
|
specs.append(volume_spec)
|
||||||
|
return specs
|
||||||
|
|
||||||
|
|
||||||
|
def uses_local_runtime_artifact(spec: str) -> bool:
|
||||||
|
# 这些挂载代表服务依赖宿主机上的 jar / 二进制,不再适合镜像化发布。
|
||||||
|
text = str(spec or "").strip()
|
||||||
|
return (
|
||||||
|
text.endswith(":/application/service.jar:ro")
|
||||||
|
or text.endswith(":/application/service.jar")
|
||||||
|
or text.endswith(":/application/service:ro")
|
||||||
|
or text.endswith(":/application/service")
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def strip_local_runtime_artifacts(volume_specs: list[str]) -> list[str]:
|
||||||
|
# 发布到 Harbor 后,需要把本地运行时制品挂载从模板里移掉。
|
||||||
|
return [spec for spec in volume_specs if not uses_local_runtime_artifact(spec)]
|
||||||
|
|
||||||
|
|
||||||
|
def build_service_definition(
|
||||||
|
host_name: str,
|
||||||
|
service_name: str,
|
||||||
|
inspect_payload: dict[str, Any],
|
||||||
|
*,
|
||||||
|
include_env_file: bool,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
# 只抽 compose 真正需要的字段,避免把 docker inspect 的噪声全部带回来。
|
||||||
|
config = inspect_payload.get("Config") or {}
|
||||||
|
host_config = inspect_payload.get("HostConfig") or {}
|
||||||
|
service_definition: dict[str, Any] = {}
|
||||||
|
|
||||||
|
image_ref = str(config.get("Image") or "").strip()
|
||||||
|
if not image_ref:
|
||||||
|
raise RuntimeError(f"missing image ref for {host_name}/{service_name}")
|
||||||
|
|
||||||
|
service_definition["image"] = image_ref
|
||||||
|
|
||||||
|
service_definition["container_name"] = str(inspect_payload.get("Name") or f"/{container_name(host_name, service_name)}").lstrip("/")
|
||||||
|
|
||||||
|
network_mode = str(host_config.get("NetworkMode") or "").strip()
|
||||||
|
if network_mode:
|
||||||
|
service_definition["network_mode"] = network_mode
|
||||||
|
|
||||||
|
restart_policy = str((host_config.get("RestartPolicy") or {}).get("Name") or "").strip()
|
||||||
|
if restart_policy:
|
||||||
|
service_definition["restart"] = restart_policy
|
||||||
|
|
||||||
|
stop_signal = str(config.get("StopSignal") or "").strip()
|
||||||
|
if stop_signal:
|
||||||
|
service_definition["stop_signal"] = stop_signal
|
||||||
|
|
||||||
|
stop_grace_period = stop_grace_period_text(host_config.get("StopTimeout"))
|
||||||
|
if stop_grace_period:
|
||||||
|
service_definition["stop_grace_period"] = stop_grace_period
|
||||||
|
|
||||||
|
if include_env_file:
|
||||||
|
service_definition["env_file"] = ["./service.env"]
|
||||||
|
|
||||||
|
volume_specs = render_volume_specs(host_name, list(inspect_payload.get("Mounts") or []))
|
||||||
|
if volume_specs:
|
||||||
|
service_definition["volumes"] = volume_specs
|
||||||
|
|
||||||
|
# 如果运行时依赖本地挂载的 jar / 二进制,就不要再强制拉镜像。
|
||||||
|
uses_local_runtime_artifact = any(
|
||||||
|
spec.endswith(":/application/service.jar:ro")
|
||||||
|
or spec.endswith(":/application/service")
|
||||||
|
or spec.endswith(":/application/service:ro")
|
||||||
|
for spec in volume_specs
|
||||||
|
)
|
||||||
|
if uses_local_runtime_artifact:
|
||||||
|
service_definition["pull_policy"] = "never"
|
||||||
|
elif image_is_pullable(image_ref):
|
||||||
|
service_definition["pull_policy"] = "always"
|
||||||
|
|
||||||
|
working_dir = str(config.get("WorkingDir") or "").strip()
|
||||||
|
if working_dir:
|
||||||
|
service_definition["working_dir"] = working_dir
|
||||||
|
|
||||||
|
entrypoint = normalize_command(config.get("Entrypoint"))
|
||||||
|
if entrypoint is not None:
|
||||||
|
service_definition["entrypoint"] = entrypoint
|
||||||
|
|
||||||
|
command = normalize_command(config.get("Cmd"))
|
||||||
|
if command is not None:
|
||||||
|
service_definition["command"] = command
|
||||||
|
|
||||||
|
healthcheck = healthcheck_block(config)
|
||||||
|
if healthcheck:
|
||||||
|
service_definition["healthcheck"] = healthcheck
|
||||||
|
|
||||||
|
return service_definition
|
||||||
|
|
||||||
|
|
||||||
|
def generate_host_compose_template(host_name: str) -> dict[str, Any]:
|
||||||
|
# 生成模板时严格按 targets.json 里的服务顺序输出,后续 diff 更稳定。
|
||||||
|
host_map = managed_host_map()
|
||||||
|
host = host_map.get(host_name)
|
||||||
|
if host is None:
|
||||||
|
raise RuntimeError(f"unsupported compose host: {host_name}")
|
||||||
|
|
||||||
|
service_names = [str(service.get("name") or "").strip() for service in host.get("services") or []]
|
||||||
|
service_names = [service_name for service_name in service_names if service_name]
|
||||||
|
if not service_names:
|
||||||
|
raise RuntimeError(f"host has no managed services: {host_name}")
|
||||||
|
|
||||||
|
inspect_items = docker_inspect_payload(host, service_names)
|
||||||
|
inspect_by_name = {
|
||||||
|
str((item.get("Name") or "").lstrip("/")): item
|
||||||
|
for item in inspect_items
|
||||||
|
}
|
||||||
|
|
||||||
|
service_map: dict[str, dict[str, Any]] = {}
|
||||||
|
missing_services: list[str] = []
|
||||||
|
for service_name in service_names:
|
||||||
|
expected_container = container_name(host_name, service_name)
|
||||||
|
inspect_item = inspect_by_name.get(expected_container)
|
||||||
|
if inspect_item is None:
|
||||||
|
missing_services.append(service_name)
|
||||||
|
continue
|
||||||
|
service_map[service_name] = build_service_definition(
|
||||||
|
host_name,
|
||||||
|
service_name,
|
||||||
|
inspect_item,
|
||||||
|
include_env_file=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
if missing_services:
|
||||||
|
raise RuntimeError(f"missing running containers on {host_name}: {', '.join(missing_services)}")
|
||||||
|
|
||||||
|
content = render_compose_yaml(service_map)
|
||||||
|
return {
|
||||||
|
"ok": True,
|
||||||
|
"host": host_name,
|
||||||
|
"services": list(service_map.keys()),
|
||||||
|
"updatedAt": utc_now(),
|
||||||
|
"content": content,
|
||||||
|
"path": str(compose_template_path(host_name)),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def save_local_host_compose_template(host_name: str) -> dict[str, Any]:
|
||||||
|
# 从远端运行态重新生成本地完整模板。
|
||||||
|
payload = generate_host_compose_template(host_name)
|
||||||
|
path = compose_template_path(host_name)
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
previous = path.read_text(encoding="utf-8") if path.exists() else ""
|
||||||
|
path.write_text(payload["content"], encoding="utf-8")
|
||||||
|
return {
|
||||||
|
**payload,
|
||||||
|
"changed": previous != payload["content"],
|
||||||
|
"source": "remote-runtime",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def load_local_host_compose_template(host_name: str) -> dict[str, Any]:
|
||||||
|
# 本地模板优先作为远端同步的稳定来源。
|
||||||
|
path = compose_template_path(host_name)
|
||||||
|
if not path.exists():
|
||||||
|
raise RuntimeError(f"local compose template missing: {host_name}")
|
||||||
|
return {
|
||||||
|
"ok": True,
|
||||||
|
"host": host_name,
|
||||||
|
"services": [str(service.get("name") or "").strip() for service in managed_host_map()[host_name].get("services") or []],
|
||||||
|
"updatedAt": utc_now(),
|
||||||
|
"content": path.read_text(encoding="utf-8"),
|
||||||
|
"path": str(path),
|
||||||
|
"source": "local-cache",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def ensure_local_host_compose_template(host_name: str) -> dict[str, Any]:
|
||||||
|
# 第一次进入服务更新时,模板可能还没在本地生成。
|
||||||
|
path = compose_template_path(host_name)
|
||||||
|
if path.exists():
|
||||||
|
return load_local_host_compose_template(host_name)
|
||||||
|
return save_local_host_compose_template(host_name)
|
||||||
|
|
||||||
|
|
||||||
|
def load_local_host_compose_model(host_name: str) -> dict[str, Any]:
|
||||||
|
# 同时返回解析后的 services,方便直接修改模板。
|
||||||
|
template_payload = ensure_local_host_compose_template(host_name)
|
||||||
|
parsed = parse_compose_yaml(str(template_payload.get("content") or ""))
|
||||||
|
return {
|
||||||
|
**template_payload,
|
||||||
|
"services": parsed["services"],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def save_local_host_compose_model(host_name: str, service_map: dict[str, dict[str, Any]]) -> dict[str, Any]:
|
||||||
|
# 模板编辑完成后,统一走这里回写磁盘并返回 diff 信息。
|
||||||
|
path = compose_template_path(host_name)
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
content = render_compose_yaml(service_map)
|
||||||
|
previous = path.read_text(encoding="utf-8") if path.exists() else ""
|
||||||
|
path.write_text(content, encoding="utf-8")
|
||||||
|
return {
|
||||||
|
"ok": True,
|
||||||
|
"host": host_name,
|
||||||
|
"updatedAt": utc_now(),
|
||||||
|
"path": str(path),
|
||||||
|
"content": content,
|
||||||
|
"changed": previous != content,
|
||||||
|
"source": "local-cache",
|
||||||
|
"services": list(service_map.keys()),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def update_local_service_image(host_name: str, service_name: str, image_ref: str) -> dict[str, Any]:
|
||||||
|
# 服务更新链路会把新镜像 digest 写回本地完整模板。
|
||||||
|
if not str(image_ref or "").strip():
|
||||||
|
raise RuntimeError("image ref is required")
|
||||||
|
|
||||||
|
model = load_local_host_compose_model(host_name)
|
||||||
|
service_map = dict(model.get("services") or {})
|
||||||
|
service_definition = dict(service_map.get(service_name) or {})
|
||||||
|
if not service_definition:
|
||||||
|
raise RuntimeError(f"missing service in local compose template: {host_name}/{service_name}")
|
||||||
|
|
||||||
|
service_definition["image"] = str(image_ref).strip()
|
||||||
|
|
||||||
|
volume_specs = [str(item) for item in service_definition.get("volumes") or []]
|
||||||
|
volume_specs = strip_local_runtime_artifacts(volume_specs)
|
||||||
|
if volume_specs:
|
||||||
|
service_definition["volumes"] = volume_specs
|
||||||
|
else:
|
||||||
|
service_definition.pop("volumes", None)
|
||||||
|
|
||||||
|
if image_is_pullable(image_ref):
|
||||||
|
service_definition["pull_policy"] = "always"
|
||||||
|
else:
|
||||||
|
service_definition.pop("pull_policy", None)
|
||||||
|
|
||||||
|
service_map[service_name] = service_definition
|
||||||
|
saved = save_local_host_compose_model(host_name, service_map)
|
||||||
|
saved["service"] = service_name
|
||||||
|
saved["image"] = str(image_ref).strip()
|
||||||
|
return saved
|
||||||
|
|
||||||
|
|
||||||
|
def verify_remote_compose(host: dict[str, Any], remote_path: str) -> list[str]:
|
||||||
|
# 下发后立即让 docker compose 解析一遍,避免把坏文件写到线上却没人发现。
|
||||||
|
script = f"""set -eu
|
||||||
|
FILE={shlex.quote(remote_path)}
|
||||||
|
docker compose -f "$FILE" config --services
|
||||||
|
"""
|
||||||
|
result_map = run_shell_command(
|
||||||
|
[host["instanceId"]],
|
||||||
|
script,
|
||||||
|
command_name=f"verify-compose-{host['host']}",
|
||||||
|
timeout_seconds=60,
|
||||||
|
)
|
||||||
|
result = result_map[host["instanceId"]]
|
||||||
|
if result.get("status") != "SUCCESS":
|
||||||
|
raise RuntimeError(f"compose verify failed on {host['host']}: {result.get('status')}")
|
||||||
|
return [line.strip() for line in str(result.get("output") or "").splitlines() if line.strip()]
|
||||||
|
|
||||||
|
|
||||||
|
def sync_remote_host_compose_template(host_name: str, *, refresh_from_remote: bool = True) -> dict[str, Any]:
|
||||||
|
# 对外暴露的主入口:优先刷新本地模板,再把完整 compose 同步到远端。
|
||||||
|
host_map = managed_host_map()
|
||||||
|
host = host_map.get(host_name)
|
||||||
|
if host is None:
|
||||||
|
raise RuntimeError(f"unsupported compose host: {host_name}")
|
||||||
|
|
||||||
|
template_payload: dict[str, Any] | None = None
|
||||||
|
refresh_error = ""
|
||||||
|
if refresh_from_remote:
|
||||||
|
try:
|
||||||
|
template_payload = save_local_host_compose_template(host_name)
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
refresh_error = str(exc)
|
||||||
|
|
||||||
|
if template_payload is None:
|
||||||
|
template_payload = load_local_host_compose_template(host_name)
|
||||||
|
template_payload["refreshError"] = refresh_error
|
||||||
|
|
||||||
|
remote_path = compose_remote_path(host_name)
|
||||||
|
try:
|
||||||
|
remote_current = read_remote_text(host, remote_path)
|
||||||
|
except Exception:
|
||||||
|
remote_current = ""
|
||||||
|
|
||||||
|
backup_path = ""
|
||||||
|
if remote_current and remote_current != template_payload["content"]:
|
||||||
|
backup_path = f"{remote_path}.hyapp.{utc_now().replace(':', '').replace('+', '_')}.bak"
|
||||||
|
write_remote_text(host, backup_path, remote_current)
|
||||||
|
|
||||||
|
changed = remote_current != template_payload["content"]
|
||||||
|
if changed:
|
||||||
|
write_remote_text(host, remote_path, template_payload["content"])
|
||||||
|
|
||||||
|
remote_after = read_remote_text(host, remote_path)
|
||||||
|
if remote_after != template_payload["content"]:
|
||||||
|
raise RuntimeError(f"compose verify content mismatch on {host_name}")
|
||||||
|
|
||||||
|
remote_services = verify_remote_compose(host, remote_path)
|
||||||
|
return {
|
||||||
|
"ok": True,
|
||||||
|
"host": host_name,
|
||||||
|
"services": template_payload["services"],
|
||||||
|
"updatedAt": utc_now(),
|
||||||
|
"path": remote_path,
|
||||||
|
"templatePath": template_payload["path"],
|
||||||
|
"changed": changed,
|
||||||
|
"backupPath": backup_path,
|
||||||
|
"remoteServices": remote_services,
|
||||||
|
"source": template_payload.get("source") or "local-cache",
|
||||||
|
"refreshError": template_payload.get("refreshError") or "",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def sync_remote_compose_templates(host_names: list[str], *, refresh_from_remote: bool = True) -> dict[str, Any]:
|
||||||
|
# 允许对一组主机做完整模板同步。
|
||||||
|
normalized_names = [str(name or "").strip() for name in host_names if str(name or "").strip()]
|
||||||
|
normalized_names = list(dict.fromkeys(normalized_names))
|
||||||
|
if not normalized_names:
|
||||||
|
normalized_names = managed_compose_host_names()
|
||||||
|
|
||||||
|
items = [sync_remote_host_compose_template(host_name, refresh_from_remote=refresh_from_remote) for host_name in normalized_names]
|
||||||
|
return {
|
||||||
|
"ok": True,
|
||||||
|
"updatedAt": utc_now(),
|
||||||
|
"hosts": items,
|
||||||
|
}
|
||||||
@ -1,6 +1,7 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import json
|
import json
|
||||||
|
import os
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
@ -9,6 +10,8 @@ from .env import env_float, env_int, env_int_list, env_str, load_env_file
|
|||||||
|
|
||||||
# 项目根目录。
|
# 项目根目录。
|
||||||
ROOT = Path(__file__).resolve().parents[1]
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
# 常见工作区根目录,便于本地开发时自动发现业务仓。
|
||||||
|
WORKSPACE_ROOT = ROOT.parent
|
||||||
# 静态资源目录。
|
# 静态资源目录。
|
||||||
STATIC_DIR = ROOT / "static"
|
STATIC_DIR = ROOT / "static"
|
||||||
# 默认 .env 路径。
|
# 默认 .env 路径。
|
||||||
@ -45,6 +48,10 @@ ROLLING_RESTART_TIMEOUT_SECONDS = env_int("ROLLING_RESTART_TIMEOUT_SECONDS", 180
|
|||||||
ROLLING_RESTART_POLL_SECONDS = env_float("ROLLING_RESTART_POLL_SECONDS", 2.0)
|
ROLLING_RESTART_POLL_SECONDS = env_float("ROLLING_RESTART_POLL_SECONDS", 2.0)
|
||||||
# 单实例恢复后额外稳定等待秒数。
|
# 单实例恢复后额外稳定等待秒数。
|
||||||
ROLLING_RESTART_STABILIZE_SECONDS = env_float("ROLLING_RESTART_STABILIZE_SECONDS", 2.0)
|
ROLLING_RESTART_STABILIZE_SECONDS = env_float("ROLLING_RESTART_STABILIZE_SECONDS", 2.0)
|
||||||
|
# 服务更新时本地构建最长等待秒数。
|
||||||
|
SERVICE_BUILD_TIMEOUT_SECONDS = env_int("SERVICE_BUILD_TIMEOUT_SECONDS", 3600)
|
||||||
|
# 服务更新时目标机预拉镜像最长等待秒数。
|
||||||
|
SERVICE_PULL_TIMEOUT_SECONDS = env_int("SERVICE_PULL_TIMEOUT_SECONDS", 600)
|
||||||
|
|
||||||
# TAT 所在区域。
|
# TAT 所在区域。
|
||||||
DEPLOY_REGION = env_str("DEPLOY_REGION", "me-saudi-arabia")
|
DEPLOY_REGION = env_str("DEPLOY_REGION", "me-saudi-arabia")
|
||||||
@ -53,6 +60,22 @@ TENCENT_SECRET_ID = env_str("TENCENT_SECRET_ID", "")
|
|||||||
# 腾讯云 SecretKey。
|
# 腾讯云 SecretKey。
|
||||||
TENCENT_SECRET_KEY = env_str("TENCENT_SECRET_KEY", "")
|
TENCENT_SECRET_KEY = env_str("TENCENT_SECRET_KEY", "")
|
||||||
|
|
||||||
|
|
||||||
|
def pick_existing_path(candidates: list[Path]) -> Path:
|
||||||
|
# 配置未显式指定时,优先挑当前机器上真正存在的路径。
|
||||||
|
for candidate in candidates:
|
||||||
|
if candidate.exists():
|
||||||
|
return candidate
|
||||||
|
return candidates[0]
|
||||||
|
|
||||||
|
|
||||||
|
def env_or_candidate_path(env_key: str, candidates: list[Path]) -> Path:
|
||||||
|
# 允许 .env 显式覆盖,也兼容本地开发和远端 deploy 两种目录布局。
|
||||||
|
override = os.environ.get(env_key, "").strip()
|
||||||
|
if override:
|
||||||
|
return Path(override).expanduser()
|
||||||
|
return pick_existing_path(candidates)
|
||||||
|
|
||||||
# 页面默认刷新间隔。
|
# 页面默认刷新间隔。
|
||||||
DEFAULT_REFRESH_INTERVAL_SECONDS = env_int("DEFAULT_REFRESH_INTERVAL_SECONDS", 10)
|
DEFAULT_REFRESH_INTERVAL_SECONDS = env_int("DEFAULT_REFRESH_INTERVAL_SECONDS", 10)
|
||||||
# 页面可选刷新间隔。
|
# 页面可选刷新间隔。
|
||||||
@ -79,6 +102,37 @@ NACOS_USERNAME = env_str("NACOS_USERNAME", "")
|
|||||||
# Nacos 可选密码。
|
# Nacos 可选密码。
|
||||||
NACOS_PASSWORD = env_str("NACOS_PASSWORD", "")
|
NACOS_PASSWORD = env_str("NACOS_PASSWORD", "")
|
||||||
|
|
||||||
|
# Harbor 镜像仓地址,服务更新会推送镜像并让目标机主动拉取。
|
||||||
|
HARBOR_REGISTRY = env_str("HARBOR_REGISTRY", "10.2.1.3:18082")
|
||||||
|
# Harbor 项目名。
|
||||||
|
HARBOR_PROJECT = env_str("HARBOR_PROJECT", "likei")
|
||||||
|
# Harbor 登录用户名。
|
||||||
|
HARBOR_USERNAME = env_str("HARBOR_USERNAME", "")
|
||||||
|
# Harbor 登录密码。
|
||||||
|
HARBOR_PASSWORD = env_str("HARBOR_PASSWORD", "")
|
||||||
|
# Docker 构建平台,和现有 Jenkins/构建脚本口径保持一致。
|
||||||
|
BUILD_DOCKER_PLATFORM = env_str("BUILD_DOCKER_PLATFORM", "linux/amd64")
|
||||||
|
# Java 服务默认拉取的 Git ref。
|
||||||
|
UPDATE_DEFAULT_JAVA_GIT_REF = env_str("UPDATE_DEFAULT_JAVA_GIT_REF", "main")
|
||||||
|
# Go 服务默认拉取的 Git ref。
|
||||||
|
UPDATE_DEFAULT_GO_GIT_REF = env_str("UPDATE_DEFAULT_GO_GIT_REF", "main")
|
||||||
|
# Java 仓库路径,默认兼容本地工作区和远端 deploy 机。
|
||||||
|
JAVA_REPO_ROOT = env_or_candidate_path(
|
||||||
|
"JAVA_REPO_ROOT",
|
||||||
|
[
|
||||||
|
WORKSPACE_ROOT / "chatapp3" / "chatapp3-java",
|
||||||
|
Path("/opt/deploy/sources/chatapp3-java"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
# Go 仓库路径,默认兼容本地工作区和远端 deploy 机。
|
||||||
|
GOLANG_REPO_ROOT = env_or_candidate_path(
|
||||||
|
"GOLANG_REPO_ROOT",
|
||||||
|
[
|
||||||
|
WORKSPACE_ROOT / "chatapp3" / "chatapp3-golang",
|
||||||
|
Path("/opt/deploy/sources/chatapp3-golang"),
|
||||||
|
],
|
||||||
|
)
|
||||||
|
|
||||||
# Mongo 连接串。
|
# Mongo 连接串。
|
||||||
MONGO_URI = env_str("MONGO_URI", "")
|
MONGO_URI = env_str("MONGO_URI", "")
|
||||||
# Mongo 默认关注的业务库名。
|
# Mongo 默认关注的业务库名。
|
||||||
|
|||||||
@ -15,6 +15,7 @@ from .nacos import (
|
|||||||
validate_nacos_config_payload,
|
validate_nacos_config_payload,
|
||||||
)
|
)
|
||||||
from .runtime_config import get_go_config_payload, save_go_config_payload, validate_go_config_payload
|
from .runtime_config import get_go_config_payload, save_go_config_payload, validate_go_config_payload
|
||||||
|
from .service_release import build_update_targets_payload, deploy_updated_services_payload
|
||||||
from .service_ops import build_restart_targets_payload, rolling_restart_payload
|
from .service_ops import build_restart_targets_payload, rolling_restart_payload
|
||||||
|
|
||||||
|
|
||||||
@ -149,6 +150,20 @@ class MonitorHandler(BaseHTTPRequestHandler):
|
|||||||
send_body=send_body,
|
send_body=send_body,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# 返回可构建并发布的业务服务。
|
||||||
|
if parsed.path == "/api/ops/update-targets":
|
||||||
|
try:
|
||||||
|
payload = build_update_targets_payload()
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
return self.send_error_payload(HTTPStatus.BAD_GATEWAY, str(exc), send_body=send_body)
|
||||||
|
|
||||||
|
return self.send_payload(
|
||||||
|
HTTPStatus.OK,
|
||||||
|
response_json(payload),
|
||||||
|
"application/json; charset=utf-8",
|
||||||
|
send_body=send_body,
|
||||||
|
)
|
||||||
|
|
||||||
# favicon 直接返回空。
|
# favicon 直接返回空。
|
||||||
if parsed.path == "/favicon.ico":
|
if parsed.path == "/favicon.ico":
|
||||||
return self.send_payload(HTTPStatus.NO_CONTENT, b"", "image/x-icon", send_body=send_body)
|
return self.send_payload(HTTPStatus.NO_CONTENT, b"", "image/x-icon", send_body=send_body)
|
||||||
@ -180,6 +195,9 @@ class MonitorHandler(BaseHTTPRequestHandler):
|
|||||||
if parsed.path == "/api/ops/rolling-restart":
|
if parsed.path == "/api/ops/rolling-restart":
|
||||||
return self.handle_rolling_restart(payload)
|
return self.handle_rolling_restart(payload)
|
||||||
|
|
||||||
|
if parsed.path == "/api/ops/update-services":
|
||||||
|
return self.handle_update_services(payload)
|
||||||
|
|
||||||
return self.send_error_payload(HTTPStatus.NOT_FOUND, "not found", send_body=True)
|
return self.send_error_payload(HTTPStatus.NOT_FOUND, "not found", send_body=True)
|
||||||
|
|
||||||
def handle_nacos_validate(self, payload: dict[str, Any]) -> None:
|
def handle_nacos_validate(self, payload: dict[str, Any]) -> None:
|
||||||
@ -284,6 +302,34 @@ class MonitorHandler(BaseHTTPRequestHandler):
|
|||||||
send_body=True,
|
send_body=True,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
def handle_update_services(self, payload: dict[str, Any]) -> None:
|
||||||
|
services = payload.get("services")
|
||||||
|
if not isinstance(services, list):
|
||||||
|
return self.send_error_payload(HTTPStatus.BAD_REQUEST, "services must be a list", send_body=True)
|
||||||
|
|
||||||
|
java_git_ref = str(payload.get("javaGitRef") or "").strip()
|
||||||
|
go_git_ref = str(payload.get("goGitRef") or "").strip()
|
||||||
|
image_tag = str(payload.get("imageTag") or "").strip()
|
||||||
|
skip_maven_build = bool(payload.get("skipMavenBuild"))
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = deploy_updated_services_payload(
|
||||||
|
services,
|
||||||
|
java_git_ref=java_git_ref,
|
||||||
|
go_git_ref=go_git_ref,
|
||||||
|
image_tag=image_tag,
|
||||||
|
skip_maven_build=skip_maven_build,
|
||||||
|
)
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
return self.send_error_payload(HTTPStatus.BAD_GATEWAY, str(exc), send_body=True)
|
||||||
|
|
||||||
|
return self.send_payload(
|
||||||
|
HTTPStatus.OK,
|
||||||
|
response_json(result),
|
||||||
|
"application/json; charset=utf-8",
|
||||||
|
send_body=True,
|
||||||
|
)
|
||||||
|
|
||||||
def query_value(self, query: dict[str, list[str]], key: str) -> str:
|
def query_value(self, query: dict[str, list[str]], key: str) -> str:
|
||||||
# 统一读取单值 query 参数。
|
# 统一读取单值 query 参数。
|
||||||
return str((query.get(key) or [""])[0]).strip()
|
return str((query.get(key) or [""])[0]).strip()
|
||||||
|
|||||||
@ -6,6 +6,7 @@ import shlex
|
|||||||
import time
|
import time
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
|
from .compose_templates import sync_remote_host_compose_template
|
||||||
from .config import (
|
from .config import (
|
||||||
ROLLING_RESTART_POLL_SECONDS,
|
ROLLING_RESTART_POLL_SECONDS,
|
||||||
ROLLING_RESTART_STABILIZE_SECONDS,
|
ROLLING_RESTART_STABILIZE_SECONDS,
|
||||||
@ -194,6 +195,7 @@ def rolling_restart_payload(service_names: list[str]) -> dict[str, Any]:
|
|||||||
|
|
||||||
records_map = service_records_by_name()
|
records_map = service_records_by_name()
|
||||||
steps: list[dict[str, Any]] = []
|
steps: list[dict[str, Any]] = []
|
||||||
|
prepared_hosts: dict[str, dict[str, Any]] = {}
|
||||||
|
|
||||||
for service_name in normalized_names:
|
for service_name in normalized_names:
|
||||||
if service_name not in records_map:
|
if service_name not in records_map:
|
||||||
@ -210,6 +212,18 @@ def rolling_restart_payload(service_names: list[str]) -> dict[str, Any]:
|
|||||||
}
|
}
|
||||||
|
|
||||||
try:
|
try:
|
||||||
|
# 单机第一次进入重启前,先把完整 compose 模板同步到远端。
|
||||||
|
if service["host"] not in prepared_hosts:
|
||||||
|
prepared_hosts[service["host"]] = sync_remote_host_compose_template(
|
||||||
|
service["host"],
|
||||||
|
refresh_from_remote=True,
|
||||||
|
)
|
||||||
|
step["compose"] = {
|
||||||
|
"path": prepared_hosts[service["host"]]["path"],
|
||||||
|
"changed": bool(prepared_hosts[service["host"]].get("changed")),
|
||||||
|
"source": str(prepared_hosts[service["host"]].get("source") or ""),
|
||||||
|
}
|
||||||
|
|
||||||
restart_result = restart_service_instance(service)
|
restart_result = restart_service_instance(service)
|
||||||
step["tatStatus"] = restart_result.get("status")
|
step["tatStatus"] = restart_result.get("status")
|
||||||
step["tatOutput"] = str(restart_result.get("output") or "").strip()[:500]
|
step["tatOutput"] = str(restart_result.get("output") or "").strip()[:500]
|
||||||
|
|||||||
766
monitor/service_release.py
Normal file
766
monitor/service_release.py
Normal file
@ -0,0 +1,766 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
# 镜像 tag 需要拼时间戳。
|
||||||
|
from datetime import datetime
|
||||||
|
# 本地构建和 git worktree 都依赖标准库 subprocess / tempfile。
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
import re
|
||||||
|
import shlex
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import tempfile
|
||||||
|
import time
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from .compose_templates import (
|
||||||
|
load_local_host_compose_model,
|
||||||
|
sync_remote_host_compose_template,
|
||||||
|
update_local_service_image,
|
||||||
|
)
|
||||||
|
from .config import (
|
||||||
|
BUILD_DOCKER_PLATFORM,
|
||||||
|
GOLANG_REPO_ROOT,
|
||||||
|
HARBOR_PASSWORD,
|
||||||
|
HARBOR_PROJECT,
|
||||||
|
HARBOR_REGISTRY,
|
||||||
|
HARBOR_USERNAME,
|
||||||
|
JAVA_REPO_ROOT,
|
||||||
|
ROLLING_RESTART_STABILIZE_SECONDS,
|
||||||
|
ROLLING_RESTART_TIMEOUT_SECONDS,
|
||||||
|
SERVICE_BUILD_TIMEOUT_SECONDS,
|
||||||
|
SERVICE_PULL_TIMEOUT_SECONDS,
|
||||||
|
UPDATE_DEFAULT_GO_GIT_REF,
|
||||||
|
UPDATE_DEFAULT_JAVA_GIT_REF,
|
||||||
|
utc_now,
|
||||||
|
)
|
||||||
|
from .service_ops import (
|
||||||
|
clear_operation_caches,
|
||||||
|
restart_service_instance,
|
||||||
|
service_records_by_name,
|
||||||
|
wait_http_ready,
|
||||||
|
wait_java_registration_ready,
|
||||||
|
)
|
||||||
|
from .tencent_tat import run_shell_command
|
||||||
|
|
||||||
|
|
||||||
|
# Java 仓里当前支持 Harbor 更新的服务集合。
|
||||||
|
JAVA_BUILDABLE_SERVICES = ("auth", "gateway", "external", "console", "other", "live", "wallet", "order")
|
||||||
|
# Go 仓目前只有主业务服务。
|
||||||
|
GOLANG_BUILDABLE_SERVICES = ("golang",)
|
||||||
|
# Java 批量 Maven 构建时,需要映射回对应的启动模块。
|
||||||
|
JAVA_SERVICE_MODULES = {
|
||||||
|
"auth": "rc-auth",
|
||||||
|
"gateway": "rc-gateway",
|
||||||
|
"external": "rc-service/rc-service-external/external-start",
|
||||||
|
"console": "rc-service/rc-service-console/console-start",
|
||||||
|
"other": "rc-service/rc-service-other/other-start",
|
||||||
|
"live": "rc-service/rc-service-live/live-start",
|
||||||
|
"wallet": "rc-service/rc-service-wallet/wallet-start",
|
||||||
|
"order": "rc-service/rc-service-order/order-start",
|
||||||
|
}
|
||||||
|
# 当前线上镜像仓路径并不完全统一,这里只给缺失模板信息时兜底。
|
||||||
|
SERVICE_IMAGE_REPOSITORY_SUFFIXES = {
|
||||||
|
"auth": "java/chatapp3-auth",
|
||||||
|
"gateway": "java/chatapp3-gateway",
|
||||||
|
"external": "java/chatapp3-external",
|
||||||
|
"console": "console",
|
||||||
|
"other": "java/chatapp3-other",
|
||||||
|
"live": "java/chatapp3-live",
|
||||||
|
"wallet": "java/chatapp3-wallet",
|
||||||
|
"order": "order",
|
||||||
|
"golang": "golang",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def buildable_service_names() -> list[str]:
|
||||||
|
# 页面和接口统一按这里判定哪些服务允许“构建并更新”。
|
||||||
|
return [*JAVA_BUILDABLE_SERVICES, *GOLANG_BUILDABLE_SERVICES]
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_service_names(service_names: list[str]) -> list[str]:
|
||||||
|
# 请求里可能重复勾选同一个服务,这里先去重并保序。
|
||||||
|
normalized = [str(name or "").strip() for name in service_names if str(name or "").strip()]
|
||||||
|
return list(dict.fromkeys(normalized))
|
||||||
|
|
||||||
|
|
||||||
|
def service_kind(service_name: str) -> str:
|
||||||
|
# 更新链路目前只分 Java / Golang 两类仓库。
|
||||||
|
if service_name in JAVA_BUILDABLE_SERVICES:
|
||||||
|
return "java"
|
||||||
|
if service_name in GOLANG_BUILDABLE_SERVICES:
|
||||||
|
return "golang"
|
||||||
|
raise RuntimeError(f"unsupported update service: {service_name}")
|
||||||
|
|
||||||
|
|
||||||
|
def ensure_repo_root(repo_root: Path, label: str) -> Path:
|
||||||
|
# 构建前先确认业务仓真的存在,避免跑到一半才发现路径错了。
|
||||||
|
resolved = Path(repo_root).expanduser().resolve()
|
||||||
|
if not resolved.exists():
|
||||||
|
raise RuntimeError(f"{label} repo root missing: {resolved}")
|
||||||
|
if not (resolved / ".git").exists():
|
||||||
|
raise RuntimeError(f"{label} repo root is not a git repo: {resolved}")
|
||||||
|
return resolved
|
||||||
|
|
||||||
|
|
||||||
|
def local_command_output(
|
||||||
|
command: list[str],
|
||||||
|
*,
|
||||||
|
cwd: Path,
|
||||||
|
label: str,
|
||||||
|
env: dict[str, str] | None = None,
|
||||||
|
input_text: str = "",
|
||||||
|
timeout_seconds: int = 120,
|
||||||
|
) -> str:
|
||||||
|
# 小输出命令直接走 capture_output,适合 git rev-parse / docker inspect 这类场景。
|
||||||
|
merged_env = os.environ.copy()
|
||||||
|
if env:
|
||||||
|
merged_env.update(env)
|
||||||
|
|
||||||
|
result = subprocess.run(
|
||||||
|
command,
|
||||||
|
cwd=str(cwd),
|
||||||
|
text=True,
|
||||||
|
input=input_text if input_text else None,
|
||||||
|
capture_output=True,
|
||||||
|
env=merged_env,
|
||||||
|
timeout=timeout_seconds,
|
||||||
|
check=False,
|
||||||
|
)
|
||||||
|
if result.returncode != 0:
|
||||||
|
detail = (result.stderr or result.stdout or "").strip()
|
||||||
|
raise RuntimeError(f"{label} failed: {detail or f'exit {result.returncode}'}")
|
||||||
|
return (result.stdout or "").strip()
|
||||||
|
|
||||||
|
|
||||||
|
def tail_text(path: Path, max_bytes: int = 8192) -> str:
|
||||||
|
# 大构建日志只回最后一小段,避免把整份 Maven 输出塞进响应体。
|
||||||
|
if not path.exists():
|
||||||
|
return ""
|
||||||
|
|
||||||
|
with path.open("rb") as handle:
|
||||||
|
handle.seek(0, os.SEEK_END)
|
||||||
|
size = handle.tell()
|
||||||
|
handle.seek(max(0, size - max_bytes))
|
||||||
|
return handle.read().decode("utf-8", errors="replace").strip()
|
||||||
|
|
||||||
|
|
||||||
|
def run_logged_command(
|
||||||
|
command: list[str],
|
||||||
|
*,
|
||||||
|
cwd: Path,
|
||||||
|
label: str,
|
||||||
|
env: dict[str, str] | None = None,
|
||||||
|
timeout_seconds: int = SERVICE_BUILD_TIMEOUT_SECONDS,
|
||||||
|
) -> str:
|
||||||
|
# Maven / docker build 输出很大,单独落临时日志再取尾部更稳。
|
||||||
|
merged_env = os.environ.copy()
|
||||||
|
if env:
|
||||||
|
merged_env.update(env)
|
||||||
|
|
||||||
|
log_file = tempfile.NamedTemporaryFile(prefix="hy-app-monitor-build-", suffix=".log", delete=False)
|
||||||
|
log_path = Path(log_file.name)
|
||||||
|
log_file.close()
|
||||||
|
|
||||||
|
try:
|
||||||
|
with log_path.open("w", encoding="utf-8") as handle:
|
||||||
|
process = subprocess.Popen(
|
||||||
|
command,
|
||||||
|
cwd=str(cwd),
|
||||||
|
stdout=handle,
|
||||||
|
stderr=subprocess.STDOUT,
|
||||||
|
text=True,
|
||||||
|
env=merged_env,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
return_code = process.wait(timeout=timeout_seconds)
|
||||||
|
except subprocess.TimeoutExpired as exc:
|
||||||
|
process.kill()
|
||||||
|
process.wait()
|
||||||
|
raise RuntimeError(f"{label} timed out: {tail_text(log_path)}") from exc
|
||||||
|
|
||||||
|
output_tail = tail_text(log_path)
|
||||||
|
if return_code != 0:
|
||||||
|
raise RuntimeError(f"{label} failed: {output_tail or f'exit {return_code}'}")
|
||||||
|
return output_tail
|
||||||
|
finally:
|
||||||
|
log_path.unlink(missing_ok=True)
|
||||||
|
|
||||||
|
|
||||||
|
def fetch_repo_refs(repo_root: Path, label: str) -> None:
|
||||||
|
# 统一先刷新远端分支和 tag,后面的 ref 解析才可信。
|
||||||
|
local_command_output(
|
||||||
|
["git", "-C", str(repo_root), "fetch", "--all", "--tags", "--prune"],
|
||||||
|
cwd=repo_root,
|
||||||
|
label=f"{label} fetch refs",
|
||||||
|
timeout_seconds=300,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def resolve_repo_commit(repo_root: Path, git_ref: str, label: str) -> str:
|
||||||
|
# 优先把分支解析到 origin/<branch>,避免本地旧分支 HEAD 干扰构建。
|
||||||
|
candidates = [
|
||||||
|
f"refs/remotes/origin/{git_ref}^{{commit}}",
|
||||||
|
f"refs/tags/{git_ref}^{{commit}}",
|
||||||
|
f"{git_ref}^{{commit}}",
|
||||||
|
]
|
||||||
|
for candidate in candidates:
|
||||||
|
try:
|
||||||
|
return local_command_output(
|
||||||
|
["git", "-C", str(repo_root), "rev-parse", "--verify", candidate],
|
||||||
|
cwd=repo_root,
|
||||||
|
label=f"{label} resolve ref",
|
||||||
|
)
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
continue
|
||||||
|
raise RuntimeError(f"{label} git ref not found: {git_ref}")
|
||||||
|
|
||||||
|
|
||||||
|
def create_temp_worktree(repo_root: Path, commit: str, label: str) -> Path:
|
||||||
|
# 用临时 worktree 构建,避免直接改业务仓当前工作区。
|
||||||
|
worktree_root = Path(tempfile.mkdtemp(prefix=f"hy-app-monitor-{repo_root.name}-"))
|
||||||
|
try:
|
||||||
|
local_command_output(
|
||||||
|
["git", "-C", str(repo_root), "worktree", "add", "--detach", str(worktree_root), commit],
|
||||||
|
cwd=repo_root,
|
||||||
|
label=f"{label} create worktree",
|
||||||
|
timeout_seconds=300,
|
||||||
|
)
|
||||||
|
return worktree_root
|
||||||
|
except Exception:
|
||||||
|
shutil.rmtree(worktree_root, ignore_errors=True)
|
||||||
|
raise
|
||||||
|
|
||||||
|
|
||||||
|
def remove_temp_worktree(repo_root: Path, worktree_root: Path) -> None:
|
||||||
|
# 构建结束后清理 worktree,避免 deploy 机累积一堆临时目录。
|
||||||
|
try:
|
||||||
|
local_command_output(
|
||||||
|
["git", "-C", str(repo_root), "worktree", "remove", "--force", str(worktree_root)],
|
||||||
|
cwd=repo_root,
|
||||||
|
label="remove worktree",
|
||||||
|
timeout_seconds=120,
|
||||||
|
)
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
shutil.rmtree(worktree_root, ignore_errors=True)
|
||||||
|
|
||||||
|
|
||||||
|
def require_harbor_credentials() -> None:
|
||||||
|
# 构建推送和目标机预拉都依赖 Harbor 账号。
|
||||||
|
if not HARBOR_REGISTRY:
|
||||||
|
raise RuntimeError("missing HARBOR_REGISTRY in .env")
|
||||||
|
if not HARBOR_USERNAME or not HARBOR_PASSWORD:
|
||||||
|
raise RuntimeError("missing HARBOR_USERNAME / HARBOR_PASSWORD in .env")
|
||||||
|
|
||||||
|
|
||||||
|
def docker_login_local(repo_root: Path) -> None:
|
||||||
|
# 本机构建前先登录 Harbor,避免 push 时才失败。
|
||||||
|
require_harbor_credentials()
|
||||||
|
local_command_output(
|
||||||
|
["docker", "login", HARBOR_REGISTRY, "-u", HARBOR_USERNAME, "--password-stdin"],
|
||||||
|
cwd=repo_root,
|
||||||
|
label="docker login harbor",
|
||||||
|
input_text=HARBOR_PASSWORD,
|
||||||
|
timeout_seconds=120,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def image_repository_ref(image_ref: str) -> str:
|
||||||
|
# 把 tag / digest 去掉,保留完整仓库地址。
|
||||||
|
text = str(image_ref or "").strip()
|
||||||
|
if "@" in text:
|
||||||
|
return text.split("@", 1)[0]
|
||||||
|
|
||||||
|
last_slash = text.rfind("/")
|
||||||
|
last_colon = text.rfind(":")
|
||||||
|
if last_colon > last_slash:
|
||||||
|
return text[:last_colon]
|
||||||
|
return text
|
||||||
|
|
||||||
|
|
||||||
|
def sanitized_ref_fragment(git_ref: str) -> str:
|
||||||
|
# Harbor tag 只保留相对安全的字符。
|
||||||
|
cleaned = re.sub(r"[^0-9A-Za-z._-]+", "-", str(git_ref or "").strip()) or "manual"
|
||||||
|
return cleaned.strip("-") or "manual"
|
||||||
|
|
||||||
|
|
||||||
|
def derive_image_tag(git_ref: str, commit: str, image_tag: str) -> str:
|
||||||
|
# 用户没指定 tag 时,默认用 ref + commit12 + 时间戳。
|
||||||
|
explicit = str(image_tag or "").strip()
|
||||||
|
if explicit:
|
||||||
|
return explicit
|
||||||
|
timestamp = datetime.utcnow().strftime("%Y%m%d-%H%M%S")
|
||||||
|
return f"{sanitized_ref_fragment(git_ref)}-{commit[:12]}-{timestamp}"
|
||||||
|
|
||||||
|
|
||||||
|
def docker_digest(image_ref: str, *, cwd: Path) -> str:
|
||||||
|
# push 完要拿 digest,后续 compose 一律按 digest 部署。
|
||||||
|
return local_command_output(
|
||||||
|
["docker", "image", "inspect", image_ref, "--format", "{{index .RepoDigests 0}}"],
|
||||||
|
cwd=cwd,
|
||||||
|
label=f"inspect digest {image_ref}",
|
||||||
|
timeout_seconds=120,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def current_template_image(service_name: str) -> str:
|
||||||
|
# 优先从本地完整模板里拿当前运行时镜像仓路径。
|
||||||
|
records = service_records_by_name().get(service_name) or []
|
||||||
|
for record in records:
|
||||||
|
model = load_local_host_compose_model(record["host"])
|
||||||
|
image_ref = str(((model.get("services") or {}).get(service_name) or {}).get("image") or "").strip()
|
||||||
|
if image_ref.startswith(f"{HARBOR_REGISTRY}/{HARBOR_PROJECT}/"):
|
||||||
|
return image_ref
|
||||||
|
return ""
|
||||||
|
|
||||||
|
|
||||||
|
def service_repository_ref(service_name: str) -> str:
|
||||||
|
# 线上已有镜像时沿用现有仓路径;没有时再走保守兜底。
|
||||||
|
current_image = current_template_image(service_name)
|
||||||
|
if current_image:
|
||||||
|
return image_repository_ref(current_image)
|
||||||
|
|
||||||
|
suffix = SERVICE_IMAGE_REPOSITORY_SUFFIXES.get(service_name, "").strip()
|
||||||
|
if not suffix:
|
||||||
|
raise RuntimeError(f"missing Harbor repository mapping for service: {service_name}")
|
||||||
|
return f"{HARBOR_REGISTRY}/{HARBOR_PROJECT}/{suffix}"
|
||||||
|
|
||||||
|
|
||||||
|
def maven_module_list(service_names: list[str]) -> str:
|
||||||
|
# Java 多服务更新时,先一次性 Maven 打包对应模块,避免重复 clean package。
|
||||||
|
modules: list[str] = []
|
||||||
|
for service_name in service_names:
|
||||||
|
module = JAVA_SERVICE_MODULES.get(service_name)
|
||||||
|
if not module:
|
||||||
|
raise RuntimeError(f"missing java maven module mapping: {service_name}")
|
||||||
|
if module not in modules:
|
||||||
|
modules.append(module)
|
||||||
|
return ",".join(modules)
|
||||||
|
|
||||||
|
|
||||||
|
def build_java_images(service_names: list[str], git_ref: str, image_tag: str, *, skip_maven_build: bool) -> dict[str, Any]:
|
||||||
|
# Java 服务共享一个仓库和 Maven 本地缓存,这里按仓统一构建。
|
||||||
|
repo_root = ensure_repo_root(JAVA_REPO_ROOT, "java")
|
||||||
|
resolved_ref = str(git_ref or "").strip() or UPDATE_DEFAULT_JAVA_GIT_REF
|
||||||
|
fetch_repo_refs(repo_root, "java")
|
||||||
|
commit = resolve_repo_commit(repo_root, resolved_ref, "java")
|
||||||
|
worktree_root = create_temp_worktree(repo_root, commit, "java")
|
||||||
|
short_commit = commit[:12]
|
||||||
|
resolved_image_tag = derive_image_tag(resolved_ref, commit, image_tag)
|
||||||
|
|
||||||
|
outputs: list[dict[str, Any]] = []
|
||||||
|
service_images: dict[str, str] = {}
|
||||||
|
|
||||||
|
try:
|
||||||
|
if not skip_maven_build:
|
||||||
|
run_logged_command(
|
||||||
|
[str(worktree_root / "ci" / "install-private-maven.sh")],
|
||||||
|
cwd=worktree_root,
|
||||||
|
label="java install private maven",
|
||||||
|
timeout_seconds=min(SERVICE_BUILD_TIMEOUT_SECONDS, 1200),
|
||||||
|
)
|
||||||
|
run_logged_command(
|
||||||
|
[
|
||||||
|
"mvn",
|
||||||
|
"-pl",
|
||||||
|
maven_module_list(service_names),
|
||||||
|
"-am",
|
||||||
|
"-Dmaven.test.skip=true",
|
||||||
|
"clean",
|
||||||
|
"package",
|
||||||
|
],
|
||||||
|
cwd=worktree_root,
|
||||||
|
label="java maven package",
|
||||||
|
timeout_seconds=SERVICE_BUILD_TIMEOUT_SECONDS,
|
||||||
|
)
|
||||||
|
|
||||||
|
for service_name in service_names:
|
||||||
|
repository_ref = service_repository_ref(service_name)
|
||||||
|
target_image_ref = f"{repository_ref}:{resolved_image_tag}"
|
||||||
|
build_tail = run_logged_command(
|
||||||
|
[str(worktree_root / "ci" / "build-service-image.sh"), service_name, target_image_ref, short_commit],
|
||||||
|
cwd=worktree_root,
|
||||||
|
label=f"java build {service_name}",
|
||||||
|
env={
|
||||||
|
"DOCKER_PLATFORM": BUILD_DOCKER_PLATFORM,
|
||||||
|
"SKIP_MAVEN_BUILD": "1" if not skip_maven_build else "1",
|
||||||
|
},
|
||||||
|
timeout_seconds=SERVICE_BUILD_TIMEOUT_SECONDS,
|
||||||
|
)
|
||||||
|
push_tail = run_logged_command(
|
||||||
|
["docker", "push", target_image_ref],
|
||||||
|
cwd=worktree_root,
|
||||||
|
label=f"java push {service_name}",
|
||||||
|
timeout_seconds=SERVICE_BUILD_TIMEOUT_SECONDS,
|
||||||
|
)
|
||||||
|
digest_ref = docker_digest(target_image_ref, cwd=worktree_root)
|
||||||
|
service_images[service_name] = digest_ref
|
||||||
|
outputs.append(
|
||||||
|
{
|
||||||
|
"service": service_name,
|
||||||
|
"repository": repository_ref,
|
||||||
|
"tag": target_image_ref,
|
||||||
|
"image": digest_ref,
|
||||||
|
"buildTail": build_tail,
|
||||||
|
"pushTail": push_tail,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
remove_temp_worktree(repo_root, worktree_root)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"kind": "java",
|
||||||
|
"repoRoot": str(repo_root),
|
||||||
|
"gitRef": resolved_ref,
|
||||||
|
"commit": commit,
|
||||||
|
"imageTag": resolved_image_tag,
|
||||||
|
"services": list(service_names),
|
||||||
|
"items": outputs,
|
||||||
|
"serviceImages": service_images,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def build_golang_images(service_names: list[str], git_ref: str, image_tag: str) -> dict[str, Any]:
|
||||||
|
# Go 服务当前是单仓构建,逻辑更简单。
|
||||||
|
repo_root = ensure_repo_root(GOLANG_REPO_ROOT, "golang")
|
||||||
|
resolved_ref = str(git_ref or "").strip() or UPDATE_DEFAULT_GO_GIT_REF
|
||||||
|
fetch_repo_refs(repo_root, "golang")
|
||||||
|
commit = resolve_repo_commit(repo_root, resolved_ref, "golang")
|
||||||
|
worktree_root = create_temp_worktree(repo_root, commit, "golang")
|
||||||
|
short_commit = commit[:12]
|
||||||
|
resolved_image_tag = derive_image_tag(resolved_ref, commit, image_tag)
|
||||||
|
|
||||||
|
outputs: list[dict[str, Any]] = []
|
||||||
|
service_images: dict[str, str] = {}
|
||||||
|
|
||||||
|
try:
|
||||||
|
for service_name in service_names:
|
||||||
|
repository_ref = service_repository_ref(service_name)
|
||||||
|
target_image_ref = f"{repository_ref}:{resolved_image_tag}"
|
||||||
|
build_tail = run_logged_command(
|
||||||
|
[str(worktree_root / "ci" / "build-image.sh"), target_image_ref, short_commit],
|
||||||
|
cwd=worktree_root,
|
||||||
|
label=f"golang build {service_name}",
|
||||||
|
env={"DOCKER_PLATFORM": BUILD_DOCKER_PLATFORM},
|
||||||
|
timeout_seconds=SERVICE_BUILD_TIMEOUT_SECONDS,
|
||||||
|
)
|
||||||
|
push_tail = run_logged_command(
|
||||||
|
["docker", "push", target_image_ref],
|
||||||
|
cwd=worktree_root,
|
||||||
|
label=f"golang push {service_name}",
|
||||||
|
timeout_seconds=SERVICE_BUILD_TIMEOUT_SECONDS,
|
||||||
|
)
|
||||||
|
digest_ref = docker_digest(target_image_ref, cwd=worktree_root)
|
||||||
|
service_images[service_name] = digest_ref
|
||||||
|
outputs.append(
|
||||||
|
{
|
||||||
|
"service": service_name,
|
||||||
|
"repository": repository_ref,
|
||||||
|
"tag": target_image_ref,
|
||||||
|
"image": digest_ref,
|
||||||
|
"buildTail": build_tail,
|
||||||
|
"pushTail": push_tail,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
finally:
|
||||||
|
remove_temp_worktree(repo_root, worktree_root)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"kind": "golang",
|
||||||
|
"repoRoot": str(repo_root),
|
||||||
|
"gitRef": resolved_ref,
|
||||||
|
"commit": commit,
|
||||||
|
"imageTag": resolved_image_tag,
|
||||||
|
"services": list(service_names),
|
||||||
|
"items": outputs,
|
||||||
|
"serviceImages": service_images,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def build_service_images(
|
||||||
|
service_names: list[str],
|
||||||
|
*,
|
||||||
|
java_git_ref: str,
|
||||||
|
go_git_ref: str,
|
||||||
|
image_tag: str,
|
||||||
|
skip_maven_build: bool,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
# 一个请求里可能同时选了 Java 和 Go,需要分仓构建后再合并 digest。
|
||||||
|
normalized = normalize_service_names(service_names)
|
||||||
|
java_services = [service_name for service_name in normalized if service_kind(service_name) == "java"]
|
||||||
|
golang_services = [service_name for service_name in normalized if service_kind(service_name) == "golang"]
|
||||||
|
|
||||||
|
if not normalized:
|
||||||
|
raise RuntimeError("services are required")
|
||||||
|
|
||||||
|
docker_login_local(ensure_repo_root(JAVA_REPO_ROOT if java_services else GOLANG_REPO_ROOT, "build"))
|
||||||
|
|
||||||
|
builds: list[dict[str, Any]] = []
|
||||||
|
service_images: dict[str, str] = {}
|
||||||
|
if java_services:
|
||||||
|
java_build = build_java_images(java_services, java_git_ref, image_tag, skip_maven_build=skip_maven_build)
|
||||||
|
builds.append(java_build)
|
||||||
|
service_images.update(java_build["serviceImages"])
|
||||||
|
if golang_services:
|
||||||
|
golang_build = build_golang_images(golang_services, go_git_ref, image_tag)
|
||||||
|
builds.append(golang_build)
|
||||||
|
service_images.update(golang_build["serviceImages"])
|
||||||
|
|
||||||
|
return {
|
||||||
|
"builds": builds,
|
||||||
|
"serviceImages": service_images,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def host_image_plan(service_names: list[str], service_images: dict[str, str]) -> list[dict[str, Any]]:
|
||||||
|
# 先按宿主机聚合 image,后面可以先预拉再做滚动重建。
|
||||||
|
records_map = service_records_by_name()
|
||||||
|
items: list[dict[str, Any]] = []
|
||||||
|
host_index: dict[str, dict[str, Any]] = {}
|
||||||
|
|
||||||
|
for service_name in service_names:
|
||||||
|
image_ref = str(service_images.get(service_name) or "").strip()
|
||||||
|
if not image_ref:
|
||||||
|
raise RuntimeError(f"missing built image for service: {service_name}")
|
||||||
|
|
||||||
|
for record in records_map.get(service_name) or []:
|
||||||
|
host_entry = host_index.get(record["host"])
|
||||||
|
if host_entry is None:
|
||||||
|
host_entry = {
|
||||||
|
"host": record["host"],
|
||||||
|
"ip": record["ip"],
|
||||||
|
"instanceId": record["instanceId"],
|
||||||
|
"services": [],
|
||||||
|
"images": [],
|
||||||
|
}
|
||||||
|
host_index[record["host"]] = host_entry
|
||||||
|
items.append(host_entry)
|
||||||
|
|
||||||
|
if service_name not in host_entry["services"]:
|
||||||
|
host_entry["services"].append(service_name)
|
||||||
|
if image_ref not in host_entry["images"]:
|
||||||
|
host_entry["images"].append(image_ref)
|
||||||
|
|
||||||
|
return items
|
||||||
|
|
||||||
|
|
||||||
|
def render_remote_login_lines() -> list[str]:
|
||||||
|
# 目标机拉 Harbor 镜像前同样需要显式登录。
|
||||||
|
require_harbor_credentials()
|
||||||
|
return [
|
||||||
|
f"printf '%s' {shlex.quote(HARBOR_PASSWORD)} | docker login {shlex.quote(HARBOR_REGISTRY)} -u {shlex.quote(HARBOR_USERNAME)} --password-stdin",
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
def remote_host_result(host_entry: dict[str, Any], script: str, command_name: str, timeout_seconds: int) -> dict[str, Any]:
|
||||||
|
# 统一执行单宿主机命令,便于预拉镜像和其他操作复用。
|
||||||
|
result_map = run_shell_command(
|
||||||
|
[host_entry["instanceId"]],
|
||||||
|
script,
|
||||||
|
command_name=command_name,
|
||||||
|
timeout_seconds=timeout_seconds,
|
||||||
|
)
|
||||||
|
result = result_map[host_entry["instanceId"]]
|
||||||
|
if result.get("status") != "SUCCESS":
|
||||||
|
raise RuntimeError(f"{host_entry['host']} {command_name} failed: {result.get('status')}")
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def prepull_host_images(host_entry: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
# 先把该宿主机上要更新的镜像拉好,减少单实例重建时的等待。
|
||||||
|
lines = ["set -eu", *render_remote_login_lines()]
|
||||||
|
for image_ref in host_entry.get("images") or []:
|
||||||
|
lines.append(f"docker pull {shlex.quote(str(image_ref))}")
|
||||||
|
script = "\n".join(lines) + "\n"
|
||||||
|
result = remote_host_result(
|
||||||
|
host_entry,
|
||||||
|
script,
|
||||||
|
f"prepull-{host_entry['host']}",
|
||||||
|
timeout_seconds=max(SERVICE_PULL_TIMEOUT_SECONDS, 120),
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"status": result.get("status"),
|
||||||
|
"output": str(result.get("output") or "").strip()[-1200:],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def build_update_targets_payload() -> dict[str, Any]:
|
||||||
|
# 页面需要展示当前可更新服务、宿主机,以及本地构建所依赖的仓路径。
|
||||||
|
records_map = service_records_by_name()
|
||||||
|
items: list[dict[str, Any]] = []
|
||||||
|
|
||||||
|
for service_name in buildable_service_names():
|
||||||
|
records = records_map.get(service_name) or []
|
||||||
|
if not records:
|
||||||
|
continue
|
||||||
|
current_images: list[str] = []
|
||||||
|
for record in records:
|
||||||
|
model = load_local_host_compose_model(record["host"])
|
||||||
|
image_ref = str(((model.get("services") or {}).get(service_name) or {}).get("image") or "").strip()
|
||||||
|
if image_ref and image_ref not in current_images:
|
||||||
|
current_images.append(image_ref)
|
||||||
|
|
||||||
|
items.append(
|
||||||
|
{
|
||||||
|
"service": service_name,
|
||||||
|
"kind": service_kind(service_name),
|
||||||
|
"repository": service_repository_ref(service_name),
|
||||||
|
"currentImages": current_images,
|
||||||
|
"hosts": [
|
||||||
|
{
|
||||||
|
"host": record["host"],
|
||||||
|
"ip": record["ip"],
|
||||||
|
"instanceId": record["instanceId"],
|
||||||
|
"port": record["port"],
|
||||||
|
"path": record["path"],
|
||||||
|
}
|
||||||
|
for record in records
|
||||||
|
],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"ok": True,
|
||||||
|
"updatedAt": utc_now(),
|
||||||
|
"defaults": {
|
||||||
|
"javaGitRef": UPDATE_DEFAULT_JAVA_GIT_REF,
|
||||||
|
"goGitRef": UPDATE_DEFAULT_GO_GIT_REF,
|
||||||
|
"dockerPlatform": BUILD_DOCKER_PLATFORM,
|
||||||
|
},
|
||||||
|
"builder": {
|
||||||
|
"harborRegistry": HARBOR_REGISTRY,
|
||||||
|
"harborProject": HARBOR_PROJECT,
|
||||||
|
"javaRepoRoot": str(Path(JAVA_REPO_ROOT).expanduser()),
|
||||||
|
"golangRepoRoot": str(Path(GOLANG_REPO_ROOT).expanduser()),
|
||||||
|
},
|
||||||
|
"items": items,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def deploy_updated_services_payload(
|
||||||
|
service_names: list[str],
|
||||||
|
*,
|
||||||
|
java_git_ref: str,
|
||||||
|
go_git_ref: str,
|
||||||
|
image_tag: str,
|
||||||
|
skip_maven_build: bool,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
# 这是“服务更新”主链路:本地 build -> Harbor digest -> 完整 compose -> 预拉 -> 滚动重建。
|
||||||
|
normalized = normalize_service_names(service_names)
|
||||||
|
if not normalized:
|
||||||
|
raise RuntimeError("services are required")
|
||||||
|
|
||||||
|
records_map = service_records_by_name()
|
||||||
|
unsupported = [service_name for service_name in normalized if service_name not in buildable_service_names()]
|
||||||
|
if unsupported:
|
||||||
|
raise RuntimeError(f"unsupported update services: {', '.join(unsupported)}")
|
||||||
|
for service_name in normalized:
|
||||||
|
if service_name not in records_map:
|
||||||
|
raise RuntimeError(f"service not found in runtime topology: {service_name}")
|
||||||
|
|
||||||
|
build_result = build_service_images(
|
||||||
|
normalized,
|
||||||
|
java_git_ref=java_git_ref,
|
||||||
|
go_git_ref=go_git_ref,
|
||||||
|
image_tag=image_tag,
|
||||||
|
skip_maven_build=skip_maven_build,
|
||||||
|
)
|
||||||
|
service_images = dict(build_result.get("serviceImages") or {})
|
||||||
|
|
||||||
|
template_updates: list[dict[str, Any]] = []
|
||||||
|
for service_name in normalized:
|
||||||
|
for record in records_map[service_name]:
|
||||||
|
template_updates.append(update_local_service_image(record["host"], service_name, service_images[service_name]))
|
||||||
|
|
||||||
|
host_plan = host_image_plan(normalized, service_images)
|
||||||
|
prepared_hosts: list[dict[str, Any]] = []
|
||||||
|
steps: list[dict[str, Any]] = []
|
||||||
|
|
||||||
|
try:
|
||||||
|
for host_entry in host_plan:
|
||||||
|
compose_result = sync_remote_host_compose_template(host_entry["host"], refresh_from_remote=False)
|
||||||
|
prepull_result = prepull_host_images(host_entry)
|
||||||
|
prepared_hosts.append(
|
||||||
|
{
|
||||||
|
"host": host_entry["host"],
|
||||||
|
"ip": host_entry["ip"],
|
||||||
|
"services": list(host_entry["services"]),
|
||||||
|
"images": list(host_entry["images"]),
|
||||||
|
"compose": {
|
||||||
|
"path": compose_result["path"],
|
||||||
|
"changed": bool(compose_result.get("changed")),
|
||||||
|
"source": str(compose_result.get("source") or ""),
|
||||||
|
"backupPath": str(compose_result.get("backupPath") or ""),
|
||||||
|
},
|
||||||
|
"prepull": {
|
||||||
|
"status": prepull_result["status"],
|
||||||
|
"output": prepull_result["output"],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
for service_name in normalized:
|
||||||
|
for service in records_map[service_name]:
|
||||||
|
started_at = time.time()
|
||||||
|
step: dict[str, Any] = {
|
||||||
|
"service": service_name,
|
||||||
|
"host": service["host"],
|
||||||
|
"ip": service["ip"],
|
||||||
|
"kind": service["kind"],
|
||||||
|
"image": service_images[service_name],
|
||||||
|
"status": "running",
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
restart_result = restart_service_instance(service)
|
||||||
|
step["tatStatus"] = restart_result.get("status")
|
||||||
|
step["tatOutput"] = str(restart_result.get("output") or "").strip()[:500]
|
||||||
|
|
||||||
|
http_result = wait_http_ready(service, timeout_seconds=ROLLING_RESTART_TIMEOUT_SECONDS)
|
||||||
|
step["http"] = {
|
||||||
|
"ok": bool(http_result.get("ok")),
|
||||||
|
"statusCode": int(http_result.get("statusCode") or 0),
|
||||||
|
"latencyMs": int(http_result.get("latencyMs") or 0),
|
||||||
|
"detail": str(http_result.get("detail") or ""),
|
||||||
|
}
|
||||||
|
|
||||||
|
if service["kind"] == "java":
|
||||||
|
step["nacos"] = wait_java_registration_ready(
|
||||||
|
service,
|
||||||
|
timeout_seconds=ROLLING_RESTART_TIMEOUT_SECONDS,
|
||||||
|
)
|
||||||
|
|
||||||
|
if ROLLING_RESTART_STABILIZE_SECONDS > 0:
|
||||||
|
time.sleep(ROLLING_RESTART_STABILIZE_SECONDS)
|
||||||
|
|
||||||
|
step["status"] = "success"
|
||||||
|
step["durationSeconds"] = round(time.time() - started_at, 2)
|
||||||
|
steps.append(step)
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
step["status"] = "failed"
|
||||||
|
step["error"] = str(exc)
|
||||||
|
step["durationSeconds"] = round(time.time() - started_at, 2)
|
||||||
|
steps.append(step)
|
||||||
|
clear_operation_caches()
|
||||||
|
return {
|
||||||
|
"ok": False,
|
||||||
|
"updatedAt": utc_now(),
|
||||||
|
"services": normalized,
|
||||||
|
"build": build_result.get("builds") or [],
|
||||||
|
"templates": template_updates,
|
||||||
|
"preparedHosts": prepared_hosts,
|
||||||
|
"steps": steps,
|
||||||
|
"error": step["error"],
|
||||||
|
}
|
||||||
|
finally:
|
||||||
|
clear_operation_caches()
|
||||||
|
|
||||||
|
return {
|
||||||
|
"ok": True,
|
||||||
|
"updatedAt": utc_now(),
|
||||||
|
"services": normalized,
|
||||||
|
"build": build_result.get("builds") or [],
|
||||||
|
"templates": template_updates,
|
||||||
|
"preparedHosts": prepared_hosts,
|
||||||
|
"steps": steps,
|
||||||
|
"message": "service update complete",
|
||||||
|
}
|
||||||
54
ops/sync_runtime_compose.py
Normal file
54
ops/sync_runtime_compose.py
Normal file
@ -0,0 +1,54 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
# 直接把项目根目录加到 import path,脚本可以从任意 cwd 调起。
|
||||||
|
ROOT = Path(__file__).resolve().parents[1]
|
||||||
|
if str(ROOT) not in sys.path:
|
||||||
|
sys.path.insert(0, str(ROOT))
|
||||||
|
|
||||||
|
from monitor.compose_templates import managed_compose_host_names, save_local_host_compose_template, sync_remote_compose_templates
|
||||||
|
|
||||||
|
|
||||||
|
def parse_args() -> argparse.Namespace:
|
||||||
|
# 默认只刷新本地完整模板,不主动改线上 compose。
|
||||||
|
parser = argparse.ArgumentParser(description="refresh full docker compose templates for runtime hosts")
|
||||||
|
parser.add_argument("--hosts", nargs="*", default=[], help="target host names, default all managed service hosts")
|
||||||
|
parser.add_argument(
|
||||||
|
"--sync-remote",
|
||||||
|
action="store_true",
|
||||||
|
help="after refreshing local templates, also sync full compose files to remote hosts",
|
||||||
|
)
|
||||||
|
return parser.parse_args()
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
args = parse_args()
|
||||||
|
host_names = [host_name for host_name in (args.hosts or managed_compose_host_names()) if host_name]
|
||||||
|
|
||||||
|
if args.sync_remote:
|
||||||
|
payload = sync_remote_compose_templates(host_names, refresh_from_remote=True)
|
||||||
|
print(json.dumps(payload, ensure_ascii=False, indent=2))
|
||||||
|
return
|
||||||
|
|
||||||
|
items = [save_local_host_compose_template(host_name) for host_name in host_names]
|
||||||
|
print(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"ok": True,
|
||||||
|
"updatedAt": items[-1]["updatedAt"] if items else "",
|
||||||
|
"hosts": items,
|
||||||
|
},
|
||||||
|
ensure_ascii=False,
|
||||||
|
indent=2,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
@ -527,6 +527,39 @@ body {
|
|||||||
min-width: 220px;
|
min-width: 220px;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.update-form-grid {
|
||||||
|
display: grid;
|
||||||
|
grid-template-columns: repeat(4, minmax(0, 1fr));
|
||||||
|
gap: 12px;
|
||||||
|
margin-top: 14px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.update-field,
|
||||||
|
.update-check {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
gap: 8px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.update-field span,
|
||||||
|
.update-check span {
|
||||||
|
color: var(--muted);
|
||||||
|
font-size: 12px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.update-check {
|
||||||
|
justify-content: center;
|
||||||
|
padding: 12px 14px;
|
||||||
|
border: 1px solid var(--line);
|
||||||
|
border-radius: 18px;
|
||||||
|
background: color-mix(in oklab, var(--panel-strong) 92%, transparent);
|
||||||
|
}
|
||||||
|
|
||||||
|
.update-check input {
|
||||||
|
width: 18px;
|
||||||
|
height: 18px;
|
||||||
|
}
|
||||||
|
|
||||||
.config-layout {
|
.config-layout {
|
||||||
display: grid;
|
display: grid;
|
||||||
grid-template-columns: minmax(260px, 340px) minmax(0, 1fr);
|
grid-template-columns: minmax(260px, 340px) minmax(0, 1fr);
|
||||||
@ -664,6 +697,10 @@ body {
|
|||||||
.config-meta-grid {
|
.config-meta-grid {
|
||||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.update-form-grid {
|
||||||
|
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@media (max-width: 720px) {
|
@media (max-width: 720px) {
|
||||||
@ -683,7 +720,8 @@ body {
|
|||||||
.host-metrics,
|
.host-metrics,
|
||||||
.summary-grid,
|
.summary-grid,
|
||||||
.metric-grid,
|
.metric-grid,
|
||||||
.config-meta-grid {
|
.config-meta-grid,
|
||||||
|
.update-form-grid {
|
||||||
grid-template-columns: 1fr;
|
grid-template-columns: 1fr;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
117
static/app.js
117
static/app.js
@ -85,6 +85,30 @@ createApp({
|
|||||||
diffKeys: [],
|
diffKeys: [],
|
||||||
hosts: [],
|
hosts: [],
|
||||||
},
|
},
|
||||||
|
updateOps: {
|
||||||
|
loading: false,
|
||||||
|
error: "",
|
||||||
|
running: false,
|
||||||
|
message: "",
|
||||||
|
items: [],
|
||||||
|
selectedServices: [],
|
||||||
|
javaGitRef: "main",
|
||||||
|
goGitRef: "main",
|
||||||
|
imageTag: "",
|
||||||
|
skipMavenBuild: false,
|
||||||
|
defaults: {
|
||||||
|
javaGitRef: "main",
|
||||||
|
goGitRef: "main",
|
||||||
|
dockerPlatform: "linux/amd64",
|
||||||
|
},
|
||||||
|
builder: {
|
||||||
|
harborRegistry: "",
|
||||||
|
harborProject: "",
|
||||||
|
javaRepoRoot: "",
|
||||||
|
golangRepoRoot: "",
|
||||||
|
},
|
||||||
|
result: null,
|
||||||
|
},
|
||||||
restartOps: {
|
restartOps: {
|
||||||
loading: false,
|
loading: false,
|
||||||
error: "",
|
error: "",
|
||||||
@ -194,6 +218,9 @@ createApp({
|
|||||||
restartTargetMap() {
|
restartTargetMap() {
|
||||||
return Object.fromEntries((this.restartOps.items || []).map((item) => [item.service, item]));
|
return Object.fromEntries((this.restartOps.items || []).map((item) => [item.service, item]));
|
||||||
},
|
},
|
||||||
|
updateTargetMap() {
|
||||||
|
return Object.fromEntries((this.updateOps.items || []).map((item) => [item.service, item]));
|
||||||
|
},
|
||||||
},
|
},
|
||||||
watch: {
|
watch: {
|
||||||
refreshIntervalSeconds() {
|
refreshIntervalSeconds() {
|
||||||
@ -284,6 +311,9 @@ createApp({
|
|||||||
restartTargetLabel(item) {
|
restartTargetLabel(item) {
|
||||||
return `${item.service} · ${item.kind} · ${item.hosts.map((host) => host.host).join(" / ")}`;
|
return `${item.service} · ${item.kind} · ${item.hosts.map((host) => host.host).join(" / ")}`;
|
||||||
},
|
},
|
||||||
|
updateTargetLabel(item) {
|
||||||
|
return `${item.service} · ${item.kind} · ${item.hosts.map((host) => host.host).join(" / ")}`;
|
||||||
|
},
|
||||||
syncRefreshSettings() {
|
syncRefreshSettings() {
|
||||||
const settings = this.payload.settings || {};
|
const settings = this.payload.settings || {};
|
||||||
const options = Array.isArray(settings.refreshIntervalOptions)
|
const options = Array.isArray(settings.refreshIntervalOptions)
|
||||||
@ -606,6 +636,92 @@ createApp({
|
|||||||
this.restartOps.loading = false;
|
this.restartOps.loading = false;
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
async refreshUpdateTargets() {
|
||||||
|
this.updateOps.loading = true;
|
||||||
|
this.updateOps.error = "";
|
||||||
|
|
||||||
|
try {
|
||||||
|
const response = await fetch("/api/ops/update-targets", { cache: "no-store" });
|
||||||
|
const payload = await this.readJsonResponse(response);
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(payload.error || `HTTP ${response.status}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
this.updateOps.items = Array.isArray(payload.items) ? payload.items : [];
|
||||||
|
this.updateOps.defaults = payload.defaults || this.updateOps.defaults;
|
||||||
|
this.updateOps.builder = payload.builder || this.updateOps.builder;
|
||||||
|
|
||||||
|
if (!this.updateOps.javaGitRef) {
|
||||||
|
this.updateOps.javaGitRef = this.updateOps.defaults.javaGitRef || "main";
|
||||||
|
} else if (this.updateOps.javaGitRef === "main" && this.updateOps.defaults.javaGitRef) {
|
||||||
|
this.updateOps.javaGitRef = this.updateOps.defaults.javaGitRef;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!this.updateOps.goGitRef) {
|
||||||
|
this.updateOps.goGitRef = this.updateOps.defaults.goGitRef || "main";
|
||||||
|
} else if (this.updateOps.goGitRef === "main" && this.updateOps.defaults.goGitRef) {
|
||||||
|
this.updateOps.goGitRef = this.updateOps.defaults.goGitRef;
|
||||||
|
}
|
||||||
|
|
||||||
|
const available = new Set(this.updateOps.items.map((item) => item.service));
|
||||||
|
this.updateOps.selectedServices = this.updateOps.selectedServices.filter((service) => available.has(service));
|
||||||
|
} catch (error) {
|
||||||
|
this.updateOps.error = error instanceof Error ? error.message : String(error);
|
||||||
|
} finally {
|
||||||
|
this.updateOps.loading = false;
|
||||||
|
}
|
||||||
|
},
|
||||||
|
toggleUpdateService(serviceName) {
|
||||||
|
const next = new Set(this.updateOps.selectedServices);
|
||||||
|
if (next.has(serviceName)) {
|
||||||
|
next.delete(serviceName);
|
||||||
|
} else {
|
||||||
|
next.add(serviceName);
|
||||||
|
}
|
||||||
|
this.updateOps.selectedServices = [...next];
|
||||||
|
},
|
||||||
|
async runServiceUpdate() {
|
||||||
|
if (this.updateOps.selectedServices.length === 0) {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
this.updateOps.running = true;
|
||||||
|
this.updateOps.error = "";
|
||||||
|
this.updateOps.message = "";
|
||||||
|
|
||||||
|
try {
|
||||||
|
const response = await fetch("/api/ops/update-services", {
|
||||||
|
method: "POST",
|
||||||
|
headers: {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
},
|
||||||
|
body: JSON.stringify({
|
||||||
|
services: this.updateOps.selectedServices,
|
||||||
|
javaGitRef: this.updateOps.javaGitRef,
|
||||||
|
goGitRef: this.updateOps.goGitRef,
|
||||||
|
imageTag: this.updateOps.imageTag,
|
||||||
|
skipMavenBuild: this.updateOps.skipMavenBuild,
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
const payload = await this.readJsonResponse(response);
|
||||||
|
if (!response.ok) {
|
||||||
|
throw new Error(payload.error || `HTTP ${response.status}`);
|
||||||
|
}
|
||||||
|
|
||||||
|
this.updateOps.result = payload;
|
||||||
|
this.updateOps.message = payload.ok
|
||||||
|
? (payload.message || "服务更新完成")
|
||||||
|
: (payload.error || "服务更新失败");
|
||||||
|
|
||||||
|
await this.refresh();
|
||||||
|
await this.refreshUpdateTargets();
|
||||||
|
await this.refreshRestartTargets();
|
||||||
|
} catch (error) {
|
||||||
|
this.updateOps.error = error instanceof Error ? error.message : String(error);
|
||||||
|
} finally {
|
||||||
|
this.updateOps.running = false;
|
||||||
|
}
|
||||||
|
},
|
||||||
toggleRestartService(serviceName) {
|
toggleRestartService(serviceName) {
|
||||||
const next = new Set(this.restartOps.selectedServices);
|
const next = new Set(this.restartOps.selectedServices);
|
||||||
if (next.has(serviceName)) {
|
if (next.has(serviceName)) {
|
||||||
@ -663,6 +779,7 @@ createApp({
|
|||||||
this.refresh();
|
this.refresh();
|
||||||
this.refreshNacosConfigs({ preserveSelection: false, reloadSelected: true });
|
this.refreshNacosConfigs({ preserveSelection: false, reloadSelected: true });
|
||||||
this.refreshGoConfig();
|
this.refreshGoConfig();
|
||||||
|
this.refreshUpdateTargets();
|
||||||
this.refreshRestartTargets();
|
this.refreshRestartTargets();
|
||||||
},
|
},
|
||||||
beforeUnmount() {
|
beforeUnmount() {
|
||||||
|
|||||||
@ -418,6 +418,156 @@
|
|||||||
</section>
|
</section>
|
||||||
</section>
|
</section>
|
||||||
|
|
||||||
|
<section class="section-head">
|
||||||
|
<h2>服务更新</h2>
|
||||||
|
</section>
|
||||||
|
<section class="infra-panel">
|
||||||
|
<div class="panel-head">
|
||||||
|
<div>
|
||||||
|
<h3>Service Update</h3>
|
||||||
|
<p>本机构建选中的 Java / Golang 服务,推到 Harbor,更新完整 compose,并只重建你选中的服务</p>
|
||||||
|
</div>
|
||||||
|
<span :class="['badge', updateOps.error ? 'bad' : updateOps.running ? 'warn' : 'ok']">
|
||||||
|
{{ updateOps.error ? '存在错误' : updateOps.running ? '执行中' : '可操作' }}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="config-toolbar">
|
||||||
|
<div class="restart-pills">
|
||||||
|
<button
|
||||||
|
v-for="item in updateOps.items"
|
||||||
|
:key="item.service"
|
||||||
|
:class="['config-item', updateOps.selectedServices.includes(item.service) ? 'active' : '']"
|
||||||
|
@click="toggleUpdateService(item.service)"
|
||||||
|
>
|
||||||
|
<strong>{{ item.service }}</strong>
|
||||||
|
<span>{{ item.kind }} · {{ item.hosts.map((host) => host.host).join(' / ') }}</span>
|
||||||
|
<small v-if="item.currentImages && item.currentImages.length > 0" class="mono">
|
||||||
|
{{ item.currentImages.length === 1 ? item.currentImages[0] : `${item.currentImages.length} 个当前镜像` }}
|
||||||
|
</small>
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
<div class="action-group">
|
||||||
|
<button class="refresh ghost" @click="refreshUpdateTargets" :disabled="updateOps.loading || updateOps.running">
|
||||||
|
{{ updateOps.loading ? '加载中...' : '刷新更新目标' }}
|
||||||
|
</button>
|
||||||
|
<button class="refresh danger" @click="runServiceUpdate" :disabled="updateOps.running || updateOps.selectedServices.length === 0">
|
||||||
|
{{ updateOps.running ? '更新中...' : `更新 ${updateOps.selectedServices.length} 项服务` }}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="host-tags">
|
||||||
|
<span class="badge neutral mono">
|
||||||
|
Harbor:{{ updateOps.builder.harborRegistry }} / {{ updateOps.builder.harborProject }}
|
||||||
|
</span>
|
||||||
|
<span class="badge neutral mono">Java 仓:{{ updateOps.builder.javaRepoRoot }}</span>
|
||||||
|
<span class="badge neutral mono">Go 仓:{{ updateOps.builder.golangRepoRoot }}</span>
|
||||||
|
<span class="badge neutral">平台:{{ updateOps.defaults.dockerPlatform }}</span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="update-form-grid">
|
||||||
|
<label class="update-field">
|
||||||
|
<span>Java Git Ref</span>
|
||||||
|
<input v-model.trim="updateOps.javaGitRef" class="search" placeholder="main / branch / tag / commit" />
|
||||||
|
</label>
|
||||||
|
<label class="update-field">
|
||||||
|
<span>Go Git Ref</span>
|
||||||
|
<input v-model.trim="updateOps.goGitRef" class="search" placeholder="main / branch / tag / commit" />
|
||||||
|
</label>
|
||||||
|
<label class="update-field">
|
||||||
|
<span>Image Tag</span>
|
||||||
|
<input v-model.trim="updateOps.imageTag" class="search" placeholder="可选,不填则自动生成" />
|
||||||
|
</label>
|
||||||
|
<label class="update-check">
|
||||||
|
<input v-model="updateOps.skipMavenBuild" type="checkbox" />
|
||||||
|
<span>复用已有 Java Jar,跳过 Maven</span>
|
||||||
|
</label>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<p v-if="updateOps.error" class="metric-error">{{ updateOps.error }}</p>
|
||||||
|
<p v-if="updateOps.message" :class="updateOps.result && updateOps.result.ok ? 'ok-text' : 'metric-error'">{{ updateOps.message }}</p>
|
||||||
|
|
||||||
|
<div v-if="updateOps.result && updateOps.result.build && updateOps.result.build.length > 0" class="table-wrap">
|
||||||
|
<table class="table compact">
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th>仓库类型</th>
|
||||||
|
<th>Git Ref</th>
|
||||||
|
<th>Commit</th>
|
||||||
|
<th>Image Tag</th>
|
||||||
|
<th>服务</th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
<tr v-for="build in updateOps.result.build" :key="build.kind + '-' + build.commit">
|
||||||
|
<td>{{ build.kind }}</td>
|
||||||
|
<td class="mono">{{ build.gitRef }}</td>
|
||||||
|
<td class="mono">{{ build.commit }}</td>
|
||||||
|
<td class="mono">{{ build.imageTag }}</td>
|
||||||
|
<td>{{ (build.services || []).join(', ') }}</td>
|
||||||
|
</tr>
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div v-if="updateOps.result && updateOps.result.preparedHosts && updateOps.result.preparedHosts.length > 0" class="table-wrap">
|
||||||
|
<table class="table compact">
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th>主机</th>
|
||||||
|
<th>服务</th>
|
||||||
|
<th>Compose</th>
|
||||||
|
<th>预拉镜像</th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
<tr v-for="item in updateOps.result.preparedHosts" :key="item.host">
|
||||||
|
<td>{{ item.host }}</td>
|
||||||
|
<td>{{ (item.services || []).join(', ') }}</td>
|
||||||
|
<td>{{ item.compose && item.compose.changed ? '已更新' : '未变化' }}</td>
|
||||||
|
<td :class="item.prepull && item.prepull.status === 'SUCCESS' ? 'ok-text' : 'bad-text'">
|
||||||
|
{{ item.prepull ? item.prepull.status : '-' }}
|
||||||
|
</td>
|
||||||
|
</tr>
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div v-if="updateOps.result && updateOps.result.steps && updateOps.result.steps.length > 0" class="table-wrap">
|
||||||
|
<table class="table compact">
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th>服务</th>
|
||||||
|
<th>主机</th>
|
||||||
|
<th>镜像</th>
|
||||||
|
<th>状态</th>
|
||||||
|
<th>HTTP</th>
|
||||||
|
<th>Nacos</th>
|
||||||
|
<th>耗时</th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
<tr v-for="step in updateOps.result.steps" :key="step.service + '-' + step.host + '-' + step.image">
|
||||||
|
<td>{{ step.service }}</td>
|
||||||
|
<td>{{ step.host }}</td>
|
||||||
|
<td class="mono">{{ step.image }}</td>
|
||||||
|
<td :class="step.status === 'success' ? 'ok-text' : 'bad-text'">{{ step.status }}</td>
|
||||||
|
<td>
|
||||||
|
<span v-if="step.http">{{ step.http.statusCode }} / {{ step.http.latencyMs }}ms</span>
|
||||||
|
<span v-else>-</span>
|
||||||
|
</td>
|
||||||
|
<td>
|
||||||
|
<span v-if="step.nacos">{{ step.nacos.source || 'ok' }}</span>
|
||||||
|
<span v-else>-</span>
|
||||||
|
</td>
|
||||||
|
<td>{{ step.durationSeconds || '-' }}s</td>
|
||||||
|
</tr>
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
|
||||||
<section class="section-head">
|
<section class="section-head">
|
||||||
<h2>滚动重启</h2>
|
<h2>滚动重启</h2>
|
||||||
</section>
|
</section>
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user