diff --git a/docs/blog/golang/architectural/4.md b/docs/blog/golang/architectural/4.md index 307f631ef6..061bed0933 100644 --- a/docs/blog/golang/architectural/4.md +++ b/docs/blog/golang/architectural/4.md @@ -3,3 +3,152 @@ title: mage跨平台构建说明 hide_title: true sidebar_position: 4 --- + +# mage跨平台构建说明 + +在 Go 项目里,“构建”常常被理解成把若干个二进制文件编译出来。但对 OpenIM 这类由多个服务、多个工具、多个部署环境组成的工程来说,只完成编译并不等于完成交付。 + +真正的问题通常有六个:目标在哪里?要编译到哪些平台?运行时需要多少实例?工具和服务的启动顺序是什么?如何判断进程真的起来了?最终如何把这些产物交给另一台机器使用? + +**gomake 的核心价值,是把这些问题收拢到一条以 Mage 为入口、以目录约定为边界、以配置和进程检查为闭环的交付链路里。** 它不是简单地包装 `go build`,而是把“构建、启动、停止、检查、导出”这些动作放进同一套路径、平台和进程模型中,让大型 Go 项目的交付行为可以被重复执行。 + +## 01. 为什么不能只停留在 go build + +单体项目里的构建目标通常很清晰:一个入口,一个产物,一个运行命令。微服务项目则不同。一个仓库里可能同时存在后台服务、同步执行的初始化工具、协议生成任务,以及不同平台下的部署产物。 + +如果每个服务都靠手写脚本维护,就会很快遇到几个问题: + +| 问题 | 表现 | +| --- | --- | +| 构建目标分散 | 新增服务后容易忘记同步脚本 | +| 平台差异外溢 | Unix、Windows 的进程和资源限制逻辑混在一起 | +| 启动顺序不稳定 | 初始化工具和后台服务没有清晰边界 | +| 状态判断模糊 | 只看进程名容易误伤或误判 | +| 产物不可迁移 | 编译出来的文件缺少可执行的启动上下文 | + +因此,gomake 没有把重点放在“如何写一条更短的编译命令”上,而是先定义工程交付的几个稳定约定:源码放在哪里、工具放在哪里、输出放在哪里、配置从哪里读、日志写到哪里、当前平台对应哪组产物。 + +有了这些约定,后面的构建和运行才不会变成散落在仓库各处的脚本片段。 + +## 02. 用路径模型统一工程坐标 + +gomake 的第一层抽象是路径模型。 + +它把项目根目录、配置目录、输出目录、二进制目录、日志目录、归档目录、临时目录、服务源码目录和工具源码目录都纳入同一个路径词汇表。这样一来,构建任务、启动任务、导出任务看到的是同一套坐标,而不是各自拼接路径。 + +这件事看起来很基础,但在跨平台构建里非常关键。因为产物并不是只有一个目录,而是会按照系统和架构分层,例如当前主机产物、目标平台产物、工具产物、服务产物、导出归档产物。只要路径没有统一,后续每个任务都会重复处理这些差异。 + +gomake 还允许通过参数覆盖根目录、输出目录、配置目录、服务目录和工具目录。这意味着它既可以服务于标准 OpenIM 仓库布局,也可以被嵌入到相近结构的 Go 项目里。对于 Kubernetes 这类部署环境,配置路径还可以切换到更适合挂载配置的布局,从而避免把本地开发路径硬编码到运行时。 + +路径模型解决的是一个底层问题:所有任务都必须先知道“同一个工程”指的是什么。只有这个坐标系统稳定,构建和编排才有可能共享状态。 + +## 03. 目录发现让服务和工具成为约定 + +gomake 没有要求开发者在脚本里逐个登记所有二进制目标,而是通过目录约定发现它们。 + +`cmd` 目录代表后台服务,`tools` 目录代表同步工具。只要某个子目录符合 Go 入口的约定,它就会被识别为一个可构建目标。服务和工具会被放进不同的输出区域,也会在启动阶段承担不同角色。 + +这个区分很重要。 + +后台服务通常需要常驻运行,需要实例编号、配置路径、进程检查和端口观测;工具则更像启动前的准备动作,例如初始化、迁移或检查。工具执行失败时,后续服务启动应该停止,而不是继续拉起一个依赖条件不满足的系统。 + +通过目录发现,gomake 把“这个入口是什么类型”从命令参数转移到了工程结构本身。新增服务时,只要放在约定目录里,构建系统就能自动纳入;新增工具时,也会天然进入同步执行链路。这种约定降低了脚本维护成本,也减少了多人协作时遗漏目标的概率。 + +## 04. 跨平台构建不是循环执行命令 + +跨平台构建表面上是为不同 `GOOS` 和 `GOARCH` 组合执行编译,实际要处理的问题更多。 + +gomake 会先确定构建选项:是否启用 CGO,是否按发布模式优化,是否压缩产物,目标平台列表来自显式配置还是当前机器自动识别。随后,它会把服务目标和工具目标分别解析出来,再按平台输出到对应目录。Windows 平台下的可执行文件命名、不同架构下的输出路径、服务与工具的分类都会在同一套模型里完成。 + +这里有两个设计取舍值得注意。 + +第一,配置优先级是明确的。代码中传入的构建选项优先于环境变量,环境变量再作为默认输入。这可以避免复杂 CI 或本地环境里的隐式变量意外覆盖上层调用者的决定。 + +第二,构建并发被主动节制。多平台、多目标同时编译时,如果完全放开并发,很容易把机器 CPU、内存和磁盘 I/O 打满,反而造成构建不稳定。gomake 选择按照机器能力控制并发,让吞吐和稳定性之间保持平衡。 + +因此,跨平台构建在 gomake 里不是一组循环命令,而是一条可解释的产物生成流水线:先识别目标,再确定平台,再分类输出,最后生成可被启动系统读取的配置基础。 + +## 05. start-config.yml 把构建结果变成运行计划 + +构建结束并不代表系统可以运行。gomake 会把已经识别出的服务和工具沉淀到 `start-config.yml` 这类运行配置里,让构建产物进一步变成可执行计划。 + +这个配置关注三类信息:哪些服务需要常驻运行,哪些工具需要在启动前同步执行,每个服务需要多少实例以及运行时的资源约束。默认情况下,服务会以一个实例开始,最大文件描述符也会有基础默认值;后续使用者可以按部署需要调整。 + +这一步的意义在于,它把构建系统和运行系统接了起来。 + +如果没有运行配置,构建脚本只能告诉你“文件已经生成”;有了运行配置,gomake 可以继续回答“应该按什么顺序运行这些文件、应该期待多少进程、应该用什么配置启动”。这也是它区别于普通构建工具的地方:构建产物不是终点,而是启动编排的输入。 + +## 06. 启动链路先处理确定性,再处理并发 + +gomake 的启动流程分为两类动作:同步工具和后台服务。 + +同步工具先执行。它们适合承载那些必须在服务启动前完成的动作,例如准备环境或生成必要数据。只要工具失败,启动流程就会中断。这种“先确定前置条件”的策略,可以避免服务已经拉起但依赖状态不完整的半成功场景。 + +后台服务随后启动。gomake 会根据配置里的实例数量,为服务注入实例标识和配置路径,并以异步方式启动进程。完整启动时,它还会先清理已经存在的同类服务,等待旧进程退出,再拉起新进程并检查数量是否符合预期。 + +这里最关键的是进程识别方式。gomake 不是只按进程名判断,而是尽量以可执行文件路径作为匹配依据。对于多服务、多版本或同名二进制并存的机器来说,只看名字很容易误判;按路径匹配则能把“这个进程是不是我刚才构建出来的那个服务”说得更清楚。 + +启动结束后,gomake 还会检查服务进程数量,并输出相关监听端口。这样用户看到的不只是“命令执行完了”,而是能进一步判断系统是否进入了预期状态。 + +## 07. stop 和 check 让进程管理形成闭环 + +如果只有启动,没有停止和检查,服务编排仍然是不完整的。 + +gomake 的停止逻辑会读取同一份运行配置,定位对应服务进程,先尝试正常终止,再在需要时使用更强制的方式结束进程。停止后还会反复等待和确认,避免命令返回时旧进程仍然残留。 + +检查逻辑则从另一侧验证系统状态:配置里声明了多少个服务实例,当前机器上是否真的存在对应数量的进程,它们是否暴露了监听端口。这个动作对于本地开发和集成环境都很实用,因为它提供了一个比“看日志猜状态”更直接的入口。 + +更重要的是,`start`、`stop`、`check` 用的是同一套路径和配置模型。启动时怎么找到服务,停止时也怎么找到;构建时输出到哪里,检查时也从哪里反推。这种闭环减少了状态漂移,让工具链不会因为多个脚本各自维护规则而互相打架。 + +## 08. 平台差异被隔离在边界上 + +跨平台工具最容易失控的地方,是把平台差异散落到业务流程里。 + +gomake 的做法是把差异隔离在边界:Unix 系统需要处理文件描述符限制,Windows 下则没有同样语义;Unix 进程优先级和 Windows 优先级也不是同一种模型;可执行文件后缀、进程控制方式、环境变量行为都可能不同。 + +这些差异并没有污染到上层的构建和启动叙事里。对于使用者来说,仍然是 `build`、`start`、`stop`、`check` 这些稳定动作;对于实现来说,平台相关能力被拆到对应文件和内部模块里。 + +这是一种很适合基础设施工具的边界设计:上层暴露稳定语义,下层承认系统差异。跨平台不是假装所有系统一样,而是让差异只出现在必须出现的位置。 + +## 09. 统一命令执行和日志,让工具可观测 + +构建和编排工具最终都要大量执行外部命令。如果每个任务都自己处理参数、环境变量、工作目录、标准输入输出、错误输出和优先级,行为很快就会变得不一致。 + +gomake 把命令执行包装成统一模型:任务只需要描述要执行什么、在哪里执行、带什么环境和输出策略,底层负责把这些行为落实到具体进程。日志也会同时面向控制台和统一文件,控制台输出负责即时反馈,日志文件负责事后排查。 + +这让 gomake 具备了一个工程工具很重要的特征:失败不是静默的。构建失败、工具失败、服务启动失败、进程检查失败,都能沿着统一的输出和日志模型被定位,而不是分散在不同脚本的临时打印里。 + +## 10. export 把产物变成可迁移的交付包 + +很多构建系统到生成二进制文件就结束了,但实际交付还需要回答一个问题:这些文件如何被带到另一台机器上继续使用? + +gomake 的 `export` 任务会在构建之后,把服务产物、工具产物、启动配置和编译后的 Mage 启动器放进同一个归档包。这样导出的不只是若干个二进制文件,而是一套带有启动入口和运行计划的交付单元。 + +这种设计降低了部署侧重新理解项目结构的成本。接收方不需要重新推断哪些文件是服务、哪些文件是工具、应该用什么配置启动;这些信息已经在归档结构中被保留下来。 + +换句话说,export 让 gomake 的链路从“本地构建”延伸到“可迁移运行”。这对于多平台交付尤其重要,因为不同平台的产物和启动器都需要在打包阶段保持对应关系。 + +## 11. 协议生成和 bootstrap 是链路的补充 + +除了核心的构建和服务编排,gomake 还提供了协议生成和首次环境准备能力。 + +协议生成任务负责围绕 Go 模块和协议目录组织生成流程,让协议文件的更新可以纳入同一个 Mage 入口。bootstrap 脚本则解决第一次使用时的基础问题:确保 Mage 可用,并准备 Go 依赖。 + +这两个能力不是主链路,但它们体现了同一个设计方向:把重复、容易遗漏、跨环境差异明显的工程动作收进统一入口。用户不需要记住一组互相独立的脚本,而是通过 Mage 任务进入同一套工程语义。 + +## 12. 总结:gomake 解决的是交付一致性 + +从源码结构看,gomake 最重要的设计并不是某个单点功能,而是把 Go 项目交付拆成几个稳定层次: + +| 层次 | 解决的问题 | +| --- | --- | +| 路径模型 | 所有任务共享同一套工程坐标 | +| 目录发现 | 服务和工具通过结构自动纳入构建 | +| 平台模型 | 多系统、多架构产物按规则输出 | +| 运行配置 | 构建结果转化为启动计划 | +| 进程控制 | 启动、停止、检查形成闭环 | +| 导出归档 | 本地产物变成可迁移交付包 | + +这些层次组合起来,gomake 才从一个构建入口变成一条交付链路。 + +对于 OpenIM 这样的项目来说,构建工具不能只追求“能编译”,还要保证不同开发者、不同机器、不同平台上的交付动作尽量一致。gomake 的架构价值就在这里:它用约定减少配置,用统一模型隔离平台差异,用运行配置连接构建和进程管理,最终让跨平台构建和服务编排不再是两套割裂的脚本。 diff --git a/i18n/en/docusaurus-plugin-content-docs-blog/current/golang/architectural/4.md b/i18n/en/docusaurus-plugin-content-docs-blog/current/golang/architectural/4.md index fd9f0262f8..45dd667edc 100644 --- a/i18n/en/docusaurus-plugin-content-docs-blog/current/golang/architectural/4.md +++ b/i18n/en/docusaurus-plugin-content-docs-blog/current/golang/architectural/4.md @@ -3,3 +3,152 @@ title: Mage Cross-platform Build Instructions hide_title: true sidebar_position: 4 --- + +# Mage Cross-platform Build Instructions + +In Go projects, “build” is often understood as compiling a set of binaries. For OpenIM, however, that is only the first step. OpenIM is made of multiple services, auxiliary tools, deployment environments, operating systems, and CPU architectures. Compiling binaries does not automatically mean the system is ready to ship. + +The real questions are broader: where are the targets? Which platforms should they be built for? How many instances should run? What should start first, tools or services? How do we know the processes are really alive? How can the generated artifacts be moved to another machine and still remain runnable? + +**The core value of gomake is that it pulls these questions into one delivery workflow: Mage is the entry point, directory conventions define the boundary, and runtime configuration plus process checks close the loop.** It is not just a wrapper around `go build`; it puts build, start, stop, check, and export into one shared model of paths, platforms, and processes. + +## 01. Why go build Is Not Enough + +In a small monolithic project, the build target is usually obvious: one entry point, one artifact, one command to run it. A microservice project is different. A single repository may contain background services, synchronous initialization tools, protocol generation tasks, and deployment artifacts for different platforms. + +If every service is maintained through handwritten scripts, several problems appear quickly: + +| Problem | Symptom | +| --- | --- | +| Scattered build targets | New services are easy to forget in scripts | +| Platform differences leak upward | Unix and Windows process or resource rules become mixed together | +| Unstable startup order | Initialization tools and long-running services have no clear boundary | +| Ambiguous status checks | Matching only by process name can kill or report the wrong process | +| Non-portable artifacts | Binaries are generated without a runnable startup context | + +So gomake does not start by making a shorter compile command. It first defines stable delivery conventions: where source targets live, where tools live, where output goes, where configuration is read from, where logs are written, and which artifacts belong to the current platform. + +Once those conventions exist, build and runtime orchestration no longer need to live as scattered script fragments across the repository. + +## 02. A Path Model as the Project Coordinate System + +The first abstraction in gomake is the path model. + +It brings the project root, configuration directory, output directory, binary directory, log directory, archive directory, temporary directory, service source directory, and tool source directory into one shared vocabulary. Build tasks, startup tasks, and export tasks all see the same coordinates instead of rebuilding their own path logic. + +This seems basic, but it matters a lot for cross-platform builds. Artifacts do not live in a single flat directory. They are split by operating system and architecture, by current-host output and target-platform output, by service binaries and tool binaries, and by exported archives. Without a shared path model, every task would have to rediscover these differences on its own. + +gomake also allows callers to override the project root, output directory, configuration directory, service source directory, and tool source directory. That means it works with the standard OpenIM layout, but can also be embedded into Go projects with similar structure. In Kubernetes-style deployment, the configuration path can switch to a layout that better fits mounted configuration, avoiding local development paths leaking into runtime. + +The path model solves a foundational problem: every task must agree on what “this project” means. Only when that coordinate system is stable can build and orchestration share state. + +## 03. Directory Discovery Turns Services and Tools into Conventions + +gomake does not require developers to manually register every binary target in a script. It discovers them through directory conventions. + +The `cmd` directory represents background services. The `tools` directory represents synchronous tools. When a subdirectory follows the Go entry-point convention, it becomes a buildable target. Services and tools are emitted into different output areas and play different roles during startup. + +That distinction is important. + +Background services are long-running processes. They need instance indexes, configuration paths, process checks, and port observation. Tools are closer to startup prerequisites: initialization, migration, validation, or other preparatory actions. If a tool fails, the service startup should stop instead of booting a system whose dependencies are not ready. + +Through directory discovery, gomake moves the question “what kind of executable is this?” out of command-line flags and into the project structure. A new service added under the convention becomes part of the build naturally. A new tool added under the convention enters the synchronous startup chain naturally. This reduces script maintenance and lowers the chance that collaborators forget to update a target list. + +## 04. Cross-Platform Builds Are Not Just Command Loops + +On the surface, cross-platform build means compiling for different `GOOS` and `GOARCH` combinations. In practice, there is more to manage. + +gomake first resolves build options: whether CGO is enabled, whether release optimization is used, whether artifacts should be compressed, and whether target platforms come from explicit configuration or automatic host detection. It then resolves service and tool targets separately, emitting each platform’s artifacts into the right output directory. Windows executable naming, architecture-specific paths, and the service/tool split all flow through the same model. + +Two design choices are especially useful. + +First, configuration precedence is explicit. Build options passed from code take priority over environment variables, while environment variables act as defaults. That prevents hidden variables in CI or local shells from accidentally overriding the caller’s intent. + +Second, build concurrency is intentionally restrained. If every service and every platform is compiled at maximum parallelism, CPU, memory, and disk I/O can be saturated, making builds less stable rather than faster. gomake controls concurrency based on machine capacity, balancing throughput with reliability. + +So cross-platform build in gomake is not a list of repeated commands. It is an explainable artifact pipeline: discover targets, resolve platforms, separate service and tool output, then generate the runtime configuration foundation that the startup system can read. + +## 05. start-config.yml Turns Build Output into a Runtime Plan + +A successful build does not mean the system can run. gomake turns the discovered services and tools into a runtime plan such as `start-config.yml`. + +This configuration focuses on three kinds of information: which services should stay running, which tools should run synchronously before startup, how many instances each service needs, and what runtime resource limits should be applied. By default, services start with one instance and a baseline file descriptor limit; users can adjust those values for their deployment. + +This step connects the build system to the runtime system. + +Without runtime configuration, a build script can only say “the files exist.” With runtime configuration, gomake can also answer “in what order should these files run, how many processes should exist, and which configuration path should be passed to them?” That is what makes it more than a normal build tool: build artifacts are not the endpoint; they become input for orchestration. + +## 06. Startup Handles Determinism Before Concurrency + +gomake splits startup into two kinds of actions: synchronous tools and background services. + +Synchronous tools run first. They are suitable for actions that must finish before services start, such as preparing the environment or generating required data. If a tool fails, startup stops. This “preconditions first” approach avoids the half-successful state where services are running but the system they depend on is incomplete. + +Background services start afterward. gomake uses the configured instance count, injects instance identity and configuration path, and launches services asynchronously. During a full startup, it first clears existing matching services, waits for old processes to exit, starts new processes, and then verifies that the expected count is present. + +The key detail is process identification. gomake does not rely only on process names. It prefers matching by executable path. On machines where multiple versions or binaries with the same name may coexist, names alone are easy to misread; paths make it clearer whether a process is the service just built by this workflow. + +After startup, gomake checks service counts and reports listening ports. Users do not just see that a command returned; they can inspect whether the system entered the expected runtime state. + +## 07. stop and check Complete the Process Management Loop + +Startup alone is not enough. Without stop and check, service orchestration remains incomplete. + +gomake’s stop flow reads the same runtime configuration, locates the corresponding service processes, attempts normal termination first, and escalates when needed. After that, it keeps waiting and checking so that old processes do not remain after the command returns. + +The check flow validates state from the other direction: how many service instances are declared in configuration, whether that number of processes exists on the machine, and which ports they listen on. This is useful for local development and integration environments because it gives a more direct entry point than guessing from logs. + +Most importantly, `start`, `stop`, and `check` all use the same path and configuration model. The same rules used to find services during startup are used to stop them. The same output locations used during build are used to infer status during checks. This loop reduces state drift and prevents multiple scripts from fighting each other with different assumptions. + +## 08. Platform Differences Stay at the Boundary + +Cross-platform tools become hard to maintain when operating-system differences leak into the main workflow. + +gomake keeps those differences at the boundary. Unix systems need file descriptor limit handling; Windows does not have the same semantics. Unix process priority and Windows priority classes are different models. Executable suffixes, process control, and environment behavior also vary by platform. + +These differences do not pollute the top-level build and startup story. For users, the stable actions remain `build`, `start`, `stop`, and `check`. For the implementation, platform-specific behavior is isolated in platform-specific files and internal modules. + +This is the right kind of boundary for infrastructure tools: expose stable semantics above, acknowledge system differences below. Cross-platform support does not mean pretending every system is identical; it means letting differences appear only where they must. + +## 09. Unified Command Execution and Logging Make the Tool Observable + +Build and orchestration tools eventually execute many external commands. If every task handles arguments, environment variables, working directories, standard input, output streams, error streams, and priority on its own, behavior quickly becomes inconsistent. + +gomake wraps command execution in one model. A task describes what should run, where it should run, what environment it needs, and how output should be handled; the lower layer turns that into an actual process. Logging targets both the console and a shared file: the console gives immediate feedback, while the log file supports later debugging. + +This gives gomake an important property for an engineering tool: failures are not silent. Build failures, tool failures, startup failures, and process-check failures can all be traced through a consistent output and logging model instead of being scattered across temporary print statements in unrelated scripts. + +## 10. export Turns Artifacts into a Portable Delivery Package + +Many build systems stop after generating binaries. Real delivery still has one more question: how do these files move to another machine and remain usable? + +gomake’s `export` task builds the target artifacts, then packages service binaries, tool binaries, startup configuration, and a compiled Mage launcher into one archive. The exported result is not just a pile of binaries; it is a delivery unit with an entry point and runtime plan. + +That lowers the deployment-side cost of understanding the repository. The receiver does not need to rediscover which files are services, which files are tools, or how they should be started. That information is preserved in the archive structure. + +In other words, export extends gomake from “local build” to “portable runtime.” This is especially important for multi-platform delivery, where artifacts and launchers must remain matched to the platform they were built for. + +## 11. Protocol Generation and Bootstrap Extend the Same Idea + +Beyond the core build and service orchestration flow, gomake also provides protocol generation and first-run environment preparation. + +The protocol generation task organizes generation around the Go module and protocol directories, making protocol updates accessible through the same Mage entry point. The bootstrap scripts solve the first-use problem: making sure Mage is available and Go dependencies are prepared. + +These two capabilities are not the main delivery path, but they follow the same design direction: repeated, easy-to-forget, environment-sensitive engineering actions should move into one stable entry point. Users should not have to remember a collection of unrelated scripts; they should enter the same engineering vocabulary through Mage tasks. + +## 12. Summary: gomake Solves Delivery Consistency + +From the source structure, the most important design in gomake is not a single feature. It is the way Go project delivery is split into stable layers: + +| Layer | Problem Solved | +| --- | --- | +| Path model | All tasks share the same project coordinates | +| Directory discovery | Services and tools enter builds through structure | +| Platform model | Multi-OS and multi-architecture artifacts follow clear rules | +| Runtime configuration | Build output becomes a startup plan | +| Process control | Start, stop, and check form a closed loop | +| Export archive | Local artifacts become a portable delivery package | + +Together, these layers turn gomake from a build entry point into a delivery workflow. + +For a project like OpenIM, a build tool cannot stop at “it compiles.” It also has to keep delivery behavior consistent across developers, machines, and platforms. That is gomake’s architectural value: it reduces configuration through conventions, isolates platform differences behind one model, connects build output to process management through runtime configuration, and makes cross-platform build plus service orchestration one coherent workflow.