|
1 | | -# DP-GEN 配置、验证与执行 |
| 1 | +# DP-GEN Configuration, Validation, and Execution |
2 | 2 |
|
3 | | -在创建、审阅、验证、启动或恢复 `dpgen run` 配置时加载本 reference。 |
| 3 | +Load this reference when creating, reviewing, validating, launching, or |
| 4 | +recovering a `dpgen run` configuration. |
4 | 5 |
|
5 | | -## 1. `param.json` 要点 |
| 6 | +## 1. `param.json` essentials |
6 | 7 |
|
7 | | -应保留或收集: |
| 8 | +Preserve or collect: |
8 | 9 |
|
9 | | -- `type_map`(有序元素符号)、可选的 `mass_map`、`init_data_prefix` 和 `init_data_sys`; |
10 | | -- `sys_configs_prefix` 与 `sys_configs`(系统列表,每个系统又是结构列表); |
11 | | -- `numb_models` 以及完整的 `default_training_param.model`、`learning_rate`、`loss` 和 `training`; |
12 | | -- `model_devi_dt`、`model_devi_skip`、信任阈值、清理标志和 `model_devi_jobs`; |
13 | | -- `fp_style`、`fp_task_min`、`fp_task_max` 及后端专用 FP 输入。 |
| 10 | +- `type_map` (ordered symbols), optional `mass_map`, `init_data_prefix`, and `init_data_sys`; |
| 11 | +- `sys_configs_prefix` and `sys_configs` (a list of systems, each a list of structures); |
| 12 | +- `numb_models` and complete `default_training_param.model`, `learning_rate`, `loss`, and `training`; |
| 13 | +- `model_devi_dt`, `model_devi_skip`, trust thresholds, cleanup flag, and `model_devi_jobs`; |
| 14 | +- `fp_style`, `fp_task_min`, `fp_task_max`, and backend-specific FP inputs. |
14 | 15 |
|
15 | | -DeePMD-kit 1.x 使用 `training.systems`,2.x/3.x 使用 |
16 | | -`training.training_data.systems`。将对应版本的 systems 留空,由 DP-GEN 根据 |
17 | | -`init_data_sys` 填充。使用版本相关 descriptor 或功能前,核对 `deepmd_version` |
18 | | -和已安装的软件栈。 |
| 16 | +DeepMD-kit 1.x uses `training.systems`; 2.x/3.x use |
| 17 | +`training.training_data.systems`. Leave the version-appropriate value empty so |
| 18 | +DP-GEN can fill it from `init_data_sys`. Verify `deepmd_version` and the |
| 19 | +installed stack before using version-specific descriptors or features. |
19 | 20 |
|
20 | | -`train_backend` 支持 `tensorflow` 和 `pytorch`,但实际兼容性仍取决于已安装的 |
21 | | -DeePMD-kit 与 descriptor。核对组合并准确拼写 `se_atten_v2`。普通 LAMMPS 探索使用 |
22 | | -默认 `model_devi_engine`;替代引擎只有在验证后才能设置。当前 `dpgen run` 不接受 |
23 | | -`fp_style: "none"`。 |
| 21 | +`train_backend` accepts `tensorflow` and `pytorch`, but compatibility still |
| 22 | +depends on the installed DeePMD-kit and descriptor. Verify the combination; |
| 23 | +spell `se_atten_v2` exactly. Ordinary LAMMPS exploration uses the default |
| 24 | +`model_devi_engine`; set an alternative only when verified. Current `dpgen run` |
| 25 | +does not accept `fp_style: "none"`. |
24 | 26 |
|
25 | | -每个 `init_data_sys` 目录必须包含 `type_map.raw`、`type.raw` 和 `set.000/`; |
26 | | -每个 `type_map.raw` 必须与 `param.json.type_map` 完全一致。 |
| 27 | +Each `init_data_sys` directory must contain `type_map.raw`, `type.raw`, and |
| 28 | +`set.000/`; every `type_map.raw` must match `param.json.type_map` exactly. |
27 | 29 |
|
28 | | -## 2. `machine.json` 要点 |
| 30 | +## 2. `machine.json` essentials |
29 | 31 |
|
30 | | -保持独立的 `train`、`model_devi` 和 `fp` block。每个 block 提供 `command`、 |
31 | | -`machine`(`batch_type`、`context_type`、`local_root` 以及需要时的 |
32 | | -`remote_root`/profile)和 `resources`(节点数、CPU/GPU 数、`group_size`、 |
33 | | -队列/分区参数及调度器 `source_list`)。通用 local、scheduler 和 SSH 结构见 |
34 | | -[machine schema 说明](../../../doc/run/example-of-machine.md)。 |
| 32 | +Keep separate `train`, `model_devi`, and `fp` blocks. Each block supplies a |
| 33 | +`command`, `machine` (`batch_type`, `context_type`, `local_root`, and any |
| 34 | +required `remote_root`/profile), and `resources` (`number_node`, CPU/GPU counts, |
| 35 | +`group_size`, queue/partition flags, and scheduler `source_list`). See the |
| 36 | +[repository machine schema walkthrough](../../../doc/run/example-of-machine.md) |
| 37 | +for generic local, scheduler, and SSH shapes. |
35 | 38 |
|
36 | | -启动 shell 与派发任务处于不同环境。调度器任务必须通过 `source_list` 激活自己的 |
37 | | -运行环境,外层激活不会可靠继承。训练命令必须选择与 `train_backend` 相同的后端 |
38 | | -(DeePMD-kit 3.x 的 PyTorch 可能需要 `dp --pt`)。应根据已安装的 DPDispatcher |
39 | | -验证命令、context、batch 别名、路径和资源,不要凭经验统一改写。 |
| 39 | +The launcher shell and dispatched tasks are different environments. Scheduler |
| 40 | +jobs must activate their own runtime through `source_list`; outer activation is |
| 41 | +not inherited reliably. The training command must select the same backend as |
| 42 | +`train_backend` (for DeePMD-kit 3.x PyTorch this may be `dp --pt`). Validate |
| 43 | +commands, context types, batch aliases, roots, and resources against installed |
| 44 | +DPDispatcher rather than normalizing by assumption. |
40 | 45 |
|
41 | | -## 3. 启动前验证生成输入 |
| 46 | +## 3. Validate generated inputs before launch |
42 | 47 |
|
43 | 48 | ```bash |
44 | 49 | dpgen -h |
45 | 50 | python -m json.tool param.json |
46 | 51 | python -m json.tool machine.json |
47 | 52 | ``` |
48 | 53 |
|
49 | | -用 `run_jdata_arginfo()` 规范化 `param.json`,用 `convert_mdata()` 转换 |
50 | | -`machine.json`。解析所有数据/结构路径,比较 type map,检查可执行文件和调度器权限, |
51 | | -并核对 FP 输入及成本上限。 |
| 54 | +Normalize `param.json` with `run_jdata_arginfo()` and convert `machine.json` with |
| 55 | +`convert_mdata()`. Resolve all data/structure paths, compare type maps, check |
| 56 | +executables and scheduler access, and verify FP inputs and cost limits. |
52 | 57 |
|
53 | | -提交前逐个检查每个阶段/任务实际生成的 `input.json`。不能从根目录 JSON 推断最终配置: |
54 | | -核对模型数、seed、checkpoint、数据系统、`sys_idx`、温度/压力和最终训练步数。存在 |
55 | | -reuse 或迭代 override 时,同时断言默认值和生效值。 |
| 58 | +Before submission, inspect the generated `input.json` for every stage/task. Do |
| 59 | +not infer the effective configuration from the root files: check model count, |
| 60 | +seed, checkpoint, data systems, `sys_idx`, temperatures/pressures, and final |
| 61 | +training steps. When reuse or iteration overrides exist, assert both the default |
| 62 | +and effective values. |
56 | 63 |
|
57 | | -将目标迭代与 `record.dpgen`、阶段目录、DPDispatcher work base/提交元数据及日志交叉核对。 |
58 | | -仅有目录或 job 并不代表阶段已完成。 |
| 64 | +Cross-check the intended iteration against `record.dpgen`, stage directories, |
| 65 | +DPDispatcher work base/submission metadata, and logs. A stage is not complete |
| 66 | +because a directory or job exists. |
59 | 67 |
|
60 | | -## 4. 恢复与重启 |
| 68 | +## 4. Restart and recovery |
61 | 69 |
|
62 | | -从 `record.dpgen` 恢复属于可能产生费用的操作。同一工作目录只能保留一个 controller。 |
63 | | -编辑状态前先停止 controller、备份状态文件、记录阶段/job ID 及新旧设置,并查询调度器。 |
| 70 | +Treat a restart from `record.dpgen` as cost-bearing. Keep one controller for a |
| 71 | +work directory. Before editing state, stop it, back up the state file, record |
| 72 | +stage/job IDs and old/new settings, and query the scheduler. |
64 | 73 |
|
65 | | -活动阶段的 command、context、remote root、队列、资源、`group_size`、flags、激活方式、 |
66 | | -任务列表或 common files 变化,都可能产生新的 DPDispatcher submission identity。新 identity |
67 | | -不会导入旧 identity 的完成状态。要区分旧提交恢复和使用新 remote root 的有意重跑;连接超时 |
68 | | -本身不是重新提交理由。 |
| 74 | +A changed active-stage command, context, remote root, queue, resources, |
| 75 | +`group_size`, flags, activation, task list, or common files can create a new |
| 76 | +DPDispatcher submission identity. It will not import completion state from the |
| 77 | +old identity. Distinguish recovery of the old submission from an intentional |
| 78 | +rerun with a new remote root; never resubmit solely because a connection timed |
| 79 | +out. |
69 | 80 |
|
70 | | -恢复的正证据包括旧 submission hash、未变化的 job ID 和恢复日志;同时不能出现新的提交日志。 |
71 | | -`record.dpgen` 是工作流指针,不是调度器完成记录。当前计费阶段推进后,才能应用后续迭代的 |
72 | | -machine 修改。 |
| 81 | +Positive recovery evidence is the old submission hash, unchanged job IDs, and a |
| 82 | +recovery message; absence of a new-submission message is required. |
| 83 | +`record.dpgen` is a workflow pointer, not a scheduler completion record. |
| 84 | +Apply later-iteration machine changes only after the current cost-bearing stage |
| 85 | +advances. |
73 | 86 |
|
74 | | -## 5. 确认与检查 |
| 87 | +## 5. Confirm and inspect |
75 | 88 |
|
76 | | -展示文件、验证摘要、未解决风险和 `dpgen run param.json machine.json`。只有明确确认后才执行。 |
77 | | -启动后检查 `iter.*`、阶段日志、失败/等待状态、选中和标注数量,以及 |
78 | | -[监控输出](monitoring.md)。 |
| 89 | +Show the exact files, validation summary, unresolved risks, and |
| 90 | +`dpgen run param.json machine.json`. Execute only after explicit confirmation. |
| 91 | +After launch, inspect `iter.*`, stage logs, failure/pending states, selected and |
| 92 | +labeled counts, and [monitoring outputs](monitoring.md). |
79 | 93 |
|
80 | | -外部参考:https://docs.deepmodeling.com/projects/dpgen/en/latest/run/index.html |
| 94 | +External references: https://docs.deepmodeling.com/projects/dpgen/en/latest/run/index.html |
0 commit comments