RunnerConfig

danling.runners.RunnerConfig

Bases: Config

Top-level configuration for DanLing runners.

RunnerConfig owns runner lifecycle settings, restore sources, and typed subsystem sections. Detailed subsystem field semantics live on the matching subconfig class, for example OptimizerConfig, SchedulerConfig, ScoreConfig, CheckpointConfig, WorkspaceConfig, DataloaderConfig, FsdpConfig, and ParallelConfig.

RunnerConfig inherits from Config and provides attribute-style access to nested values:

config = RunnerConfig()
config.workspace.experiment = "resnet50"
config.dataloader.batch_size = 32
config["optim"] = {"type": "adamw", "lr": 1e-3}

Command-line integration is built in:

config = MyConfig()
config.parse()  # Parse CLI args, e.g., --epochs 20 --optim.lr 0.01

Core attributes

seed, deterministic: Reproducibility controls. steps, epochs: Mutually exclusive training boundaries. accum_steps: Number of micro-batches per optimizer step. train_splits, evaluate_splits: Optional split selection overrides. precision: Optional autocast precision. max_grad_value, max_grad_norm, skip_nonfinite_loss, skip_nonfinite_grad: Gradient safety controls. checkpoint, resume, pretrained: Restore sources. Source priority is checkpoint > resume > pretrained. deepspeed: Optional raw DeepSpeed config mapping.

Nested sections

optim, sched, score, workspace, logging, tensorboard, wandb, mlflow, ft, compile, dist, gc, profiling, heartbeat, ckpt, dataloader, performance, activation_checkpoint, fsdp, and parallel.

Examples:

Basic usage:

# Create a config
config = RunnerConfig()
config.workspace.experiment = "resnet18"
config.dataloader.batch_size = 32
config["optim"] = {"type": "adamw", "lr": 1e-3}
config.epochs = 10

# Use in a runner
runner = Runner(config)

Custom config class with typed attributes:

class TrainingConfig(RunnerConfig):
    # Type annotations provide auto-completion and validation
    model: str = "resnet18"
    epochs: int = 100
    precision: str = "bf16"

    def __init__(self):
        super().__init__()
        self.dataloader.batch_size = 32
        self["optim"] = {"type": "adamw", "lr": 1e-3}

    def post(self):
        # Called after parsing CLI args
        super().post()
        # Create derived settings
        lr = self.get("optim.lr")
        self.workspace.experiment = f"{self.model}_bs{self.dataloader.batch_size}_lr{lr}"

Command-line integration:

# Override config settings via CLI
python train.py --epochs 50 --dataloader.batch_size 64 --optim.lr 0.0005

Note

Always store all parameters needed to reproduce a run in the RunnerConfig. The RunnerConfig is automatically saved with checkpoints, enabling exact resumption.

See Also

• Runner: Main runner class that uses this config.
• chanfig.Config: Base config implementation.

Source code in danling/runners/config.py

class RunnerConfig(chanfig.Config):  # pylint: disable=too-many-instance-attributes
    r"""
    Top-level configuration for DanLing runners.

    `RunnerConfig` owns runner lifecycle settings, restore sources, and typed
    subsystem sections. Detailed subsystem field semantics live on the matching
    subconfig class, for example `OptimizerConfig`, `SchedulerConfig`,
    `ScoreConfig`, `CheckpointConfig`, `WorkspaceConfig`, `DataloaderConfig`,
    `FsdpConfig`, and `ParallelConfig`.

    `RunnerConfig` inherits from [`Config`][chanfig.Config] and provides
    attribute-style access to nested values:

    ```python
    config = RunnerConfig()
    config.workspace.experiment = "resnet50"
    config.dataloader.batch_size = 32
    config["optim"] = {"type": "adamw", "lr": 1e-3}
    ```

    Command-line integration is built in:

    ```python
    config = MyConfig()
    config.parse()  # Parse CLI args, e.g., --epochs 20 --optim.lr 0.01
    ```

    Core attributes:
        stack: Runner stack selector used by `danling.runners.Runner`. Supported
            values include `"auto"`, `"ddp"`/`"torch"`, `"graph"`,
            `"deepspeed"`/`"ds"`, and `"parallel"`.
        seed, deterministic: Reproducibility controls.
        steps, epochs: Mutually exclusive training boundaries.
        accum_steps: Number of micro-batches per optimizer step.
        train_splits, evaluate_splits: Optional split selection overrides.
        precision: Optional autocast precision.
        max_grad_value, max_grad_norm, skip_nonfinite_loss, skip_nonfinite_grad:
            Gradient safety controls.
        checkpoint, resume, pretrained: Restore sources. Source priority is
            `checkpoint` > `resume` > `pretrained`.
        deepspeed: Optional raw DeepSpeed config mapping.

    Nested sections:
        `optim`, `sched`, `score`, `workspace`, `logging`, `tensorboard`,
        `wandb`, `mlflow`, `ft`, `compile`, `dist`, `gc`, `profiling`, `heartbeat`,
        `ckpt`, `dataloader`, `performance`, `activation_checkpoint`,
        `fsdp`, and `parallel`.

    Examples:
        Basic usage:
        ```python
        # Create a config
        config = RunnerConfig()
        config.workspace.experiment = "resnet18"
        config.dataloader.batch_size = 32
        config["optim"] = {"type": "adamw", "lr": 1e-3}
        config.epochs = 10

        # Use in a runner
        runner = Runner(config)
        ```

        Custom config class with typed attributes:
        ```python
        class TrainingConfig(RunnerConfig):
            # Type annotations provide auto-completion and validation
            model: str = "resnet18"
            epochs: int = 100
            precision: str = "bf16"

            def __init__(self):
                super().__init__()
                self.dataloader.batch_size = 32
                self["optim"] = {"type": "adamw", "lr": 1e-3}

            def post(self):
                # Called after parsing CLI args
                super().post()
                # Create derived settings
                lr = self.get("optim.lr")
                self.workspace.experiment = f"{self.model}_bs{self.dataloader.batch_size}_lr{lr}"
        ```

        Command-line integration:
        ```bash
        # Override config settings via CLI
        python train.py --epochs 50 --dataloader.batch_size 64 --optim.lr 0.0005
        ```

    Note:
        Always store all parameters needed to reproduce a run in the RunnerConfig.
        The RunnerConfig is automatically saved with checkpoints, enabling exact resumption.

    See Also:
        - [`Runner`][danling.runners.Runner]: Main runner class that uses this config.
        - [`chanfig.Config`](https://github.com/ultmaster/chanfig): Base config implementation.
    """

    # Defining mapping equality disables inherited hashing for this mutable class.
    __eq__ = dict.__eq__

    stack: str = "auto"
    name: Optional[str] = None

    seed: Optional[int] = 1016
    deterministic: bool = False

    steps: Optional[int] = None
    epochs: Optional[int] = None
    accum_steps: int = 1
    train_splits: Union[Sequence[str], str, None] = None
    evaluate_splits: Union[Sequence[str], str, None] = None
    precision: Optional[str] = None
    max_grad_value: Optional[float] = None
    max_grad_norm: Optional[float] = None
    skip_nonfinite_loss: bool = False
    skip_nonfinite_grad: bool = False

    checkpoint: Optional[str] = None
    resume: bool = False
    pretrained: Optional[str] = None

    optim: Optional[OptimizerConfig]
    sched: Optional[SchedulerConfig]
    fp8: Fp8Config = Fp8Config()
    deepspeed: Optional[Mapping[str, Any]] = None

    score: ScoreConfig = ScoreConfig()
    workspace: WorkspaceConfig = WorkspaceConfig()
    logging: LoggingConfig = LoggingConfig()
    tensorboard: TensorboardConfig = TensorboardConfig()
    wandb: WandbConfig = WandbConfig()
    mlflow: MlflowConfig = MlflowConfig()
    ft: FaultToleranceConfig = FaultToleranceConfig()

    compile: CompileConfig = CompileConfig()
    dist: DistributedConfig = DistributedConfig()
    ddp: DdpConfig = DdpConfig()
    gc: GcConfig = GcConfig()
    profiling: ProfilingConfig = ProfilingConfig()
    heartbeat: HeartbeatConfig = HeartbeatConfig()
    ckpt: CheckpointConfig = CheckpointConfig()
    dataloader: DataloaderConfig = DataloaderConfig()
    performance: PerformanceConfig = PerformanceConfig()
    activation_checkpoint: ActivationCheckpointConfig = ActivationCheckpointConfig()
    fsdp: FsdpConfig = FsdpConfig()
    parallel: ParallelConfig = ParallelConfig()

    def __post_init__(self, *args, **kwargs) -> None:
        super().__post_init__(*args, **kwargs)
        self.validate()

    def post(self) -> None:
        super().post()
        self.validate()

    def validate(self) -> None:
        if self.steps is not None and self.epochs is not None:
            raise ValueError("`steps` and `epochs` are mutually exclusive; set only one training boundary")

    @staticmethod
    def _semantic_section(section: Any, defaults: chanfig.Config) -> chanfig.NestedDict:
        if not isinstance(section, Mapping):
            return chanfig.NestedDict()
        return defaults.difference(section)

    def canonical(self) -> chanfig.NestedDict:
        canonical = chanfig.NestedDict(self.dict())
        stack = normalize_stack_name(canonical.get("stack", "auto"))
        canonical["stack"] = stack
        for key in NON_SEMANTIC_CONFIG_KEYS:
            canonical.pop(key, None)

        ckpt_backend = canonical.get("ckpt.backend")
        canonical.pop("ckpt", None)
        if ckpt_backend is not None:
            ckpt_backend = str(ckpt_backend).strip().lower()
            if ckpt_backend != "auto":
                canonical["ckpt"] = chanfig.NestedDict({"backend": ckpt_backend})

        for key, defaults in DEFAULT_FILTERED_CONFIG_SECTIONS:
            semantic_section = self._semantic_section(canonical.get(key), defaults)
            if semantic_section:
                canonical[key] = semantic_section
            else:
                canonical.pop(key, None)

        if stack != "parallel":
            canonical.pop("fsdp", None)
            canonical.pop("parallel", None)
        return canonical

    def fingerprint(self) -> int:
        """Return an unsigned 64-bit fingerprint of the current canonical config.

        Hash the UTF-8 canonical YAML with SHA-1 and interpret its first eight
        bytes as a big-endian integer. Runtime-only settings are excluded by
        ``canonical()``. The configuration remains mutable and unhashable;
        this method recomputes the fingerprint on each call.

        Returns:
            int: A value in ``[0, 2**64)`` derived from the current canonical YAML.
        """
        digest = hashlib.sha1(self.canonical().yamls().encode("utf-8")).digest()
        return int.from_bytes(digest[:8], byteorder="big", signed=False)

fingerprint

Python

fingerprint() -> int

Return an unsigned 64-bit fingerprint of the current canonical config.

Hash the UTF-8 canonical YAML with SHA-1 and interpret its first eight bytes as a big-endian integer. Runtime-only settings are excluded by canonical(). The configuration remains mutable and unhashable; this method recomputes the fingerprint on each call.

Returns:

Name	Type	Description
int	int	A value in [0, 2**64) derived from the current canonical YAML.

Source code in danling/runners/config.py

def fingerprint(self) -> int:
    """Return an unsigned 64-bit fingerprint of the current canonical config.

    Hash the UTF-8 canonical YAML with SHA-1 and interpret its first eight
    bytes as a big-endian integer. Runtime-only settings are excluded by
    ``canonical()``. The configuration remains mutable and unhashable;
    this method recomputes the fingerprint on each call.

    Returns:
        int: A value in ``[0, 2**64)`` derived from the current canonical YAML.
    """
    digest = hashlib.sha1(self.canonical().yamls().encode("utf-8")).digest()
    return int.from_bytes(digest[:8], byteorder="big", signed=False)