Skip to content

Conversation

@hhaAndroid
Copy link
Collaborator

@hhaAndroid hhaAndroid commented Dec 30, 2025

         dataset_config = []
        _data_cfg = {"dataset": DatasetConfig(name='apach',
                                              anno_path=ALPACA_PATH),
                     "tokenize_fn": OpenaiTokenizeFunctionConfig(
                         chat_template='qwen3',
                         max_length=32768
                     )
                     }
        dataset_config.append(_data_cfg)

        sft_dataloader_cfg = DataloaderConfig(
            dataset_config_list=dataset_config,
            pack_max_length=32768,
            pack_to_max_length=True,
            num_workers=0,
        )
        sft_global_batch_size = 8
        loss_reduction = "square"
        sft_loss_cfg = CELossConfig(mode="chunk", chunk_size=1024, loss_reduction=loss_reduction)

        worker_cfg: WorkerConfig = WorkerConfig(
            sft_dataloader_cfg=sft_dataloader_cfg,
            sft_global_batch_size=sft_global_batch_size,
            sft_loss_cfg=sft_loss_cfg,
            seed=42,
            ...
        )

@hhaAndroid hhaAndroid changed the title add sft trainging with RL add sft training with RL Dec 31, 2025
@hhaAndroid hhaAndroid merged commit 8dae899 into InternLM:main Dec 31, 2025
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants