Tags: inclusionAI/AReaL
Tags
[WIP][BugFix] Fix forward bug of LLM and VLM (#218) * 0707_1 * 0707_1 * 0707_2 * 0707_2 * 0703_3 * 0703_3 * r * add api * add directory structure * add tests template * p * format * fix * fix * checkout previous impl * checkout previous implementations * checkout prev impl * refactor * add remote sglang engine * format * 0707_6 * 0707_7 * add readme * refactor1 * 0707_undone * 0708_1 * 0708_2 * 0708_3 * 0708_7 * 0708_4 * 0709_1 * 0709_2 * 0709_3 * PullRequest: 331 [lite] Support remote sglang engine with corresponding testcases. Merge branch fw/lite of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/331 Reviewed-by: 晓雷 <[email protected]> * . * add test for sglang remote engine * fix * 0709_4 * 0709_5 * 0709_ * 0709_6 * 0709_7 * PullRequest: 336 add wrapper Merge branch lite-util-wrapper of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/336 Reviewed-by: 博惟 <[email protected]> * add wrapper * 0709_7 * 0709_8 * 0709_9 * PullRequest: 332 [lite] Support FSDP engines Merge branch mzy/lite/engines of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/332 Reviewed-by: 晓雷 <[email protected]> * fsdp2 engine * fix utils * add fsdp engine test * . * fsdp engine test passed * unsqueeze immediately before model inputs and after model outputts * add optimizer save/load, add position id calculation for input * . * format * not to squeeze * add train and eval api * . * . * improve fsdp engine data preprocessing * format * PullRequest: 337 [lite] Add SFT trainer example. * trainer log * minor changes * add update weights from disk * fix type annotation * PullRequest: 339 [Fix] Fix some minor issues to pass all tests. Merge branch fw/lite of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/339 Reviewed-by: 晓雷 <[email protected]> * support fsdp engine and sglang remote engine * minor fix * . * chore: empty commit * ci: build images on demand * ci: fix on demand condition * ci: fix env sha * PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/340 Reviewed-by: 晓雷 <[email protected]> * support fsdp engine and sglang remote engine * minor fix * . * refactor trainer * add close * rm mb_spec * fix * 0710_1 * 0710_2 * [Fix] Merge previous contributions from fw/refactor to lite (#163) * initial proposal * add arealite * . * change api * . * remove LOG_ROOT * remove MODEL_SAVE_PATH * remove PARAM_REALLOC_PATH, DATASET_CACHE * prepare for testing * prepare for testing * ready for run * local run * tests mainly pass * format * . * amend cluster.py * . * . * client test pass * pass rollout test * remove unused imports * add arealite readme * change api * . * . * . * . * . * . * . * . * format * . * implement iteraptable generation (#112) Co-authored-by: zhaochenyang <[email protected]> * . * fix * . * . * . * pass controller generate batch test * . * refactor rollout controller into worker and controller * . * . * . * change to async rollout * pass rollout controller test * pass test * . * update readme * . * sft debug * . * add lisence * remove unused files * remove unsed args in ppo * add hf engine wrapper (#116) * add hf engine * fix issues * fix ppo bugs and add test * add hf client interface and modify cli args * fix bugs * fix issues * Merge fw/refactor * Finish hf wrapper test * add test --------- Co-authored-by: Wei Fu <[email protected]> * format * format * . * refine hf engine * . * fix * add fsdp engine and sft tests * . * . * . * pass ppo unittest * pass ppo and rollout controller tests * clear unused imports * rename ppo to grpo * change reward function organization * reorganize code * add dataset api * . * . * . * format * chmod fix * . * rename workflow to collector * refactor llm_client location * . * . * fix llm server api * refactor config structure * . * fix tests * . * . * . * Fix unresolved issue in SFTTrainer PR (#139) * . * . * efficient loading * format * . * . * . * . * . * . * Add CI for testing AReaLite (#150) * ci: add test-arealite * ci: add checkout before running test-arealite * ci: add USERNAME * ci: add test script * ci: add GitHub mirror * ci: fix typo * ci: clone one commit * ci: fix condition * ci: set command timeout to 60m * ci: enable pip cache * ci: optimize container lifecycle * ci: split into many stages * ci(test-arealite): fix typo * ci: fix wrong env * ci: fix pytest * ci: uninstall transformer-engine * ci: uninstall transformer-engine * ci: fix model paths * ci: show stdout/stderr * ci: fix not clean up * ci: backup sglang * ci: remove tmp repo dir when run * ci: fix docker run exit 1 condition * ci(test-arealite): limit the concurrency and extend command timeout * . * merge fw/refactor * revert some changes * fix --------- Co-authored-by: meizhiyu.mzy <[email protected]> Co-authored-by: Chayenne <[email protected]> Co-authored-by: zhaochenyang <[email protected]> Co-authored-by: Jayon02 <[email protected]> Co-authored-by: root <meizhiyu.mzy> Co-authored-by: Zijian Zhang <[email protected]> * 0710_2 * 0710_3 * [Doc] Add an instruction about how to run the SFT example. (#164) * 0710_3 * 0710_3 * 0710_5 * 0710_4 * merge_2 * merge_3 * 0711_1 * 0711_2 * 0711_3 * 0711_4 * 0711_6 * 0711_7 * 0711_8 * 0711_8 * 0711_9 * 0711_10 * 0711-11 * [Fix] Fix CI running condition for lite. (#172) * . * fix * . * . * fix * fix * fix * PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/353 Reviewed-by: 博惟 <[email protected]> * add gradient checkpointing * PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/354 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * 0714_1 * 0714_2 * 0714_3 * 0714_3 * 0714_5 * PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * . * fix * . * PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * fix destroy process group * added LocalSGlangEngine and test (#170) * added LocalSGLangEngine * upload test file * add build args * fix sgl_local generate * improved sgl local robustness * test * test updated * added fallback when sgl engine isn't initialized * finish test local engine * added LocalSGlangEngine and test * format and fix format and fix, raise when generate missing field format * change cli_args.py * add comment header format --------- Co-authored-by: ChangyiYang <[email protected]> * format (#174) * PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/358 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * fix loss mask * fix * . * 0715_1 * 0715_2 * 0715_2 * [Feat][Refactor]Support DeepSpeed AutoTP; Refactor hf_engine.py and unit test. (#161) * refactor hf engine * format file * revert file format * Squashed commit of the following: commit 8d4b8dc90fceb144bf00f8125f938b57f696e166 Author: Wei Fu <[email protected]> Date: Thu Jul 10 13:14:10 2025 +0800 [Doc] Add an instruction about how to run the SFT example. (#164) commit 3bf9c85e400a2fe1c6a1e488d27d458ede8cea45 Author: Wei Fu <[email protected]> Date: Thu Jul 10 12:56:24 2025 +0800 [Fix] Merge previous contributions from fw/refactor to lite (#163) * initial proposal * add arealite * . * change api * . * remove LOG_ROOT * remove MODEL_SAVE_PATH * remove PARAM_REALLOC_PATH, DATASET_CACHE * prepare for testing * prepare for testing * ready for run * local run * tests mainly pass * format * . * amend cluster.py * . * . * client test pass * pass rollout test * remove unused imports * add arealite readme * change api * . * . * . * . * . * . * . * . * format * . * implement iteraptable generation (#112) Co-authored-by: zhaochenyang <[email protected]> * . * fix * . * . * . * pass controller generate batch test * . * refactor rollout controller into worker and controller * . * . * . * change to async rollout * pass rollout controller test * pass test * . * update readme * . * sft debug * . * add lisence * remove unused files * remove unsed args in ppo * add hf engine wrapper (#116) * add hf engine * fix issues * fix ppo bugs and add test * add hf client interface and modify cli args * fix bugs * fix issues * Merge fw/refactor * Finish hf wrapper test * add test --------- Co-authored-by: Wei Fu <[email protected]> * format * format * . * refine hf engine * . * fix * add fsdp engine and sft tests * . * . * . * pass ppo unittest * pass ppo and rollout controller tests * clear unused imports * rename ppo to grpo * change reward function organization * reorganize code * add dataset api * . * . * . * format * chmod fix * . * rename workflow to collector * refactor llm_client location * . * . * fix llm server api * refactor config structure * . * fix tests * . * . * . * Fix unresolved issue in SFTTrainer PR (#139) * . * . * efficient loading * format * . * . * . * . * . * . * Add CI for testing AReaLite (#150) * ci: add test-arealite * ci: add checkout before running test-arealite * ci: add USERNAME * ci: add test script * ci: add GitHub mirror * ci: fix typo * ci: clone one commit * ci: fix condition * ci: set command timeout to 60m * ci: enable pip cache * ci: optimize container lifecycle * ci: split into many stages * ci(test-arealite): fix typo * ci: fix wrong env * ci: fix pytest * ci: uninstall transformer-engine * ci: uninstall transformer-engine * ci: fix model paths * ci: show stdout/stderr * ci: fix not clean up * ci: backup sglang * ci: remove tmp repo dir when run * ci: fix docker run exit 1 condition * ci(test-arealite): limit the concurrency and extend command timeout * . * merge fw/refactor * revert some changes * fix --------- Co-authored-by: meizhiyu.mzy <[email protected]> Co-authored-by: Chayenne <[email protected]> Co-authored-by: zhaochenyang <[email protected]> Co-authored-by: Jayon02 <[email protected]> Co-authored-by: root <meizhiyu.mzy> Co-authored-by: Zijian Zhang <[email protected]> commit d48bf007cfb0821c35a18a673f3c2e8f51a8ee45 Merge: 42c717b b9dbd4a Author: 博惟 <[email protected]> Date: Thu Jul 10 12:53:30 2025 +0800 Merge branch 'main' of https://github.com/inclusionAI/AReaL into lite commit 42c717b6e45b3b345f29cff5882ba5374b79faee Merge: c38cffc a203c7c Author: 博惟 <[email protected]> Date: Thu Jul 10 11:15:01 2025 +0800 Merge branch 'lite' of https://github.com/inclusionAI/AReaL into lite commit c38cffc02385acf382196ba6b720b2d8b91a5675 Author: 博惟 <[email protected]> Date: Thu Jul 10 11:10:10 2025 +0800 PullRequest: 340 [lite] Refactor trainer API into utilities and remove mb_spec in engine methods Merge branch fw/lite-dev of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/340 Reviewed-by: 晓雷 <[email protected]> * support fsdp engine and sglang remote engine * minor fix * . * refactor trainer * add close * rm mb_spec * fix commit b9dbd4a2c18539648dca373c9c8c1347e4e9db68 Author: Wei Fu <[email protected]> Date: Wed Jul 9 10:50:19 2025 +0800 Update to persistent wechat QR code. (#159) commit 17ea7fe94dc0bd9299082a069383d7372ed13ac9 Author: xssstory <[email protected]> Date: Mon Jul 7 15:49:13 2025 +0800 fix math reward verifier (#156) * PullRequest: 293 fix get_param_realloc_path Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/293 Reviewed-by: 博惟 <[email protected]> * fix get_param_realloc_path * PullRequest: 297 bugfix: reward is always -5 Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/297 Reviewed-by: 博惟 <[email protected]> * bugfix: reward is always -5 * PullRequest: 321 fix checkpoint save dir Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/321 Reviewed-by: 博惟 <[email protected]> * fix checkpoint save dir * PullRequest: 328 [Doc] update installation Merge branch sxj/doc of [email protected]:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/328 Reviewed-by: 博惟 <[email protected]> * [Doc] update installation * PullRequest: 329 bugfix: math verifier blocks the async training Merge branch xss/debug of [email protected]:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/329 Reviewed-by: 博惟 <[email protected]> * bugfix: math verifier block the async training * format --------- Co-authored-by: 冰临 <[email protected]> Co-authored-by: garrett4wade <[email protected]> * add autotp for hf * refactor test * fix bugs * fix issues * format files * Squashed commit of the following: commit 9ed043f6ab83cf9c9b38e0251ad08009a73243e8 Author: Wei Fu <[email protected]> Date: Tue Jul 15 10:24:48 2025 +0800 format (#174) commit 8cc9b1feb5ed03cfa42f3ed588dd5f6ec16e5e55 Author: Night <[email protected]> Date: Mon Jul 14 19:22:00 2025 -0700 added LocalSGlangEngine and test (#170) * added LocalSGLangEngine * upload test file * add build args * fix sgl_local generate * improved sgl local robustness * test * test updated * added fallback when sgl engine isn't initialized * finish test local engine * added LocalSGlangEngine and test * format and fix format and fix, raise when generate missing field format * change cli_args.py * add comment header format --------- Co-authored-by: ChangyiYang <[email protected]> --------- Co-authored-by: Jayon02 <[email protected]> Co-authored-by: Wei Fu <[email protected]> * fix ci (#175) * [Feature] Add pre-commit (#178) * fix ci * . * . * . * [lite] [refactor] Add GSM8k GRPO example. (#179) * PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/353 Reviewed-by: 博惟 <[email protected]> * add gradient checkpointing * PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/354 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * . * fix * . * PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * fix destroy process group * fix ci * PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/358 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * fix loss mask * fix * . --------- Co-authored-by: 晓雷 <[email protected]> * 0716_1 * 0716_2 * PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/368 Reviewed-by: 晓雷 <[email protected]> * . * . * PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/371 Reviewed-by: 晓雷 <[email protected]> * . * change doc (#180) * [Fix] [lite] Merge from the internal repo to fix GRPO bugs and refactor the train engine (#181) * PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/353 Reviewed-by: 博惟 <[email protected]> * add gradient checkpointing * PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/354 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * . * fix * . * PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * fix destroy process group * PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/358 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * fix loss mask * fix * . * PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/368 Reviewed-by: 晓雷 <[email protected]> * . * . * PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/371 Reviewed-by: 晓雷 <[email protected]> * . --------- Co-authored-by: 晓雷 <[email protected]> * 0716_3 * 0716_4 * 0716_4 * 0716_5 * 0717_1 * 0717_3 * 0717_3 * 0717_4 * 0717_5 * 0717_6 * 0717_6 * 0717_6 * 0718_2 * 0718_4 * 0718_5 * PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/370 Reviewed-by: 博惟 <[email protected]> * . * . * . * fix * merge_0721 * 0721_1 * PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results. Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/392 Reviewed-by: 晓雷 <[email protected]> * support fsdp engine and sglang remote engine * minor fix * . * refactor trainer * add close * rm mb_spec * . * fix * . * qwen2 grpo works * fix * fix * async works * fix * slurm launcher not tested * fix arg parse * . * sglang server wrapper * . * . * slurm run * ready for boba * debug * 32k run * . * . * fix * . * . * . * . * . * fix * . * fix * . * . * . * . * fix * . * . * . * . * . * . * . * refactor train engine * refactor train engine * . * fix update weight error * . * . * match train * format * . * fix * seems to work * . * . * . * . * 0721_2 * 0721_3 * Implement fsdp distributed update (#183) * PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/353 Reviewed-by: 博惟 <[email protected]> * add gradient checkpointing * PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/354 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * . * fix * . * PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * fix destroy process group * PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/358 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * fix loss mask * fix * . * PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/368 Reviewed-by: 晓雷 <[email protected]> * . * . * PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/371 Reviewed-by: 晓雷 <[email protected]> * . * added remote nccl weight update feat: implement update_weights_from_distributed in fsdp_engine.py unfinishd test, raise PR first coroutine for each server chore: change uploads weights behavior, change test order fix small bug fixed test * fix rebase * add test.sh * updated, test stil fails * . * . * . * fix: full_tensor() should happen in all rank (#187) Co-authored-by: ChangyiYang <[email protected]> --------- Co-authored-by: 晓雷 <[email protected]> Co-authored-by: 博惟 <[email protected]> Co-authored-by: ChangyiYang <[email protected]> Co-authored-by: ChangyiYang <[email protected]> * [Feature] [lite] Merge from internal dev repo (#189) * PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/353 Reviewed-by: 博惟 <[email protected]> * add gradient checkpointing * PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/354 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * . * fix * . * PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * fix destroy process group * PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/358 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * fix loss mask * fix * . * PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/368 Reviewed-by: 晓雷 <[email protected]> * . * . * PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/371 Reviewed-by: 晓雷 <[email protected]> * . * PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/370 Reviewed-by: 博惟 <[email protected]> * . * . * . * fix * PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results. Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/392 Reviewed-by: 晓雷 <[email protected]> * support fsdp engine and sglang remote engine * minor fix * . * refactor trainer * add close * rm mb_spec * . * fix * . * qwen2 grpo works * fix * fix * async works * fix * slurm launcher not tested * fix arg parse * . * sglang server wrapper * . * . * slurm run * ready for boba * debug * 32k run * . * . * fix * . * . * . * . * . * fix * . * fix * . * . * . * . * fix * . * . * . * . * . * . * . * refactor train engine * refactor train engine * . * fix update weight error * . * . * match train * format * . * fix * seems to work * . * . * . * . * format * format * . --------- Co-authored-by: 晓雷 <[email protected]> * [Refactor] Rename files in arealite before release. (#190) * PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/353 Reviewed-by: 博惟 <[email protected]> * add gradient checkpointing * PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/354 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * . * fix * . * PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * fix destroy process group * PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/358 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * fix loss mask * fix * . * PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/368 Reviewed-by: 晓雷 <[email protected]> * . * . * PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/371 Reviewed-by: 晓雷 <[email protected]> * . * PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/370 Reviewed-by: 博惟 <[email protected]> * . * . * . * fix * PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results. Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/392 Reviewed-by: 晓雷 <[email protected]> * support fsdp engine and sglang remote engine * minor fix * . * refactor trainer * add close * rm mb_spec * . * fix * . * qwen2 grpo works * fix * fix * async works * fix * slurm launcher not tested * fix arg parse * . * sglang server wrapper * . * . * slurm run * ready for boba * debug * 32k run * . * . * fix * . * . * . * . * . * fix * . * fix * . * . * . * . * fix * . * . * . * . * . * . * . * refactor train engine * refactor train engine * . * fix update weight error * . * . * match train * format * . * fix * seems to work * . * . * . * . * format * format * . * rename * rename --------- Co-authored-by: 晓雷 <[email protected]> * 0721_4 * . * 0721_formal * 0721_formal * 0721_merge4 * 0721_merge5 * 0721_6 * 0721_merge6 * 0721_merge7 * 0721_8 * 0722_1 * 0722_2 * 0722_3 * add quickstart (#194) * [doc] [lite] Add customization docs for AReaLite. (#191) * PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/353 Reviewed-by: 博惟 <[email protected]> * add gradient checkpointing * PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/354 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * . * fix * . * PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * fix destroy process group * PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/358 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * fix loss mask * fix * . * PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/368 Reviewed-by: 晓雷 <[email protected]> * . * . * PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/371 Reviewed-by: 晓雷 <[email protected]> * . * PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/370 Reviewed-by: 博惟 <[email protected]> * . * . * . * fix * PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results. Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/392 Reviewed-by: 晓雷 <[email protected]> * support fsdp engine and sglang remote engine * minor fix * . * refactor trainer * add close * rm mb_spec * . * fix * . * qwen2 grpo works * fix * fix * async works * fix * slurm launcher not tested * fix arg parse * . * sglang server wrapper * . * . * slurm run * ready for boba * debug * 32k run * . * . * fix * . * . * . * . * . * fix * . * fix * . * . * . * . * fix * . * . * . * . * . * . * . * refactor train engine * refactor train engine * . * fix update weight error * . * . * match train * format * . * fix * seems to work * . * . * . * . * . * . * . * . * . * . * . --------- Co-authored-by: 晓雷 <[email protected]> * 0722_4 * 0722_4 * 0722_5 * 0722_6 * 0722_7 * 0723_1 * reformatted * clang-reformatted * clang-reformatted2 * 0723_1 * 0723_1 * 0723_1 * 0723_merge3 * 0723_4 * 0723_reformatted_5 * 0724_1 * 0724_1 * 0724_merge1 * 0724_merge2 * 0724_merge3 * 0724_merge3 * 0724_merge4 * [lite] [feature] Bump to SGLang v0.4.9.post2 and use NCCL to update weights (#196) * PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/353 Reviewed-by: 博惟 <[email protected]> * add gradient checkpointing * PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/354 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * . * fix * . * PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * fix destroy process group * PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/358 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * fix loss mask * fix * . * PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/368 Reviewed-by: 晓雷 <[email protected]> * . * . * PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/371 Reviewed-by: 晓雷 <[email protected]> * . * PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/370 Reviewed-by: 博惟 <[email protected]> * . * . * . * fix * PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results. Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/392 Reviewed-by: 晓雷 <[email protected]> * support fsdp engine and sglang remote engine * minor fix * . * refactor trainer * add close * rm mb_spec * . * fix * . * qwen2 grpo works * fix * fix * async works * fix * slurm launcher not tested * fix arg parse * . * sglang server wrapper * . * . * slurm run * ready for boba * debug * 32k run * . * . * fix * . * . * . * . * . * fix * . * fix * . * . * . * . * fix * . * . * . * . * . * . * . * refactor train engine * refactor train engine * . * fix update weight error * . * . * match train * format * . * fix * seems to work * . * . * . * . * . * PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2 Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/408 Reviewed-by: 晓雷 <[email protected]> * . * bump arealite to sglang 0.4.9.post2 * . * PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta` * PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9 Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/422 Reviewed-by: 晓雷 <[email protected]> * . * bump arealite to sglang 0.4.9.post2 * . * PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta` * . * PullRequest: 423 [lite] Remove the boba example for github release. Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/423 Reviewed-by: 晓雷 <[email protected]> * . * . --------- Co-authored-by: 晓雷 <[email protected]> * 0724_merge5 * 0724_merge6 * 0724_merge7 * [Docs] [lite] Add example code walkthrough documentation. (#197) * . * . * fix grammar and minor revise * [lite] [doc] Add AReaLite design doc as README (#198) * PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/353 Reviewed-by: 博惟 <[email protected]> * add gradient checkpointing * PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/354 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * . * fix * . * PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * fix destroy process group * PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/358 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * fix loss mask * fix * . * PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/368 Reviewed-by: 晓雷 <[email protected]> * . * . * PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/371 Reviewed-by: 晓雷 <[email protected]> * . * PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/370 Reviewed-by: 博惟 <[email protected]> * . * . * . * fix * PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results. Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/392 Reviewed-by: 晓雷 <[email protected]> * support fsdp engine and sglang remote engine * minor fix * . * refactor trainer * add close * rm mb_spec * . * fix * . * qwen2 grpo works * fix * fix * async works * fix * slurm launcher not tested * fix arg parse * . * sglang server wrapper * . * . * slurm run * ready for boba * debug * 32k run * . * . * fix * . * . * . * . * . * fix * . * fix * . * . * . * . * fix * . * . * . * . * . * . * . * refactor train engine * refactor train engine * . * fix update weight error * . * . * match train * format * . * fix * seems to work * . * . * . * . * . * PullRequest: 408 [Feature] Bump SGLang version to v0.4.9.post2 Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/408 Reviewed-by: 晓雷 <[email protected]> * . * bump arealite to sglang 0.4.9.post2 * . * PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta` * PullRequest: 422 [lite] Fix tests and scripts after updating sgl to 0.4.9 Merge branch fw/sgl049 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/422 Reviewed-by: 晓雷 <[email protected]> * . * bump arealite to sglang 0.4.9.post2 * . * PullRequest: 412 [lite] Minor refactor on `UpdateWeightMeta` * . * PullRequest: 423 [lite] Remove the boba example for github release. Merge branch fw/remove-boba of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/423 Reviewed-by: 晓雷 <[email protected]> * . * . * update readme * . --------- Co-authored-by: 晓雷 <[email protected]> * 0724_4 * 0724-merge8 * 0724_merge8 * 0725_1 * 0725_6 * 0725_7 * 0725_4padded_image * 0725_9padded_image * 0725_10padded_image * 0725 * 0725_12 * 0725_format * Add self-hosted runner support (#199) * ci: add testsz for self hosted runner * ci: use self-hosted runner * ci: run ci on push * ci: remove cd to tmp dir * ci: fix * ci: add environment back * ci: set concurrency * ci: fix build env image * Update .github/workflows/test-arealite.yml Co-authored-by: Copilot <[email protected]> * Update ci/build_env_image.sh Co-authored-by: Copilot <[email protected]> * ci: add some test code * ci: simplify * ci: try to fix * ci: test * ci: test2 * ci: test3 * ci: test3 * ci: tidy up * ci: simplify * ci: do not trigger on push * ci: test arealite/tests/ * ci: use UUID as run id * ci: use openssl rand to genreate run id --------- Co-authored-by: Copilot <[email protected]> * [WIP][feat] Initial support for VLMs, add Qwen2VL SFT test and Qwen2.5VL GRPO test (#188) * vlm_sft_test * vlm_sft_test * . * . * Fix unresolved issue in SFTTrainer PR (#139) * . * . * efficient loading * format * . * . * Fix unresolved issue in SFTTrainer PR (#139) * . * . * efficient loading * format * . * . * image_process0701 * image_process0701 * image_process0701_2 * image_process0701_2 * image_process0701_3 * image_process0701_3 * . * . * . * . * . * . * imageprocess0702 * imageprocess0702 * image_process0702_2 * image_process0702_2 * image_process0702_3 * image_process0702_3 * image_process0702_4 * image_process0702_4 * image_process0702_5 * image_process0702_5 * image_process0703_1 * image_process0703_1 * 0703_2 * 0703_2 * 0703_3 * 0703_3 * 0703_4 * 0703_4 * 0703_4 * 0703_4 * 0703_5 * 0703_5 * 0703_6 * 0703_6 * 0703_7 * 0703_7 * 0703_8 * 0703_8 * 0703_9 * 0703_9 * 0703_11 * 0703_11 * 0703_12 * 0703_12 * 0703_13 * 0703_13 * 0703_14 * 0703_14 * 0703_15 * 0703_15 * 0703_16 * 0703_16 * 0703-17 * 0703-17 * 0703_18 * 0703_18 * 0703_18 * 0703_18 * 0703_19 * 0703_19 * 0704_1 * 0704_1 * 0704_2 * 0704_2 * 0704_3 * 0704_3 * . * . * 0707_1 * 0707_1 * 0707_2 * 0707_2 * 0703_3 * 0703_3 * r * p * fix * fix * refactor * 0707_6 * 0707_7 * refactor1 * 0707_undone * 0708_1 * 0708_2 * 0708_3 * 0708_7 * 0708_4 * 0709_1 * 0709_2 * 0709_3 * 0709_4 * 0709_5 * 0709_ * 0709_6 * 0709_7 * 0709_7 * 0709_8 * 0709_9 * 0710_1 * 0710_2 * 0710_2 * 0710_3 * 0710_3 * 0710_3 * 0710_5 * 0710_4 * merge_2 * merge_3 * 0711_1 * 0711_2 * 0711_3 * 0711_4 * 0711_6 * 0711_7 * 0711_8 * 0711_8 * 0711_9 * 0711_10 * 0711-11 * PullRequest: 353 [Lite] Add gradient checkpointing to FSDPEngine Merge branch mzy/add-gradient-ckpt of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/353 Reviewed-by: 博惟 <[email protected]> * add gradient checkpointing * PullRequest: 354 [lite] GRPO pre-commit: minor changes in FSDP engine Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/354 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * 0714_1 * 0714_2 * 0714_3 * 0714_3 * 0714_5 * PullRequest: 355 [Lite] GRPO pre-commit 2: Refactor RemoteSGLangEngine thread and SGLang configuration Merge branch fw/lite-fix1 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/355?tab=commit Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * . * fix * . * PullRequest: 357 [lite] GRPO pre-commit 3: Fix typos and experiment utilities Merge branch fw/lite-fix2 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/357?tab=comment Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * . * fix destroy process group * PullRequest: 358 [lite] Support GRPO training locally with the GSM8k dataset Merge branch fw/lite-fix3 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/358 Reviewed-by: 晓雷 <[email protected]> * . * . * . * . * fix loss mask * fix * . * 0715_1 * 0715_2 * 0715_2 * 0716_1 * 0716_2 * PullRequest: 368 [lite] Refactor train engine after merging contributions from GitHub Merge branch fw/lite-train-engine of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/368 Reviewed-by: 晓雷 <[email protected]> * . * . * PullRequest: 371 [lite] [fix] fix misc bugs in GRPO implementation Merge branch fw/lite-fix0716 of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/371 Reviewed-by: 晓雷 <[email protected]> * . * 0716_3 * 0716_4 * 0716_4 * 0716_5 * 0717_1 * 0717_3 * 0717_3 * 0717_4 * 0717_5 * 0717_6 * 0717_6 * 0717_6 * 0718_2 * 0718_4 * 0718_5 * PullRequest: 370 [lite] Add Slurm Launcher and Ray Launcher Merge branch mzy/lite/launcher of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/370 Reviewed-by: 博惟 <[email protected]> * . * . * . * fix * merge_0721 * 0721_1 * PullRequest: 392 [lite] Fix several bugs regarding RL learning and add an example to reproduce boba-math results. Merge branch fw/lite-boba of [email protected]:inclusionAI/AReaL.git into lite https://code.alipay.com/inclusionAI/AReaL/pull_requests/392 Reviewed-by: 晓雷 <[email protected]> * support fsdp engine and sglang remote engine * minor fix * . * refactor trainer * add close * rm mb_spec * . * fix * . * qwen2 grpo works * fix * fix * async works * fix * slurm launcher not tested * fix arg parse * . * sglang server wrapper * . * . * slurm run * ready for boba * debug * 32k run * . * . * fix * . * . * . * . * . * fix * . * fix * . * . * . * . * fix * . * . * . * . * . * . * . * refactor train engine * refactor train engine * . * fix update weight error * . * . * match train * format * . * fix * seems to work * . * . * . * . * 0721_2 * 0721_3 * 0721_4 * . * 0721_formal * 0721_formal * 0721_merge4 * 0721_merge5 * 0721_6 * 0721_merge6 * 0721_merge7 * 0721_8 * 0722_1 * 0722_2 * 0722_3 * 0722_4 * 0722_4 * 0722_5 * 0722_6 * 0722_7 * 0723_1 * reformatted * clang-reformatted * clang-reformatted2 * 0723_1 * 0723_1 * 0723_1 * 0723_merge3 * 0723_4 * 0723_reformatted_5 * 0724_1 * 0724_1 * 0724_merge1 * 0724_merge2 * 0724_merge3 * 0724_merge3 * 0724_merge4 * 0724_merge5 * 0724_merge6 * 0724_merge7 * 0724_4 * 0724-merge8 * 0724_merge8 * 0725_1 * 0725_6 * 0725_7 * 0725_4padded_image * 0725_9padded_image * 0725_10padded_image * 0725 * 0725_12 * 0725_format --------- Co-authored-by: bowei.fw <[email protected]> Co-authored-by: nuzant <[email protected]> Co-authored-by: 朱晗 <[email protected]> * 0731 * 0731_1 * 0731_2 * 0731_2 * 0731_3 * 0731_4 * 0801_1 * 0801_2 * 0804_1 * 0804_2 * 0804_2 * 0804_5 * 0805_3 * 0805_2 * 0806 * 0806_merge1 * 0806_merge2 * 0806_format1 * 0806_merge3 * 0806_4 * 0806_6 * 0806_7 * 0806_formatted2 * fix * revert examples * . * . --------- Co-authored-by: 朱晗 <[email protected]> Co-authored-by: 博惟 <[email protected]> Co-authored-by: garrett4wade <[email protected]> Co-authored-by: Wei Fu <[email protected]>
PreviousNext