113 Commits

Author SHA1 Message Date
hoshi-hiyouga
302e4e22bf Merge pull request #6078 from wtmlon/support-efficient-tokens-calculation
support effective tokens calculation on sft/dpo

Former-commit-id: bd639a137e6f46e1a0005cc91572f5f1ec894f74
2024-11-20 13:43:15 +08:00
Ting
e27a0c3d53 code refactor
Former-commit-id: 40627c601efc9f144a227dded8c6b40babff4e8b
2024-11-19 20:33:18 +08:00
Ting
32656bc50d update
Former-commit-id: f566ecc8d1f04615351acbe4f8480b75b2daed42
2024-11-19 19:12:10 +08:00
Ting
bf2b8df540 update
Former-commit-id: ef6e14550dd76810285cee9c268590d1d9423e54
2024-11-19 19:10:07 +08:00
Ting
7ad5b5c088 support efficient tokens calculation on sft/dpo
Former-commit-id: b9f00286d8a017ed9fd2876986da3b4d7034ef07
2024-11-19 17:15:47 +08:00
hoshi-hiyouga
9815d1712c fix #6050
Former-commit-id: dc828218726704ff0453a2d13535663ac6ad7833
2024-11-16 16:11:16 +08:00
hiyouga
c2766af6f4 fix dpo metrics
Former-commit-id: 4270f7dfb9a12471c91f6c03dce7ca6fd88566e1
2024-11-02 20:59:01 +08:00
hiyouga
e83cb17f97 support rank0 logger
Former-commit-id: c38aa29336f286266553da4909a7267d7ef21f37
2024-11-02 18:31:04 +08:00
hiyouga
3f7c874594 update tests
Former-commit-id: 93d3b8f43faf4a81b809d2f7d897e39bdb5475c3
2024-11-02 12:41:44 +08:00
hiyouga
584ce3a105 fix incorrect loss value for vlms
Former-commit-id: 30567a1487727473950104718e626ff660f10cbb
2024-10-30 08:56:46 +00:00
hiyouga
13c7e873e0 fix #5749
Former-commit-id: 23dbe9a09999fe0f9eb2902a40e33b36db4ca584
2024-10-29 13:02:13 +00:00
hiyouga
d183966a5d fix pissa
Former-commit-id: 51e5f962474739bbf396782afdaa68743636fe90
2024-10-29 12:18:45 +00:00
hiyouga
825ea1c72d fix #5747
Former-commit-id: ae045c884f8ac2aa0ea27592e0757b7bca2dba13
2024-10-29 10:47:04 +00:00
hiyouga
0d8aa6e6ef use pre-commit
Former-commit-id: 21db8ed2f4a0eba203754a92ce0741538e8ee709
2024-10-29 09:07:46 +00:00
hoshi-hiyouga
bdb77bc85a fix test
Former-commit-id: 74a79cc0599b047a691c427d16344a824b21e0f3
2024-10-22 12:35:36 +08:00
hiyouga
4464a6ff5b tiny fix
Former-commit-id: 451d271718a8026056d0f7d7b8ab333391d24ad4
2024-10-08 17:48:56 +08:00
Chengcheng Pei
e80c98367e 1, log exceptions in details; 2, check processor is None before calling it.
Former-commit-id: 573e3183e644e8da61a409d96b9adcfacbfc3a7a
2024-09-25 12:59:48 -07:00
hiyouga
009500bc6d fix #5411
Former-commit-id: c7e51ff187658eb472c2b234f75d8934c6f7c782
2024-09-11 17:36:42 +08:00
hiyouga
3aefdad4ec release v0.9.0 (real)
Former-commit-id: 90d6df622252c6fad985f68b97771c979357e2fc
2024-09-09 01:00:25 +08:00
hiyouga
7ccb86b215 add docstrings, refactor logger
Former-commit-id: 54c69059379d77dc9046c144cbe2d0253de3a4da
2024-09-08 00:56:56 +08:00
hoshi-hiyouga
f6014742fa fix #5366
Former-commit-id: e9bda48c6d7bde135df6456513708a997ada916c
2024-09-05 18:08:09 +08:00
hiyouga
666013d09d fix ci
Former-commit-id: 1173f7fc1dbdcf814650bfdf854ade5212fc4738
2024-09-05 03:02:59 +08:00
hiyouga
d5ea05cfff update get template
Former-commit-id: dabad5570bf4a6b1044c963d8f27717030f373ef
2024-09-04 22:36:20 +08:00
hiyouga
22deca0e9e lazy image load
Former-commit-id: 47ea97fb1ba77de2e8a561904aa8fdc27c3f5025
2024-09-04 02:27:08 +08:00
hiyouga
982585e375 lint
Former-commit-id: 22959bcdd3b124a642e2acaadc050e36d0520f52
2024-09-03 00:46:25 +08:00
hiyouga
6e98872622 fix #5324
Former-commit-id: a61c8c4890962f3847b19eff31b170cd7f54316c
2024-09-02 23:56:21 +08:00
hoshi-hiyouga
5af92971bc fix trainer predict
Former-commit-id: 99fd9637bdc25f41fd1abc8a162f1069cb9060d4
2024-09-02 10:15:29 +08:00
hoshi-hiyouga
5c9972a2d5 remove .cpu()
Former-commit-id: a6c6750e8af5bc1ece1dfe6111d3e484fd19ee75
2024-09-02 10:10:53 +08:00
hiyouga
bfdcc6bacf add rlhf-v dataset
Former-commit-id: 8e49940746c1a6ff910f07dbefbec14af9d0f3c6
2024-09-01 22:57:41 +08:00
hiyouga
236f97b35c tiny fix
Former-commit-id: 55027282cdaa59a470ac89bfb3860504ba9075ff
2024-09-01 21:15:44 +08:00
hiyouga
cb776752f6 fix mixed mm inputs and rlhf-v
Former-commit-id: 9967ccb3aef3ca557ad6eafb78c6c99866857008
2024-09-01 20:52:47 +08:00
hiyouga
f31e7e0dfc remove visual_inputs, fix qlora
Former-commit-id: a025c3df61db154bef13033518903bbf846f4fc8
2024-08-31 00:24:51 +08:00
hiyouga
51a0016873 optimize predict vram
Former-commit-id: a244f143f48a01910ce1cd56c0855ef11d62a72a
2024-08-30 23:08:45 +08:00
hiyouga
a83756b5e9 refactor mm training
Former-commit-id: 3382317e32f88ed377d3e7759bdeaf0f2559d22a
2024-08-30 02:14:31 +08:00
hiyouga
21d3976eea fix #5295
Former-commit-id: ad72f3e06593f124d661d61774def336511716e0
2024-08-29 20:30:18 +08:00
hiyouga
1494fa1f18 fix #5305
Former-commit-id: 364b757e306f7a154359a2bc8245a839f39c4fab
2024-08-29 20:16:01 +08:00
hiyouga
7b5834b2dd tiny fix
Former-commit-id: f6ae4e75ddaeb4ac4a527f0141ac5b1afefde10e
2024-08-27 12:49:32 +08:00
hiyouga
daebca2368 tiny fix
Former-commit-id: c8b4c7fee5398654683b713ad5c03b5daf13218a
2024-08-20 00:10:52 +08:00
liu-zichen
8a7ab8ab21 fix lr not change
Former-commit-id: ddee718b31a5bf3cb39c5adf3f8e0be8fddf9dbb
2024-08-13 16:33:34 +08:00
hiyouga
5eacd17090 add adam_mini to readme
Former-commit-id: e2a28f51c635d64ff9de65a37087d89356bdedcc
2024-08-09 20:02:03 +08:00
moontidef
44f7c4dd56 feat: add support for adammini
Former-commit-id: 82bc15dc795f95768b81c25eaaabdc613da30cd8
2024-08-07 10:08:22 +08:00
moontidef
b0d32b2041 fix: rename optimzer to optimizer
Former-commit-id: 40908a36fae3393715f75156867c11e6373fabad
2024-08-07 10:05:01 +08:00
hiyouga
20013e130b fix #5048
Former-commit-id: b7ca6c8dc14f689d0df16684a6121cc0ec24f8ba
2024-08-05 23:48:19 +08:00
codingma
8132725f2e fix pissa save
Former-commit-id: 2c1ca9f7425b84e158fef527fd6e13297c8253c6
2024-07-29 10:44:34 +08:00
hiyouga
e1e01d7efd add unittest
Former-commit-id: 608de799a21f37319bf31c04c0aa50c4542ec757
2024-07-19 01:06:27 +08:00
hiyouga
3c7b10b1fa fix metrics #4786
Former-commit-id: beec77a0898a39d94f41c23920415f5b4873a23a
2024-07-17 00:47:00 +08:00
hiyouga
e90fae61f4 support batch_eval_metrics, fix #4826
Former-commit-id: d774b94f124923829b2eae428e25199d503ebfcb
2024-07-17 00:33:00 +08:00
hiyouga
84e6715423 fix #4820
Former-commit-id: fd8cc490084aba9b5155eaaaf26129efd2871fa3
2024-07-15 22:32:07 +08:00
hiyouga
14bc7b0551 fix up
Former-commit-id: 29ebcd75d55f70f2891632eba187b643cc3a9e51
2024-07-15 01:04:56 +08:00
hoshi-hiyouga
2b22a7da48 Merge pull request #4691 from codemayq/feature-suppot-eval-dataset
add eval dataset support

Former-commit-id: 15b399a82f45b08fc07d2957884fb7821eba9fd9
2024-07-15 01:00:34 +08:00