33 Commits

Author SHA1 Message Date
hiyouga
dc64166d13 update scripts
Former-commit-id: f2aa02c070b1442bfb58a075f22af25386192bfa
2024-09-08 14:17:41 +08:00
hiyouga
0daee7cb39 support activation offloading via unsloth gc
Former-commit-id: fb72a3adb0916232cc9ac9f0c725c02d07b9354c
2024-09-08 01:22:19 +08:00
hiyouga
7ccb86b215 add docstrings, refactor logger
Former-commit-id: 54c69059379d77dc9046c144cbe2d0253de3a4da
2024-09-08 00:56:56 +08:00
hoshi-hiyouga
4d8b782268 fix
Former-commit-id: 842c5455b0e7c4b8d307af950304d4ed52bbb3e8
2024-09-08 00:41:45 +08:00
hoshi-hiyouga
4f28e0e5d2 Update cal_mfu.py
Former-commit-id: e71133cab14eed2c071cfd4ce044ae9905447427
2024-09-08 00:39:48 +08:00
-.-
ab1775cd95 update cal_mfu.py
Former-commit-id: 66ec36522c9bf8dfffc1065202362801875a104d
2024-09-07 23:21:35 +08:00
hiyouga
d5ea05cfff update get template
Former-commit-id: dabad5570bf4a6b1044c963d8f27717030f373ef
2024-09-04 22:36:20 +08:00
hiyouga
25b9cfa163 update scripts
Former-commit-id: 86f7099fa3fadd9c5a2059361ab5a5e1dbf5b1a2
2024-08-09 19:16:23 +08:00
hiyouga
0aad78694a tiny fix
Former-commit-id: ace1d44857368dbbc868e4bba2d7ff17329921f5
2024-07-24 18:33:39 +08:00
hiyouga
019c6dad84 fix #4925
Former-commit-id: 935b22d93ed461e829cd1e6201de1fea697f4d1e
2024-07-24 16:56:58 +08:00
hiyouga
14bc7b0551 fix up
Former-commit-id: 29ebcd75d55f70f2891632eba187b643cc3a9e51
2024-07-15 01:04:56 +08:00
codingma
74f0d02eb8 1. add custom eval dataset support
2. merge load dataset and split dataset function


Former-commit-id: 76f3bbcfc0e11aa41f8f5cbebc60b77b987f7901
2024-07-05 15:52:10 +08:00
hiyouga
e5c89890b1 update scripts
Former-commit-id: 1e0c860c8c5ae8958d7105acafdac5d253a585f9
2024-07-03 20:07:44 +08:00
hiyouga
654116c0b1 fix #4556
Former-commit-id: 59e0b4f616736ede37cc37a13346b547f5a2d4e7
2024-06-26 19:43:16 +08:00
hiyouga
19bf21efba lint
Former-commit-id: 24c160df3d575843e5ad5f1b47246d04430a79f0
2024-06-17 22:35:56 +08:00
hiyouga
004f289074 tiny fix
Former-commit-id: 2bf2863a58c93206f271de17d7dfcbcd6375cd73
2024-06-17 17:47:25 +08:00
hiyouga
f25b8626bf support pissa
Former-commit-id: 8c1046d78ac6c8f9429b73617e35e1eccb35138f
2024-06-16 01:08:12 +08:00
hiyouga
c0c6b8075a tiny fix
Former-commit-id: 38b6b0f52edeb8ba45aa03b415b3c0c1b0e0c1e4
2024-06-16 01:06:41 +08:00
hiyouga
2946153cea add license
Former-commit-id: d87108daa68bd40174b262be1ca65fe6e1b7ab56
2024-06-15 17:54:33 +08:00
hiyouga
4f3e680b57 init unittest
Former-commit-id: 1c7f0ab51906b20190f8d4db932623cff76efc01
2024-06-08 01:35:58 +08:00
hiyouga
0eff6a66d5 tiny fix
Former-commit-id: 5a13b3baa63225e7f79e024610722de0f87e0acc
2024-06-04 00:31:10 +08:00
hiyouga
cae823ddf0 rename package
Former-commit-id: 308edbc4260d45907b4a9d3a45ec21d83e48aacb
2024-05-16 18:39:08 +08:00
hiyouga
6e6267f17c fix #3694
Former-commit-id: 2a67ab3925f0c17c4cb5e8c5a5e2cc6a9dc7d47e
2024-05-16 00:35:28 +08:00
hiyouga
9bbb5c846d update webui
Former-commit-id: af596988b1e8988fd3f1376a8e090ed8b80ee810
2024-05-05 00:17:54 +08:00
hiyouga
df43fbb029 update scripts
Former-commit-id: c1a53a0debf98be26f0f64d710a13c3b25140f75
2024-05-04 23:05:17 +08:00
hiyouga
5f8d83b630 add avg ppl
Former-commit-id: 25aeaae51b6d08a747e222bbcb27e75c4d56a856
2024-05-04 22:35:31 +08:00
hiyouga
4df26e7439 update ppl script
Former-commit-id: 76a077bdcea127fe1a9ae9e334f443d0ea7c5c7b
2024-05-04 22:13:14 +08:00
hiyouga
23f9efdf7d add cal_ppl script
Former-commit-id: 3a666832c119606a8d5baf4694b96569bee18659
2024-05-04 22:02:25 +08:00
hiyouga
70bf2a2247 update readme
Former-commit-id: 27ba1b63ce22d6fc2198c517141051705b2e77f6
2024-04-26 05:44:30 +08:00
hiyouga
54a4a8217a fix bug in latest gradio
Former-commit-id: 5ddcecda50ccff93d51bebc9ac72c2a0dd483e9b
2024-04-04 00:55:31 +08:00
hiyouga
2ecf2bcbf0 fix resize vocab at inference #3022
Former-commit-id: 148bda353f0b53af022c51da9a9e59a56f341510
2024-04-03 18:14:24 +08:00
hiyouga
34533b2f35 support vllm
Former-commit-id: d07ad5cc1cdbc13879afd84f653afdfee03a6933
2024-03-07 20:26:31 +08:00
hiyouga
e887aface7 fix version checking
Former-commit-id: 3016e6565708637c1d760f2cd5a67cbd8a5a6c26
2024-03-06 14:51:51 +08:00