hiyouga
|
ec81d45d27
|
fix mod stuff
Former-commit-id: f58425ab45727f7859583d4b9fda776715e27ff6
|
2024-04-21 18:11:10 +08:00 |
|
Marco
|
639297a5ef
|
Added Mixture of Depths
Former-commit-id: 620add7b9f634de1a711f7b87b16050adf735e9b
|
2024-04-18 20:31:24 +02:00 |
|
hiyouga
|
2dc3343b1c
|
support cohere commandR #3184
Former-commit-id: e0dbac28450a0e1e0b84e1577ef785fc762c0b46
|
2024-04-15 23:26:42 +08:00 |
|
hiyouga
|
ceccad3419
|
fix #3273
Former-commit-id: efc345c4b0095ec959ea23bbe54c344278780cbe
|
2024-04-15 15:32:58 +08:00 |
|
hiyouga
|
f4be51f356
|
add moe aux loss control #3085
Former-commit-id: b267aeb53fc49d2eeb0f3fc5ebe55e643f5db377
|
2024-04-02 14:26:31 +08:00 |
|
hiyouga
|
b7468ea0a8
|
support infer 4bit model on GPUs #3023
Former-commit-id: eb259cc5738dfb383e4cc5d32579501c580e11b1
|
2024-04-01 17:34:04 +08:00 |
|
hiyouga
|
a74426df0f
|
fix kv cache
Former-commit-id: 96ce76cd2753bc91c781ad13aa8f7a972abe815a
|
2024-03-13 01:21:50 +08:00 |
|
hiyouga
|
0b7e870b07
|
fix #2802
Former-commit-id: 8d8956bad542c0e1c0f7edbf4ffc22bb0f8788ae
|
2024-03-12 17:08:34 +08:00 |
|
hiyouga
|
7124b71676
|
fix #2782 #2798
Former-commit-id: 07f9b754a7418b489e839bd674aa47094583a92d
|
2024-03-12 15:53:29 +08:00 |
|
hiyouga
|
868444e124
|
allow non-packing pretraining
Former-commit-id: bdb496644ce2c18806fc4fdae1fedcb3e5b5f808
|
2024-03-09 22:21:46 +08:00 |
|
hiyouga
|
7443ac3116
|
fix chat engine, update webui
Former-commit-id: 5d956e2a5167201aecdfce2794c25d8a2d84e234
|
2024-03-08 03:01:53 +08:00 |
|
hiyouga
|
34533b2f35
|
support vllm
Former-commit-id: d07ad5cc1cdbc13879afd84f653afdfee03a6933
|
2024-03-07 20:26:31 +08:00 |
|
hiyouga
|
37e40563f1
|
fix #2735
Former-commit-id: f74f804a715dfb16bf24a056bc95db6b102f9ed7
|
2024-03-07 16:15:53 +08:00 |
|
hiyouga
|
9561809ce9
|
improve aqlm optim
Former-commit-id: 259af60d28985b919911587716c24a3ac7f7de64
|
2024-03-05 20:49:50 +08:00 |
|
hiyouga
|
96265ec154
|
support llama pro #2338 , add rslora
Former-commit-id: 7924ffc55d98e33bfbfbca303e46c8f476435673
|
2024-02-15 02:27:36 +08:00 |
|
hiyouga
|
23dd337ac2
|
lint
Former-commit-id: 88a1bc97736bf06f292cd768fc8b61503aca1988
|
2024-02-07 01:10:04 +08:00 |
|
hiyouga
|
b27e91222c
|
format style
Former-commit-id: 638234ceee1b19716e45b6e5f4ea54d9122da4df
|
2024-01-20 20:15:56 +08:00 |
|
hiyouga
|
edf725208d
|
add upcast_lmhead option
Former-commit-id: 8cbe4e960983ace47f8b956cdc31411347592129
|
2024-01-19 23:54:25 +08:00 |
|
hiyouga
|
6a954cc075
|
support export push_to_hub #2183
Former-commit-id: 42859f073434eab0928940e8a9c52f275a2fc93a
|
2024-01-16 23:59:42 +08:00 |
|
hiyouga
|
cae66bce3d
|
fix args
Former-commit-id: 65c5b0477c0e62691a1f8790670ba04d7f6d2804
|
2023-12-28 18:47:19 +08:00 |
|
hiyouga
|
16688b773a
|
fix export format
Former-commit-id: e165354facf7e69f535f9b7d99438f03dbf0293d
|
2023-12-28 18:40:46 +08:00 |
|
hiyouga
|
2a3980d6ba
|
update loader
Former-commit-id: 6629087e12f64f2635f24311234202077814083c
|
2023-12-24 19:10:23 +08:00 |
|
hiyouga
|
5d440f978e
|
update patcher
Former-commit-id: e44b82ee245a7ee99057c7b58b1edef5c222dc1f
|
2023-12-23 15:24:27 +08:00 |
|
hiyouga
|
f0d405f392
|
support unsloth
Former-commit-id: 7aad0b889d9a316fffd65f32a419078418fc0986
|
2023-12-23 00:14:33 +08:00 |
|
hiyouga
|
5a199af387
|
fix tokenizer for Yi chat models #1617 #1875
Former-commit-id: 71a9c1617181b7df46cfb193464fb7e56e6399b1
|
2023-12-18 17:18:11 +08:00 |
|
hiyouga
|
bd03307bbd
|
refactor adapter hparam
Former-commit-id: 0716f5e470afffd2df5a815712b552a4b4797153
|
2023-12-15 20:53:11 +08:00 |
|
hoshi-hiyouga
|
b67085e13a
|
Merge branch 'main' into feat/support_ms
Former-commit-id: 6382efec52f6be3daa5db0bd280a96162009fca1
|
2023-12-12 17:55:32 +08:00 |
|
xingjun.wang
|
879209829e
|
update args for MsDataset.load
Former-commit-id: 09533e95edc5fa65a38b2f04c6d88506196021b3
|
2023-12-12 13:02:54 +08:00 |
|
hiyouga
|
c60e79c12e
|
patch modelscope
Former-commit-id: bd42c229b01a0bf3ceadb8cee5ad49a060cc2d13
|
2023-12-01 22:53:15 +08:00 |
|
yuze.zyz
|
e08e0e5814
|
support ms
Former-commit-id: d38a2e7341100902b6c761895b1fe6191c905d06
|
2023-11-29 20:36:55 +08:00 |
|
hiyouga
|
f441932bd1
|
support full-parameter PPO
Former-commit-id: ce783036001397a20b0b4c5da2fea6d0c03389d2
|
2023-11-16 02:08:04 +08:00 |
|
hiyouga
|
55e097aaac
|
add todo
Former-commit-id: a0c31c68c4909637b86c90c319c321fd887c4910
|
2023-11-10 14:38:18 +08:00 |
|
hiyouga
|
91f406cc99
|
fix ppo train and dpo eval
Former-commit-id: 01260d975477ebb8570933a1bd7f547b4dba607f
|
2023-11-07 22:48:51 +08:00 |
|
hiyouga
|
3d40bdb600
|
upgrade peft, fix #1088 #1411
Former-commit-id: b2a60905f384ada92618bf21301fe96dac1c10bf
|
2023-11-07 16:13:36 +08:00 |
|
hiyouga
|
d6c77d9196
|
reimplement neftune
Former-commit-id: 7b4acf7265b04cc4a674b3dcafdb90e76f149e39
|
2023-10-22 16:15:08 +08:00 |
|
hiyouga
|
f3fa47fa7d
|
refactor export, fix #1190
Former-commit-id: ea82f8a82a7356bbdf204190d596d0b1c8ef1a84
|
2023-10-15 16:01:48 +08:00 |
|
hiyouga
|
e585c789ce
|
fix #1184
Former-commit-id: af18b0dce7a4ef10b30da069d454010eddd269af
|
2023-10-14 19:20:11 +08:00 |
|
hiyouga
|
c9d1cd108d
|
refactor model_dtype, fix PPO trainer
Former-commit-id: 2818af0b0967d7695f27658acac0b7e2c2728e5d
|
2023-10-11 23:16:01 +08:00 |
|
hiyouga
|
deb17942ab
|
fix layer norm dtype
Former-commit-id: 84b7486885c600e5e65c5ba9095d56ecc2502977
|
2023-09-28 00:25:55 +08:00 |
|
hiyouga
|
108c31e1fc
|
support LongLoRA
Former-commit-id: 90375f600d5601866836123597fa3ef52008eeef
|
2023-09-27 21:55:50 +08:00 |
|
hiyouga
|
4581d09fa6
|
fix #944
Former-commit-id: 338b8664edea5ae65192ac657bb013581245ae15
|
2023-09-21 19:51:02 +08:00 |
|
hiyouga
|
8ab5566dc0
|
support FlashAttention2
Former-commit-id: d8aa1404bee9842f3e4cd037ad8d66c85470ac37
|
2023-09-10 20:43:56 +08:00 |
|
hiyouga
|
9ed4bb63d4
|
change to right-padding, update reward score #803
Former-commit-id: 8ea32e4046d75ddfa9517669e9de9f48fea720c6
|
2023-09-08 20:04:31 +08:00 |
|
hiyouga
|
a4fd976048
|
refactor dataset_attr, add eos in pt, fix #757
Former-commit-id: a9d1fb72f791ae57a4d12f4e3a7e2abccf6a7077
|
2023-09-01 19:00:45 +08:00 |
|
hiyouga
|
3f0a2d6adc
|
support rope scaling, fix #475 #476 #478
Former-commit-id: fa940c17b8d3e379af08804003f1a522c1cd6ac4
|
2023-08-12 20:46:27 +08:00 |
|
hiyouga
|
79f4ba0d26
|
Release v0.1.6
Former-commit-id: a48cb0d474ef0648a97387daf5f623498b5e3ee6
|
2023-08-11 23:25:57 +08:00 |
|
hiyouga
|
abdfa26d06
|
support DPO training (2305.18290)
Former-commit-id: 3ec4351cfdaf2aefcc7d13345e19d79874ed61d3
|
2023-08-11 03:02:53 +08:00 |
|
jiongxuc
|
7ffd961b8b
|
huggingface login for projects must login while running
Former-commit-id: 3e000c2b60c2e29bcafcf8d39c1a5d567ae2491c
|
2023-08-10 14:57:12 +08:00 |
|
hiyouga
|
a696148d6b
|
modity code structure
Former-commit-id: f75137661358f9070bc70c341dfa2cc5fd69cf94
|
2023-07-15 16:54:28 +08:00 |
|