hiyouga
|
be0a807e8c
|
fix ORPO loss
Former-commit-id: 5544ddde9087f00f9e20b78d0079f20c2f5d1604
|
2024-04-01 14:42:41 +08:00 |
|
hiyouga
|
52d402e2a9
|
fix IPO and ORPO loss
Former-commit-id: fc27955732aedbb12003faf19b760e2768b228f2
|
2024-04-01 14:37:53 +08:00 |
|
hiyouga
|
c5a46f9113
|
fix plots
Former-commit-id: 81355671296b84d438967463bb2a92934ff31aae
|
2024-03-31 19:43:48 +08:00 |
|
hiyouga
|
00e17a377c
|
use log1p in orpo loss
https://github.com/huggingface/trl/pull/1491
Former-commit-id: 3b15d495264b00a4f8716bafea334778874963d7
|
2024-03-31 19:27:08 +08:00 |
|
hiyouga
|
d764cd8736
|
support ORPO
Former-commit-id: f44a4c27e2461cdaa1b16865f597a31033c0e6d9
|
2024-03-31 18:29:50 +08:00 |
|
marko1616
|
a6858a36c0
|
Fix Llama model save for full param train
Former-commit-id: ca17b5db4f97c3ec9fe2004877f150e8f51ab4b5
|
2024-03-30 23:45:04 +08:00 |
|
hiyouga
|
fbd0584391
|
release v0.6.1
Former-commit-id: a59d823f554505b2e649e6e111b9dee8306d3ad8
|
2024-03-29 11:36:08 +08:00 |
|
hiyouga
|
9408366a36
|
fix #2982
Former-commit-id: e5e6a0c50c7a1c0052ed6b459450b9735ff2c9a1
|
2024-03-28 20:22:31 +08:00 |
|
hiyouga
|
59e6ebf039
|
update trainers
Former-commit-id: d0dd6eefed0b86895ed00a7cafb331e5193db645
|
2024-03-28 18:16:27 +08:00 |
|
hoshi-hiyouga
|
dc540dfaa8
|
fix ds optimizer
Former-commit-id: 2675127070a1e7584e71039a11c1ebac54ddd1db
|
2024-03-26 23:39:56 +08:00 |
|
hiyouga
|
3336422760
|
fix #2961
Former-commit-id: 616917bb3be7f71073b56ad8c7bc4e164b08b9b5
|
2024-03-26 17:26:14 +08:00 |
|
hiyouga
|
04423b916f
|
release v0.6.0 (real)
Former-commit-id: 34e06bf408ccd21e674f896703f1c7b62e97e1ca
|
2024-03-25 23:37:48 +08:00 |
|
hiyouga
|
daab85e3e6
|
release v0.6.0
Former-commit-id: 51910d5803eb718e4976da0b3bfcdc5eeeea48eb
|
2024-03-25 22:38:56 +08:00 |
|
hiyouga
|
04e0fe9147
|
tiny fix
Former-commit-id: c39cf3439a3025f703d50ac414c10ef3c8486a1f
|
2024-03-25 21:18:08 +08:00 |
|
marko1616
|
7f99cb1817
|
pass ruff check
Former-commit-id: 8534b069a05121eb041371a6becccf0a1a23f268
|
2024-03-24 16:12:10 +08:00 |
|
marko1616
|
c555b2cce3
|
fix Llama lora merge crash
Former-commit-id: 46f7d8e6b85f73fb0c51c8b08bd9955c3b171d93
|
2024-03-24 03:06:11 +08:00 |
|
marko1616
|
2eba1c6851
|
fix Llama lora merge crash
Former-commit-id: a8bd8e9149ff79a2707fec9c6d006761cfdd0dee
|
2024-03-24 02:55:23 +08:00 |
|
marko1616
|
edeed55664
|
fix Llama lora merge crash
Former-commit-id: c29a2893f58cf7a916ff05b2725fadf1ad2c4c9a
|
2024-03-24 02:44:35 +08:00 |
|
hiyouga
|
c7af26a9e3
|
fix #2777 #2895
Former-commit-id: 54d5f62d29456a8d9d0c0dd3d0bbfffe48935803
|
2024-03-20 17:59:45 +08:00 |
|
hiyouga
|
ffdacaa618
|
fix packages
Former-commit-id: 2f9f334a123d43267bfb3dd26aaa1ad285ffe7a5
|
2024-03-17 22:32:03 +08:00 |
|
hiyouga
|
ed020579dc
|
fix export
Former-commit-id: 4e996f194406d7eb27b2bde290a12c82c41219d0
|
2024-03-15 15:06:30 +08:00 |
|
hiyouga
|
623ee1bd88
|
tiny fix
Former-commit-id: bf8123669be334338b4268d0a8f7703ff2cf6255
|
2024-03-14 21:19:06 +08:00 |
|
hiyouga
|
aabe90343e
|
fix export
Former-commit-id: c9b968b84c97c9a00fbb43194c3adc9354d74f3b
|
2024-03-14 18:17:01 +08:00 |
|
hiyouga
|
764cfb506d
|
fix bug
Former-commit-id: 38c618b797ec219c2c45de960c9cbe50ec524c94
|
2024-03-13 23:55:31 +08:00 |
|
hiyouga
|
249ad56075
|
fix bug
Former-commit-id: 47ee0276830adbed65bc111d5a83049e77ad360a
|
2024-03-13 23:43:42 +08:00 |
|
hiyouga
|
46f99ff277
|
improve lora+ impl.
Former-commit-id: 332bad25455a70ad9204e7dd384bb086d789aa39
|
2024-03-13 23:32:51 +08:00 |
|
齐保元
|
3c91e86268
|
[FEATURE]: ADD LORA+ ALGORITHM
Former-commit-id: c35b3c3b1e27171f8a703f88ede1dc8a84c80a56
|
2024-03-13 19:43:27 +08:00 |
|
hiyouga
|
4f9a47c026
|
fix #2775
Former-commit-id: a5c7feb3e8089f4deff760b00a9f84425957c419
|
2024-03-11 00:42:54 +08:00 |
|
hiyouga
|
7ff8a064f3
|
support layerwise galore
Former-commit-id: d43a4da0947897d0be3f62fad3107754d4c89f2b
|
2024-03-10 00:24:11 +08:00 |
|
hiyouga
|
4881f4e631
|
allow non-packing pretraining
Former-commit-id: 3fee5cc5a3db9ce874ad90f2500ec092d904bd4e
|
2024-03-09 22:21:46 +08:00 |
|
hiyouga
|
c631799f5d
|
fix #2766
Former-commit-id: a8cd556230c1d0bc4e090acc2276c035910ce6f6
|
2024-03-09 21:35:24 +08:00 |
|
hiyouga
|
43b2ede0f8
|
fix #2756 , patch #2746
Former-commit-id: 627d1c91e675f1d9ebf47bad123cbbf29821da4d
|
2024-03-09 02:01:26 +08:00 |
|
hiyouga
|
e416cecf62
|
fix galore
Former-commit-id: 62a3ceeef8f60caef43ccc7f971a0c9184e21296
|
2024-03-08 00:44:51 +08:00 |
|
hiyouga
|
1e6fb6c8aa
|
support galore
Former-commit-id: b67a4a46a88d83bb2a3459b3317b66cda15e0171
|
2024-03-07 22:41:36 +08:00 |
|
hiyouga
|
e93fb3cc6c
|
tiny fix
Former-commit-id: c3145afa4164dd28888f17599a154f7dddbe9326
|
2024-03-06 17:25:08 +08:00 |
|
hiyouga
|
02b838b9b0
|
fix export model
Former-commit-id: 7ba2f7bf8da3c559e05d8dde20e93cd1d3d4e8ef
|
2024-03-05 11:05:41 +08:00 |
|
hiyouga
|
59a9a5994e
|
fix #2649
Former-commit-id: 1c850de660c671d92f0bc63f230d338b60b7c0bd
|
2024-03-01 13:02:41 +08:00 |
|
hoshi-hiyouga
|
88f3358320
|
Merge pull request #2525 from stephen-nju/main
update project_kwargs for ppo config
Former-commit-id: e7a6910141cc8d8dd966c1f54388d9ef764418d0
|
2024-02-25 15:54:00 +08:00 |
|
hiyouga
|
a274900188
|
fix #2532
Former-commit-id: 23a8e64f1c47cd473c627effbe271233c136369c
|
2024-02-21 21:55:14 +08:00 |
|
stephen
|
823f618cba
|
update project_kwargs for ppo config
Former-commit-id: 14f106962fc0a87802ae9ecffff00d52f7f5f046
|
2024-02-21 13:47:38 +08:00 |
|
hiyouga
|
596b6828cb
|
support llama pro #2338 , add rslora
Former-commit-id: 40d659b7f30dd5a004703c176ec1f22dc864e505
|
2024-02-15 02:27:36 +08:00 |
|
hiyouga
|
de0ebab464
|
fix #2189
Former-commit-id: b3d81b229d376671e1c12229aeb487b0d84f2548
|
2024-02-04 00:47:37 +08:00 |
|
hiyouga
|
1ace676170
|
fix #2320
Former-commit-id: e0b0c4415aaf80e75f6dd4f3777a0616b0e60f84
|
2024-01-24 16:19:18 +08:00 |
|
hoshi-hiyouga
|
bf075c075c
|
Update tuner.py
Former-commit-id: 691420661f7115f809e76484c1f29f74637e7cd0
|
2024-01-21 12:39:38 +08:00 |
|
yhyu13
|
cd1cb8b83c
|
Remove manully set use_cache; torch_dtype is not str, save model as bfloat16 used to fail;
Former-commit-id: 75557fb5df16fd6eda7586cf041a16822dcfee8e
|
2024-01-21 11:12:15 +08:00 |
|
hiyouga
|
66e0e651b9
|
format style
Former-commit-id: 53b683531b83cd1d19de97c6565f16c1eca6f5e1
|
2024-01-20 20:15:56 +08:00 |
|
hiyouga
|
1750218057
|
fix tests
Former-commit-id: 23f97bd437424ef43b2b84743d56acc5d1ca70d5
|
2024-01-20 19:58:04 +08:00 |
|
hiyouga
|
80637fc06d
|
support longlora for main branch
Former-commit-id: f869501ad4c368df26534c41f62c6d63c6be17dd
|
2024-01-20 19:25:22 +08:00 |
|
hiyouga
|
42a13fec46
|
Update tuner.py
Former-commit-id: db30107385f100f88c9370abea6692ce6030a0c9
|
2024-01-18 15:06:02 +08:00 |
|
hiyouga
|
a423274fd9
|
support function calling
Former-commit-id: 66533b3f65babf2429c92c0f8fafe4eff5e0ff63
|
2024-01-18 09:54:23 +08:00 |
|