1216 Commits

Author SHA1 Message Date
hiyouga
d8deb0f99e update readme and examples
Former-commit-id: a1f1fac33b2a727b38e8ba52d68a224814d4848b
2024-04-22 00:37:32 +08:00
hiyouga
92e24a73cb remove extras
Former-commit-id: ddbd29d77702f7b82051d930e3eac1b47f5c6d35
2024-04-22 00:35:41 +08:00
hiyouga
ecc62ff3be Update wechat.jpg
Former-commit-id: 1197513a16f004c6af871b517762a2ecf5e7252c
2024-04-22 00:22:58 +08:00
hiyouga
c0d8d530dd update readme
Former-commit-id: a83e7587a05e388916a3b9bb6fe42cffcdcf0886
2024-04-22 00:21:01 +08:00
hiyouga
31f96ce22a set dev version
Former-commit-id: 3e1bf8325c7ae3ad8e6e3ccc1c644d37030c83ff
2024-04-21 23:14:30 +08:00
hiyouga
869777b990 release v0.6.3
Former-commit-id: e0d6b72d569f2edeccdb05dc8e8ef2b0f1d525b0
v0.6.3
2024-04-21 23:13:23 +08:00
hiyouga
1ce9db5654 fix #3366
Former-commit-id: 4469a3237db8b40e07bec6de36c9ed5b2347f854
2024-04-21 21:34:25 +08:00
hiyouga
6a3ee1edd9 fix optimizers
Former-commit-id: fbbe0dba2f244557fff8ac0076a9f312d0aa01ab
2024-04-21 20:40:54 +08:00
hiyouga
79666c298d fix #3365
Former-commit-id: a1d31ffc8cb7a6a477704efe779d485d83b8b9fb
2024-04-21 19:20:18 +08:00
hiyouga
9e45f82be7 fix bug in galore optimizer
Former-commit-id: 5c62881c5a59cfcc5a76d365263c8ad8c817ce49
2024-04-21 18:53:22 +08:00
hiyouga
ec81d45d27 fix mod stuff
Former-commit-id: f58425ab45727f7859583d4b9fda776715e27ff6
2024-04-21 18:11:10 +08:00
hoshi-hiyouga
7c63a9b5fd Merge pull request #3338 from astramind-ai/main
Adding Mixture of Depth

Former-commit-id: d0273787be481fb2cbed993580f8239e63d74f7f
2024-04-21 18:05:52 +08:00
hoshi-hiyouga
e9b1aff447 fix #3348
Former-commit-id: 1fa287fd637aad0c5e8893046515a54bbff4c009
2024-04-20 10:34:09 +08:00
hiyouga
12290955d8 add dpo mix dataset
Former-commit-id: 6339edefff4eb23a4052fd273d1348f5ab59b47c
2024-04-20 01:31:38 +08:00
hiyouga
97c3f9e22c fix #3352
Former-commit-id: ba559a659a82d1e2ffb8ea7939fe4d6b4b37fd92
2024-04-19 22:40:01 +08:00
hiyouga
ac7ec7ed10 fix llama3 template
Former-commit-id: 14a605a2da16367ff64219b9f301c352b4812c4c
2024-04-19 15:46:51 +08:00
Marco
b6a87e0a3f fix small typo
Former-commit-id: 4fb7e046b3599aebeeba5b150acb6369f70ecb49
2024-04-18 20:33:29 +02:00
Marco
639297a5ef Added Mixture of Depths
Former-commit-id: 620add7b9f634de1a711f7b87b16050adf735e9b
2024-04-18 20:31:24 +02:00
hoshi-hiyouga
15d17a1e86 support llama3
Former-commit-id: 2aaaede2470a0bb8ce3aabb19c1e11a5f20e36e8
2024-04-19 01:13:50 +08:00
hiyouga
9aa62ffb57 fix #3324
Former-commit-id: 942362d0087345e468e0ae541dcca9b684d74d1a
2024-04-18 15:34:45 +08:00
hiyouga
e2e0bbde12 tiny fix
Former-commit-id: 3b43a3b7c5f894064afb1dd2b313962bd851156f
2024-04-18 00:22:17 +08:00
hiyouga
a49dd8b4f3 update readme
Former-commit-id: e2f1c6fc6a08750f1bb149ef13cb027cc4352910
2024-04-17 23:40:49 +08:00
hiyouga
8a369cc084 add mixtral 8x22B models
Former-commit-id: cab0598fd0a7e01ca2b40cb8f556992c0626cba1
2024-04-17 23:35:59 +08:00
hiyouga
3b491b4e1a add CodeQwen models
Former-commit-id: 5f86053d7555423cd35bd2b8610adf423ba3dbd0
2024-04-17 23:27:22 +08:00
hiyouga
0170ef83a6 fix #3316
Former-commit-id: c9a477322df82fecdb268ed385e3e0c376c0baeb
2024-04-17 22:54:34 +08:00
hiyouga
f9c859e97b fix #3317
Former-commit-id: 6d641af70361756131eaee456362909bd82a6c58
2024-04-17 22:17:19 +08:00
hiyouga
dac0c1a52b Update wechat.jpg
Former-commit-id: e3ca57602c85b6e558c6af2a8f3dc2b39b301403
2024-04-17 21:14:37 +08:00
hiyouga
bdd253228e lint
Former-commit-id: 278c5e10c475b37f082821d6ffbb7b260d72c4e6
2024-04-16 18:21:09 +08:00
hoshi-hiyouga
7daaae2005 Merge pull request #3291 from codemayq/main
support for previewing custom dataset in directory format

Former-commit-id: aa3206ec26236e8e176a3aa14229e0b0b31eb585
2024-04-16 18:12:09 +08:00
hiyouga
038d524fcd Update parser.py
Former-commit-id: c00f0771a5ab2a422e0300dcb6d88e8609a3b997
2024-04-16 18:09:31 +08:00
hiyouga
8beb7a9239 update readme and gradio version
Former-commit-id: 5d62a51c12061c59b509db8fe367817f4e48f737
2024-04-16 18:09:16 +08:00
hiyouga
0a94fab357 support badam for all stages
Former-commit-id: e3d8fc75eb2cfc54efd35bfd9ad6c4ac5acc458c
2024-04-16 17:44:48 +08:00
hoshi-hiyouga
496396b3bc Merge pull request #3287 from Ledzy/badam
[Feature] Add BAdam algorithm

Former-commit-id: 4d660c5ade384df4444fa0543a39edce6220903d
2024-04-16 17:32:16 +08:00
hoshi-hiyouga
6d954b2733 Update utils.py
Former-commit-id: c9828f4c6e6c150c884e02d0213dff0c09801e77
2024-04-16 17:30:12 +08:00
hoshi-hiyouga
5226c4fa97 Update trainer.py
Former-commit-id: 6700a1b9fa0cbd965ac45d3f2de1088727235c25
2024-04-16 17:29:52 +08:00
hoshi-hiyouga
b92f690190 Update utils.py
Former-commit-id: 38a56706e0f52297501d351d38b51bee73e881dc
2024-04-16 17:29:30 +08:00
hoshi-hiyouga
48fb0be1b9 Update patcher.py
Former-commit-id: a950f3b81de701f5f23ce3efa60ff0382bb40dfe
2024-04-16 17:29:19 +08:00
hoshi-hiyouga
ce56ff22af Update adapter.py
Former-commit-id: 750cdf2e74097c8775d03ddf55646cc14d4a686f
2024-04-16 17:28:12 +08:00
hoshi-hiyouga
0f6d691aa6 Update parser.py
Former-commit-id: 4660703674233949a8ba8c76bdb17dafc9d620d4
2024-04-16 17:27:25 +08:00
hoshi-hiyouga
d68212d016 Update parser.py
Former-commit-id: 5b59ff421204115ced405a2e3d56ac0ee8c5b788
2024-04-16 17:27:02 +08:00
hoshi-hiyouga
5f48dd545e Update finetuning_args.py
Former-commit-id: ec899cccf3b8710510e496a3cd8e4c302bb99a19
2024-04-16 17:26:30 +08:00
hoshi-hiyouga
507ab397f5 Update sft.sh
Former-commit-id: 57dcd91e17833a0eeb8d99af92ac73c132a77648
2024-04-16 17:25:40 +08:00
hoshi-hiyouga
4133811434 Update requirements.txt
Former-commit-id: 50ced257e0a1737dbd73d569f8f706ff452f70aa
2024-04-16 17:10:17 +08:00
hoshi-hiyouga
4aa17105e2 Update setup.py
Former-commit-id: f4b4a26c6435aa7fdccd11fb92795b8b26dd989a
2024-04-16 17:10:02 +08:00
Jonery
0853312593 remove badam from core requirements
Former-commit-id: 86900210d32e0d4cd9c6fb411a3692178411e970
2024-04-16 12:25:50 +08:00
Jonery
b3260c7456 resolve gradient checkpointing issue.
Former-commit-id: 7ecb61822b37f5d71060d696495830ff98edaa06
2024-04-16 12:05:27 +08:00
codingma
64f96cfd6d add check
Former-commit-id: 62294289dc6f594f37c5c6b73585c2403248cc55
2024-04-16 10:56:39 +08:00
codingma
43d1730aa5 support for previewing custom dataset in directory format
Former-commit-id: 75aa6392e86e965dc4a04b288ec60cff4e83f1af
2024-04-16 10:43:14 +08:00
hiyouga
627f281e5b add empty template
Former-commit-id: b3ac14ffc4c31cb54689c26b66963c20db64f21f
2024-04-16 03:10:02 +08:00
hiyouga
01b0913f08 update readme
Former-commit-id: cf52911fed8326dbe3f2781431c25959b175260e
2024-04-16 02:36:54 +08:00