hiyouga
|
ca106d1f1b
|
improve rlhf
Former-commit-id: e441780e3db256ca09a442ea9254e7ce16898a07
|
2024-07-02 22:23:08 +08:00 |
|
ancv
|
260f55ea47
|
move efficient_packing from data_args to model_args
Former-commit-id: 7b61659c707480bcf8c802c73e10d12ad5b9b965
|
2024-07-02 18:37:55 +07:00 |
|
hoshi-hiyouga
|
9174675ba9
|
Merge branch 'main' into main
Former-commit-id: 7be442f37d53a0c6324728fa1fa8e2c84d7f0fa5
|
2024-07-01 21:01:09 +08:00 |
|
hiyouga
|
a43f518389
|
fix #4402 #4617
Deprecate reserved_label_len arg
Former-commit-id: 4b6568984c0be4b31e7aa91b7c0d52b7f7b12b0b
|
2024-07-01 01:19:27 +08:00 |
|
hiyouga
|
0265a98942
|
fix #4556
Former-commit-id: 81faa9a985c14e83e38f42aedd228edb676b0695
|
2024-06-26 19:43:16 +08:00 |
|
hiyouga
|
f9d00ce051
|
tiny fix
Former-commit-id: bb57478366a70a0871af30ab31c890f471e27ff4
|
2024-06-25 01:15:19 +08:00 |
|
hoshi-hiyouga
|
1dcb792c82
|
Merge pull request #4417 from mMrBun/main
Add tool_format parameter to rewrite templates for different function call formats.
Former-commit-id: 8d1460cad5bff5e4626fdd675046021e0a3d1947
|
2024-06-24 23:17:55 +08:00 |
|
hoshi-hiyouga
|
d9529b749e
|
Update template.py
Former-commit-id: d53517bff6f8734221d7df9982f3bdd4d2eb2cab
|
2024-06-24 23:12:59 +08:00 |
|
hoshi-hiyouga
|
70410aedc1
|
Update loader.py
Former-commit-id: afa59d61844595e6b615227e6bfdc0b16c8015dd
|
2024-06-24 23:06:18 +08:00 |
|
hiyouga
|
a27d4bb4be
|
fix #4410
Former-commit-id: f49adc4ab5eade21d7a9e029212f17688ee9b0cf
|
2024-06-24 22:34:31 +08:00 |
|
mMrBun
|
15a104e26d
|
Add tool_format to overwrite tool formatter template
Former-commit-id: af08971ca50443fd5597e5e4412a3aa17214502f
|
2024-06-22 02:13:23 +08:00 |
|
hiyouga
|
c173a1e9ca
|
remove dup template
Former-commit-id: 5fec12203b24608af4d4993f44a657eb5a0348e5
|
2024-06-22 01:31:32 +08:00 |
|
hiyouga
|
11f88c4fdc
|
fix jinja template
Former-commit-id: 0ebf2e2ee23918d28b0cbb20ba456732d6eedfbb
|
2024-06-19 20:03:50 +08:00 |
|
hiyouga
|
da3b0aab6d
|
fix templates
Former-commit-id: 6f357d59b73309c5955683008632e7f320e7dcb1
|
2024-06-19 17:44:05 +08:00 |
|
hiyouga
|
15959d522c
|
fix bug
Former-commit-id: 412139eaa2fde98ba19e1257d21144382a59f0d6
|
2024-06-19 03:49:23 +08:00 |
|
hiyouga
|
5d7dfa75ca
|
use prefix to replace force system
Former-commit-id: 731d9a964f1c3dbfb83825524d697831e691fb9d
|
2024-06-19 03:39:52 +08:00 |
|
hiyouga
|
b11e57157f
|
fix tool formatter, allow parallel function #4362
Former-commit-id: b8f16c976db4ecec1cc8558851c8cbfb6a5b7e9c
|
2024-06-19 03:23:51 +08:00 |
|
hoshi-hiyouga
|
eb0bcfc860
|
Merge pull request #4173 from mMrBun/main
Implemented the tool_formatter and tool_extractor for glm4 and Qwen2 tool_format
Former-commit-id: 36b02ceed40198ecd5d559ee4ebef9205442ded2
|
2024-06-19 03:18:55 +08:00 |
|
hiyouga
|
640372cb66
|
tiny fix
Former-commit-id: f7f440986b0ae3b38ea9f2da80789629d4f79ea1
|
2024-06-16 01:06:41 +08:00 |
|
ancv
|
c5e1dfb3a0
|
remove some unused params
Former-commit-id: fef8132c50505a5fb6a246bd024491bd31798a3c
|
2024-06-15 23:00:55 +07:00 |
|
hiyouga
|
acfae2e677
|
add license
Former-commit-id: 69cfc98d7c81756a5ab6bf962240e393e449fef0
|
2024-06-15 17:54:33 +08:00 |
|
hiyouga
|
e8885443a9
|
fix #4221
Former-commit-id: 05a3be4853b941909e7d193c31e8d62c8c5f879b
|
2024-06-13 02:48:21 +08:00 |
|
ancv
|
4463a5227a
|
implement efficient packing without cross-contamination attention
Former-commit-id: a64a5305c0da5ef092d4cc26faf829bb44de65d1
|
2024-06-12 11:56:01 +07:00 |
|
hoshi-hiyouga
|
6625bf6b33
|
Update pretrain.py
Former-commit-id: e2317b2a84149e39fddfd6366be3de23dfb71f82
|
2024-06-11 17:02:14 +08:00 |
|
d
|
dfac202c7d
|
经过大量的增量预训练，进行对比试验，发现这个bug：llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ，这里在每条数据后面也得用这个，而不是'<|eot_id|>'，否则很容易导致严重的性能下降
Former-commit-id: ef470561f742b16eaa0f99c4cadecd7c84ce6bd2
|
2024-06-11 16:23:40 +08:00 |
|
mMrBun
|
680a340a40
|
Optimize the handling of QWEN2 in scenarios involving multiple tool calls.
Former-commit-id: 48f870edc96ada40360f7e6e67cbf58805295b33
|
2024-06-10 02:00:14 +08:00 |
|
mMrBun
|
f86bc621be
|
Removed unnecessary comments.
Former-commit-id: 2b81252aa693871098931cd7873ef83ef4922ba5
|
2024-06-09 18:25:22 +08:00 |
|
mMrBun
|
10e62fb974
|
Merge branch 'hiyouga:main' into main
Former-commit-id: c25734d874a36222e0a540a2c994bbda73008b27
|
2024-06-09 18:17:24 +08:00 |
|
mMrBun
|
8d0df82d0e
|
Implemented the tool_formatter and tool_extractor for glm4 tool_format
Former-commit-id: db7fa4490ea7f6966418d2879c895cbc1763b16d
|
2024-06-09 18:16:15 +08:00 |
|
hiyouga
|
cabe5ca7d0
|
release v0.8.0
Former-commit-id: 004db680b9e3996ec511ee818df6c0c02bf13603
|
2024-06-08 05:20:54 +08:00 |
|
hiyouga
|
72c91be943
|
update data processors
Former-commit-id: 04b138cbcb8b9a72e4bbda6c65843bb459e525e7
|
2024-06-07 04:15:40 +08:00 |
|
hoshi-hiyouga
|
a4cc4413e3
|
Merge pull request #4009 from AlongWY/main
supervised packing with greedy knapsack algorithm
Former-commit-id: 5ded166b39a75a98ded5733678f5a1eab7d4cc71
|
2024-06-07 03:48:46 +08:00 |
|
hoshi-hiyouga
|
4a4b8d0b39
|
Update supervised.py
Former-commit-id: 04b6c2a754e602e0b698cfe6c255c2f2486d8865
|
2024-06-07 03:42:08 +08:00 |
|
hoshi-hiyouga
|
9de42c4b3a
|
Update supervised.py
Former-commit-id: 49993c4f4e1f871a22ff0196afe60026b668a4dc
|
2024-06-07 03:38:23 +08:00 |
|
hoshi-hiyouga
|
b5421ac8eb
|
Update supervised.py
Former-commit-id: 67625b5278a839c12a3e4245f9e90af67d8b11b4
|
2024-06-07 03:38:04 +08:00 |
|
hiyouga
|
0b1f4a34f8
|
rename files
Former-commit-id: e1a8431770fc36c0c9ee7fed4abbc3d7fdcc5efd
|
2024-06-07 00:09:06 +08:00 |
|
hiyouga
|
56a6db6d84
|
fix ppo dataset bug #4012
Former-commit-id: 7fc51b2e93698ae5e012566af8481f4d861c873d
|
2024-06-06 19:03:20 +08:00 |
|
hiyouga
|
04a7065830
|
support glm-4
Former-commit-id: a10f4718fbf3f3c89dc7eb31cb8e1a46ca6adda5
|
2024-06-05 15:16:38 +08:00 |
|
hiyouga
|
1cc9508fb3
|
tiny fix
Former-commit-id: f9d50501aac1f60a3b445ca3fee9aa60995461ee
|
2024-06-04 00:31:10 +08:00 |
|
hiyouga
|
920b091581
|
fix #3992
Former-commit-id: a48321fbf5196b88a11106cf74a74fbcea2ea50b
|
2024-06-04 00:17:36 +08:00 |
|
hiyouga
|
2e843a4cf6
|
fix data loader hint
Former-commit-id: 25b56126a11591b0155e2f72b673dd8f45a6c8c9
|
2024-06-03 18:28:27 +08:00 |
|
ylfeng
|
7226a1b431
|
remove empty line
Former-commit-id: 3164710971a6d6545629f5bf133f98de5ff0991a
|
2024-05-31 21:43:08 +08:00 |
|
ylfeng
|
cc62c22559
|
fix eos
Former-commit-id: 6e236c952958cbfe50b5dcb7b8eff6aea8477922
|
2024-05-31 21:40:41 +08:00 |
|
ylfeng
|
3f849f2314
|
supervised packing with greedy knapsack algorithm
Former-commit-id: 24d12396c9aabd49da0b08719068f24679111cc6
|
2024-05-31 15:33:54 +08:00 |
|
hoshi-hiyouga
|
8ff3e53457
|
Merge pull request #3829 from seanzhang-zhichen/add_dataset_sample_num
Add dataset sample num
Former-commit-id: ab38cf74ce48ea4f1800e077ca287f2eb9336135
|
2024-05-30 00:25:45 +08:00 |
|
hoshi-hiyouga
|
ae773f9355
|
Update loader.py
Former-commit-id: 0aa59322906d91c5e385c9c02ebb5dd64ba060f3
|
2024-05-30 00:20:20 +08:00 |
|
hoshi-hiyouga
|
88f4c583d3
|
Update loader.py
Former-commit-id: aa7f335e3ad5a78e4ed5f99c120be28e9733ea2e
|
2024-05-30 00:17:21 +08:00 |
|
hoshi-hiyouga
|
d5ee485440
|
Update loader.py
Former-commit-id: 19d8fd62c18ee3ba0e431fc241f7d315cb716fef
|
2024-05-30 00:12:12 +08:00 |
|
hoshi-hiyouga
|
b221962ccf
|
Update parser.py
Former-commit-id: 310cc11e8c83f16fc5bccc349c38fea347ea9a97
|
2024-05-30 00:05:20 +08:00 |
|
hiyouga
|
50940fe912
|
fix cohere system
Former-commit-id: 5d629b29e705c8ff8dd4521719d9c0e67a3fe0a2
|
2024-05-29 20:58:23 +08:00 |
|