ancv
|
20fdf177e8
|
move efficient_packing from data_args to model_args
Former-commit-id: 7b61659c707480bcf8c802c73e10d12ad5b9b965
|
2024-07-02 18:37:55 +07:00 |
|
hoshi-hiyouga
|
a715490c2a
|
Merge branch 'main' into main
Former-commit-id: 7be442f37d53a0c6324728fa1fa8e2c84d7f0fa5
|
2024-07-01 21:01:09 +08:00 |
|
hiyouga
|
67d2eb6b2a
|
fix #4402 #4617
Deprecate reserved_label_len arg
Former-commit-id: 4b6568984c0be4b31e7aa91b7c0d52b7f7b12b0b
|
2024-07-01 01:19:27 +08:00 |
|
hiyouga
|
cf2dc4c444
|
fix #4556
Former-commit-id: 81faa9a985c14e83e38f42aedd228edb676b0695
|
2024-06-26 19:43:16 +08:00 |
|
hiyouga
|
135bfbf7c1
|
tiny fix
Former-commit-id: bb57478366a70a0871af30ab31c890f471e27ff4
|
2024-06-25 01:15:19 +08:00 |
|
hoshi-hiyouga
|
5cfcb8262e
|
Merge pull request #4417 from mMrBun/main
Add tool_format parameter to rewrite templates for different function call formats.
Former-commit-id: 8d1460cad5bff5e4626fdd675046021e0a3d1947
|
2024-06-24 23:17:55 +08:00 |
|
hoshi-hiyouga
|
5d6cf55208
|
Update template.py
Former-commit-id: d53517bff6f8734221d7df9982f3bdd4d2eb2cab
|
2024-06-24 23:12:59 +08:00 |
|
hoshi-hiyouga
|
9a1ec19845
|
Update loader.py
Former-commit-id: afa59d61844595e6b615227e6bfdc0b16c8015dd
|
2024-06-24 23:06:18 +08:00 |
|
hiyouga
|
a79e93f335
|
fix #4410
Former-commit-id: f49adc4ab5eade21d7a9e029212f17688ee9b0cf
|
2024-06-24 22:34:31 +08:00 |
|
mMrBun
|
43a065bb07
|
Add tool_format to overwrite tool formatter template
Former-commit-id: af08971ca50443fd5597e5e4412a3aa17214502f
|
2024-06-22 02:13:23 +08:00 |
|
hiyouga
|
4513a2cc75
|
remove dup template
Former-commit-id: 5fec12203b24608af4d4993f44a657eb5a0348e5
|
2024-06-22 01:31:32 +08:00 |
|
hiyouga
|
c65f7e9bd5
|
fix jinja template
Former-commit-id: 0ebf2e2ee23918d28b0cbb20ba456732d6eedfbb
|
2024-06-19 20:03:50 +08:00 |
|
hiyouga
|
3e0fa4a8da
|
fix templates
Former-commit-id: 6f357d59b73309c5955683008632e7f320e7dcb1
|
2024-06-19 17:44:05 +08:00 |
|
hiyouga
|
235ed85b0f
|
fix bug
Former-commit-id: 412139eaa2fde98ba19e1257d21144382a59f0d6
|
2024-06-19 03:49:23 +08:00 |
|
hiyouga
|
1ca639a777
|
use prefix to replace force system
Former-commit-id: 731d9a964f1c3dbfb83825524d697831e691fb9d
|
2024-06-19 03:39:52 +08:00 |
|
hiyouga
|
e36a994fe6
|
fix tool formatter, allow parallel function #4362
Former-commit-id: b8f16c976db4ecec1cc8558851c8cbfb6a5b7e9c
|
2024-06-19 03:23:51 +08:00 |
|
hoshi-hiyouga
|
19ffcfea76
|
Merge pull request #4173 from mMrBun/main
Implemented the tool_formatter and tool_extractor for glm4 and Qwen2 tool_format
Former-commit-id: 36b02ceed40198ecd5d559ee4ebef9205442ded2
|
2024-06-19 03:18:55 +08:00 |
|
hiyouga
|
05f3a3c944
|
tiny fix
Former-commit-id: f7f440986b0ae3b38ea9f2da80789629d4f79ea1
|
2024-06-16 01:06:41 +08:00 |
|
ancv
|
f91fe10985
|
remove some unused params
Former-commit-id: fef8132c50505a5fb6a246bd024491bd31798a3c
|
2024-06-15 23:00:55 +07:00 |
|
hiyouga
|
bb88536166
|
add license
Former-commit-id: 69cfc98d7c81756a5ab6bf962240e393e449fef0
|
2024-06-15 17:54:33 +08:00 |
|
hiyouga
|
49b58fd6af
|
fix #4221
Former-commit-id: 05a3be4853b941909e7d193c31e8d62c8c5f879b
|
2024-06-13 02:48:21 +08:00 |
|
ancv
|
c7ab302c69
|
implement efficient packing without cross-contamination attention
Former-commit-id: a64a5305c0da5ef092d4cc26faf829bb44de65d1
|
2024-06-12 11:56:01 +07:00 |
|
hoshi-hiyouga
|
cc9717e2f2
|
Update pretrain.py
Former-commit-id: e2317b2a84149e39fddfd6366be3de23dfb71f82
|
2024-06-11 17:02:14 +08:00 |
|
d
|
77bf3d66c7
|
经过大量的增量预训练，进行对比试验，发现这个bug：llama3在预训练时使用的tokenizer.eos_toke是'<|end_of_text|>' ，这里在每条数据后面也得用这个，而不是'<|eot_id|>'，否则很容易导致严重的性能下降
Former-commit-id: ef470561f742b16eaa0f99c4cadecd7c84ce6bd2
|
2024-06-11 16:23:40 +08:00 |
|
mMrBun
|
bc04ca464a
|
Optimize the handling of QWEN2 in scenarios involving multiple tool calls.
Former-commit-id: 48f870edc96ada40360f7e6e67cbf58805295b33
|
2024-06-10 02:00:14 +08:00 |
|
mMrBun
|
44829df762
|
Removed unnecessary comments.
Former-commit-id: 2b81252aa693871098931cd7873ef83ef4922ba5
|
2024-06-09 18:25:22 +08:00 |
|
mMrBun
|
94ddfa66c0
|
Merge branch 'hiyouga:main' into main
Former-commit-id: c25734d874a36222e0a540a2c994bbda73008b27
|
2024-06-09 18:17:24 +08:00 |
|
mMrBun
|
8db8ed5a41
|
Implemented the tool_formatter and tool_extractor for glm4 tool_format
Former-commit-id: db7fa4490ea7f6966418d2879c895cbc1763b16d
|
2024-06-09 18:16:15 +08:00 |
|
hiyouga
|
c0c387e4db
|
release v0.8.0
Former-commit-id: 004db680b9e3996ec511ee818df6c0c02bf13603
|
2024-06-08 05:20:54 +08:00 |
|
hiyouga
|
8c4c2e580c
|
update data processors
Former-commit-id: 04b138cbcb8b9a72e4bbda6c65843bb459e525e7
|
2024-06-07 04:15:40 +08:00 |
|
hoshi-hiyouga
|
07f33e7641
|
Merge pull request #4009 from AlongWY/main
supervised packing with greedy knapsack algorithm
Former-commit-id: 5ded166b39a75a98ded5733678f5a1eab7d4cc71
|
2024-06-07 03:48:46 +08:00 |
|
hoshi-hiyouga
|
1998c641af
|
Update supervised.py
Former-commit-id: 04b6c2a754e602e0b698cfe6c255c2f2486d8865
|
2024-06-07 03:42:08 +08:00 |
|
hoshi-hiyouga
|
be1e5f9d62
|
Update supervised.py
Former-commit-id: 49993c4f4e1f871a22ff0196afe60026b668a4dc
|
2024-06-07 03:38:23 +08:00 |
|
hoshi-hiyouga
|
fdeec6db52
|
Update supervised.py
Former-commit-id: 67625b5278a839c12a3e4245f9e90af67d8b11b4
|
2024-06-07 03:38:04 +08:00 |
|
hiyouga
|
fcb134e144
|
rename files
Former-commit-id: e1a8431770fc36c0c9ee7fed4abbc3d7fdcc5efd
|
2024-06-07 00:09:06 +08:00 |
|
hiyouga
|
ca95e98ca0
|
fix ppo dataset bug #4012
Former-commit-id: 7fc51b2e93698ae5e012566af8481f4d861c873d
|
2024-06-06 19:03:20 +08:00 |
|
hiyouga
|
7528bc1bc0
|
support glm-4
Former-commit-id: a10f4718fbf3f3c89dc7eb31cb8e1a46ca6adda5
|
2024-06-05 15:16:38 +08:00 |
|
hiyouga
|
2ac2cde03e
|
tiny fix
Former-commit-id: f9d50501aac1f60a3b445ca3fee9aa60995461ee
|
2024-06-04 00:31:10 +08:00 |
|
hiyouga
|
f4f5d7e3ce
|
fix #3992
Former-commit-id: a48321fbf5196b88a11106cf74a74fbcea2ea50b
|
2024-06-04 00:17:36 +08:00 |
|
hiyouga
|
123a845209
|
fix data loader hint
Former-commit-id: 25b56126a11591b0155e2f72b673dd8f45a6c8c9
|
2024-06-03 18:28:27 +08:00 |
|
ylfeng
|
b1b7d735b3
|
remove empty line
Former-commit-id: 3164710971a6d6545629f5bf133f98de5ff0991a
|
2024-05-31 21:43:08 +08:00 |
|
ylfeng
|
230c69f7ce
|
fix eos
Former-commit-id: 6e236c952958cbfe50b5dcb7b8eff6aea8477922
|
2024-05-31 21:40:41 +08:00 |
|
ylfeng
|
bfc43558ef
|
supervised packing with greedy knapsack algorithm
Former-commit-id: 24d12396c9aabd49da0b08719068f24679111cc6
|
2024-05-31 15:33:54 +08:00 |
|
hoshi-hiyouga
|
4f7c850115
|
Merge pull request #3829 from seanzhang-zhichen/add_dataset_sample_num
Add dataset sample num
Former-commit-id: ab38cf74ce48ea4f1800e077ca287f2eb9336135
|
2024-05-30 00:25:45 +08:00 |
|
hoshi-hiyouga
|
391eca66cf
|
Update loader.py
Former-commit-id: 0aa59322906d91c5e385c9c02ebb5dd64ba060f3
|
2024-05-30 00:20:20 +08:00 |
|
hoshi-hiyouga
|
a67199246d
|
Update loader.py
Former-commit-id: aa7f335e3ad5a78e4ed5f99c120be28e9733ea2e
|
2024-05-30 00:17:21 +08:00 |
|
hoshi-hiyouga
|
5f67fdaac9
|
Update loader.py
Former-commit-id: 19d8fd62c18ee3ba0e431fc241f7d315cb716fef
|
2024-05-30 00:12:12 +08:00 |
|
hoshi-hiyouga
|
05e6fe4287
|
Update parser.py
Former-commit-id: 310cc11e8c83f16fc5bccc349c38fea347ea9a97
|
2024-05-30 00:05:20 +08:00 |
|
hiyouga
|
f90c4ca672
|
fix cohere system
Former-commit-id: 5d629b29e705c8ff8dd4521719d9c0e67a3fe0a2
|
2024-05-29 20:58:23 +08:00 |
|
hiyouga
|
a922e85a5c
|
fix #3965
Former-commit-id: 37d15ac55d0be0ff47d6a88f07e2d823117a4a36
|
2024-05-29 20:55:51 +08:00 |
|