173 Commits

Author SHA1 Message Date
hiyouga
380c7741f5 fix test
Former-commit-id: 173921419dc24506527cf506bfbfc01541fd2b66
2024-07-14 15:44:30 +08:00
hiyouga
dfd2d912cd fix #4699
slow tokenizer for yi models


Former-commit-id: 88a20ba7972c533d650967a118d612471fe2b2e8
2024-07-14 15:34:22 +08:00
hiyouga
3baf1d1f7e fix #4792
Former-commit-id: 6b48308ef9be34d072f3e6bb2444e186a38c2779
2024-07-13 22:07:58 +08:00
codingma
74f0d02eb8 1. add custom eval dataset support
2. merge load dataset and split dataset function


Former-commit-id: 76f3bbcfc0e11aa41f8f5cbebc60b77b987f7901
2024-07-05 15:52:10 +08:00
hiyouga
9aa3403687 fix #4683
Former-commit-id: e43809bced009323b3bac9accdd3baa3a2836fdb
2024-07-05 00:58:05 +08:00
hiyouga
aa14a625e4 update tests
Former-commit-id: 636bb9c1e65e72c3a27049dacb3200234d1c2782
2024-07-04 04:00:12 +08:00
hiyouga
d7657d772d tiny fix
Former-commit-id: 0c699de39de06eac96af67e8dd4fc4c53335b17e
2024-07-04 03:47:05 +08:00
hiyouga
bfdaadcc40 update packing
Former-commit-id: cce7083024bed4c7429ddc8288d1c9190fde29f5
2024-07-04 01:10:55 +08:00
hiyouga
28c8e083f4 test
Former-commit-id: a4a1ddbcb987422cd04125ff3f36f8c739061b5c
2024-07-03 23:05:39 +08:00
hiyouga
ca7b65439d fix #4402 #4617
Deprecate reserved_label_len arg


Former-commit-id: 1771251ce3f6887b301dac10f3de7a253c5e5884
2024-07-01 01:19:27 +08:00
hiyouga
f3f25ae3b7 lint
Former-commit-id: 555ca8d780a1fbaf42e73450f5eb33048329d921
2024-06-25 02:55:50 +08:00
hiyouga
80effa2993 fix test case
Former-commit-id: c244af0dc3478532de02271667e7af4ad8f54228
2024-06-25 02:51:49 +08:00
hoshi-hiyouga
709bbc1d92 Merge pull request #4417 from mMrBun/main
Add tool_format parameter to rewrite templates for different function call formats.

Former-commit-id: def6d280db3a9fe468b05503bcd9929c83c6c19b
2024-06-24 23:17:55 +08:00
hoshi-hiyouga
18863245df Update test_formatter.py
Former-commit-id: 672152d2ce6b49d7668c70100d877a1c34c08eae
2024-06-24 23:14:36 +08:00
hoshi-hiyouga
a9f10a9abd Update test_attention.py
Former-commit-id: a9b3d91952dd5a51ff97fbb40a2dd88885d380b8
2024-06-24 21:35:34 +08:00
hiyouga
030b4811c7 update patcher
Former-commit-id: 3b040e8e0f78dbb6bc1409a1b2b788e1affc7458
2024-06-19 21:27:00 +08:00
hiyouga
3d72b1a856 fix jinja template
Former-commit-id: 2b596fb55ff689d2e488d9a9bbab98f70f356c3c
2024-06-19 20:03:50 +08:00
hiyouga
bccc852f76 fix tool formatter, allow parallel function #4362
Former-commit-id: cd75b1fe9d91fb52a9ae6de7435302ff06b4d933
2024-06-19 03:23:51 +08:00
hiyouga
004f289074 tiny fix
Former-commit-id: 2bf2863a58c93206f271de17d7dfcbcd6375cd73
2024-06-17 17:47:25 +08:00
hiyouga
ce4a27a5f7 fix tol
Former-commit-id: 46093b5786611d99adf1fd3d42926a728fc629f8
2024-06-16 01:38:44 +08:00
hiyouga
2d2c78d66c increase tol
Former-commit-id: de43bee0b004c7e90811100474b3113590d0f130
2024-06-16 01:21:06 +08:00
hiyouga
f25b8626bf support pissa
Former-commit-id: 8c1046d78ac6c8f9429b73617e35e1eccb35138f
2024-06-16 01:08:12 +08:00
hiyouga
c0c6b8075a tiny fix
Former-commit-id: 38b6b0f52edeb8ba45aa03b415b3c0c1b0e0c1e4
2024-06-16 01:06:41 +08:00
hiyouga
96b82ccd4d use fixture
Former-commit-id: 80a9e6bf94cf14fa63e6b6cdf7e1ce13722c8b5e
2024-06-15 20:06:17 +08:00
hiyouga
8053929b20 add tests
Former-commit-id: 1b834f50be64ae9b5123da0e6f528cfbd5167477
2024-06-15 19:51:20 +08:00
hiyouga
2946153cea add license
Former-commit-id: d87108daa68bd40174b262be1ca65fe6e1b7ab56
2024-06-15 17:54:33 +08:00
hiyouga
a3f4925c2c add test cases
Former-commit-id: b27269bd2b52fb9d43cde8a8b7f293099b0127a2
2024-06-15 04:05:54 +08:00
hiyouga
144544cd37 update evaluator
Former-commit-id: 0012762b04d4082a3a7d98889cc3024adc92fc44
2024-06-10 23:56:00 +08:00
hiyouga
3b244a69dc fix #2666
Former-commit-id: c907d816670975daa900898660d3503708b7fc37
2024-06-10 21:24:15 +08:00
hiyouga
de3400a521 set dev version
Former-commit-id: 3ac11e77cccf686e0da499bd152997133b49a265
2024-06-08 06:46:09 +08:00
hiyouga
ce40d12692 release v0.8.0
Former-commit-id: 5aa4ce47567146cd97c61623018153b41d7c1278
2024-06-08 05:20:54 +08:00
hiyouga
d3eb985bb6 fix ci
Former-commit-id: 7f20e4722ae6ac907b36a3219dcd09d2ff5d071a
2024-06-08 01:57:36 +08:00
hiyouga
4f3e680b57 init unittest
Former-commit-id: 1c7f0ab51906b20190f8d4db932623cff76efc01
2024-06-08 01:35:58 +08:00
hiyouga
34f1de0574 fix #3077
Former-commit-id: aee634cd20e6dfdfbe2fbb47ae57f62b2da2bf9a
2024-04-01 21:35:18 +08:00
hiyouga
34533b2f35 support vllm
Former-commit-id: d07ad5cc1cdbc13879afd84f653afdfee03a6933
2024-03-07 20:26:31 +08:00
hiyouga
e887aface7 fix version checking
Former-commit-id: 3016e6565708637c1d760f2cd5a67cbd8a5a6c26
2024-03-06 14:51:51 +08:00
hiyouga
6032d3fe7d add test scripts
Former-commit-id: 26912cd816db6281a796cdca9fef73dedf0015ad
2024-02-19 02:09:13 +08:00
hiyouga
c673595c66 fix safetensors
Former-commit-id: e4e86a73f107710cddeef7efb669fb9987c65720
2024-02-18 18:12:16 +08:00
hiyouga
96265ec154 support llama pro #2338 , add rslora
Former-commit-id: 7924ffc55d98e33bfbfbca303e46c8f476435673
2024-02-15 02:27:36 +08:00
hoshi-hiyouga
ac89ec9ced Update test_toolcall.py
Former-commit-id: b5007afb193647c905e44037dc77ab8bfa5da832
2024-02-05 22:51:03 +08:00
hoshi-hiyouga
e3e7b4dc33 Update test_toolcall.py
Former-commit-id: d4994235f01fccf1498c7cc973cc535247611aa6
2024-02-05 22:50:43 +08:00
tao.jun
cb1a7e5b6b Update test_toolcall.py
Add openai version notes

Former-commit-id: 4081c88723391bfc0e229b2d1b69ac7d890eb1f9
2024-02-05 20:49:23 +08:00
hiyouga
e606832a3c update tool test
Former-commit-id: 37ec5deaecb98d0cc06dc46dabffa86b09b7751b
2024-01-21 19:41:46 +08:00
hiyouga
daea73cf2b fix api
Former-commit-id: f51d48ea081001ff2338a6a78231bfb13cfb0465
2024-01-21 19:15:27 +08:00
hiyouga
b27e91222c format style
Former-commit-id: 638234ceee1b19716e45b6e5f4ea54d9122da4df
2024-01-20 20:15:56 +08:00
hiyouga
2f7684a8ee fix tests
Former-commit-id: f6d6e00337ebef8740d180836dcb18d0e6a3c59a
2024-01-20 19:58:04 +08:00
hiyouga
4e3bfb799d support function calling
Former-commit-id: d9f1cae35150cce594a7abd96dd2beb811fa33f2
2024-01-18 09:54:23 +08:00
hiyouga
71306bbfb1 Update llamafy_internlm2.py
Former-commit-id: 28135d787d62ddcec67fbec0aa78bbd2bdccad1e
2024-01-18 01:12:31 +08:00
hiyouga
6fa117a0ff Update llamafy_internlm2.py
Former-commit-id: 484becae1bc38457d6cd8755bbfcb9f0e2e13f5d
2024-01-18 01:00:16 +08:00
hiyouga
925b61ff99 Update llamafy_internlm2.py
Former-commit-id: c84a387c2c0e6ac5a33fd233bcfba4363eccb4cb
2024-01-18 00:49:31 +08:00