2089 Commits

Author SHA1 Message Date
hiyouga
d5ea05cfff update get template
Former-commit-id: dabad5570bf4a6b1044c963d8f27717030f373ef
2024-09-04 22:36:20 +08:00
hoshi-hiyouga
1dfd1aaf82 Merge pull request #5323 from naem1023/feat/add-dataset-map-batch-size-argument
Add batch size of map function in the preprocessed dataset

Former-commit-id: 8f441c2b3a5bb84dec2c037a541084c0201726c6
2024-09-04 22:09:36 +08:00
hoshi-hiyouga
8ac74c8ccb fix #5228
Former-commit-id: 44d6947e554cd61cff23c297248fff32a5f554da
2024-09-04 19:10:30 +08:00
hiyouga
53aeacae81 fix #5252
Former-commit-id: ebddce5d4a2ebb40ce71462f81701e274eb3c29f
2024-09-04 03:17:54 +08:00
hiyouga
af8c4b4e20 add vl_feedback dataset
Former-commit-id: 57497135bf0a956af9c6893177ee97504b9f34ac
2024-09-04 03:13:03 +08:00
hiyouga
a3d47818b7 fix #5344
Former-commit-id: d41d43a7c37cd10e34c9f399d1a346ffaee641c3
2024-09-04 03:06:06 +08:00
hoshi-hiyouga
a65b5061af Merge pull request #5346 from hiyouga/lazy_image
[exp] Lazyload for multimodal inputs

Former-commit-id: ce7ed6eb99f075e4a3337a86fd6a245bc98c479e
2024-09-04 03:00:53 +08:00
hiyouga
22deca0e9e lazy image load
Former-commit-id: 47ea97fb1ba77de2e8a561904aa8fdc27c3f5025
2024-09-04 02:27:08 +08:00
hiyouga
5ef58eb655 fix #5334
Former-commit-id: 59d2b31e968677263f005f57ae8a56fc758307a7
2024-09-03 19:09:42 +08:00
hiyouga
f837ae8cb5 fix #5338
Former-commit-id: 69d0acacc38dc90a97ce18447f865f525fff086a
2024-09-03 17:45:17 +08:00
hiyouga
982585e375 lint
Former-commit-id: 22959bcdd3b124a642e2acaadc050e36d0520f52
2024-09-03 00:46:25 +08:00
hiyouga
6e98872622 fix #5324
Former-commit-id: a61c8c4890962f3847b19eff31b170cd7f54316c
2024-09-02 23:56:21 +08:00
naem1023
46695e42cc feat: add batch size of map function in the preprocessed dataset
Former-commit-id: 209313eeeab8d1a7c320bd9aa90a5f4656082b7c
2024-09-02 13:52:47 +09:00
hoshi-hiyouga
5af92971bc fix trainer predict
Former-commit-id: 99fd9637bdc25f41fd1abc8a162f1069cb9060d4
2024-09-02 10:15:29 +08:00
hoshi-hiyouga
5c9972a2d5 remove .cpu()
Former-commit-id: a6c6750e8af5bc1ece1dfe6111d3e484fd19ee75
2024-09-02 10:10:53 +08:00
hiyouga
b2a5f49a24 fix mm inference
Former-commit-id: 60fc6b926ead923dbb487b595ed4aa4cbfd94805
2024-09-02 01:47:40 +08:00
hiyouga
f13e974930 tiny fix
Former-commit-id: 3a6f19f017539a2be31e348845a8ed8712275dcb
2024-09-02 01:33:22 +08:00
hiyouga
859823eb23 add image num check
Former-commit-id: ce8c5a264742161101da2094d0fc31c7fa7fb641
2024-09-02 01:31:36 +08:00
hiyouga
549adc888b add pokemon dataset
Former-commit-id: 194064fdae0226dd22522586c9d47c5866a71a8e
2024-09-02 01:02:25 +08:00
hiyouga
65425aeb63 update readme
Former-commit-id: a8f8a2ad8a8c026de528c9bcfb23b65d004bfc96
2024-09-01 23:32:39 +08:00
hiyouga
fd2a99ee5c update wechat
Former-commit-id: 753c5fb36c8b3f9a4488626b0a69396ffcfbbd37
2024-09-01 23:30:57 +08:00
hoshi-hiyouga
69fbecf1fb Merge pull request #5317 from ByronHsu/patch-1
Add liger kernel link

Former-commit-id: 6f9e455af481790aaf16af620ee36f76293e93d0
2024-09-01 23:30:12 +08:00
hiyouga
bfdcc6bacf add rlhf-v dataset
Former-commit-id: 8e49940746c1a6ff910f07dbefbec14af9d0f3c6
2024-09-01 22:57:41 +08:00
hiyouga
236f97b35c tiny fix
Former-commit-id: 55027282cdaa59a470ac89bfb3860504ba9075ff
2024-09-01 21:15:44 +08:00
hiyouga
413a206652 fix bug
Former-commit-id: 64cb947c60398dfdfc2877f898147b0240089ea3
2024-09-01 21:07:49 +08:00
hiyouga
cb776752f6 fix mixed mm inputs and rlhf-v
Former-commit-id: 9967ccb3aef3ca557ad6eafb78c6c99866857008
2024-09-01 20:52:47 +08:00
Byron Hsu
7c0d1a5ff1 Add liger kernel link
Former-commit-id: b8a9cb554efc4c2dedacb48833c5152d2cd2fec5
2024-08-30 17:16:16 -07:00
hiyouga
34dc36462c fix ci (temp)
Former-commit-id: b5063b41449d1f81d2031980d6480a285e359ebf
2024-08-31 02:03:56 +08:00
hiyouga
09a2ecebc4 add test mm plugin
Former-commit-id: a2a8c0b92c49fb1ee65de271aec651e011dcabc4
2024-08-31 01:53:38 +08:00
hiyouga
f31e7e0dfc remove visual_inputs, fix qlora
Former-commit-id: a025c3df61db154bef13033518903bbf846f4fc8
2024-08-31 00:24:51 +08:00
hiyouga
51a0016873 optimize predict vram
Former-commit-id: a244f143f48a01910ce1cd56c0855ef11d62a72a
2024-08-30 23:08:45 +08:00
hiyouga
c883542583 add examples
Former-commit-id: e08045a9468986edf1e84001e6043db0ee2e5265
2024-08-30 21:43:19 +08:00
hiyouga
92c398166d tiny fix
Former-commit-id: bee1bd43b946501690d70e4980205f9d82404296
2024-08-30 03:21:50 +08:00
hiyouga
913ee05e74 fix #5307
Former-commit-id: 8b588c7224e631a96c09f3460effc5dbfca6ec14
2024-08-30 02:45:40 +08:00
hiyouga
a83756b5e9 refactor mm training
Former-commit-id: 3382317e32f88ed377d3e7759bdeaf0f2559d22a
2024-08-30 02:14:31 +08:00
hoshi-hiyouga
98b0c7530c Merge pull request #5290 from simonJJJ/qwen2_vl
support qwen2-vl

Former-commit-id: 727e1848401d306274fb60ba78f66fed577b7b55
2024-08-30 02:10:36 +08:00
hoshi-hiyouga
5730243179 fix bug
Former-commit-id: a8f22d8895c07ce37e8cb34b7014f3e9bcdbf6a3
2024-08-30 02:05:26 +08:00
hiyouga
0e4ee9d9a3 update liger kernel
Former-commit-id: a7dd7d325e68c92c7470c1e9ef83a7c8abcbc616
2024-08-29 20:46:08 +08:00
hiyouga
f153ee13be fix #5292
Former-commit-id: aa1afdc75614868172bd2f9c052647b8f226d3f2
2024-08-29 20:37:47 +08:00
hiyouga
21d3976eea fix #5295
Former-commit-id: ad72f3e06593f124d661d61774def336511716e0
2024-08-29 20:30:18 +08:00
hiyouga
1494fa1f18 fix #5305
Former-commit-id: 364b757e306f7a154359a2bc8245a839f39c4fab
2024-08-29 20:16:01 +08:00
simonJJJ
d9fd8f26cc update
Former-commit-id: 734e019cc1c120c23305d6f6bc1fb5b8e0d21b30
2024-08-28 20:22:46 +08:00
simonJJJ
8a09b1e732 initial-commit
Former-commit-id: aeb85f200bd824748008dae6047c2607dfcdf174
2024-08-28 16:51:35 +08:00
hiyouga
efd60f0306 update wechat
Former-commit-id: 0f5a0f64f798eccd1fe76a74cf537002135e7e85
2024-08-27 12:55:23 +08:00
hiyouga
c2df70e925 add extra requires
Former-commit-id: d14edd350ddf268cfeea0f4e9e3c43f38516b848
2024-08-27 12:52:12 +08:00
hiyouga
7b5834b2dd tiny fix
Former-commit-id: f6ae4e75ddaeb4ac4a527f0141ac5b1afefde10e
2024-08-27 12:49:32 +08:00
hoshi-hiyouga
7cbfd64289 Merge pull request #5237 from marko1616/patch-1
Fix mllm api

Former-commit-id: dbe886ae5cf613f537d70c352cb758a524a14279
2024-08-27 12:24:43 +08:00
marko1616
388c3e4e47 ruff pass.
Former-commit-id: df8d5b6985bb209e841347ee51fc6787d9bcd069
2024-08-27 11:30:16 +08:00
marko1616
53a2b16c38 Update chat.py
Former-commit-id: 1545684c3f176a95554afa64f313f00b53a32a77
2024-08-27 11:27:56 +08:00
hiyouga
c765292093 support liger kernel
Former-commit-id: 72bc8f01111ad69b92a647b54b4af988515d9c34
2024-08-27 11:20:14 +08:00