{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":627561173,"defaultBranch":"main","name":"web-llm","ownerLogin":"mlc-ai","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2023-04-13T18:11:59.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/106173866?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1717363464.0","currentOid":""},"activityList":{"items":[{"before":"439e62b3cb21c16dc6a1af0d6970369ef3672db8","after":"907633f9aa8a9c2eb3fc2e832ab9266c6c0de493","ref":"refs/heads/gh-pages","pushedAt":"2024-06-08T01:20:19.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Build at Sat Jun  8 01:20:19 UTC 2024","shortMessageHtmlLink":"Build at Sat Jun 8 01:20:19 UTC 2024"}},{"before":"93681432753c75c8df15d8f923d476c895adee4f","after":"30209061dc3e6233dfffab46df49ec9c7e108618","ref":"refs/heads/main","pushedAt":"2024-06-08T01:19:30.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"CharlieFRuan","name":"Charlie Ruan","path":"/CharlieFRuan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/53290280?s=80&v=4"},"commit":{"message":"Update README.md","shortMessageHtmlLink":"Update README.md"}},{"before":"bc60b812dce0f742b5e24a2b4dec3f8de053fec6","after":"439e62b3cb21c16dc6a1af0d6970369ef3672db8","ref":"refs/heads/gh-pages","pushedAt":"2024-06-08T01:17:25.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Build at Sat Jun  8 01:17:24 UTC 2024","shortMessageHtmlLink":"Build at Sat Jun 8 01:17:24 UTC 2024"}},{"before":"33aaa570b46e8c15e0154e8a6eb35dd1f3a3413c","after":"93681432753c75c8df15d8f923d476c895adee4f","ref":"refs/heads/main","pushedAt":"2024-06-08T01:16:32.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"CharlieFRuan","name":"Charlie Ruan","path":"/CharlieFRuan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/53290280?s=80&v=4"},"commit":{"message":"[Version] Bump version to 0.2.43 (#464)\n\n### Changes\r\n- New models:\r\n  - Qwen2 0.5B, 1.5B, 7B via https://github.com/mlc-ai/web-llm/pull/463\r\n- Add deprecation warning for `runtimeStatsText()` and suggest using\r\n`.usage`\r\n  - https://github.com/mlc-ai/web-llm/pull/460\r\n- Do not use sliding window for mistral models by default, but still can\r\nbe overridden by user\r\n  - https://github.com/mlc-ai/web-llm/pull/461\r\n\r\n### Model WASM Version\r\nUpdated to https://github.com/mlc-ai/binary-mlc-llm-libs/pull/126 for\r\ntrivial rename changes in MLC's WASM runtime.\r\n\r\n### TVMjs\r\nCompiled at\r\nhttps://github.com/apache/tvm/commit/5d077c5a0900a6b98934b9e9d813da14ba0fc24b\r\nwith no changes","shortMessageHtmlLink":"[Version] Bump version to 0.2.43 (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2341337724\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlc-ai/web-llm/issues/464\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlc-ai/web-llm/pull/464/hovercard\" href=\"https://github.com/mlc-ai/web-llm/pull/464\">#464</a>)"}},{"before":"3b5651409ef985374ca6723f9910edf866fd61df","after":"bc60b812dce0f742b5e24a2b4dec3f8de053fec6","ref":"refs/heads/gh-pages","pushedAt":"2024-06-08T01:07:52.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Build at Sat Jun  8 01:07:51 UTC 2024","shortMessageHtmlLink":"Build at Sat Jun 8 01:07:51 UTC 2024"}},{"before":"2a30d4400140ff4ba4715e0c559f8011fb043260","after":"33aaa570b46e8c15e0154e8a6eb35dd1f3a3413c","ref":"refs/heads/main","pushedAt":"2024-06-08T01:06:56.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"CharlieFRuan","name":"Charlie Ruan","path":"/CharlieFRuan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/53290280?s=80&v=4"},"commit":{"message":"[Model] Support Qwen2 0.5B, 1.5B, 7B, bump wasm version to 0.2.43 (#463)\n\nThis PR adds QWen2 0.5B, 1.5B, and 7B, hence deprecating Qwen1.5-1.8B\r\n\r\nThis PR also updates the model wasm version to 0.2.43 due to recent\r\nchanges in MLC's renaming of TVM global function. As a result, we update\r\nhow `grammar.ts` gets the global function.\r\n\r\nAdditionally, we remove \r\n```typescript\r\n    if (system_prompt) {\r\n      system_prompt += this.config.seps[0];\r\n    }\r\n```\r\nto reflect the up-to-date conv template in `mlc-chat-config.json`.","shortMessageHtmlLink":"[Model] Support Qwen2 0.5B, 1.5B, 7B, bump wasm version to 0.2.43 (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2341333595\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlc-ai/web-llm/issues/463\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlc-ai/web-llm/pull/463/hovercard\" href=\"https://github.com/mlc-ai/web-llm/pull/463\">#463</a>)"}},{"before":"14ba23749d6f40184442496680e18716893882da","after":"3b5651409ef985374ca6723f9910edf866fd61df","ref":"refs/heads/gh-pages","pushedAt":"2024-06-07T20:38:35.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Build at Fri Jun  7 20:38:34 UTC 2024","shortMessageHtmlLink":"Build at Fri Jun 7 20:38:34 UTC 2024"}},{"before":"5b970f91d64dea2ef36847f3a04fe597aa32d56e","after":"2a30d4400140ff4ba4715e0c559f8011fb043260","ref":"refs/heads/main","pushedAt":"2024-06-07T20:37:47.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Neet-Nestor","name":"Nestor Qin","path":"/Neet-Nestor","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/23090573?s=80&v=4"},"commit":{"message":"[Docs] Update README","shortMessageHtmlLink":"[Docs] Update README"}},{"before":"5c6cf86e1096fa8a722bfe118b32d4f1035f718c","after":"14ba23749d6f40184442496680e18716893882da","ref":"refs/heads/gh-pages","pushedAt":"2024-06-07T20:30:19.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Build at Fri Jun  7 20:30:19 UTC 2024","shortMessageHtmlLink":"Build at Fri Jun 7 20:30:19 UTC 2024"}},{"before":"09098181a0dda991c8d64318b8ce125e3cd27709","after":"5b970f91d64dea2ef36847f3a04fe597aa32d56e","ref":"refs/heads/main","pushedAt":"2024-06-07T20:29:28.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"Neet-Nestor","name":"Nestor Qin","path":"/Neet-Nestor","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/23090573?s=80&v=4"},"commit":{"message":"[Docs] Update README","shortMessageHtmlLink":"[Docs] Update README"}},{"before":"fc8e52b027af4c1ad388e14b1b3ed3fea99d03eb","after":"5c6cf86e1096fa8a722bfe118b32d4f1035f718c","ref":"refs/heads/gh-pages","pushedAt":"2024-06-06T22:29:18.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Build at Thu Jun  6 22:29:18 UTC 2024","shortMessageHtmlLink":"Build at Thu Jun 6 22:29:18 UTC 2024"}},{"before":"a6d63c2349aa4b2daddad080d216f0db24b3b356","after":"09098181a0dda991c8d64318b8ce125e3cd27709","ref":"refs/heads/main","pushedAt":"2024-06-06T22:28:31.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"CharlieFRuan","name":"Charlie Ruan","path":"/CharlieFRuan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/53290280?s=80&v=4"},"commit":{"message":"Update README for new doc link","shortMessageHtmlLink":"Update README for new doc link"}},{"before":"97d07890cd3d7e76e331ad5646816595862537c7","after":"fc8e52b027af4c1ad388e14b1b3ed3fea99d03eb","ref":"refs/heads/gh-pages","pushedAt":"2024-06-06T18:33:01.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Build at Thu Jun  6 18:33:01 UTC 2024","shortMessageHtmlLink":"Build at Thu Jun 6 18:33:01 UTC 2024"}},{"before":"e532f34c2af80648b7f8e6df7385a64b86844ead","after":"a6d63c2349aa4b2daddad080d216f0db24b3b356","ref":"refs/heads/main","pushedAt":"2024-06-06T18:32:12.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"CharlieFRuan","name":"Charlie Ruan","path":"/CharlieFRuan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/53290280?s=80&v=4"},"commit":{"message":"[Prebuilts] Reorder prebuilt model list (#461)\n\nThis PR reorders the prebuilt model list to move the popular models to\r\nthe front.\r\n\r\nBesides, we also make Mistral variants to not use sliding window by\r\ndefault, since starting from `v0.2`, the default is to not use sliding\r\nwindow. User can still use the sliding window with attention sink by\r\neither modifying `ModelRecord.overrides` or passing in `chatOptions`.","shortMessageHtmlLink":"[Prebuilts] Reorder prebuilt model list (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2338908433\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlc-ai/web-llm/issues/461\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlc-ai/web-llm/pull/461/hovercard\" href=\"https://github.com/mlc-ai/web-llm/pull/461\">#461</a>)"}},{"before":"421706d7182340578e974b12ee06e59001704c75","after":"97d07890cd3d7e76e331ad5646816595862537c7","ref":"refs/heads/gh-pages","pushedAt":"2024-06-06T18:21:25.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Build at Thu Jun  6 18:21:25 UTC 2024","shortMessageHtmlLink":"Build at Thu Jun 6 18:21:25 UTC 2024"}},{"before":"051b2f1cd59b50cedef85979e563d25549665a41","after":"e532f34c2af80648b7f8e6df7385a64b86844ead","ref":"refs/heads/main","pushedAt":"2024-06-06T18:20:35.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"tqchen","name":"Tianqi Chen","path":"/tqchen","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/2577440?s=80&v=4"},"commit":{"message":"[Streaming] Replace runtimeStatsText() with usage in examples (#460)\n\nRecently we added `usage` to the last chunk in streaming if the user\r\nspecifies `streamOptions: { include_usage: True}` in the request, to be\r\ncompatible with the latest OpenAI API; for more see\r\nhttps://github.com/mlc-ai/web-llm/pull/456.\r\n\r\nThis PR updates our streaming examples to use `chunk.usage` instead of\r\n`runtimeStatsText()`. It is expected that we will deprecate\r\n`runtimeStatsText()` in the future. Currently, only low-level API\r\n`forwardTokenAndSample()` still needs it as they do not use OpenAI API\r\n(e.g. `examples/logit-processor`).\r\n\r\nSuch examples include:\r\n- simple-chat-ts\r\n- simple-chat-js\r\n- next-simple-chat\r\n- streaming (already updated)","shortMessageHtmlLink":"[Streaming] Replace runtimeStatsText() with usage in examples (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2338817271\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlc-ai/web-llm/issues/460\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlc-ai/web-llm/pull/460/hovercard\" href=\"https://github.com/mlc-ai/web-llm/pull/460\">#460</a>)"}},{"before":"6005fafeac02f5bc544b67070453807c9de0ce38","after":"421706d7182340578e974b12ee06e59001704c75","ref":"refs/heads/gh-pages","pushedAt":"2024-06-05T12:48:45.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Build at Wed Jun  5 12:48:44 UTC 2024","shortMessageHtmlLink":"Build at Wed Jun 5 12:48:44 UTC 2024"}},{"before":"6a7fe3e6181ce540acb618a8c5a39dacb90fd329","after":"051b2f1cd59b50cedef85979e563d25549665a41","ref":"refs/heads/main","pushedAt":"2024-06-05T12:47:54.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"CharlieFRuan","name":"Charlie Ruan","path":"/CharlieFRuan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/53290280?s=80&v=4"},"commit":{"message":"[Version] Bump version to 0.2.42 (#459)\n\n### Changes\r\n- New models:\r\n  - Mistral-7B-Instruct-v0.3-q4f16_1-MLC (we had v0.2 before)\r\n  - Mistral-7B-Instruct-v0.3-q4f32_1-MLC\r\n  - TinyLlama-1.1B-Chat-v1.0-q4f16_1-MLC (we had v0.4 before)\r\n  - TinyLlama-1.1B-Chat-v1.0-q4f32_1-MLC (we had v0.4 before)\r\n- https://github.com/mlc-ai/web-llm/pull/457\r\n- **[Breaking] renamed `max_gen_len` to `max_tokens` in\r\nChatCompletionRequest**\r\n- Remove usage of `mean_gen_len` and `shift_fill_factor`, throw error\r\nwhen request's prompt exceed contextWindowSize\r\n- Terminate generation with `\"length\"` stop reason when decoding exceed\r\ncontextWindowSize\r\n- https://github.com/mlc-ai/web-llm/pull/456\r\n- Add `include_usage` in streaming, add prefill/decode speed to `usage`\r\nto replace `runtimeStatsText()`\r\n- https://github.com/mlc-ai/web-llm/pull/455\r\n- Allow overriding KVCache settings via `ModelRecord.overrides` or\r\n`chatOptions`\r\n- Can use sliding window on any model now by specifying\r\n`sliding_window_size` and `attention_sink_size`\r\n\r\n### TVMjs\r\nCompiled at\r\nhttps://github.com/apache/tvm/commit/140062705ea6fad1e1f34eedc4a9b2a62fda6751\r\nwith no changes\r\n\r\n### WASM version\r\nNo change -- 0.2.39","shortMessageHtmlLink":"[Version] Bump version to 0.2.42 (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2335815044\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlc-ai/web-llm/issues/459\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlc-ai/web-llm/pull/459/hovercard\" href=\"https://github.com/mlc-ai/web-llm/pull/459\">#459</a>)"}},{"before":"c8e9985efad377cae046ee95bc59313a4367ee2b","after":"6005fafeac02f5bc544b67070453807c9de0ce38","ref":"refs/heads/gh-pages","pushedAt":"2024-06-05T12:36:06.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Build at Wed Jun  5 12:36:06 UTC 2024","shortMessageHtmlLink":"Build at Wed Jun 5 12:36:06 UTC 2024"}},{"before":"ee2745df81c453c4561d86c9521d0a141c11b269","after":"6a7fe3e6181ce540acb618a8c5a39dacb90fd329","ref":"refs/heads/main","pushedAt":"2024-06-05T12:35:15.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"CharlieFRuan","name":"Charlie Ruan","path":"/CharlieFRuan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/53290280?s=80&v=4"},"commit":{"message":"[OpenAI] Handle exceeding context window size, rename max_gen_len to max_tokens (#457)\n\n- **[Breaking]** Rename `max_gen_len` to `max_tokens` in\r\n`ChatCompletionRequest`\r\n- Remove reading `max_gen_len` from `mlc-chat-config.json`, hence remove\r\nit from `ChatOptions`\r\n- Remove reading `mean_gen_len` and `shift_fill_factor` from\r\n`mlc-chat-config.json`, hence:\r\n  - Remove these two fields from `ChatOption`\r\n- Throw error when the input prompt tokens exceed `context_window_size`,\r\nprompting user to truncate input, or increase `context_window_size`, or\r\nuse sliding window\r\n- Finish reason with `length` when decoding exceed `context_window_size`\r\n(when `prompt_size + max_tokens > context_window_size`)\r\n- Hence this error is never shown to user:\r\nhttps://github.com/mlc-ai/web-llm/issues/385\r\n\r\nThis change is needed in order to be compatible with new\r\n`mlc-chat-config.json` in hugginface due to\r\nhttps://github.com/mlc-ai/mlc-llm/pull/2493","shortMessageHtmlLink":"[OpenAI] Handle exceeding context window size, rename max_gen_len to …"}},{"before":"b2718de853174d3c07618228c35abadef107fd2e","after":"ee2745df81c453c4561d86c9521d0a141c11b269","ref":"refs/heads/main","pushedAt":"2024-06-05T12:35:06.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"CharlieFRuan","name":"Charlie Ruan","path":"/CharlieFRuan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/53290280?s=80&v=4"},"commit":{"message":"[OpenAI] Add include_usage in streaming, add prefill decode speed to usage (#456)\n\nThis change is not breaking. Behavior only changes when `stream_options:\r\n{include_usage: True}` is set for streaming, during which we yield\r\nanother chunk with empty `choices` after the last chunk.\r\n\r\nAlso include `prefill_tokens_per_s` and `decode_tokens_per_s` for\r\n`usage` in both streaming and non-streaming, hence replacing the\r\nprevious usage of `runtimeStatsText()`.","shortMessageHtmlLink":"[OpenAI] Add include_usage in streaming, add prefill decode speed to …"}},{"before":"94958921ec77e0f63721266a55aa600caa9d7fee","after":"c8e9985efad377cae046ee95bc59313a4367ee2b","ref":"refs/heads/gh-pages","pushedAt":"2024-06-04T20:07:28.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Build at Tue Jun  4 20:07:28 UTC 2024","shortMessageHtmlLink":"Build at Tue Jun 4 20:07:28 UTC 2024"}},{"before":"9097442ac47e048d1aaaf0240c4d8d665a601232","after":"b2718de853174d3c07618228c35abadef107fd2e","ref":"refs/heads/main","pushedAt":"2024-06-04T20:06:33.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"CharlieFRuan","name":"Charlie Ruan","path":"/CharlieFRuan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/53290280?s=80&v=4"},"commit":{"message":"[Models] Add Mistral v0.3 and TinyLlama v1.0; bump web-tokenizer version (#458)\n\nAdd new models:\r\n- `Mistral-7B-Instruct-v0.3-q4f16_1-MLC` (we had v0.2 before)\r\n- `Mistral-7B-Instruct-v0.3-q4f32_1-MLC`\r\n- `TinyLlama-1.1B-Chat-v1.0-q4f16_1-MLC` (we had v0.4 before)\r\n- `TinyLlama-1.1B-Chat-v1.0-q4f32_1-MLC` (we had v0.4 before)\r\n\r\nTo support Mistral v0.3, we also bump web-tokenizers to 0.1.4\r\n\r\nNote: Mistral v0.3's WASM should be identical to v0.2; hence we can use\r\nit for models like Mistral v0.2, OpenHermes, WizardMath, etc.","shortMessageHtmlLink":"[Models] Add Mistral v0.3 and TinyLlama v1.0; bump web-tokenizer vers…"}},{"before":"68b17e6ad494e12137d62f3a6db7ce82bdc50732","after":"94958921ec77e0f63721266a55aa600caa9d7fee","ref":"refs/heads/gh-pages","pushedAt":"2024-06-04T13:06:02.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Build at Tue Jun  4 13:06:01 UTC 2024","shortMessageHtmlLink":"Build at Tue Jun 4 13:06:01 UTC 2024"}},{"before":"22f0d37c5e68515059ba63ff5ba56f54324bc820","after":"9097442ac47e048d1aaaf0240c4d8d665a601232","ref":"refs/heads/main","pushedAt":"2024-06-04T13:05:05.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"CharlieFRuan","name":"Charlie Ruan","path":"/CharlieFRuan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/53290280?s=80&v=4"},"commit":{"message":"[KVCache] Add KVCache settings to ChatOptions, add overrides to ModelRecord (#455)\n\nThere are three KVCache-related fields: `context_window_size`,\r\n`sliding_window_size`, and `attention_sink_size`.\r\n\r\nPrior to this PR, we read these three fields from the WASM, because a\r\nwhile back these affect the model's compilation. Now these things can be\r\nchanged in runtime, so we read these in from `mlc-chat-config.json`.\r\n\r\nThus, we add these three fields into `ChatConfig`, hence `ChatOptions`,\r\nso users can override these in `reload()`.\r\n\r\nAlternatively, users can override these in `ModelRecord.overrides` (i.e.\r\na \"default override\"). As a result, we no longer need both `ctx4k.wasm`\r\nand `ctx1k.wasm`; instead we simply add `overrides:\r\n{context_window_size: 1024}` in the model record for the `-1k` models\r\nand share the same wasm.\r\n\r\nWe first override with `overrides`, then with `chatOpts` (i.e.\r\nprioritizing `chatOpts`), since `ModelRecord.overrides` is considered as\r\na \"default override\"\r\n```typescript\r\n    this.config = {\r\n      ...(await configCache.fetchWithCache(configUrl, \"json\")),\r\n      ...modelRecord.overrides,\r\n      ...chatOpts,\r\n    } as ChatConfig;\r\n```\r\n\r\nUsage is demonstrated in `examples/get-started`. As a result, we can use\r\nsliding window with attention sink in models like llama3.","shortMessageHtmlLink":"[KVCache] Add KVCache settings to ChatOptions, add overrides to Model…"}},{"before":"21288ec68c7d75073c8408dee725a978fee17763","after":"68b17e6ad494e12137d62f3a6db7ce82bdc50732","ref":"refs/heads/gh-pages","pushedAt":"2024-06-03T02:21:16.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Build at Mon Jun  3 02:21:16 UTC 2024","shortMessageHtmlLink":"Build at Mon Jun 3 02:21:16 UTC 2024"}},{"before":"6581ae72c18dfc8530bc7716673bc5d62d36cf13","after":"22f0d37c5e68515059ba63ff5ba56f54324bc820","ref":"refs/heads/main","pushedAt":"2024-06-03T02:20:23.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"CharlieFRuan","name":"Charlie Ruan","path":"/CharlieFRuan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/53290280?s=80&v=4"},"commit":{"message":"[type] Add signature for overloading getToolCallFromOutputMessage() (#454)\n\nThis PR is a tiny one, updating the signature of\r\ngetToolCallFromOutputMessage() function to overload it so that\r\nTypeScript can correctly inference the retrun type based on parameter\r\nautomatically.","shortMessageHtmlLink":"[type] Add signature for overloading getToolCallFromOutputMessage() (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2329870849\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlc-ai/web-llm/issues/454\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlc-ai/web-llm/pull/454/hovercard\" href=\"https://github.com/mlc-ai/web-llm/pull/454\">#…</a>"}},{"before":null,"after":"193f599116ceca61bd9a4db64058de8f168e405f","ref":"refs/heads/overload","pushedAt":"2024-06-02T21:24:24.000Z","pushType":"branch_creation","commitsCount":0,"pusher":{"login":"Neet-Nestor","name":"Nestor Qin","path":"/Neet-Nestor","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/23090573?s=80&v=4"},"commit":{"message":"[type] Add signature for overloading getToolCallFromOutputMessage()","shortMessageHtmlLink":"[type] Add signature for overloading getToolCallFromOutputMessage()"}},{"before":"e56fe03252f3dd2bdf5fe1b179c5fe2823e636ca","after":"21288ec68c7d75073c8408dee725a978fee17763","ref":"refs/heads/gh-pages","pushedAt":"2024-06-02T20:01:09.000Z","pushType":"push","commitsCount":1,"pusher":{"login":"github-actions[bot]","name":null,"path":"/apps/github-actions","primaryAvatarUrl":"https://avatars.githubusercontent.com/in/15368?s=80&v=4"},"commit":{"message":"Build at Sun Jun  2 20:01:09 UTC 2024","shortMessageHtmlLink":"Build at Sun Jun 2 20:01:09 UTC 2024"}},{"before":"9182c4e762490edf0265777b204273a81a2220f3","after":"6581ae72c18dfc8530bc7716673bc5d62d36cf13","ref":"refs/heads/main","pushedAt":"2024-06-02T20:00:19.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"CharlieFRuan","name":"Charlie Ruan","path":"/CharlieFRuan","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/53290280?s=80&v=4"},"commit":{"message":"[Fix] Small fix on tool_calls (#453)\n\nTrivial fix. Fix how we condition on `isFunctionCalling` when populating\r\nmessages for non-streaming cases, so that non-function calling request\r\nwould not have an undefined `tool_calls` field.","shortMessageHtmlLink":"[Fix] Small fix on tool_calls (<a class=\"issue-link js-issue-link\" data-error-text=\"Failed to load title\" data-id=\"2329838688\" data-permission-text=\"Title is private\" data-url=\"https://github.com/mlc-ai/web-llm/issues/453\" data-hovercard-type=\"pull_request\" data-hovercard-url=\"/mlc-ai/web-llm/pull/453/hovercard\" href=\"https://github.com/mlc-ai/web-llm/pull/453\">#453</a>)"}}],"hasNextPage":true,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAAEX7L7ewA","startCursor":null,"endCursor":null}},"title":"Activity · mlc-ai/web-llm"}