Skip to content

Issues: niieani/gpt-tokenizer

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

18 Open 13 Closed
18 Open 13 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Support New o200k_base
#42 opened May 14, 2024 by RedwindA
1
Picture tokens
#41 opened Jan 27, 2024 by arthurwolf
vercel/pkg Error with gpt-tokenizer
#40 opened Nov 28, 2023 by Ranork
Is the algorithm of this repo consistent with openai tiktoken?
#38 opened Nov 12, 2023 by Zwe1
ChatMessage[] TypeScript warning
#36 opened Oct 17, 2023 by thdoan
2
Huge memory consumsation of isWithinTokenLimit
#35 opened Oct 10, 2023 by aminsol
How to set the model in encode / encode generator like encodeChat or encodeChatGenerator?
#34 opened Oct 7, 2023 by MiniSuperDev
Error in NextJS edge function: Attempted import error: './index.js' does not contain a default export (imported as 'cjs')
#30 opened Aug 17, 2023 by gablabelle
ReferenceError: Property 'TextDecoder' doesn't exist, js engine: hermes
#28 opened Jul 31, 2023 by YunySun
Error: Invariant: Method expects to have requestAsyncStorage, none available
#27 opened Jul 26, 2023 by chenqilin70
isWithinTokenLimit fails due to lack of model being provided
#26 opened Jul 16, 2023 by emfastic
5
Improving documentation about the use of api vs importing function directly
#22 opened Jul 2, 2023 by nahumzs
2
Support function calls?
#21 opened Jun 29, 2023 by seoker
2
Add support for gpt-3.5-turbo-0613 and gpt-3.5-turbo-16k-0613
#19 opened Jun 23, 2023 by nilsreichardt
avoid expensive initialization
#18 opened Jun 20, 2023 by mimoo
4
最近官方更新了GPT3.5-16K的模型，请问计算方式跟gpt-3.5-turbo是一样的吗
#17 opened Jun 15, 2023 by xiaoqixiaoqi1113
This module is not ready for CJK characters
#16 opened Jun 8, 2023 by mashihua
2
Dependency Dashboard
#14 opened Jun 1, 2023 by renovate bot
15 tasks
ProTip! What’s not been updated in a month: updated:<2024-04-20.