Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python worker can't access env.AI methods like run after binding #1992

Open
neeravkumar opened this issue Apr 8, 2024 · 1 comment
Open

Comments

@neeravkumar
Copy link

neeravkumar commented Apr 8, 2024

Can't create a python worker that can access the AI bindings

from js import Response

async def on_fetch(request, env):
    simple = "{'prompt': 'Tell me a random quote'}"
    response = await env.AI.run("@cf/meta/llama-2-7b-chat-int8", simple)
    return Response.new(response)

returns

Traceback (most recent call last):
  File \"/session/metadata/entry.py\", line 5, in on_fetch
    response = await env.AI.run(\"@cf/meta/llama-2-7b-chat-int8\", simple)
                     ^^^^^^^^^^
AttributeError: run
@hoodmane
Copy link
Contributor

hoodmane commented Apr 8, 2024

For context, in JavaScript you would write:

import { Ai } from '@cloudflare/ai'

export interface Env {
  // If you set another name in wrangler.toml as the value for 'binding',
  // replace "AI" with the variable name you defined.
  AI: any;
}

async function fetch(request: Request, env: Env) {
  const ai = new Ai(env.AI);
  const response = await ai.run('@cf/meta/llama-2-7b-chat-int8', {
      prompt: "What is the origin of the phrase Hello, World"
    }
  );
}

So the run method is on the Ai wrapper and not on env.AI itself. We need some plan for how to allow Python workers to access equivalent functionality.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants