Skip to content

Simple proompting library built on top of the XML spec

License

Notifications You must be signed in to change notification settings

raunakdoesdev/xmlai

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

51 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

XML AI

XML AI is the fastest and most ergonomic way to get structured input and output out of your large language model.

Just write your prompt in JSON and we'll autoconvert it to XML that the model can work with. The best part? You can stream the response back as a JSON object in real time. No more sacrificing streaming for function calling or schema following capabilities. Built and optimized for Anthropic's Claude models, but also including OpenAI support.

The library is designed to be as lightweight as possible, and with nearly identical APIs whether you are operating in Python or Javascript/Typescript.

Typescript

Installation

pnpm install xmlai

Getting Started

import { anthropic_prompt } from "xmlai/llm";

const prompt = anthropic_prompt(
    {
        question: "what is the answer to the ultimate question of life?",
        reference: "The Hitchhiker's Guide to the Galaxy",
    },
    "answer"
);

const completion = await anthropic.completions.create({
    model: "claude-instant-1",
    max_tokens_to_sample: 300,
    temperature: 0.1,
    ...prompt,
});

completion.completion // 42

The generated prompts look like this:

{
   "prompt":"\n\nHuman:<question>what is the answer to the ultimate question of life?</question>
   <reference>The Hitchhiker's Guide to the Galaxy</reference>\n\nAssistant:<answer>",
   "stop_sequences":[
      "</answer>"
   ]
}

Note that we feed the opening tag to the beginning of the assistant's response! This combined with the closing tag as the stop token almost always ensures that the response is valid XML.

Why another prompting library?

Anthropic's LLM Claude is trained on lots and lots of XML data. It is quite good at following XML schemas. In fact at the Anthropic Hackathon, the prompting workshop specifically presented some extra tips on how to get the best out of Claude when it comes to XML. I incorporated those tricks into this library to make it easier for others to take advantage of.

Also, the regex for dealing with XML streams is surprisingly grotesque. I figured I'd limit the monstrosity to one codebase where it can be tested and maintained.

About

Simple proompting library built on top of the XML spec

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published