Grafbase AI

Grafbase AI simplifies AI integration in products by allowing developers to run models directly in their GraphQL API with a single line of code, eliminating the need for separate workflows, reducing response times by executing models on the edge.

You can enable the experimental AI by adding the following to your grafbase.config.ts config export object:

export default config({
  schema: g,
  experimental: {
    ai: true,
  },
})

Make sure to enable the experimental AI support inside your project's configuration file.

1. Import ai from the context argument inside a JavaScript or TypeScript resolver:

export default async function MyResolver(_, __, { ai }) {
  // ...
}

2. Invoke the model using the ai import:

export default function MyResolver(_, __, { ai }) {
  const answer = ai.textLlm({
    model: 'meta/llama-2-7b-chat-int8',
    prompt: '...',
  })

  return answer
}

3. Optionally enable Grafbase KV to cache the response:

export default async function MyResolver(_, { prompt }, { ai }) {
  try {
    const { value } = await kv.get(prompt)

    if (value === null) {
      const answer = ai.textLlm({
        model: 'meta/llama-2-7b-chat-int8',
        prompt,
      })

      await kv.set(prompt, answer, { ttl: 60 })

      return answer
    } else {
      return value
    }
  } catch (e) {
    return 'error'
  }
}

Further usage and instructions for using the supported models below can be found using the Workers AI documentation.

• meta/llama-2-7b-chat-int8
• openai/whisper
• meta/m2m100-1.2b
• huggingface/distilbert-sst-2-int8
• microsoft/resnet-50
• baai/bge-base-en-v1.5