Interface EvaluateItem

One thing to judge: a state, with images or audio, or the media alone.

interface EvaluateItem {
    audio?: AudioLike;
    images?: ImageLike | ImageLike[];
    sampleRate?: number;
    state?: unknown;
}

Properties

One audio clip (up to 30 s) to judge along with the state. Needs a model that accepts audio.

Images to judge along with the state, in order. Needs a model that accepts images, such as d1-omni-600m.

sampleRate?: number

Sample rate of audio when it is a Float32Array at a rate other than 16 kHz.

state?: unknown

What to judge: a string, or JSON (serialized with sorted keys; d1 models read it as JSON). Optional when images or audio are the whole state.