Batch Requests

Batch Requests

When you need to generate many text outputs, doing so manually through the app can become cumbersome. Especially if you already know all the prompts you'll be using to build off of. Batch requests automate that process and can speed it up significantly as well.

With batch requests, you choose a generator to use (the Standard Generator or one of your custom generators). Then you decide on 1) a single prompt, 2) no prompt, or 3) upload a CSV file of prompts (one on each line). Finally, you specify how many outputs you want to generate per prompt and set some optional generation settings.

Before you run the request, you'll see an estimate of the maximum number of characters that it might generate based on your settings. When the request is done, you'll get a notification email and can download the outputs in CSV format.

Limitations

Since the generator can only comprehend a certain length of text at a time, only the last 3000 characters (currently) of each of your prompts will be used. The prompt column of the output CSV will only show these.

Each request is currently limited to 25 million characters, which is calculated by multiplying (number of prompts) × (outputs per prompt) × (maximum characters per output). For the CSV option, your file must be no larger than 2 GB.

Credit usage

Batch requests use character credits while they run. If a batch request doesn't complete for any reason (e.g. manually canceled, or an error), it will use approximately as many credits as it took to generate the outputs so far.

Each output counts as a minimum of 100 characters, but besides that only what's generated will deduct from your credits.

Batch requests using your custom generators will require the generator to be deployed and will consume deployment hour credits, as when you use the app. If you don't deploy it yourself, the batch request will automatically deploy it for you.

Expiry

Batch requests are automatically deleted ("expire") some time after they're created (currently 1 week). You'll need to download the output CSV before then.

Uniqueness

Since the text generation is random, it's not guaranteed that every output for a given prompt will be different. As when you generate through the app, it's possible to get similar or identical results. When the outputs are longer, they're more likely to diverge from each other.