Terminal
Terminal/Skill
Runs large offline batches of model requests on Together AI at lower cost, covering JSONL upload, job polling, and downloading results.
By Together AI
Works with
npx skills add togethercomputer/skills --skill together-batch-inferenceUse Together AI's Batch API for large offline workloads where latency is not the primary concern.
Typical fits:
together-chat-completions for real-time requests or tool-calling appstogether-evaluations for managed LLM-as-a-judge workflowstogether-embeddings for retrieval-specific vector generationcustom_id and body.purpose="batch-api".input_file_id=... and the target endpoint.custom_id.together>=2.0.0). If the user is on an older version, they must upgrade first: uv pip install --upgrade "together>=2.0.0".input_file_id, not legacy file parameters.custom_id stable and meaningful so result reconciliation is easy.client.batches.create() returns a wrapper; access the batch object via response.job (e.g., response.job.id). client.batches.retrieve() returns the batch object directly.max_tokens low (e.g., 4), use temperature: 0, and constrain the system prompt to return only the label. This minimizes output tokens and cost.