Terminal
Terminal/Skill
Runs LLM as a judge evaluations on Together AI to classify, score, or compare model outputs, then download the results.
By Together AI
Works with
npx skills add togethercomputer/skills --skill together-evaluationsUse Together AI evaluations when the user wants a managed LLM-as-a-judge workflow rather than an ad hoc prompt loop.
Core evaluation types:
This skill also covers external providers used as judges or targets when the workflow still runs through Together AI's evaluation system.
together-chat-completions for one-off inference or manual judge promptstogether-batch-inference for bulk offline generation rather than evaluationtogether-fine-tuning when the user wants to improve the model instead of just measure ittogether-dedicated-model-inference only if the evaluation target itself is a dedicated endpoint--eval-column, --model-a-column, or --model-b-column in the scriptsimage_data_urls column (base64 data URLs) to the dataset; the model and judge must be vision-capable--judge-model-source external, --eval-model-source external, or compare-side source flags--download-results in the scripts when you want the per-row JSONL locallytogether>=2.0.0). If the user is on an older version, they must upgrade first: uv pip install --upgrade "together>=2.0.0".check=False for eval uploads because local file validation can misclassify eval datasets.