Terminal
Terminal/Skill
Checks a SageMaker fine tuning dataset for file format and schema problems, and reports whether it is ready for training or evaluation.
By Amazon Web Services
Works with
npx skills add awslabs/agent-plugins --skill dataset-evaluationFollow the workflow shown below. Locate the dataset, check the file type, and resolve any issues with missing files or wrong file types. Determine the fine-tuning model and fine-tuning strategy. Run the appropriate validation based on the model family. Summarize the results: is the dataset ready for fine-tuning?
sdk-getting-started skill first.Locate Dataset:
Determine strategy and model:
Check File Formatting: Run the tool format_detector.py to make sure the file conforms to formatting requirements.
Summarize Results: Tell the user if their data is ready
references/strategy_data_requirements.mdreferences/custom-scorer-evaluation-dataset-formats.md and validate against the scorer-specific schema. The scorer type should be known from conversation context (determined in the model-evaluation skill).# With the file path argument identified in workflow step 1
python scripts/format_detector.py local_path/to/dataset
scripts/format_detector.py — Self-contained format validation scriptreferences/strategy_data_requirements.md — Data format requirements per strategy