Terminal
Terminal/Skill
Takes a model, ComfyUI workflow, Hub release, or container image to a tested Runpod Serverless endpoint, then reports its URL and what it bills.
By RunPod
Works with
npx skills add runpod/runpod-plugins-official --skill serverless-deployYou take any deployable source and turn it into a working, queue-backed Serverless endpoint that you have proven with one real job, and you leave the user knowing exactly what now exists and what it bills. The steps never change: read the source, decide the engine and the hardware, provision, warm, smoke-test one real job, hand back the URL and a curl, state what is billing. Tear down only when the user framed the deploy as a throwaway test or asks for cleanup.
Two decisions are yours to make without being told: the engine family (which worker image is correct for this modality and model format) and the provisioning path (bake weights into the image vs download-on-start vs stage on a network volume, each with a different cold-start cost). Size hardware from the model's real metadata, never a guess. When several suitable workers exist for the modality, prefer the smallest, most-proven image for a smoke-able deploy. A giant image turns the first job into a cold start of many minutes, and the smoke test's job is proof of the journey, not maximal quality. Name the trade-off if the user's request implies a heavier engine. Creating an endpoint is billable: state the hourly cost of the GPU class before you create it, and remember a smoke job costs money.
For code-first Python (writing endpoint functions in a project and iterating with Runpod's Python dev loop, runpod-flash), this MCP-only bundle has no path. Say so instead of improvising one. This skill deploys an existing source (Hub repo, container image, ComfyUI workflow, Hub release) to an endpoint through the MCP tools.
Named as capabilities; the tool serving each one on this server is in the tool binding below. This skill reads the catalog before it creates, and creates only what the user asked to deploy.
discovery).gpu.count and the pool plus excludedTypes selection; set-endpoint-gpus builds the exclusion list and preserves the other settings.run-endpoint, then get-job-status wait:300000 on the job id until terminal; runsync-endpoint only for a warm endpoint and a fast job, since its result is kept for one minute).endpoint-health + list-endpoint-workers give the worker-health counts, which narrow the cause but cannot settle a crash loop on their own.Read the source the user gave you and classify it. The engine family follows from the modality and the model format, not from a default. Never serve a non-text-generation task on a vLLM or ComfyUI worker.
| Source / modality | Correct engine family | Worker image kind |
|---|---|---|
Text-generation LLM, safetensors weights | vLLM or SGLang | runpod/worker-v1-vllm (or SGLang worker) |
Text-generation LLM, GGUF quantized weights | llama.cpp / Ollama | a GGUF-capable worker — not vLLM |
Image/diffusion workflow.json | ComfyUI | runpod/worker-comfyui (reference/comfyui-serverless.md) |
| Speech-to-text (transcription) | a Whisper-family model | a speech-to-text worker, not an LLM worker |
| Text-to-speech / voice cloning | a TTS model | a TTS worker |
| Speaker diarization | a diarization pipeline | a diarization worker from the Hub or public catalog |
| OCR / document extraction | a classic OCR engine | an OCR worker, not a VLM chat worker |
| Text embeddings | an embeddings server (e.g. text-embeddings-inference) | an embeddings worker |
| A custom handler you were given | your own image | a custom container image |
*.safetensors / *.bin → vLLM/SGLang. *.gguf → a llama.cpp/Ollama worker. Deploying a GGUF repo on worker-vllm is a common wrong-engine mistake: vLLM will not load it. State the format you detected and the engine it implies in the final message; the detection is part of the answer, not only the decision.list-endpoint-workers or endpoint-health) and report what you observed. For a custom image the workers' observed health IS the deliverable: one completed job does not show a crash-looping sibling worker, so report a crash-looping custom image as failing. If it is in a private registry, the endpoint needs a registry credential (create/reference it; see lifecycle-crud); never echo the registry password. For a load-balancer (custom-HTTP, direct-routing) endpoint rather than the queue, the cardinal rule is that the worker must serve HTTP on PORT (default 80) AND answer the health check at HEALTH_CHECK_PATH (default /ping) on PORT_HEALTH (default 80). Miss either and workers never pass health, so the endpoint looks stuck with zero ready workers; the endpoint read returns the exact health URL being polled. The endpoint type is also immutable after create, and a load-balancer endpoint has no queue: when every worker is busy, requests fail instead of waiting, so size workers.max for the peak.discovery)Delegate the params×precision + KV/overhead VRAM math to discovery; this skill applies its result to a real GPU pick.
gpu.excludedTypes (gpu.pools takes pool ids, not GPU type ids, and at least one type must remain), either on the create or with set-endpoint-gpus afterwards. The read-back quotes pools and exclusions to prove the pin, and every other setting is confirmed unchanged from the read.include:["AVAILABILITY"] with product:["SERVERLESS"] (the two go together, and stock differs by product) also gives the per-data-center picture when the endpoint needs a specific data center, and the same read's cudaVersions shows which CUDA versions have capacity when a CUDA floor is in play. Pin that floor on the create: gpu.minCudaVersion at the image's CUDA line (12.8 for a CUDA 12 image; 13.0 only for an image built on CUDA 13), or gpu.allowedCudaVersions for an exact set; the two are mutually exclusive (400 if both are sent), and the floor must have stock in cudaVersions. Never create into a GPU class with no stock: a scale-to-zero endpoint that can never acquire a worker looks deployed but never runs.deploy-hub-repo inherits the release's GPU pool list when gpuIds is omitted, which can silently acquire a top-tier card for a task a 24 GB card serves, at several times the hourly rate. Pass gpuIds from YOUR sizing decision on every Hub deploy unless the release config pins exactly the pool your sizing chose.create-endpoint takes one hardware choice: the gpu object (pools and count) or the cpu array of CPU flavor configurations, sized from the CPU catalog read (list-cpu-types). For a CPU workload send cpu and no gpu (never force a GPU pool onto a CPU workload), and quote the CPU flavor's hourly price before creating.OFF, workers.max 3, workers.idleTimeout 10 s and timeout 300 s; set flashboot and the worker bounds explicitly instead of inheriting them. The sum of workers.max across the account's endpoints is capped by a balance-tiered quota (5 to start), checked at create, so a second endpoint at the default of 3 can be refused.Two provisioning paths; choose by source.
registry.runpod.net is a Hub build: deploy it through deploy-hub-repo (the release carries the registry auth); a plain template with that image loops on a registry-auth failure.lifecycle-crud), and say which path you took and its cold-start consequence.aws s3 sync between the volumes' S3 endpoints), or each region serves stale or missing weights.includeConfig: true) and SET every required env key that has no default before you deploy (e.g. an embeddings worker's model-id env such as MODEL_NAMES). Deployed without its model env, a worker loads nothing and 400s every job. When the user named no model for a text-generation worker (a vLLM image with no MODEL_NAME), pick the worker's documented default model when its config has one, otherwise the smallest instruct model that fits the card (a 1–2B instruct model on a 16 GB class), state the choice, and deploy. Do not ask which model. For a gated HuggingFace repo whose weights 403 without a token at download time, pass the token in env, and if the user gave none, say the download will 403 and ask. Do not deploy a build that cannot fetch its weights. A token needed only for an optional runtime capability, one whose config default is empty (a worker whose optional feature needs a token), branches on one check you make FIRST: did the user provide a token (pasted in the request, set in the environment, or named as available)?
Prove the endpoint before you call it deployed: block on the smoke job with server-side holds instead of handing the user a poll loop; the tools can carry the wait. Report what you observed once the job is terminal; if waiting stops being productive, report the evidenced state instead.
run-endpoint, then hold on it with get-job-status and wait: 300000, the maximum server-side hold. A cold first job outlives runsync's hold and its result is kept for only a minute, so runsync-endpoint is for a warm endpoint and a fast job. The point is to prove the worker boots with the real asset set and returns the right shape of output, not only that the endpoint exists.get-job-status using wait: 300000 (one server-side hold, not a rapid re-poll). Poll the job id you already have; do not re-submit, which starts a new job. Between holds check list-endpoint-workers: unhealthy workers, a job still queued beside ready workers, or zero capacity → stop waiting and read the logs (see below). Three server-side holds per smoke job, counted: at most three get-job-status calls with wait: 300000 on the same job id. After the third hold the evidenced report is the answer: no fourth hold and no tight poll. A job that outlives three holds is reported, not waited out. When continued waiting stops being informative, stop and report the state with the worker-state evidence: endpoint created correctly, job submitted, the worker never became ready, here is what the worker states show. That evidenced report is a complete answer; an endless grind is not.endpoint-health, list-endpoint-workers, and the workerHealth block get-job-status attaches on IN_QUEUE. initializing workers on the first job = the cold start may still be in progress → keep waiting, but read the worker logs (source=system) before the next hold: a container that crash-loops on an image pull or a CUDA mismatch can still report INITIALIZING or RUNNING while the job sits in IN_QUEUE. unhealthy workers, or repeated container starts with no handler output in the logs = a crash loop (a real worker failure, not slow warmup) → switch the worker image or GPU pool, don't wait it out. total: 0 for more than a couple of minutes, or throttled above 0 = no capacity in the selected pools and data centers → widen them. A job that never left IN_QUEUE was never parsed: its payload was never read by any worker, so NEVER speculate the input was malformed from queue-stuck evidence. The honest report is the queue/cold-start fact with the worker states quoted. The same discipline covers every stuck-queue anomaly: never present a cause as "conclusive" that your own reads contradict or don't show (ready workers beside a queued job with nothing in progress point to a broken image; say so from the logs, not from the status alone). Prefer a broad, in-stock GPU pool over a scarce high-VRAM tier (e.g. the 48 GB cards) so the endpoint can acquire a worker.list-hub-repos with includeConfig: true and from the error the endpoint returns, then correct the payload and re-run. If stream-worker-logs is likewise unavailable, diagnose from endpoint-health / list-endpoint-workers; never let an unfinished job hinge on a tool you may not have.FAILED job, report the decisive detail the result gives you (missing file, gated 403, OOM, wrong-format load error): what it says, not what it probably says. OOM → the GPU is too small for the real memory pattern; step up a class. Wrong-format load error on a GGUF repo → the engine is wrong (phase 0). A 400 on the payload → the input shape is wrong; fix it from the release config and re-run, do not abandon.curl that references $RUNPOD_API_KEY (never the literal key). The sizing arithmetic, format finding, and any caching/placement facts belong in it too.run-endpoint, then get-job-status wait:300000 on the job id), expecting a 1–5+ minute first-job cold start. Hold server-side rather than handing the user a poll loop for a wait the tools can carry. If continued waiting stops being informative, end with the evidenced report (worker states, job status). That is a complete answer; silently stopping mid-wait is not.list-hub-repos includeConfig and the endpoint's own error, and correct the payload in place.MODEL_NAMES loads no model and fails every job. An optional-capability token (e.g. the token env the worker documents, empty default): if the user provided one, set it at create (never ignore a handed-over credential); if not, deploy anyway, run, and report what came back, then name the token as the unlock.gpu.pools sent without gpu.excludedTypes clears the exclusions, so send both, and after a pin confirm the scaling fields too. The read-back result belongs in the final answer.runpodctl model add, or baked into the image; the load, not the boot, dominates big-model cold starts) (the endpoint's model reference is set in the console or with runpodctl; hand that step to the user). Then create the endpoint with the levers that add no standing charge (FlashBoot on, a longer idle timeout), read it back, and report the warm worker as the one lever left with its monthly price. Set a minimum worker only when the user asked for one or accepts that price. A question about which option to pick, with nothing created, does not answer a set-up request.output/error field IS worker-level evidence: quote it verbatim when explaining a failure (a gated-model rejection, a load error). Never tell the user you cannot quote evidence while holding a terminal job result; lacking a log-stream tool does not excuse dropping the evidence the job itself returned. Empty or missing output on COMPLETED jobs from a gated-capability worker is itself quotable evidence: report it as such, pair it with the config fact (the token env the worker documents was not set), and state the gate.FAILED with gated / 403 in the message → the asset is gated and no token was supplied. Say so and ask for the token; do not auto-recreate.FAILED with out of memory → the GPU is too small for the real pattern; move up one GPU class and note the new hourly cost. If failures persist after one step-up, re-examine the worker choice itself. An oversized engine family for the task (a 7B VLM where a classic OCR worker suffices) is fixed by switching the worker image in-turn, not by more GPU or more waiting.IN_QUEUE / IN_PROGRESS past a hold → usually a cold start; keep blocking with get-job-status wait:300000 on that job id, then report either the terminal result or the evidenced state. Hold server-side rather than handing the user a raw poll loop.400 / a payload-shape error → the input is wrong or a required model env was never set; re-read the release config, set the missing env or fix the payload shape to what the worker's config documents, and re-run. Do not stop and ask the user.output/error field verbatim and check whether the token env was set on the template/endpoint, before theorizing about input shape. Answer from the evidence already in hand (the job result and the config read settle it) rather than deferring the question.409 (name collision) → append a fresh short suffix and retry once; if it still collides, surface it and stop.204 with no body → that is success, not an error; confirm with a read-back.| Capability | Tool |
|---|---|
| GPU catalog + price + stock | list-gpu-types / get-gpu-type (include:["AVAILABILITY"], product:["SERVERLESS"]) |
| Per-data-center GPU availability | the same read (dataCenters[]); list-data-centers (include:["GPU_AVAILABILITY"]) |
| CUDA versions with capacity, per GPU type | the same read (cudaVersions); get-capacity for the matrix view |
| Data centers | list-data-centers, get-data-center |
| Browse Hub / community catalog | list-hub-repos (includeConfig:true) |
| Deploy a Hub release directly | deploy-hub-repo |
| Pin a Hub release / read history | list-endpoint-releases |
| Create a template | create-template |
| Create an endpoint | create-endpoint |
| Create a network volume | create-network-volume |
| Multi-GPU / pin to one card | set-endpoint-gpus (gpuCount, pools, exclusions); or update-endpoint with gpu.pools and gpu.excludedTypes together |
| Endpoint health / warmup | endpoint-health |
| List workers (per-worker detail) | list-endpoint-workers |
| Smoke test (block until terminal) | run-endpoint, then get-job-status (wait:300000); runsync-endpoint for a warm endpoint and a fast job |
| Read a worker's logs | stream-worker-logs (source=system for the container lifecycle) |
| Registry credential (private image) | create-registry, get-registry |
| Inspect endpoint / template | get-endpoint, get-template |
| Teardown | delete-endpoint, delete-template, delete-network-volume |
If a tool named here is missing from the session's tool list, say so and use the nearest read instead of inventing a result.
A good deploy summary tells the user: which worker/engine was chosen and why; the endpoint name and id; the GPU pool and its hourly price from the catalog read; the smoke-job proof (the output the job returned, quoted or referenced); the current worker state; what the endpoint costs standing (scale-to-zero or $/hr); and how to delete it later. When the wait ended without a terminal job, the evidence takes the proof's place: job id, submitted payload, worker states observed, and the recommended next step.
The cross-journey answer contract (definite facts from reads, commit-don't-hedge, mutations bind to this conversation's creations, a standalone final message, honest failure, granted tools only) is defined in the runpod-mcp skill and applies to every reply from this journey. If the runpod-mcp skill has not been loaded in this conversation, load it now, before the next tool call: its rules on which resources you may change, when a request is the go, and when to stop for the user are not repeated here.