Terminal
Terminal/Skill
Walks through root cause analysis of failing, slow or costly BigQuery jobs, from symptoms and query plan stages to fixes.
By Google
Works with
npx skills add google/skills --skill bigquery-troubleshootingBefore running diagnostic queries or investigating incident telemetry:
Google Cloud SDK: Ensure the Google Cloud SDK is installed and configured.
Project Selection: Set the active Google Cloud project:
gcloud config set project {project_id}
API Enablement: Ensure BigQuery and Cloud Monitoring APIs are enabled:
gcloud services enable bigquery.googleapis.com monitoring.googleapis.com
Authentication: Authenticate the environment:
bq show -j): gcloud auth logingcloud auth application-default loginGOOGLE_APPLICATION_CREDENTIALS="/path/to/key.json"Billing & IAM Roles:
{project_id}.roles/bigquery.jobUser: Executing diagnostic queries.roles/bigquery.resourceViewer or roles/bigquery.admin:
Inspecting reservation and job execution telemetry.roles/monitoring.viewer: Cloud Monitoring metrics.roles/billing.viewer: Cloud Billing reports and cost attribution.Companion Skills Installation:
This skill is part of a 3-pillar operations suite (bigquery-observability,
bigquery-optimization, bigquery-troubleshooting). If any companion skill
is not yet installed in your environment, install the full suite:
npx skills add google/skills --skill bigquery-observability --skill bigquery-optimization --skill bigquery-troubleshooting
(If bigquery-observability is not installed, use the self-contained
baseline formulas and query templates provided directly in the reference
sections below).
project_id, region, reservation_id, or job_id, and domain
(Performance, Compute Cost, or Storage Cost). If the request falls outside
incident diagnosis or asks for a sibling domain, follow Routing
Boundaries below.bigquery_observability) to select the appropriate telemetry interface
(REST API bq show --location={location} -j {project_id}:{job_id} for
single-job stage bottlenecks vs. INFORMATION_SCHEMA for system-wide
factors). Diagnostic workflows, symptom-to-cause mappings, key
tables/fields, CLI triage commands, and remediation levers are fully defined
in this skill. For pre-composed SQL query templates and full schema
dictionaries, consult bigquery-observability.INFORMATION_SCHEMA best practices to isolate the root cause via
comparative analysis against a normal baseline.Performance is relative. Always approach performance troubleshooting as a comparative exercise: identify a comparable past execution, compare the statistics, and isolate which dimension shifted between a fast baseline and the slow execution:
Cost troubleshooting requires tracing physical resource consumption (Slot-Hours, TiB Billed, GiB Stored) rather than fluctuating contract rates:
INFORMATION_SCHEMA.RESERVATION_CHANGES,
INFORMATION_SCHEMA.CAPACITY_COMMITMENT_CHANGES_BY_PROJECT,
INFORMATION_SCHEMA.SCHEMATA_OPTIONS, or actor user_email /
query_hash).references/performance_resource_contention.md): Diagnostic workflows for
isolating single-job stage bottlenecks (slot_contention, spill_to_disk),
cohort baseline comparisons (normalized_literals), incident window
discovery, 1-second reservation slot saturation, timeframe contention
comparisons, fleet performance variance, and table-level concurrency.references/performance_config_changed.md): Diagnostic workflows for
auditing reservation slot_capacity and autoscale.max_slots edits,
tracking active capacity commitment timelines, diagnosing reservation
assignment modifications, and evaluating autoscaling headroom saturation.references/query_plan_execution_graph.md): Diagnostic workflows for
investigating single-job stage bottlenecks (bq show point-lookups),
isolating slowest stages (end_ms - start_ms), diagnosing join explosions
(records_written >> records_read, high_cardinality_joins), shuffle
spills (shuffle_output_bytes_spilled), and partition skew, substep
intermediate variable disambiguation ($1, $2), mandatory bytes scanned
vs records read corrections, and UI execution graph grounding concepts.references/cost_compute_ondemand.md):
Diagnostic workflows for unpartitioned runaway scans (the "Bully Query"),
hidden Row-Level Security (RLS) redaction gaps, BigQuery ML (BQML) 50x model
training rate multipliers, and user/service account query quotas.references/cost_compute_capacity.md): Diagnostic workflows for uncovered
baseline slot penalties (baseline > commitments), reservation baseline
reductions triggering autoscale surges (RESERVATION_BASELINE_CHANGED),
autoscaler thrashing from batch cron spikes, and serverless Apache Spark
stored procedure slot-hours.references/cost_storage.md):
Diagnostic workflows for historical partition 90-day timer resets (the DML
trap), unpartitioned table active data traps, physical Time Travel and
Fail-Safe churn on daily overwrites, and dropped table Fail-Safe drain
periods.If a user request shifts outside incident diagnosis during troubleshooting, execute the corresponding handoff:
SELECT *), hand off to
bigquery-optimization.INFORMATION_SCHEMA queries without an active performance regression or
incident (e.g. general telemetry queries), hand off to
bigquery-observability.bigquery-optimization.