Terminal
Terminal/Skill
Moves Synapse Data Factory pipeline JSON into Microsoft Fabric Data Factory, converting connections, datasets, and global parameters.
By Microsoft
Works with
npx skills add microsoft/skills-for-fabric --skill pipeline-migrationTelemetry — MANDATORY. Every
api.fabric.microsoft.comcall must carryx-ms-fabric-skill: pipeline-migration(az rest:--headers "x-ms-fabric-skill=pipeline-migration"), including every LRO poll,fabric_lroand retry. Snippets omit it — add it anyway.
CRITICAL NOTES
- To find workspace details (including its ID) from a workspace name: list all workspaces, then use JMESPath filtering
- To find item details (including its ID) from workspace ID, item type, and item name: list all items of that type in that workspace, then use JMESPath filtering
- Fabric does NOT have a "Dataset" item type — all dataset properties are inlined into activity
typeProperties- Linked Services map to Fabric Connections — in pipeline activity JSON,
referenceNameon the activity'slinkedServiceblock uses the Fabric connection display name (not the connection GUID); the connection GUID is used only on Fabric REST API calls- Notebook activities change from
SynapseNotebooktoTridentNotebookand reference notebooks by GUID, not by name- Synapse global parameters become a Variable Library item in Fabric, referenced as
@pipeline().libraryVariables.<name>. Variable LibraryNumbertypes are not consumable in pipelines — SynapseFloat/Doubleare mapped toStringfor runtime compatibility- The
Validationactivity type does not exist in Fabric — it must be rewritten asGetMetadata+IfCondition- Triggers are intentionally excluded from this skill — recreate schedules manually in Fabric after migration
- SSIS package execution, SHIR-exclusive connectors, and Databricks activities are parked — see pipeline-gotchas.md
These companion documents provide general Fabric REST patterns. Do NOT read them upfront — reference only when a specific phase requires a pattern not already covered in this skill's resource files:
az rest / az login CLI patterns, authentication recipes, pipeline run/schedule operationsDataPipeline and VariableLibrary item definition structures (pipeline-content.json, variables.json)For the notebook side of the migration (the notebooks that pipeline activities call), use the companion synapse-migration skill to migrate the notebook content itself.
| Topic | Reference |
|---|---|
| Pre-Migration Assessment (run first) | pipeline-assessment.md |
| Migration Orchestrator | pipeline-orchestrator.md |
| API-Driven Migration Workflow | § API-Driven Migration Workflow |
| Activity Type Mapping | activity-mapping.md |
| Notebook Activity Migration (primary focus) | notebook-activity-migration.md |
| Linked Services → Connections | linked-service-to-connection.md |
| Dataset Inlining | dataset-inlining.md |
| Global Parameters → Variable Library | global-parameters-to-variable-library.md |
| Pipeline Gotchas & Parked Activities | pipeline-gotchas.md |
| Validation & Testing | validation-testing.md |
| Migration Report | migration-report.md |
IMPORTANT — Load only what you need. Do NOT read all resource files upfront.
| When | Read This File |
|---|---|
| User asks for an assessment, scope, or plan before migrating | pipeline-assessment.md |
| User asks to migrate a full pipeline workspace | pipeline-orchestrator.md |
| User asks about activity type mapping or unsupported activities | activity-mapping.md |
| User has SynapseNotebook activities (most common) | notebook-activity-migration.md |
| User asks about linked services or connections | linked-service-to-connection.md |
| User asks about Copy, Lookup, or GetMetadata with datasets | dataset-inlining.md |
| User has global parameters to convert | global-parameters-to-variable-library.md |
| User hits SSIS, SHIR, Databricks, or other blockers | pipeline-gotchas.md |
| Post-migration verification | validation-testing.md |
| Generating a migration summary | migration-report.md |
| Target | Token Audience |
|---|---|
| Synapse Data Plane (pipelines, datasets, linked services) | https://dev.azuresynapse.net |
| Synapse ARM (global parameters, workspace properties) | https://management.azure.com |
| Fabric REST API (create pipelines, connections, Variable Libraries) | https://api.fabric.microsoft.com |
Use
az account get-access-token --resource <audience> --query accessToken -o tsvto acquire tokens.
| Operation | Endpoint |
|---|---|
| List all pipelines | GET https://{ws}.dev.azuresynapse.net/pipelines?api-version=2020-12-01 |
| Get pipeline definition | GET https://{ws}.dev.azuresynapse.net/pipelines/{name}?api-version=2020-12-01 |
| List all datasets | GET https://{ws}.dev.azuresynapse.net/datasets?api-version=2020-12-01 |
| Get dataset definition | GET https://{ws}.dev.azuresynapse.net/datasets/{name}?api-version=2020-12-01 |
| List all linked services | GET https://{ws}.dev.azuresynapse.net/linkedservices?api-version=2020-12-01 |
| Get linked service | GET https://{ws}.dev.azuresynapse.net/linkedservices/{name}?api-version=2020-12-01 |
| Get workspace (global parameters) | GET https://management.azure.com/subscriptions/{subId}/resourceGroups/{rg}/providers/Microsoft.Synapse/workspaces/{ws}?api-version=2021-06-01 |
| Operation | Endpoint |
|---|---|
| List connections | GET https://api.fabric.microsoft.com/v1/connections |
| Create connection | POST https://api.fabric.microsoft.com/v1/connections |
| Create pipeline item | POST https://api.fabric.microsoft.com/v1/workspaces/{wsId}/items |
| Update pipeline definition | POST https://api.fabric.microsoft.com/v1/workspaces/{wsId}/items/{id}/updateDefinition |
| Get pipeline definition | POST https://api.fabric.microsoft.com/v1/workspaces/{wsId}/items/{id}/getDefinition |
| Create Variable Library | POST https://api.fabric.microsoft.com/v1/workspaces/{wsId}/items (type: VariableLibrary) |
| List notebooks in workspace | GET https://api.fabric.microsoft.com/v1/workspaces/{wsId}/notebooks |
Before creating any items in Fabric, run the pipeline assessment to understand scope, complexity, and blockers. The assessment is read-only — it queries Synapse APIs only and produces a markdown report.
Copilot workflow — no Python script file needed:
az account show and az synapse workspace show| When to Use | Action |
|---|---|
| User wants to understand migration scope before committing | Ask for workspace name → load pipeline-assessment.md → run inline |
| User asks "what will and won't migrate?" | Run assessment, present the Executive Summary section |
| User asks for a migration plan or scoping document | Run assessment, print report in chat |
| User has already decided to migrate | Skip assessment — go straight to Migration Phases below |
To also save the report to disk, pass
output_path=f"pipeline-assessment-{SYNAPSE_WS}.md"togenerate_assessment_report(). ThePipelineAssessmentobjects it produces feed directly into the migration scripts in pipeline-orchestrator.md.
Copilot performs the migration directly from the terminal. No Python files to save or run manually.
Ask the user for:
* for all_migrated) — leave blank to keep the original nameEverything else is auto-discovered:
az account show + az synapse workspace showGET /v1/workspaces filtered by display nameGET /v1/workspaces/{wsId}/notebooks (required for SynapseNotebook → TridentNotebook)GET /v1/connections (when datasets reference linked services)What is fully automated inline:
SynapseNotebook → TridentNotebook — type rename, GUID lookup, remove sparkPool/sessionConfiguration, fix timeout to 12h maxtypeProperties@pipeline().libraryVariables.<name>Copilot checks before starting — will pause and report if:
SynapseNotebook activity does not exist in the Fabric workspace yetLoad pipeline-orchestrator.md for the complete inline runner.
| Phase | Source | Target | Resource |
|---|---|---|---|
| Phase 0 | Synapse notebooks (referenced by pipeline activities) | Fabric Notebooks | synapse-migration skill |
| Phase 1 | Synapse global parameters | Fabric Variable Library | global-parameters-to-variable-library.md |
| Phase 2 | Synapse linked services | Fabric Connections | linked-service-to-connection.md |
| Phase 3 | Synapse datasets | Inlined into activities | dataset-inlining.md |
| Phase 4 | Synapse pipeline activities | Fabric pipeline activities | activity-mapping.md + notebook-activity-migration.md |
| Phase 5 | Assembled pipeline JSON | Fabric DataPipeline item | pipeline-orchestrator.md |
| Final | — | Validation | validation-testing.md |
Phase 0 must precede Phase 4: Fabric Notebook GUIDs are needed before TridentNotebook activities can be written. Phase 2 must precede Phase 3: Connection names must exist before they can be referenced in inlined datasets.
Full mapping table, before/after examples, and parking decisions are in activity-mapping.md.
| Synapse Activity | Fabric Equivalent | Status |
|---|---|---|
SynapseNotebook | TridentNotebook | ✅ Migrated — see notebook-activity-migration.md |
Copy | Copy | ✅ Migrated — datasets inlined |
Lookup | Lookup | ✅ Migrated — datasets inlined |
GetMetadata | GetMetadata | ✅ Migrated — datasets inlined |
Validation | GetMetadata + IfCondition | ✅ Migrated — split into 2 activities |
ForEach | ForEach | ✅ Compatible |
IfCondition | IfCondition | ✅ Compatible |
Switch | Switch | ✅ Compatible |
Until | Until | ✅ Compatible |
Wait | Wait | ✅ Compatible |
Fail | Fail | ✅ Compatible |
SetVariable | SetVariable | ✅ Compatible |
AppendVariable | AppendVariable | ✅ Compatible |
ExecutePipeline | ExecutePipeline | ✅ Compatible — add workspaceId |
WebActivity | WebActivity | ✅ Compatible |
Script | Script | ✅ Compatible — update connection refs |
Delete | Delete | ✅ Migrated — datasets inlined |
Filter | Filter | ✅ Compatible |
SparkJobDefinition (Synapse SJD) | SparkJobDefinition | ✅ Update GUID refs |
HDInsightSpark | TridentNotebook or SparkJobDefinition | ⚠️ Rewrite required |
AzureMLBatchExecution | WebActivity | ⚠️ Rewrite as REST call |
AzureFunctionActivity | WebActivity | ⚠️ Rewrite — use function URL + key |
DatabricksNotebook | ⛔ Parked | See pipeline-gotchas.md |
DatabricksSparkJar | ⛔ Parked | See pipeline-gotchas.md |
DatabricksSparkPython | ⛔ Parked | See pipeline-gotchas.md |
ExecuteSSISPackage | ⛔ Parked | See pipeline-gotchas.md |
AzureBatch | ⛔ Parked | No Fabric equivalent |
Custom | ⛔ Parked | No Fabric equivalent |
inputs/outputs dataset properties must be embedded in each activity@pipeline().globalParameters.<name> with @pipeline().libraryVariables.<name> after creating the Variable LibraryGetMetadata + IfCondition pair — Validation does not exist as an activity type in FabricsparkPool and sessionConfiguration from migrated notebook activities — pool selection and session config belong in the Fabric Environment attached to the notebook@pipeline().libraryVariables.<name> with environment-specific Value Sets instead of pipeline-level parameters for environment promotionnotebookParameters over hardcoded values in TridentNotebook activities — mirrors Synapse parameterized notebook patternTridentNotebook activities — use the GUID from the Fabric workspace notebook list@pipeline().globalParameters syntax after migration — this expression path does not exist in Fabric; all migrated global parameters must be accessed via @pipeline().libraryVariablesFull troubleshooting guide is in pipeline-gotchas.md.
| # | Flag ID | Issue | Severity | Resolution Summary |
|---|---|---|---|---|
| PG1 | NOTEBOOK_GUID_NOT_FOUND | Notebook not yet migrated to Fabric when building TridentNotebook activity | High | Run synapse-migration skill first; get GUID from GET /v1/workspaces/{wsId}/notebooks |
| PG2 | DATASET_NOT_INLINED | Activity still references a named dataset (not valid in Fabric) | High | Apply dataset-inlining.md patterns to embed dataset properties into activity source/sink |
| PG3 | GLOBAL_PARAM_EXPRESSION | @pipeline().globalParameters.<name> expression left in migrated pipeline | High | Replace with @pipeline().libraryVariables.<name> after creating Variable Library |
| PG4 | VALIDATION_ACTIVITY_UNSUPPORTED | Validation activity type left in pipeline JSON | High | Rewrite as GetMetadata + IfCondition — see activity-mapping.md |
| PG5 | SHIR_CONNECTOR_PARKED | Activity uses a linked service backed by a Self-Hosted Integration Runtime | Medium | Must set up on-premises data gateway in Fabric; see pipeline-gotchas.md |
| PG6 | SSIS_ACTIVITY_PARKED | ExecuteSSISPackage activity cannot be migrated | High | Parked — no Fabric equivalent; see pipeline-gotchas.md for alternatives |
| PG7 | DATABRICKS_ACTIVITY_PARKED | Databricks activity type has no Fabric native equivalent | High | Parked — use Databricks REST API via WebActivity as workaround; see pipeline-gotchas.md |
| PG8 | SPARKPOOL_REF_ORPHANED | sparkPool / targetBigDataPool reference left in TridentNotebook activity | Medium | Remove sparkPool and sessionConfiguration blocks; pool config belongs in Fabric Environment |
| PG9 | EXECUTE_PIPELINE_NO_WORKSPACE | ExecutePipeline activity missing workspaceId for referenced pipeline | Medium | Add workspaceId to typeProperties; required even for same-workspace child pipelines — omitting it causes runtime failures |
| PG10 | LINKED_SERVICE_NO_CONNECTION | Linked service has no matching Fabric Connection | High | Create connection manually or via API; update connection reference in inlined dataset |
After pipeline migration, hand off to these companion skills and tools:
| Task | Skill / Tool |
|---|---|
| Migrate notebook content (mssparkutils → notebookutils, linked services) | synapse-migration skill |
| Schedule migrated pipelines | COMMON-CLI.md § Job Scheduling |
| Monitor pipeline runs | Fabric workspace → Monitor hub |
| Build new Fabric pipelines | Refer to ITEM-DEFINITIONS-CORE.md § DataPipeline |
| Explore migrated Lakehouse data post-pipeline run | spark-cli or sqldw-cli skill |
SynapseNotebook → TridentNotebook activity (before/after)
{
"name": "Run_Notebook",
"type": "SynapseNotebook",
"typeProperties": {
"notebook": {"referenceName": "MyNotebook", "type": "NotebookReference"},
"sparkPool": {"referenceName": "BigPool", "type": "BigDataPoolReference"}
}
}
After migration (GUID from GET /v1/workspaces/{wsId}/notebooks):
{
"name": "Run_Notebook",
"type": "TridentNotebook",
"typeProperties": {
"notebookId": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
"workspaceId": "yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy"
}
}
Global parameter expression rewrite
Before (Synapse): @pipeline().globalParameters.batchDate
After (Fabric): @pipeline().libraryVariables.batchDate
Dataset inlining: Copy activity — dataset reference → connection display name
Synapse dataset InputBlobDataset (will be inlined and removed):
{
"type": "AzureBlob",
"linkedServiceName": {"referenceName": "AzureBlobLinkedService"},
"typeProperties": {"folderPath": "input/", "fileName": "data.csv"}
}
Synapse Copy activity (before — references dataset by name):
{
"name": "CopyData", "type": "Copy",
"inputs": [{"referenceName": "InputBlobDataset", "type": "DatasetReference"}],
"typeProperties": {"source": {"type": "BlobSource"}, "sink": {"type": "BlobSink"}}
}
After migration (connection display name for AzureBlobLinkedService is My ADLS Connection):
{
"name": "CopyData", "type": "Copy",
"typeProperties": {
"source": {"type": "BlobSource", "storeSettings": {"type": "AzureBlobStorageReadSettings"}},
"sink": {"type": "BlobSink", "storeSettings": {"type": "AzureBlobStorageWriteSettings"}}
},
"linkedService": {"referenceName": "My ADLS Connection", "type": "LinkedServiceReference"}
}
See activity-mapping.md and notebook-activity-migration.md for full before/after examples.