Synapse has been a great tool so far for many companies. Like Fabric, it combines the best of the worlds into a single product: Azure Data Factory, SQL Data Warehouse, Spark Pools, Power Query, etc., in a single product for you to design both Warehouses and Lakehouses. You can have multiple personas use Synapse differently, and integrate with the rest of the world as you wish. So, it’s understandable that the companies have already invested a lot into Synapse Analytics.
We’ve discussed if you should migrate from Synapse to Fabric in our recent article, but I would like to discuss another option here: Start by using Fabric and Synapse together and see where it leads you. You can adopt both tools and position them strategically in your landscape: Synapse to make the data available and Fabric to gain insights. Synapse can handle the challenges of bringing the data, and Fabric can further refine it. Until Fabric receives many of the features that Synapse and other Azure components provide, you must supplement Fabric. Why not hit the ground running?
Also, both Synapse and Fabric can talk to each other’s data lakes, as they are both ADLS Gen2-based. Synapse can connect to OneLake, and Fabric can connect to ADLS Gen2 using Shortcuts. Let’s start from here.
For reference, let’s see what your Synapse analytics would look like right now and how it would evolve:
Step 1: Start By Utilising OneLake Shortcuts
OneLake’s silver bullet in quick adoption is the ability to link it to existing data lakes. You can use the OneLake Shortcuts and mount your existing lake folders into OneLake. That would make your specific data lake folder available as files or tables for your Workspace consumption. When you link an ADLS Gen2 path with a Shortcut, you can:
- Link it under Tables: OneLake would automatically resolve your Delta/Parquet files and make them available as Tables under your Lakehouse.
- Link it under Files: Anywhere on the OneLake, you can mount your ADLS Gen2 and start reading from your lake. The Tables won’t be automatically discovered, but you can run a Notebook to create tables that work through these folders.
Why is this so important in your adoption journey? Because Power BI now has Direct Lake capability, it can directly query the data on your OneLake. By creating Shortcuts, you can connect your Power BI queries directly to your Delta files/tables, eliminating a lot of boilerplate code that would convert the data into CSV/Parquet files and import it into a Power BI Dataset.
This way, you can immediately access your Silver and Gold layers on Fabric without needing to copy your data initially.
Step 2: Start Pointing At OneLake From Synapse
If you’d like a more powerful adoption of Fabric, you can start publishing your Gold datasets to OneLake instead of your ADLS Gen2. This would make the Gold data physically available in Fabric instead of Shortcuts, which is just a pointer. Strategically, you can have your Synapse be in charge of the Bronze and Silver layers, and Fabric owns the Gold layer. This is optional, depending on your desire to put Fabric in the critical path.
However, Fabric and OneLake currently don’t support authentication methods other than Azure AD (a.k.a. Microsoft Entra ID). If you have other apps or platforms depending on your Gold data through Shared Access Signature tokens, this will break your integrations. Refer to our OneLake vs. ADLS Gen2 article for a more detailed comparison.
Harun Legoz
You can do this part gradually and maintain the Shortcuts if you wish. Having a hybrid environment and progressively moving forward in adoption would make sense if you’re hesitant about Fabric.
To illustrate, this is how your landscape would look like in this state:
Step 3: Unleash Fabric's Full Power
After settling into your Fabric tenant and gaining some value, you might want to start using all of its capabilities. You don’t need to ditch your Synapse for this at all. You can have some pipelines in Synapse, some of them in Fabric.
You can create more Gold data from other Gold and Silver layer datasets or connect to third-party data sources using Dataflows Gen2 and process the data within Fabric. You can bring streaming data directly into your Fabric Workspace if you need to. Or, you can migrate your Synapse Data Warehouse into your Fabric Workspace.
The sky is the limit, depending on how much you would like to adopt Fabric. You can roll it out team by team or start new data products right on Fabric. You can go crazy with Power BI Datamarts or even implement ML Models right in the comfort of your Fabric Workspace.
Step 4: Team Fabric Forever
At this point, you should have enough know-how around Fabric and must’ve created an excellent operating model. You’re in a good position to adopt Fabric entirely, thinking it fits your strategic plan. It is a brave decision, and I wouldn’t recommend this with the current feature set of Fabric (after it goes GA, of course), but you can adopt Fabric entirely.
With some support from Azure tools like Data Share, Functions, APIM, etc., you might achieve an operating model around Fabric and make it a permanent home. You can move your data entirely onto Fabric and ditch Synapse Analytics & ADLS Gen2.
Conclusion
It would be best to wait to see Fabric and how Microsoft positions it in the market in the long run. You can find more details of my opinion on the Should You Migrate to Fabric? article, but a side-by-side run of Fabric and Synapse makes much more sense.
Harun Legoz
That way, you can use the right tool for the job on a case-by-case basis and have more options available to you in the future.
