Microsoft at its Build 2024 developer conference today announced several updates to Fabric, the company’s cloud-based suite of tools for data analytics. The additions and enhancements include a new Real-Time Intelligence module, a tool kit for customizing Fabric workflows, and the general availability of Copilot for Power BI.
Other Copilots for Fabric remain in preview.
Fabric, which was released last year in May, combines the company’s existing data warehousing, business intelligence, and data analytics products into a single offering in order to help enterprises combine workloads while reducing IT integration overhead, complexity, and costs.
The cloud-based offering, which provides a software-as-a-service (SaaS) experience to developers to help in extracting insights from raw data and present it to business users, comes with seven core modules and tool sets including data connectors, data engineering tools, data workflows for data science, and analytics tools.
The Fabric modules include Data Factory, Synapse Data Engineering, Synapse Data Science, Synapse Data Warehouse, Synapse Real-Time Analytics, Power BI, and Data Activator—all built atop the company’s data lake offering, OneLake.
Synapse Real-Time Analytics combined with Data Activator
As part of the updates to Fabric, Microsoft has combined two of its modules—Synapse Real-Time Analytics and Data Activator—to launch a new module or workload, dubbed Real-Time Intelligence, available in preview.
Real-Time Intelligence combines the analytics and activator workloads and offers additional features, such as a low-code interface, to help enterprises generate insights from real-time data, Microsoft said.
“With Real-Time Intelligence, enterprise users can ingest streaming data with high granularity, dynamically transform streaming data, query data in real time for instant insights, and trigger actions like alerting a production manager when equipment is overheating or rerunning jobs when data pipelines fail,” the company explained.
The new module is underpinned by the Real-Time Hub—a window to discover, manage, and use event streaming data from Fabric and other data sources.
“From the Real-Time Hub, users can gain insights through the data profile, configure the right level of endorsement, set alerts on changing conditions, and more, all without leaving the hub,” the company said.
Fabric Workload Development Kit debuts
Microsoft has also introduced the Fabric Workload Development Kit, a tool kit designed to help developers build interoperable applications within Fabric.
“Applications built with this kit will appear as a native workload within Fabric, providing a consistent experience for users directly in their Fabric environment without any manual effort,” Microsoft said, adding that developers can publish and monetize these workloads via the Azure Marketplace.
Further, Microsoft said that it was working on creating a workload hub experience in Fabric where enterprise users can discover, add, and manage these workloads without leaving Fabric.
Neo4j has already used the Fabric Workload Development Kit to make its offering available inside Fabric.
Microsoft is adding two other features for developers in preview—the API for GraphQL and user data functions.
“API for GraphQL is a flexible and powerful RESTful API that allows data professionals to access data from multiple sources in Fabric with a single query API,” the company said, adding that the API can help enterprises reduce network overhead and accelerate response rates.
User data functions for developers, according to Microsoft, are user-defined functions built for Microsoft Fabric experiences across all data services, such as notebooks, pipelines, and event streams, to help developers build applications easily using varied data sources, such as lakehouses, data warehouses, and mirrored databases among others, using the native code ability and custom logic.
Microsoft has also updated the Data Factory module within Fabric with a feature called data workflows. The new feature, now in preview, allows enterprises to define directed acyclic graphs (DAG) for complex data workflow orchestration within Fabric.
“Data workflows is powered by Apache Airflow and designed to help enterprises author, schedule, and monitor workflows or data pipelines using Python,” Microsoft said.
Copilot for Power BI turns GA
Microsoft on Tuesday said that the first Copilot for Fabric has been made generally available. While launching Fabric last year, the company said it would be integrating its generative AI-powered Copilot to its unified analytics platform.
The Copilot now can be accessed via the Power BI module inside Fabric, Microsoft said, adding that the Copilots inside the Data Factory, Synapse Data Engineering, Synapse Data Science, and Synapse Data Warehouse modules were still in preview.
The Copilot inside the Power BI experience can be used to generate reports and summarize them. Moreover, the Copilots in other experiences within Fabric can be used to create data flows, generate code, and build machine learning models using natural language.
In addition, Microsoft said that it was announcing another new Copilot for its Real-Time Intelligence module in preview that would enable enterprise users to query real-time data.
New data sources and Snowflake, Databricks integrations
Some of the other updates to Fabric include the addition of OneLake shortcuts to connect to data sources beyond just Azure Data Lake Service Gen2. These OneLake shortcuts are in preview.
“With an on-premises data gateway, you can now create shortcuts to Google Cloud Storage, Amazon S3, and S3-compatible storage buckets that are either on-premises or otherwise network-restricted,” Microsoft said.
Further, Microsoft said that it was partnering with Snowflake to create full interoperability between Snowflake and OneLake.
“We are excited to announce future support for Apache Iceberg in Fabric OneLake and bi-directional data access between Snowflake and Fabric,” Microsoft said. The integration will enable users to analyze their Fabric and Snowflake data in any engine within either platform, Microsoft said, and to access data across apps like Microsoft 365, Power Platform, and Azure AI Studio.
The shortcuts for Iceberg in OneLake will be made available soon, Arun Ulagaratchagan, corporate vice president of Azure Data, said in a statement.
Additionally, Microsoft said that enterprise users will be able to access Azure Databricks Unity Catalog tables directly in Microsoft Fabric in the coming months.
“From the Fabric portal, users can create and configure a new Azure Databricks Unity Catalog item in Fabric with just a few clicks. They can add a full catalog, a schema, or even individual tables,” the company said.
When the data is modified or tables are added, removed, or renamed in Azure Databricks, the data in Fabric will remain in sync, Ulagaratchagan added.
In addition, according to Microsoft, enterprise users soon will be able to access Fabric data items like lakehouses as a catalog in Azure Databricks.
“While the data remains in OneLake, you can access and view data lineage and other metadata in Azure Databricks and leverage the full power of Unity Catalog,” Ulagaratchagan said. This functionality will include the ability to extend Unity Catalog’s unified governance over data and AI into Azure Databricks Mosaic AI.
This means that enterprise users will be able to combine this data with other native and federated data in Azure Databricks, perform analysis, and publish the aggregated data back to a Power BI workspace.