![]() Navigate to workspace “Data Science and Engineering” and select the compute which you have been using for data transformation. ![]() Once you login to your account, you will notice the Unified environment for different workspaces Or you can sign up from the hyperscaler marketplace such as AWS marketplace for the same. ![]() If you are interested in getting access, you can sign up for the free trial for 14 days. You have downloaded the JDBC driver from Databricks website.Īs mentioned in the previous section, I assume you are already working on Databricks topics.You have access to Databricks Clusters as well as SQL warehouse.Access to Virtual Machine or On-Premise system where you install Data provisioning agent.You have access to SAP Datasphere with authorization to create a new Data provisioning agent.Then we utilize the delta live table framework for building data pipelines and storing the transformed data in Delta format on cloud storage, which can subsequently be accessed by Databricks SQL(DB SQL).Īs referred to in the integration scenario below, SAP Datasphere will connect to Databricks SQL with the existing data federation capabilities and users can blend the data with SAP sources for reporting/BI workloads based on SAP Analytics Cloud(SAC).Īssuming you process the incoming data and persist as tables in Databricks SQL, you will then perform the following steps to establish connectivity to SAP Datasphere Data Federation with Databricks SQL Prerequisites Note that Databricks has an autoloader feature to efficiently process data files from different cloud storages as they arrive and ingest them into Lakehouse seamlessly. We will discuss the Lakehouse platform features and capabilities in future blogs but as mentioned before we are going to focus on a data federation scenario to access data from Databricks SQL into SAP Datasphere.Ĭonsider a scenario where the data from a non-SAP source is continuously ingested into cloud object storage say AWS S3. And as mentioned on their site, the platform is simple, open & multi-cloud. Operating on multi-cloud environments and many more.Īnd Databricks is one such Lakehouse platform that takes a unified approach by integrating disparate workloads to execute data engineering, Analytics, Data Science & Machine Learning use cases.Support for open source & commercial tooling.Simplified security with a single source of truth.Enables Direct Data Access across SAP and Non-SAP sources.And by utilizing a combined data management platform such as lakehouse has the following benefits In simple terms, a lakehouse is a Data Management architecture that enables users to perform diverse workloads such as BI, SQL Analytics, Data Science & Machine Learning on a unified platform. Brief Introduction to the Lakehouse Platform There will be additional ways of integrating with Databricks in the future. Please note that what I am about to discuss further is the data federation scenarios between SAP Datasphere and Databricks that works as of today. ![]() There was an article posted by The Register regarding the SAP Datasphere and it exactly resonated with the SAP messaging But the current partnership with Databricks will focus to simplify and integrate the hybrid landscapes efficiently. Previously, there is a need to replicate the data completely out of SAP environments for customers to adopt the lakehouse platform. With the rise of the Lakehouse platform that combines both Data Warehouses & Data Lakes, there has been a trend with SAP customers exploring Unified Analytics Platforms or say unified environments that address different perspectives of data management, governance, development, and finally deployments of analytic workloads based on diverse data sources and formats. However, this blog focuses on the latest announcement related to open data partners and I am going to start by focusing on Databricks. I am sure there will be lots of blogs that will be published soon discussing the latest offerings, roadmaps, and features. Our Datasphere has been enriched with new features thereby delivering a unified service for data integration, cataloging, semantic modeling, data warehousing, and virtualizing workloads across SAP and non-SAP data. With the announcements in the SAP Data Unleashed, SAP introduced the successor of SAP Data Warehouse Cloud, SAP Datasphere – a powerful Business Technology Platform data service that addresses the Data to value strategy of every Enterprise organization to deliver seamless access to business data. ![]()
0 Comments
Leave a Reply. |
Details
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |