Well Architected AI-Powered Lakehouse

-

 

Well Architected AI-Powered Lakehouse






Q&A

Cloud Data Lake Vs Delta lake vs Lakehouse , what is the difference between them

Lakehouse is a platform where you get both benefits of Data warehousing and Data Lake, so you can process any data at any scale, which support bot h batch and streaming

The AI engine refers here is Mosaic AI?

AI engine refers to the AI capablities on Databricks for all your AI needs, including the underlying optimisation by AI to process your data and build AI products

Can you please suggest the material to learn deep on Unity Catalog and Governance and how to enable policies?

try these links to start with: https://www.databricks.com/blog/databricks-databricks-kicking-journey-governance-unity-catalog https://community.databricks.com/t5/technical-blog/understanding-unity-catalog/ba-p/93478

Can you please provide some examples of open interfaces and open formats?

Please refer to this. https://www.databricks.com/blog/2022/06/30/open-sourcing-all-of-delta-lake.html

What kind of return on investment should you expect if you invest in a Databricks?

This is a good question. Please contact your databricks account team and they will provide a POV (proof of value) for your platform and use cases

we are using fivetran for data injection, whats the better option to tanfer and aggregate to silver and gold layer? can we use fivetran or DLT or Spark notebooks ? pls advise

Fivetran is for ingestion, for sliver and gold layer you could simply use databricks workflows, and the databricks workflow will support DLT, notebook and even SQL scripts for your transformation and aggregation.

Difference between Databricks SQL and Serverless SQL?

Databricks SQL include Serverless SQL, which is one of SKU within Databricks SQL, you also for SQL Pro and SQL Classic, but serverless SQL is much more efficient and provide better user experiences

What's a good DQ tool to be able to address underlying data quality issues in the source data that's compatible with databricks

you could use DLT expectations to ensure data quality during loading, or else you could Data Quality metrics provided by Unity Catalog

are databricks only support delta lake?

Databricks also support Iceberg, we could processing any data, e.g. Parquet, CSV, flat flies, JSON, etc.

What is a Phtoton engine?

Please refer to this: https://www.databricks.com/product/photon

Do we have any option of using Presto or Trino in Databricks platform?

Please refer to this https://www.youtube.com/watch?v=-0FtEHGfZTI

if we have any Handson tutorials on these topic much appreciate

we could provide that in the future stage, please refer to https://docs.databricks.com/aws/en/lakehouse-architecture/

Can you elaborate more on lakehousw

Please refer to this https://www.databricks.com/glossary/data-lakehouse

Can you elaborate more on Lakehouse Monitoring and quality checks?

Please refer to this: https://docs.databricks.com/aws/en/lakehouse-monitoring/

are databricks can use other data ingestion tools like airbyte?

Please refer to this: https://www.databricks.com/blog/2022/07/11/using-airbyte-for-unified-data-integration-into-databricks.html

Can you provide access to the assessment and review playbooks on slide 26?

we could share this information to you later

How dose databricks ai assistant falls under Data Governance ?

In the context of Data Governance, AI assistant could help to add comments and generate metadata for your data for easier discovery

does lakeshouse support tag/classification based application securityy policies?

In the Lakehouse, all the components support tags, such as computing resources, pipelines, jobs, tables, etc.

How you measure the best practices.. i.e where these numbes comes from 4, 6, .... ?

In each pillar, you will have principles, the number of principles that are used indicate the numbers

is iceberg table supported in databricks

yes, it's supported as managed iceberg and it's performance and reliable

pls share the link for Assessment and Review playbooks?

we will share that later

The presenter voice is breaking

could you pls refresh and see if it works

Are reviews and assessments always conducted by Databricks PS or can a customer do it themselves or along with a partner

It could be done by the partners as well

Comments

Post a Comment

Popular posts from this blog

Bookmark

-
Good Blog about Architecture & design Patterns:   http://martinfowler.com/bliki/ http://www.sharepoint-videos.com/ http://go.cloudshare.com/Linkedin_ProPlus_SharePoint.html A Technical Q/A site: http://www.bookmarkresource.com http://blog.krisvandermast.­ com/ASPNET20PageLifeCycle.aspx http://blogs.vertigosoftware.­ com/shannonf/archive/2005/08/­ 07/1355.aspx http://john-sheehan.com/blog/­ net-cheat-sheets/ http://www.eggheadcafe.com/­ articles/20051227.asp http://duartes.org/gustavo/­ articles/Asp.net-Runtime-­ Cheat-Sheet-HttpRequest-­ HttpRuntime.aspx http://www.addedbytes.com/­ cheat-sheets/javascript-cheat- ­sheet/ http://www.antsmagazine.com/­ 2009/04/100-useful-cheat-­ sheets-for-web-designers-­ developers/ Nice link abt .Net 3.5: http://www.simple-talk.com/ dotnet/.net-framework/.net-3. 5-language-enhancements/ nice articles about JQuery. http://encosia.com/2008/05/29/ using-jquery-to-directly-call- aspnet-ajax-page-methods/ http://encosi...
Read more

Cloud Computing in simple

-
Cloud Computing =   Infrastructure as a Service (IaaS) + Platform as a Service (PaaS) + Software as a Service (SaaS) Cloud computing, also known as Cloud Technology, Cloud Applications or Cloud-based services. Cloud Computing is the practice of using a network of remote servers hosted on the internet to store, manage and process data, rather than a local server or a personal computer. Examples of cloud-based services include customer self-scheduling, data storage, online payment, and accounting software Different Type of Clouds Infrastructure as a Service (IaaS)  – Allows users with the services like Virtual servers, Virtual storage, Physical Database, Virtual Network, Physical Network. E.g. Google Storage ,  IBM Cloud Computing ,  Nimbus Platform as a Service (PaaS)  – Provides computational resources via a platform upon which applications and services can be developed and hosted. E.g. Databases, Data Stores,  Force.com ,  Google App Engine ,...
Read more

Azure SQL Databases: Backups, Disaster Recovery, Import and Export

-
Image
Azure SQL Database is a managed cloud database-as-a-service. It provides application developers with SQL Server databases which are hosted in the cloud and fully managed by Microsoft. The service is very easy to start with. Several clicks in the portal and you have a database running. Now you can copy the connection string to your application config file, and boom - you have all up and running. No installation, no license to buy - just pay the hourly fee. Any production database is a very important asset, so we are used to give it a good care in self-hosted scenario. A number of questions appear when you try to apply those practices to the cloud offering: How do I make a backup of my database? Where should I store it? How do I move my database including schema and data from on-premise to the cloud? How do I move it from the cloud to my local server? What is a point-in-time restore offered by Azure? Should I use geo-replication? What is geo-restore? In this post I'll ...
Read more