You must forgive me for this seemingly pseudo technical question. Last week I came across a client manager who mentioned developing 4th generation data warehousing with Spark. And I was wondering whether the individual pointedly made a reference to the new data lakehouse concept and how it was different with the current concept of Real time data pipeline, Batch data pipeline, Lambda Architecture or just plain Data enrichment. Spark can be used for all these. Can anyone throw some light on the notion of 4th Generation Data Warehousing with Spark? The D in Spark RDD for Dataset can handle structured, semi-structured and equally unstructured data so what is new?
Thanks Mich view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction.
