The Engineering Side of Data

Alex Debrie and Bob Haffner recap their favorite announcements from 2023 re:Invent

#data #dataengineering #aws

Connect with Alex Twitter: @alexbdebrie Blog: alexdebrie.com Book: dynamodbbook.com Podcast: youtube.com/@SoftwareHuddle

Connect with Bob Twitter - @bobhaffner LinkedIn - linkedin.com/in/bobhaffner

Alex’s talk https://www.youtube.com/watch?v=PVUofrFiS_A

Juan Pablo (JP) Urrutia and Bob Haffner discuss Data Quality #data #dataengineering #dataquality

Connect with JP

Twitter - @the_datachef Linkedin - https://www.linkedin.com/in/jpurrutia

Substack - https://substack.com/@thedatachef

Connect with Bob

Twitter - @bobhaffner

LinkedIn - https://www.linkedin.com/in/bobhaffner

Follow the show on Twitter @EngSideOfData

Data quality camp slack community:https://join.slack.com/t/dataqualitycamp/shared_invite/zt-2452i21wm-iZjqABfR7Hr8gcsshYuh2A

Data Quality Fundamentals Book

https://www.oreilly.com/library/view/data-quality-fundamentals/9781098112035/

Arjun Bansil and Bob Haffner chat about Ajun's start in Data Engineering #data #dataengineering Connect with Arjun Linkedin - https://www.linkedin.com/in/arjun-bansil/ Connect with Bob Twitter - @bobhaffner LinkedIn - https://www.linkedin.com/in/bobhaffner

Follow the show on Twitter

@EngSideOfData

Joseph Machado and Bob Haffner discuss the many aspects of developer ergonomics and how they relate to data engineering.  

#data #dataengineering 

Connect with Joseph 

Blog - https://www.startdataengineering.com/

Linkedin - https://www.linkedin.com/in/josephmachado1991/ 

Connect with Bob

Twitter - @bobhaffner

LinkedIn - https://www.linkedin.com/in/bobhaffner

Alex Debrie and Bob Haffner recap their favorite announcements from 2022 re:Invent  

#data #dataengineering #aws   

Connect with Alex

Twitter: @alexbdebrie 

Blog: alexdebrie.com 

Book: dynamodbbook.com  

Connect with Bob 

Twitter - @bobhaffner 

LinkedIn - linkedin.com/in/bobhaffner   

Top Announcements 

https://aws.amazon.com/blogs/aws/top-announcements-of-aws-reinvent-2022/  

Yan Cui’s Choreography vs Orchestration blog post 

https://theburningmonk.com/2020/08/choreography-vs-orchestration-in-the-land-of-serverless/

Join Alex Merced and Bob Haffner for a discussion about the Open Data Lakehouse concept  

#data #dataengineering #datalake #datalakehouse   

Connect with Alex 

Twitter - @amdatalakehouse  

Connect with Bob 

Twitter - @bobhaffner 

LinkedIn - linkedin.com/in/bobhaffner   

Show notes  

The DataNation Podcast  

Available on iTunes/Spotify/Stitcher  

The Subsurface Data Lakehouse Community 

dremio.com/subsurface  

Dremio 

dremio.com  

Follow the podcast on Twitter @EngSideOfData

Sven Balnojan and Bob Haffner discuss the goals and principles of Functional Data Engineering  

#data #dataengineering #functionaldataengineering  

Connect with Sven 

Twitter - @sbalnojan 

LinkedIn - linkedin.com/in/dr-sven-balnojan-a55b4072 

Blog - https://medium.com/three-data-point-thursday  

Connect with Bob 

Twitter - @bobhaffner 

LinkedIn - linkedin.com/in/bobhaffner  

Follow the podcast on Twitter @EngSideOfData  

Show Notes  

https://github.com/sbalnojan/easy-functional-data-engineering  

http://svenbalnojan.com/  

https://maximebeauchemin.medium.com/functional-data-engineering-a-modern-paradigm-for-batch-data-processing-2327ec32c42a  

Infrastructure as Code by Keif Morris 

https://infrastructure-as-code.com/book/

Michael Meyer and Bob Haffner chat about data catalogs  

#data #dataengineering #datacatalogs 

Connect with Mike 

Twitter - @dataguyatheart 

LinkedIn - linkedin.com/in/michael-meyer-6972286  

Connect with Bob 

Twitter - @bobhaffner 

LinkedIn - linkedin.com/in/bobhaffner  

Follow the podcast on Twitter @EngSideOfData

Johnny and Bob discuss Glue the serverless ETL tool from AWS  

#dataengineering #etl #aws   

Connect with Johnny  

Youtube Channel 

https://www.youtube.com/c/JohnnyChivers  

The QuestionBank - Completely free Bank of community Questions AWS Certifications developed on AWS Amplify 

https://www.thequestionbank.io/  

Personal Website for Contact, forum and free AWS learning resource 

https://johnnychivers.co.uk/  

Connect with Bob  

Twitter - @bobhaffner 

LinkedIn - linkedin.com/in/bobhaffner

https://twitter.com/EngSideOfData

Mário Pereira and Bob Haffner chat about Data Engineering at the Edge  

#data #dataengineering #iot #edge #edgecomputing   

Connect with Mário

LinkedIn - linkedin.com/in/xmariopereira/  

Connect with Bob Twitter - @bobhaffner 

LinkedIn - linkedin.com/in/bobhaffner  

Vopak is hiring (https://www.werkenbijvopak.nl/vacatures?lastchecked=discipline&discipline=9)

Adi Polak and Bob Haffner host a Twitter Spaces conversation on Careers and Community in Data Engineering.  

#data #dataengineering 

Connect with Adi 

Twitter - @AdiPolak  

Connect with Bob 

Twitter - @bobhaffner 

LinkedIn - linkedin.com/in/bobhaffner  

Join the Community - https://lakefs.io/community/

Cindi, Mike and Bob are back with the sequel to their 2021 Data Vault conversation.  

Check out the upcoming World Wide Data Vault Consortium! 

https://wwdvc.com/ 

Part 1 of this discussion

https://www.youtube.com/watch?v=4U9HcFS93tM

Connect with Cindi 

Twitter - @Data_Rebels  

Connect with Mike

Twitter - @dataguyatheart 

LinkedIn - linkedin.com/in/michael-meyer-6972286  

Connect with Bob 

Twitter - @bobhaffner 

LinkedIn - linkedin.com/in/bobhaffner  

Show notes  

Learn More: https://www.datavaultalliance.com - THE Standards Board for DV2 and Certification Education  

https://www.datarebels.com - Authorized Training Partner, CDVP2 Certified Instruction, and Consulting Services  

https://www.danLinstedt.com - Extensive Blogs by Daniel Linstedt, Data Vault Inventor and Found  

https://kentgraziano.com/ - Kent Graziano's Data Warrior Blogs  

https://twitter.com/dlinstedt 

Participation and Getting Involved: 

https://www.wwdvc.com - Annual Data Vault Conference 

https://www.meetup.com/data-vault-north-american-user-group - Data Vault North American User' Group 

https://www.linkedin.com/groups/9050802/ - Data Vault North American Users' Group LinkedIn Profile  

Certified DV2 Training and Education: 

https://datarebels.com/get-schooled/ 

https://datavaultalliance.com/certification/authorized-trainers/ 

https://datavaultalliance.com/event-directory/ 

https://www.arihovi.com/en/courses/data-vault-2-0-bootcamp-certification/ 

Other Resources: 

Building a Scalable Data Warehouse with Data Vault 2.0 Published by Elsevier Paperback ISBN: 9780128025109 eBook ISBN: 9780128026489 

Amazon:  https://www.amazon.com/Building-Scalable-Data-Warehouse-Vault/dp/0128025107/ref=sr_1_1?crid=335M6BS7BRLQA&dchild=1&keywords=building+a+scalable+data+warehouse+with+data+vault+2.0&qid=1633796921&sprefix=building+a+scala%2Caps%2C148&sr=8-1

#data. #dataengineering #datavault

Brian McMillan and Bob Haffner talk about Data Engineering for Data Discovery  

#data #dataengineering #elt #etl #analysticsengineering 

Connect with Brian 

Like to discuss this further? First 5 people to schedule a meeting with me and receive a free copy of the book (https://calendly.com/building-data-products).  

Purchase the book: https://www.minimumviablearchitecture.com   

Brian McMillan – Brian [at] minimumviablearchitecture.com 

Twitter – @brianmcmillan01 

LinkedIn – linkedin.com/in/brianmcmillan01 

GitHub – https://github.com/brianmcmillan/intro_to_data_and_analytics_engineering

Schedule a discussion – https://www.minimumviablearchitecture.com/contact.html 

The Book: Building Data Products: Introduction to Data and Analytics Engineering for non-programmers – https://www.minimumviablearchitecture.com  

Full Length Demo - https://www.youtube.com/watch?v=xLjQhz1KJoQ&t=4s 

Connect with Bob 

Twitter - @bobhaffner 

LinkedIn - linkedin.com/in/bobhaffner   

Links: 3X - Kent Beck – https://www.youtube.com/watch?v=FlJN6_4yI2A

Why Use Make (Mike Bostock) – https://bost.ocks.org/mike/make/

SQLite – https://www.sqlite.org/index.htm, https://blog.wesleyac.com/posts/consider-sqlite

Building Production Applications Using Go & SQLite – https://www.youtube.com/watch?v=XcAYkriuQ1o) 

CSVkit – https://csvkit.readthedocs.io/en/latest/

SQLite-Utils – https://sqlite-utils.datasette.io/en/stable/

Datasette – https://datasette.io

Vega Lite –https://vega.github.io/vega-lite/

Dan Silberman and Bob Haffner discuss the traditional approach of Extract, Transform and Load (ETL) vs Extract, Load and Transform (ELT).   What each one means, why has ELT become so popular and how ELT will evolve.

#data #dataengineering #elt #etl #analysticsengineering 

Check out Mozart Data 

https://www.mozartdata.com/ 

Connect with Bob

Twitter - @bobhaffner 

LinkedIn - linkedin.com/in/bobhaffner

Alex DeBrie joins Bob Haffner to recap the 2021 re:Invent from the data perspective  

#data #dataengineering 

Connect with Alex 

Twitter: @alexbdebrie 

Blog - alexdebrie.com 

Book - dynamodbbook.com   

Connect with Bob 

Twitter - @bobhaffner 

LinkedIn - linkedin.com/in/bobhaffner

Arjun Narayan and Bob Haffner discuss the state of streaming.  

Connect with Arjun

Twitter - @narayanarjun 

LinkedIn - linkedin.com/in/arjunravinarayan/  

Connect with Bob 

Twitter - @bobhaffner 

LinkedIn - linkedin.com/in/bobhaffner  

Show Notes 

Materialize website - materialize.com 

Materialize Twitter - @MaterializeInc 

Materialize LinkedIn - https://www.linkedin.com/company/materializeinc

Kafka is not a database blog post - https://materialize.com/kafka-is-not-a-database/

Streaming SQL blog post - https://materialize.com/streaming-sql-intro/ 

#data #dataengineering #streaming

Cindi Meyersohn joins Michael Meyer and Bob Haffner for a discussion about Data Vault.    

Connect with Cindi 

Twitter - @Data_Rebels  

Connect with Mike 

Twitter - @dataguyatheart 

LinkedIn - linkedin.com/in/michael-meyer-6972286  

Connect with Bob 

Twitter - @bobhaffner 

LinkedIn - linkedin.com/in/bobhaffner   

Show notes 

 Learn More: 

https://www.datavaultalliance.com - THE Standards Board for DV2 and Certification Education  

https://www.datarebels.com - Authorized Training Partner, CDVP2 Certified Instruction, and Consulting Services  

https://www.danLinstedt.com - Extensive Blogs by Daniel Linstedt, Data Vault Inventor and Found  

https://kentgraziano.com/ - Kent Graziano's Data Warrior Blogs  

https://twitter.com/dlinstedt  

Participation and Getting Involved: 

https://www.wwdvc.com - Annual Data Vault Conference 

https://www.meetup.com/data-vault-north-american-user-group - Data Vault North American User' Group 

https://www.linkedin.com/groups/9050802/ - Data Vault North American Users' Group LinkedIn Profile  

Certified DV2 Training and Education: 

https://datarebels.com/get-schooled/

https://datavaultalliance.com/certification/authorized-trainers/ 

https://datavaultalliance.com/event-directory/ 

https://www.arihovi.com/en/courses/data-vault-2-0-bootcamp-certification/ 

Other Resources: 

Building a Scalable Data Warehouse with Data Vault 2.0 Published by Elsevier Paperback ISBN: 9780128025109 eBook ISBN: 9780128026489 

Amazon: 

 https://www.amazon.com/Building-Scalable-Data-Warehouse-Vault/dp/0128025107/ref=sr_1_1?crid=335M6BS7BRLQA&dchild=1&keywords=building+a+scalable+data+warehouse+with+data+vault+2.0&qid=1633796921&sprefix=building+a+scala%2Caps%2C148&sr=8-1

Caleb Welchans and Bob Haffner discuss GIS.  GIS is a topic that everyone in data should be familiar with.    

Connect with Caleb 

Twitter - @calebwelchans 

LinkedIn - linkedin.com/in/welchans       

Intro/Outro music by John Yasut  

#data #dataengineering #gis 

Connect with Bob 

Twitter - @bobhaffner 

LinkedIn - linkedin.com/in/bobhaffner  

Please Like, Subscribe and Comment!

A podcast about Data Engineering and the edges of this critical and ever-changing domain

Intro/Outro by John Yasut