The Data Life Podcast

We talk with Michel Tricot, who is the Founder and CEO of Airbyte, which is an open source data integration Y Combinator startup. It has raised over $30M in capital and has been growing quite fast. It was a great conversation and I think you will also enjoy it. 🎉

We cover lots of things in the podcast including: 

1. Technical aspects of what Airbyte does, how it sits in the ETL/ ELT landscape, how it differs from other tools such as Fivetran, Stich etc. 

2. Data Warehouses being a canonical source of data and how Airbyte helps with bringing the data into the warehouse. 

3. How Airbyte works as an open source data tool. 

4. Life aspects of running a fast growing start-up including raising capital, hiring etc. 

Links to the tools/ services mentioned: 

1. Airbyte: airbyte.io

2. Airbyte Slack where you can talk with the team: slack.airbyte.io 

3. Dbt for transformation in ELT: getdbt.com 

4. Airflow which is a data orchestration tool: https://airflow.apache.org/

5. Astronomer which can host Airflow: https://astronomer.io/ 

Pay as you use data warehouses: 

6. Snowflake Data Warehouse: https://www.snowflake.com/

7. BigQuery Data Warehouse: https://cloud.google.com/bigquery 

Set up your own infrastructure: 

8. Redshift Data Warehouse: https://aws.amazon.com/redshift/ 

Imagine you are at a beach and you are hanging out and seeing all the waves come and go and all the shells on the beach. And you get an idea. How about you collect these shells and make necklaces to sell? Well how would you go about doing this? Maybe you’d collect a few shells and make a small necklace and try to show to your friend. This is where we begin our journey on learning about data engineering pipelines. 

Using an example of running a necklace business from shells - we learn about the following data engineering concepts: 

1. ETL - Extract Transform Load vs ELT - Extract Load Transform concepts. Why Data Warehouses are great for analytics. 

2. Spark for large data processing and hosting / running

3. Data orchestration using Airflow

My blog on Towards Data Science about moving from Pandas to Spark: https://towardsdatascience.com/moving-from-pandas-to-spark-7b0b7d956adb 

Great book to learn about Spark: https://www.amazon.com/dp/1492050040/?tag=omnilence-20 

Tools covered in the episode: 

dbt: https://www.getdbt.com/ 

Databricks: https://databricks.com/

EMR: https://aws.amazon.com/emr/

AWS Redshift: https://aws.amazon.com/redshift/

Snowflake: https://www.snowflake.com/

Delta Lake: https://databricks.com/product/delta-lake-on-databricks 

In this episode, we are talking about women in tech with Rupal Gupta. Rupal, a recent graduate from Online MS in CS from Georgia Tech, is a data engineer in the industry and is passionate to help promote women in tech. She also has some great tips and resources for anyone trying to break into data science and tech! 

In this episode we talk about things that can help promote women in tech, women in tech conferences such as Grace Hopper, looking for jobs, resources to prepare for the interviews etc. 

If you want to reach out to Rupal for any help or to collaborate with her project womenmentors.co, here is her LinkedIn: https://www.linkedin.com/in/rupalgupta15/ 

FREE Women in Tech Conference by Manning Publications on Oct 13th at 12pm ET on Twitch: https://freecontent.manning.com/livemanning-conferences-women-in-tech/ 🎉 There will be women in tech speakers from Dropbox, Microsoft, Warby Parker and more.

🌟 Programs and conferences covered in the episode:
OMSCS program at Georgia Tech: https://omscs.gatech.edu/
Grace Hopper conference: https://ghc.anitab.org/
Anita Borg Institute: https://anitab.org/

🌟 Interviewing resources:
1. Pramp: https://www.pramp.com/#/
2. Interviewing.io: https://interviewing.io/
3. Educative "Grokking the System Design Interview": https://www.educative.io/courses/grokking-the-system-design-interview
4. AWS Certifications: https://aws.amazon.com/certification/

Disclaimer: All opinions on this podcast are our own and not the views of our employers or organizations.

~Thanks for listening~

As a data scientist, you will work on machine learning models that are deployed on websites - usually wrapped around a REST API, these days they also call this approach a “micro-service”. It is for this reason it is important to know how backends and front ends work and how to build them. In this episode, we talk about building a note app which is a Single Page Application or SPA using Pythons flask library for backend and Vue.js for frontend. We use REST API to communicate between them. 

We cover following topics in Q and A format:

1. Why should data scientists care about building frontend and backend and rest api? 

2. What is a single page application? 

3. Why Vue.js? 

4. Why do we need server side code? 

5. What is REST API?

6. How does Flask help with building rest api? 

Then we go into the exact mechanics of building the SPA:

Step 1: Database setup 

Step 2: Write REST API in flask 

Step 3: Postman setup and testing of the API

Step 4: Build frontend and write forms to get information 

Step 5: Build routing and login pages 

Step 6: Front end design and UI/UX 

Finally you can deploy both the server and client separately on AWS or Heroku so that other users can see it and use it. 

Dependencies: 

1) Flask to build server side REST APIs

2) Sqlalchemy which is ORM to access database 

3) Bcrypt for hashing user passwords to store in your database 

4) Vue for building frontend 

5) Bootstrap-Vue for using bootstrap with Vue.js

6) Axios to communicate via AJAX between client and server 

7) Vue CLI 3 to manage the tooling of the client 

Really awesome resources:

1) Learn Vue.JS from scratch by the awesome teacher Net Ninja - YouTube https://www.youtube.com/watch?v=5LYrN_cAJoA&list=PL4cUxeGkcC9gQcYgjhBoeQH7wiAyZNrYa&index=1

2) Building book recording app using Vue and Flask https://testdriven.io/blog/developing-a-single-page-app-with-flask-and-vuejs/#bootstrap-vue

3) Managing state in Vue.js including Vuex and simple global store: https://medium.com/fullstackio/managing-state-in-vue-js-23a0352b1c87

4) Authenticating a Flask API Using JSON Web Tokens - YouTube https://www.youtube.com/watch?v=J5bIPtEbS0Q

5) Really nice tutorial for using databases with Flask by Corey Schafer - YouTube https://www.youtube.com/watch?v=cYWiDiIUxQc&list=PL-osiE80TeTs4UjLw5MM6OjgkjFeUxCYH&index=4

If this has been of value please consider supporting me by buying me a coffee at the Anchor link at the end. If you support, I will provide extra bonus content for you. Thanks for listening!

Ever wonder how to automatically detect language from a script? How does Google do it? 

Ever wonder how Amazon knows whether you are searching for a product or a SKU on its search bar? 

We look into character-based text classifiers in this episode. We cover 2 types of models. First is the bag-of-words models such as Naive Bayes, logistic regression and vanilla neural network. Second we cover sequence models such as LSTMs and how to prepare your characters for the LSTMs including things like one-hot encoding, padding, creating character embeddings and then feeding these into LSTMs. We also cover how to set up and compile these sequence models. 

Thanks for listening, and if you find this content useful, please leave a review and consider supporting this podcast from the link below. 

You and your team might spend a lot of time building a new feature. But how do you know if this feature will be liked by the users? One of the ways to statistically prove this is by using A/B testing. Listen to this episode to get tips, tricks and intuition behind hypothesis testing, alpha, beta, p-values, two-sample t-tests and more. 

These understandings have been learnt from experiences deploying A/B tests in the field, and talking to experts. 

These ideas are typically not covered in traditional A/B testing texts which tend to focus a lot on math without the intuition, and that's why I really wanted to cover it in this podcast episode. Thanks for listening! I'd really appreciate your support for this podcast. Follow the link below. 

In this episode, we will talk about the importance of business impact in data science.  "Your users don't care how smart you are" was a quote I read that got me started in thinking about this. 

The right way to do data science is to think of users, revenue impact, business value and go for the simplest solution possible.  The wrong way to do data science is to just find a nail to hit the hammer with rather than the other way around. 

We will cover about all this and more! 

Amazon link of Inspired by Marty Cagan (a great read to get better at product thinking): https://www.amazon.com/dp/1119387507/?tag=omnilence-20

Please consider buying me a coffee if you find this content useful. Refer to link at the bottom.

In this episode, we will talk about things like Maslow's Hierarchy of Needs, and focussing on higher level needs such as satisfaction and achieving full potential. In the area of tech, data science and software development, admitting your interest could involve "shyness" as the next shiny cool thing is pursued by everyone. But if your interest is in a niche, don't let others stop you from putting in an effort to become great at it. 

Thanks for listening, and please show your support to keep this podcast going! 

You and your team might spend weeks or even months building a model. These are the 3 mistakes to avoid in your next machine learning project! This can save you a lot of time and effort in your next project. 

These tips have been learnt from experiences deploying ML models in production as well as hearing from experts in the field. 

These tips and mistakes are typically not covered in traditional machine learning texts and courses, and that's why I really wanted to cover it in this podcast episode. I'd really appreciate your support for this podcast. Please visit the podcast webpage and support, so that I can continue to develop podcast episodes. Thanks for listening!

To kick things off, I talk about the kind of topics you can expect to hear in this podcast. Welcome to The Data Life!