Skip to main content
Data on Kubernetes Community

Data on Kubernetes Community

By Data on Kubernetes Community

The Data on Kubernetes Community (DoKC) is where users go to run data on Kubernetes. We facilitate the creation and sharing of best practices to help users advance in their DoK journey. Here you can enjoy the audio from our livestreams and meetups. Learn more at https://dok.community/
Available on
Google Podcasts Logo
Overcast Logo
Pocket Casts Logo
RadioPublic Logo
Spotify Logo
Currently playing episode

#35 DoK Community: Make Kubernetes your development environment // Ramiro Berrelleza

Data on Kubernetes CommunityMar 18, 2021

00:00
01:02:49
Implementing Data & Databases on K8s within the Dutch Government | DoKC Town Hall

Implementing Data & Databases on K8s within the Dutch Government | DoKC Town Hall

Implementing Data & Databases on K8s within the Dutch Government
Presented by Sebastiaan Mannem, Director at Mannem Solutions

A small walkthrough of projects within the Dutch government running databases on OpenShift. This talk shares success stories, provides a proven recipe to `get it done,` and debunks some of the FUD. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w

Feb 13, 202444:54
Unsticking Ourselves from Glue: Migrating PayIt’s Data Pipelines to Argo Workflows and Hera | DoKC Town Hall

Unsticking Ourselves from Glue: Migrating PayIt’s Data Pipelines to Argo Workflows and Hera | DoKC Town Hall

Unsticking Ourselves from Glue: Migrating PayIt’s Data Pipelines to Argo Workflows and Hera
Presented by Matt Menzenski, Senior Software Engineering Manager, Payitgov

At PayIt, we’ve been deploying applications to Kubernetes almost since the beginning of the company. Our data workloads, however, have run instead in AWS Glue. This has worked well enough for the reporting use cases that have been the main focus of this team historically. However, at the beginning of 2022, the PayIt data team began building out a new data platform, and in the process, ran into a number of challenges with Glue. In this talk, I will share the difficulties that we encountered with building, deploying, and orchestrating ETL pipelines in AWS Glue, our decision process for moving those workloads into Kubernetes, and the ELT architecture that we’ve arrived at today. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w

Feb 06, 202423:18
Repel Boarders! How to find a Kubernetes operator that really protects your data | DoKC Town Hall

Repel Boarders! How to find a Kubernetes operator that really protects your data | DoKC Town Hall

Repel Boarders! How to find a Kubernetes operator that really protects your data
Presented by Robert Hodges, Altinity
Operators are a godsend for managing data in Kubernetes. But how about protecting it? We'll explore security threats to cloud native databases and show what protection you should look for in operators. Finally we'll introduce a new Data on Kubernetes Community project to develop security standards for database operators in Kubernetes. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w

Jan 30, 202419:22
DoK + Apache Spark | DoKC Town Hall

DoK + Apache Spark | DoKC Town Hall

DoK + Apache Spark
Presented by Holden Karau, Spark Committer and Open Source Engineer at Netflix
In this brief talk, Holden will cover some of the best practices from trying to deploy both small and large scale Spark on Kube. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w

Jan 23, 202419:52
DoK @ Comcast - Deliver Business Outcomes & Improved DevX with Data Services on K8s | DoKC Town Hall

DoK @ Comcast - Deliver Business Outcomes & Improved DevX with Data Services on K8s | DoKC Town Hall

DoK @ Comcast: Delivering Business Outcomes & Improved DevX with Data Services Running on Kubernetes Presented by Greg Otto, Executor Director, DevX Platforms & Charles Ju, Principal Engineer Transforming how to deliver measurable value using data on Kubernetes, while providing psychological safety. If you just sighed, you’re one of the many people leaders and engineers who know the challenges of making these changes. In this talk, we will share our transformation journey, the “Months to Minutes” outcomes we achieved, the architecture approach, and the human journey from one of our engineers. Related Links DoKC Website - https://dok.community/ DoKC Meetups - https://www.meetup.com/data-on-kubernetes-community/ Join Slack - https://join.slack.com/t/dokcommunity/shared_invite/zt-1vgv7ymz7-YtLFvZicrcLP9fS3o_r2_w

Jan 03, 202416:44
DoK Talks - What is Kafka? The rise of one of the world's most used streaming data technologies // Abbey Russell

DoK Talks - What is Kafka? The rise of one of the world's most used streaming data technologies // Abbey Russell

Abbey Russell, PM at Cockroach Labs, shared the backstory on how and why Kafka was created.   

Along the way, you'll learn about  
- Who Franz Kafka was  
- Kafka's earliest use at Linkedin in 2010
- Why organizations like Uber/Coursera/Mailchimp use it today
- Future of Data Streaming  

To find out more about how organizations are benefitting from running streaming workloads on Kubernetes, check out our 2022 research report- https://dok.community/data-on-kubernetes-2022-report/ 

https://dokcommunity.slack.com/
https://dok.community/ 

About Data on Kubernetes Community (DoKC)  
DoKC is an independent tech community where practitioners go to run data workloads on Kubernetes. It is managed by Constantia.io. We produce a range of resources for end users looking to run DoK including:  

- DoK Landscape  An interactive landscape of products and services for running data workloads on Kubernetes. (https://dok.community/landscape/)  

- Videos and Blogs Hundreds of videos and blogs including how tos, tutorials, use cases, case studies, and more. (https://dok.community/landscape/) (https://dok.community/blog/)  

- DoK Reports DoKC works with an independent research firm to annually produce the industry’s only DoK-focused report. (https://dok.community/dok-reports/)  

- Livestreams and Local Meetups  Weekly livestreams on DoK-related topics including demos, tutorials, visionary talks, panels, and more. (https://www.meetup.com/data-on-kubern...)  

- DoK Days @ KubeCon  We host hybrid events twice a year for people to get together and learn in-person or online. Details are published on our main website. (https://dok.community/)  

Connect With Us:  
https://dok.community/     
https://twitter.com/DoKcommunity
https://www.linkedin.com/company/6667...
https://github.com/dokc

Mar 09, 202315:28
DoK Talks - (almost)Everything you need to know about stateful cloud native network applications // W Watson
Mar 02, 202343:39
The Outer Nerd #001 - Dungeons & Dragons - Why should you care? // Abhi Vaidyanatha, Fabian Met & Chase Christensen

The Outer Nerd #001 - Dungeons & Dragons - Why should you care? // Abhi Vaidyanatha, Fabian Met & Chase Christensen

https://dokcommunity.slack.com/
https://dok.community/

ABSTRACT OF THE TALK  

Fabian, Chris and Abhi will discuss their passion for roleplaying games, and what they can teach us about the power of community, improvisation, and using our creativity.

Dec 13, 202258:25
DoK Talks #155 - Databases at the edge with K3s and ARM devices // Sergio Méndez

DoK Talks #155 - Databases at the edge with K3s and ARM devices // Sergio Méndez

https://go.dok.community/slack
https://dok.community/
https://youtu.be/KjiK6eXYO34

ABSTRACT OF THE TALK  

In this talk Sergio is going to present different ways to store data at the edge using different databases and Long Horn as a storage class. All this running on a Raspberry Pi and showing and small application using a database running at the edge.  

BIO  

Sergio Méndez is a systems engineer and professor of operating systems at USAC Guatemala university. His work at the university is related to teaching and researching cloud native technologies with his students.
He has experience working on DevOps, and MLOps using open source technologies at work. About open source communities, he is involved in the CNCF Community, promoting students into the CNCF Ecosystem and hosting a Cloud Native meetup in Guatemala. He has been a speaker at several conferences such as KubeCon, WTFisCloudNative, and Kubernetes Community Days.
He is also a Linkerd Ambassador and author of Edge Computing systems with Kubernetes with Packt publisher.  

KEY TAKE-AWAYS FROM THE TALK  

How to storage data at the edge using databases

Nov 29, 202249:40
DoK Talks #154 - StatefulSets in K8 // Srinivas Karnati

DoK Talks #154 - StatefulSets in K8 // Srinivas Karnati

https://go.dok.community/slack
https://dok.community/
Link: https://youtu.be/n_thXwyJNSU

ABSTRACT OF THE TALK  

Deploying Stateless applications is easy but this is not the case for Stateful applications. StatefulSets are the K8s API object that helps to manage stateful application. Learn about what Stateful sets are, how to create, How it differs from Deployments.   

BIO  

Passionate about Cloud Native, Kubernetes, Developer relations  

KEY TAKE-AWAYS FROM THE TALK  

This talk is focused on basics of StatefulSet, how StatefulSet differs from Deployments, How to manage Stateful app using StatefulSet

Nov 23, 202231:55
Data-driven Diversity, Equity, and Inclusion // Lisa-Marie Namphy, Melissa Logan, Tiffany Jachja, Audra Montenegro & Cortney Nickerson (DoK Day North America 2022)
Nov 02, 202219:50
 Formula 1 telemetry processing using Apache Kafka on Kubernetes // Paolo Patierno (DoK Day North America 2022)

Formula 1 telemetry processing using Apache Kafka on Kubernetes // Paolo Patierno (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)  

Video - https://youtu.be/4cPVRWOK-_E


ABSTRACT

Apache Kafka is the de facto data streaming platform used for ingesting vast amounts of data and processing them in real-time. Low latency analytics are vital if users are to react to events as fast as possible and to effectively shape future decision making. Together with Kubernetes, it allows to develop cloud oriented analytics solutions which are highly scalable.
During this session, we will explore an Apache Kafka based solution running on Kubernetes to help a Formula 1 team gain insights during a race. The “real” data will come from a well-known racing game and will be processed by our application in real-time, making us feel like real track-side F1 engineers!

Nov 02, 202215:36
Choosing Kubernetes for Stateful Applications // Akshay Ram & Peter Schuurman (DoK Day North America 2022)

Choosing Kubernetes for Stateful Applications // Akshay Ram & Peter Schuurman (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)  
Video - https://youtu.be/Y4tdy9lctEI


ABSTRACT

Learn how customers are increasingly deploying stateful applications on Kubernetes to benefit from portability, economies of scale, and built-in orchestration capabilities. This talk will include how customers choose between using Kuberentes, or a data Software as a Service (SaaS) and stateful capabilities of Kubernetes across two dimensions - the application orchestration and the storage layer. Also learn about MariaDB SKYSQL, a database software as a service that runs thousands of StatefulSet Pods across multiple zones and regions on Kubernetes.

Nov 02, 202218:31
Kubernetes 360º - Data driven observability - from Secrets to logs // Ben Hirschberg (DoK Day North America 2022)

Kubernetes 360º - Data driven observability - from Secrets to logs // Ben Hirschberg (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)
Video - https://youtu.be/A1ch4AhKoeQ


ABSTRACT

If there’s one thing that everyone can agree on - it’s that the sheer scale and complexity of Kubernetes operations is growing constantly. What’s more, cloud native environments are becoming more and more expensive to operate and manage, as well as increasingly difficult to secure. On the bright side, there is a growing ecosystem of exceptional open source tools to help overcome this complexity, and provide greater situational awareness to what’s happening in your many and multiple Kubernetes clusters.

In this talk, we’ll take a deep dive on how to understand everything that is happening with stateful workloads in your Kubernetes clusters from health to security - including the management and monitoring of your clusters, troubleshooting and debugging, through full security scanning in multiple data perspectives- from configMaps, Secrets, hostPaths and many more security driven checks for proper role-based access control, vulnerabilities and misconfigurations. We will demonstrate how you can have a 360º-view of your Kubernetes clusters in a single open source dashboard leveraging tools like Lens and Kubescape, and ultimately enforce policies with Open Policy Agent.



Nov 02, 202217:12
Shifting Left Stateful Applications In Kubernetes // Viktor Farcic (DoK Day North America 2022)

Shifting Left Stateful Applications In Kubernetes // Viktor Farcic (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)
Video - https://youtu.be/LymPjH6HA3E

ABSTRACT

Stateless apps are easy to manage. More often than not, a Kubernetes Deployment, with a Service, Ingress, and Horizontal Pod Autoscaler (HPA) is enough. Almost everyone can do it. But, when it comes to stateful applications, things become a bit more complicated. We might need a database and storage. We might need to manage database users and schema. We might need to consider quite a few other things. Stateful apps are harder for everyone, especially if we want to shift left and enable developers to do it themselves.

In this talk, we'll try to make the management of stateful applications easy for everyone. We'll accomplish that by creating easy-to-consume services that are made specifically for the needs of our organizations. We'll see how to create new Kubernetes Custom Resource Definitions (CRDs) and controllers using Crossplane. Those controllers will envelop all the tools, resources, and processes we might need. As a result, ops can focus on creating such services while everyone else can consume them (create and manage everything related to stateful apps) without opening JIRA tickets and waiting for others to complete their tasks.

Nov 02, 202215:53
Medical - Healthcare Data on Kubernetes // Olyvia Rakshit & Prasad Dorbala (DoK Day North America 2022)

Medical - Healthcare Data on Kubernetes // Olyvia Rakshit & Prasad Dorbala (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)  

ABSTRACT  

Healthcare organizations are transforming their applications and embracing digital platforms for efficient patient care. Today, compute at the edge, plays a critical role in deploying innovative healthcare applications that promise new approaches to patient care. Connected care platforms at scale are turning to Kubernetes infrastructure for deployment of telehealth and telemedicine apps. These kinds of applications often need real time image processing and inferencing at the edge, which in turn, require a distributed application platform on Kubernetes. They also need secure foundations for a regulatory compliant framework for wider adoption.  

To elaborate further, a distributed platform or a multi-cluster infrastructure is necessary for healthcare applications. All medical application platforms are data intense. For eg. Medical applications that need to stream video data, i.e move large quantities of data need secure connections from edge to cloud. Smart healthcare applications are also seeing the value of multi-cloud deployments.   

The ability to choose different technologies like AI, data storage systems or some specific relational database systems is driving multi-cloud adoption in healthcare solutions. A multi-cluster connectivity fabric is thus needed for data intensive healthcare applications to distribute across geographical boundaries. The fabric needs to have inbuilt automation and the smarts to make the best use of compute resources in a distributed infrastructure, for improved patient healthcare.

Nov 02, 202213:41
Highly Available Postgres Clusters In Kubernetes // John Long & Jonathan Gonzalez (DoK Day North America 2022)

Highly Available Postgres Clusters In Kubernetes // John Long & Jonathan Gonzalez (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)  

ABSTRACT  

A practical session about running Highly Available PostgreSQL in Kubernetes. The primary objective will be to demonstrate how to set up a reliable architecture in a Kubernetes cluster to achieve low RTO and RPO. This will be covered by going over the various Kubernetes native components of our fully open source operator CloudNativePG[0]. It will be demonstrated how these components allow for a simple adoption of PostgreSQL into Kubernetes environments. If time and technical constraints permit there will be a live demo at the end of the presentation; if for some reason that is not possible there will be a link to a recorded demo.  

https://cloudnative-pg.io/

Nov 02, 202215:05
Inter-Cluster PostreSQL on Kubernetes // Julian Fischer (DoK Day North America 2022)

Inter-Cluster PostreSQL on Kubernetes // Julian Fischer (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)


ABSTRACT  
In this talk you’ll explore how to run a PostgreSQL cluster across multiple Kubernetes clusters. Learn what challenges arise when using asynchronous streaming replication in a set of Kubernetes clusters spanning across several geographical regions. It will be discussed how such a topology can be established using Kubernetes and Kubernetes Add-Ons. You will learn which qualities the system has, where such a distributed system plays its strengths and where it may be problematic to use it. At the end of the talk you will have an increased understanding about how to create multi-regional data service instances using PostgreSQL as an example.

Nov 02, 202217:08
Open Source Databases on Kubernetes- Best Practices // Peter Zaitsev (DoK Day North America 2022)

Open Source Databases on Kubernetes- Best Practices // Peter Zaitsev (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)  

ABSTRACT  

So you’re looking to run your Open Source Database on Kubernetes. What best practices should you follow and what pitfalls should you avoid ? In this presentation we will look at how to run stateful applications on Kubernetes overall as well as what is particularly important for databases - we will cover high availability, security, backups and disaster recovery. Finally we will show how these practices can be implemented with Percona Operators for MySQL, MongoDB, PostgreSQL - one of the leading solutions to run Open Source Databases on Kubernetes

Nov 02, 202216:05
The Kubernetes Native Database // Jeffrey Carpenter (DoK Day North America 2022)

The Kubernetes Native Database // Jeffrey Carpenter (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)

ABSTRACT

In the software industry we’re fond of terms that define major trends, like “cloud native”, “Kubernetes native” and “serverless”. As more and more organizations move stateful workloads to Kubernetes, we’ve started to see these terms applied to data infrastructure, where they can get overtaken by marketing hype unless we work to define them.

In this talk, we’ll examine two different databases, TiDB and Apache Cassandra, in order to identify what it means for a database to be Kubernetes native and why it matters. We’ll look at points including:
- The differences between cloud native, Kubernetes native, and serverless
- How databases become Kubernetes native
- Benefits of Kubernetes native databases
- How Kubernetes can better support databases

Nov 02, 202216:27
Databases on Kubernetes: Why are they important? // With Bhavin Shah, Xing Yang, Gabriele Bartolini & Patrick McFadin (DoK Day North America 2022)

Databases on Kubernetes: Why are they important? // With Bhavin Shah, Xing Yang, Gabriele Bartolini & Patrick McFadin (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)

ABSTRACT

Kubernetes has crossed the chasm, but what about stateful applications and databases? Join us for this panel discussion and learn more about how organizations are deploying different databases like PostgreSQL and Cassandra on Kubernetes, what are the benefits of running databases on Kubernetes, and how the ecosystem is working towards making these things boring, so you can focus on your applications! We will have an interactive discussion with the hosts and guests of the Kubernetes Bytes podcast, and open it up to the audience to ask questions and learn more about the what, why, and how about Databases on Kubernetes!

Nov 02, 202234:52
Data streaming on Kubernetes // Yaniv Ben Hemo (DoK Day North America 2022)

Data streaming on Kubernetes // Yaniv Ben Hemo (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)


ABSTRACT

I will cover what is the current data streaming on k8s landscape, why it is important, use cases, and what are the challenges needed to solve

Nov 02, 202213:51
Architecting Your First Event Driven Serverless Streaming Applications on K8 // Timothy Spann (DoK Day North America 2022)

Architecting Your First Event Driven Serverless Streaming Applications on K8 // Timothy Spann (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)

ABSTRACT

Once you have built a topic in Apache Pulsar, you will quickly see the need to build event-driven applications. This can require a lot of decisions on what framework to use, where to run it, how to deploy it, and how to manage these applications on Kubernetes cloud natively.

I will walk you through step-by-step in building Pulsar Functions which is the easy way to design, test, develop, integrate, deploy, monitor, and manage serverless streaming applications in Java and Python.

Together we will build a full application as an Apache Pulsar function and enjoy the power of running it in the cloud for IoT events and add any routing, transformation, or machine learning that we need to accomplish our business requirements.

Through FunctionMesh we run on Kubernetes natively.

In this talk, you will deploy ML functions to transform real-time data on Kubernetes.

Nov 02, 202213:30
Fybrik - A Kubernetes based platform for governed data use // Flora Gilboa-Solomon, Alexey Roytman, Maryna Strelchuk & Barry Hijkoop (DoK Day North America 2022)

Fybrik - A Kubernetes based platform for governed data use // Flora Gilboa-Solomon, Alexey Roytman, Maryna Strelchuk & Barry Hijkoop (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)

ABSTRACT

Data is the foundation for business value. However, in many enterprises, it is spread across different data stores, public/private clouds, and on-premises. The use of data is governed by regulatory requirements and enterprise policies and enterprises face dynamic data residency challenges.
Fybrik (https://fybrik.io) is an open-source, Kubernetes-based infrastructure that enables easier access to data distributed across hybrid and multi-cloud environments, while enforcing data governance policy according to the business’s needs.

In this talk, we will share use cases from a pilot project carried out with ING (a multinational banking corporation). Fybrik enabled ING the use of their data across geographical borders, across clusters in a secure, automated, and compliant manner.

Nov 01, 202220:59
The Challenges of Data Processing On Kubernetes - A look at Spark, Flink, Dask, and Ray // Holden Karau (DoK Day North America 2022)

The Challenges of Data Processing On Kubernetes - A look at Spark, Flink, Dask, and Ray // Holden Karau (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)

ABSTRACT

This talk will go through both the improvements that have been made in Kubernetes for batch analytic workloads as well as some of the current pain experienced by users and developers moving their workloads to Kube. In this talk you will learn about how we “cheated” back in the YARN and Mesos days to make things go fast, why Kubernetes doesn’t like those cheats, and what some alternatives are.

Oct 31, 202220:10
Scaling our SaaS offering to thousands of clusters // Dax McDonald (DoK Day North America 2022)

Scaling our SaaS offering to thousands of clusters // Dax McDonald (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)

ABSTRACT

Sourcegraph is a code intelligence platform that helps our customers to understand their code better. As we have scaled up, we are starting to run hundreds of instances for our customers in separate kubernetes clusters.

Running dozens of distinct clusters with a stateful application that scales horizontally and vertically has been challenging. Sourcegraph's inherently stateful nature requires thoughtful planning to achieve resilience while making tradeoffs to keep costs down. This talk will discuss our methodologies to allow our cluster to scale while minimizing drift across multiple customer deployments. I will also cover our use of Terraform, ArgoCD, and Helm to manage the application and infrastructure. Finally, this talk will discuss our disaster recovery strategies that allowed our organization to achieve SOCII compliance while iterating quickly.

If you are looking to scale out a managed offering at your organization, learn from our mistakes and come to this talk!

Oct 29, 202221:05
Why we decided to migrate our Jaeger storage to ClickHouse on Kubernetes // Arul Jegadish Francis (DoK Day North America 2022)

Why we decided to migrate our Jaeger storage to ClickHouse on Kubernetes // Arul Jegadish Francis (DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)  

Abstract

We at OpsVerse provide a DevOps tools platform with fully-managed open source-based tools. One of our key offerings is a holistic observability platform. Metrics and logs are straightforward to aggregate, however traces – which are collected using CNCF Jaeger – were left with some holes in advanced insights. 

 In this talk, I will explain why we decided to switch our Jaeger storage backend to ClickHouse, and why we decided to run it on Kubernetes. I will also share tips and tricks to migrate and run ClickHouse on Kubernetes.

Oct 28, 202213:49
Building a Digital Factory for the Sheet Metal Industry // Elie Assi (From the DoK Day North America 2022)

Building a Digital Factory for the Sheet Metal Industry // Elie Assi (From the DoK Day North America 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)  


Abstract

We develop systems to digitize the sheet metal industry with the belief that they should cooperate with each other in an open way. We are convinced that the future lies in creating a software ecosystem that interconnects all levels of the company and even manages to communicate with supplier and customer systems, making for more agile management throughout the entire value chain.  

One of our newer projects is providing a multi-tenant cloud service to our clients. The chosen database for the service we are developing is MongoDB. After testing with docker containers on virtual machines, we decided to finally host the service in Kubernetes because we wanted something to be able to scale at large for our clients being a service that will be consumed by a multitude of our clients, so to the database in order to reduce latency.

Oct 27, 202220:48
How we built our Big Data Stack (almost) entirely on top of Kubernetes // Neylson Crepalde (From DoK Day NA 2022)

How we built our Big Data Stack (almost) entirely on top of Kubernetes // Neylson Crepalde (From DoK Day NA 2022)

From the DoK Day North America 2022 (https://youtu.be/YWTa-DiVljY)  


Abstract 

Working with Terabytes of data is a major challenge for organizations both in terms of architecture and cost. In recent years, a new paradigm has emerged in the world of Big Data, that is, implementing the entire architecture for processing massive data from a microservices approach. This architecture proved to be scalable, efficient, with excellent performance, failsafe, portable between different cloud providers and at a lower cost than conventional architectures. In this talk, I will present how my team managed to implement this architecture based on Kubernetes and what were the main gains we had with the innovation.

Oct 26, 202216:00
Dok Talks #153 - CRD Panel // Eyar Zilberman & Álvaro Hernández

Dok Talks #153 - CRD Panel // Eyar Zilberman & Álvaro Hernández

https://go.dok.community/slack
https://dok.community

We are going to speak about CRDs, and discuss considering them as higher level entities that we normally consider them.

CRDs normally are kind of a byproduct of an operator. But in reality, they can be considered as the user-facing API of the operator surface. And as such, we would like to introduce conversations about even standardizing some of them, or creating "specifications" out of some CRDs.

Take this for example: there are some CRDs that could be, potentially, common across multiple operators. Think of a CRD to represent an S3 (or any other storage object) bucket. For operators that store data in object stores, each and every one of them are creating their own versions. But shouldn't we have some "standardized" `ObjectStorage` CRD, without a controller, that all could use?

We'll be joined by John Sanda of DataStax who works on the K8ssandra project, Alvaro Hernandez who is the CEO of Ongres and Postgres superhero, and Eyar Zilberman Co-Founder and Chief Product Officer at Datree.

Oct 14, 202258:05
Dok #152-Running PostgreSQL in Kubernetes:from day 0 to day 2 with CloudNativePG // Gabriele Bartolini

Dok #152-Running PostgreSQL in Kubernetes:from day 0 to day 2 with CloudNativePG // Gabriele Bartolini

https://go.dok.community/slack
https://dok.community

With:
Gabriele Bartolini - Vice President/CTO of Cloud Native and Kubernetes, EDB
Bart Farrell - Head of Community, Data on Kubernetes Community

ABSTRACT OF THE TALK

Imagine this: you have a virtual infrastructure based on Kubernetes, made up of virtual data centers, possibly spread across multiple Kubernetes clusters and regions. Your infrastructure could even be hosted on premises or on different cloud service providers. Infrastructure as Code is a requirement. You’ve been tasked to run Postgres databases, alongside your applications.

The good news is that you can leverage a fully open source stack with Kubernetes, PostgreSQL and the CloudNativePG operator, and deploy your Postgres database in the same way you deploy applications.
Join me in this webinar to discover the key role that you have to make this succeed, starting from day 0 through day 2 operations.
I’ll share some examples and best practices for running Postgres databases in Kubernetes, before peeking at the new features we are developing for the months to come.

BIO

A long time open source programmer and entrepreneur, Gabriele has a degree in Statistics from the University of Florence. After having consistently contributed to the growth of 2ndQuadrant and its members through nurturing a lean and devops culture, he is now leading the Cloud Native initiative at EDB.

Gabriele lives in Prato, a small but vibrant city located in the northern part of Tuscany, Italy - known for having hosted the first European PostgreSQL conferences. His second home is Melbourne, Australia, where he studied at Monash University and worked in the ICT sector. He loves playing the Blues with his Fender Stratocaster, but his major passions are called Elisabeth and Charlotte!

Sep 28, 202201:03:50
Dok Talks #148 - Cost and Kubernetes // Chris Love

Dok Talks #148 - Cost and Kubernetes // Chris Love

https://go.dok.community/slack
https://dok.community

With:
Chris Love - Managing Partner, LionKube
Bart Farrell - Head of Community, Data on Kubernetes Community

ABSTRACT OF THE TALK

Using Kubernetes to run data workloads costs less than running the same workloads on separate servers. But how do we save at least twenty to thirty percent more? We need the data and a cost-savings culture. We will explore a project called Open Cost: cost monitoring for cloud-native environments. Secondly, we will explore why company culture is more important than cost savings tools.

BIO

Chris Love is a Google Cloud Certified hybrid Multi-cloud Fellow and a Co-founder of Lionkube. Chris co-authored the book Core Kubernetes. He has over 25 years of software and IT engineering experience with companies including Google, Oracle, VMWare, Cisco, Johnson & Johnson, and others. As a thought leader within Kubernetes and the DevOps community, Chris Love has contributed to many open source projects, including Kubernetes, kops (former AWS SIG lead), Bazel (contributed to Kubernetes rules), and Terraform (an early contributor to the VMware plugin). His professional interests include Kubernetes, IT culture transformation, containerization technologies, automated testing frameworks and practices, and DevOps. Love also enjoys speaking around the world about DevOps, Kubernetes, and Technology and mentoring people in the IT and Software industry.

Outside of work Love enjoys skiing, volleyball, yoga, and other outdoor activities that come with living in Colorado. He’s also been a practicing martial artist for over 20 years.

If you’re interested in having virtual coffee or have questions for Chris, you can contact him at @chrislovecnm on Twitter or LinkedIn.

KEY TAKE-AWAYS

Two things will save us money running a stateful workload hosted in Kubernetes; data and company culture.

Sep 27, 202245:25
Dok Talks #151 - Analytics with Apache Superset and ClickHouse // Vijay Anand Ramakrishnan

Dok Talks #151 - Analytics with Apache Superset and ClickHouse // Vijay Anand Ramakrishnan

https://go.dok.community/slack
https://dok.community

With:
Vijay Anand Ramakrishnan - Database Administrator, ChistaDATA
Bart Farrell - Head of Community, Data on Kubernetes Community

ABSTRACT OF THE TALK

This talk concerns performing analytical tasks with Apache Superset with ClickHouse as the data backend. ClickHouse is a super fast database for analytical tasks, and Apache Superset is an Apache Software foundation project meant for data visualization and exploration. Performing analytical tasks using this combo is super fast since both the software are designed to be scalable and capable of handling data of petabyte scale.

BIO

Vijay Anand is based out of Chennai (India), working as a Database Administrator in ChistaDATA. He has extensive experience in ClickHouse, Python and has contributed as a technical lead in multiple organizations building ClickHouse based solutions. His areas of interest include database design, building software solutions using open source technologies. He is the author of a book on ClickHouse titled "Up and Running with ClickHouse".

KEY TAKE-AWAYS

Real time analytics, Data exploration and Visualization

Sep 23, 202233:00
Dok Talks #150 - Building a Simple Postgres Async Streaming Cluster // Julian Fischer

Dok Talks #150 - Building a Simple Postgres Async Streaming Cluster // Julian Fischer

https://go.dok.community/slack
https://dok.community

With:
Julian Fischer - CEO, anynines GmbH
Bart Farrell - Head of Community, Data on Kubernetes Community

ABSTRACT OF THE TALK

In this talk you will learn how to build the a Postgres service with Kubernetes. See how asynchronous replication is set up using a Kubernetes resources including, a headless Service, StatefulSet, ConfigMap and Secret.

BIO

Julian Fischer, CEO of anynines, has dedicated his career to the improvement and automation of software operations. In more than fifteen years,he has built several application platforms with various automation tools. His latest passions are data service automation, Cloud Foundry, BOSH and Kubernetes.

KEY TAKE-AWAYS

At the end of the talk you will know the Kubernetes resources a controller had to manage as input to building an actual Operator.


Sep 23, 202201:04:45
DoK Talks #149 - Overcoming challenges with protecting and migrating data in multi-cloud K8s environments // Sebastian Glab & Martin Phan

DoK Talks #149 - Overcoming challenges with protecting and migrating data in multi-cloud K8s environments // Sebastian Glab & Martin Phan

https://go.dok.community/slack
https://dok.community/

With:
Sebastian Glab - Cloud Architect, CloudCasa by Catalogic
Martin Phan - Field CTO – Americas, CloudCasa by Catalogic
Bart Farrell - Head of Community, Data on Kubernetes Community

ABSTRACT OF THE TALK  

If you are running or planning a multi-cloud or even a multi-cluster environment, there are several considerations in implementing a data protection solution – especially if you plan on an organic home-grown, do-it-yourself option. This talk will highlight challenges and best practices around centralized management of configuration, credentials, compliance across multiple accounts, regions, providers etc. We will also highlight the deviations in CSI driver implementations of various storage vendors and cloud providers. Finally, we will cover the various recovery options available in the market today.

Kubernetes cloud services are popular since they mitigate, but do not eliminate, the difficulties of operating a Kubernetes environment. This is especially true for protecting the stateful configuration and data of your Kubernetes applications, where the inherent high-availability and infrastructure as code are not a substitute for have cloud-native backup and disaster recovery capabilities. Further, many companies now have multi-cloud strategies for their cloud-native applications. These challenges can be addressed with backup applications that are both Kubernetes managed service and multi-cloud aware in order to snapshot, copy, restore, and migrate Kubernetes workloads (resources and data) running on AKS, EKS and GKE. Capturing  information from cloud accounts and how the cluster and storage resources are configured allows 1) centralized visibility into all cloud accounts and the clusters and resources in the accounts including for compliance; 2) cross-account, cross-cluster, and cross-region data restores; 3) automation of the cluster and data restores including for Dev, Test, and Production recovery use cases.

BIO  

Sebastian Glab is a Cloud Architect for CloudCasa and he resides in Poland. He is responsible for integrating the different cloud providers with the CloudCasa service, and making sure that all clusters in the cloud service get discovered and protected. In his free time, he plays volleyball and develops his own projects.

Martin Phan is the Field CTO in North America for CloudCasa by Catalogic Software. With over 20+ years of experience in the software-industry, he takes pride in supporting, developing, implementing, and selling enterprise software and data protection solutions to help customer solve their backup and recovery challenges.

KEY TAKE-AWAYS FROM THE TALK  

1) Challenges and best practices around centralized management of configuration, credentials, compliance across multiple accounts, regions, providers etc.  

2) Advantages of cloud awareness and Kubernetes managed service awareness for application and data recovery and security 

3) Examples of overcoming Container Storage Interface (CSI) deviations 

4) Various recovery options available in the market today.

Sep 16, 202247:40
DoK Talks #147 - Evaluating Cloud Native Storage Vendors // Dinesh Majrekar

DoK Talks #147 - Evaluating Cloud Native Storage Vendors // Dinesh Majrekar

https://go.dok.community/slack
https://dok.community/


With:
Dinesh Majrekar - CTO, Civo
Bart Farrell - Head of Community, Data on Kubernetes Community


ABSTRACT OF THE TALK  

In a continuation of a talk given at DoK day at KubeCon EU 2022, join Dinesh Majrekar, Civo's CTO as they walk through their evaluation process of the CNCF Storage market.

Civo offers managed Kubernetes clusters powered by K3s to customers around the world. We manage thousands of Virtual Machines and stateful customer data within multiple data centres across several continents.  
In late 2021, Civo had the opportunity to evaluate the CNCF storage landscape to move to a new technology stack. During the migration project, Civo evaluated Mayastor, Ondat, Ceph and Longhorn against the following metrics:

Scalability
Performance
Ease of Support

Attendants will see practical examples on how they could carry out their own similar evaluation and see some of the results of the Civo research project.


BIO  

Dinesh is CTO at Civo. Having worked in the hosting industry for many years, Dinesh has a passion for creating solutions that operate at scale. This not only applies to the technology stack, but for nurturing engineers through their career.

Sep 05, 202201:00:03
Dok Talks #146 - OpenFeature - Making feature flags a commodity // Oleg Nenashev

Dok Talks #146 - OpenFeature - Making feature flags a commodity // Oleg Nenashev

https://go.dok.community/slack
https://dok.community/  

With:
Oleg Nenashev - Community Builder and Developer Advocate, Dynatrace
Bart Farrell - Head of Community, Data on Kubernetes Community

ABSTRACT OF THE TALK

With the evolution of software delivery methods, feature flags became one of the ways to deliver features and get user feedback. Modern applications may have hundreds of feature flags describing customizations, preview features and environment specifics. Feature flags are not just a single toggle anymore, their values are often determined at runtime depending on the instance, user context and inputs from the control plane.

In this session we will talk about OpenFeature - a new collaboration by many feature flag vendors and end users. OpenFeature, a CNCF sandbox project, is an open standard for feature flag management, created to support a robust feature flag ecosystem using cloud native technologies and to basically make feature flags a commodity. OpenFeature provides a unified API and SDKs, and a developer-first, cloud-native implementation, with extensibility for open source and commercial offerings.

BIO

Oleg works on open source programs and Keptn community at the Dynatrace Open Source Program Office. He is a TOC chair and community ambassador in the Continuous Delivery Foundation. Oleg is a core maintainer and board member in Jenkins where he writes code, mentors contributors, and organizes community events and Google Summer of Code in the project. Open source software and open hardware advocate. Oleg has a PhD degree in hardware design and volunteers in the Free and Open Source Silicon Foundation.

KEY TAKE-AWAYS FROM THE TALK

Feature flags at scale 101. Feature flags are an important topic for stateful apps in K8s, and for progressive delivery and other modern approaches. OpenFeature is like OTel but for feature flags. You can try it now, here is a live demo to follow.

Aug 26, 202201:01:30
DoK Talks #145 - Making Hard Things Easy is Hard // Kurt Rinehart

DoK Talks #145 - Making Hard Things Easy is Hard // Kurt Rinehart

https://go.dok.community/slack
https://dok.community/
https://youtu.be/6eSWOUzCb4w

With:
Kurt Rinehart - Director of Information Engineering, Section
Bart Farrell - Head of Community, Data on Kubernetes Community

ABSTRACT OF THE TALK  

Making complicated things simple is a prime objective of technology. In addition to technical challenges there is an interfacing challenge. At Section we're automating all dimensions of hosting containerized workloads and solving the "location challenge" is one aspect of this. Making it simple to get the locations you want evokes many trade-offs and can collide with past habits that may no longer be suitable.

BIO  

Dr. Kurt Rinehart is the Director of Information Engineering at Section. Kurt holds a PhD in Natural Resource Management, specialising in applying quantitative decision analysis to complex conservation problems. After working as a quantitative consultant, Kurt found and fell in love with startup life and Section. His current responsibilities include designing, developing, and managing autonomic computing systems and providing analytics and data science support across every facet of Section.

Kurt is married with 2 kids and a dog, and plays guitar for The Intolerables (theintolerables.com).

KEY TAKE-AWAYS FROM THE TALK  

- Good agile practice and feedback loops is critical.  
- "Trust but verify" is a good philosophy to keep in mind.  
- When developers are your users, there are extra challenges in finding the happy balance.

Aug 19, 202257:40
DoK Talks #144 - We will Dok You! - The journey to adopt stateful workloads on k8s // Guy Menahem

DoK Talks #144 - We will Dok You! - The journey to adopt stateful workloads on k8s // Guy Menahem

https://go.dok.community/slack
https://dok.community/
https://youtu.be/AjvwG53yLMY

With:
Guy Menahem - Solution Architect, Komodor
Bart Farrell - Head of Community, Data on Kubernetes Community

ABSTRACT OF THE TALK  

Stateful workloads are the heart of any application, yet they remain confusing and complicated even to daily K8s practitioners. That’s why many organizations shy away from migrating their data - their prized possession - to the unfamiliar stateful realm of Kubernetes.

After meeting with many organizations in the adoption phase, I discovered what works best, what to avoid, and how critical it is to gain confidence and the right knowledge in order to successfully adopt stateful workloads.

In this talk I will demonstrate how to optimally adopt Kubernetes and stateful workloads in a few steps, based on what I’ve learned from observing dozens of different adoption journeys. If you are taking your first steps in data on K8s or contemplating where to start - this talk is for you!

BIO

- A Developer turned Solution Architect.
- Working at Komodor, a startup building the first K8s-native troubleshooting platform.
- Love everything in infrastructure: storage, networks & security - from 70’s era mainframes to cloud-native.
- All about “plan well, sleep well”.

KEY TAKE-AWAYS FROM THE TALK  

- Understand how critical stateful workloads are for any system, and that the key challenges to migrating it to Kubernetes are knowledge and confidence.

- How to build the foundational knowledge required to overcome adoption challenges by creating a learning path for individuals and teams.

- How to gain confidence to run stateful workloads on Kubernetes with support from the community (and yourself!)

Aug 18, 202201:06:30
DoK Talks #142 - Kubernetes Cluster Upgrade Strategies and Data: Best Practices for your Stateful Workload // Peter Schuurman

DoK Talks #142 - Kubernetes Cluster Upgrade Strategies and Data: Best Practices for your Stateful Workload // Peter Schuurman

https://go.dok.community/slack
https://dok.community/

ABSTRACT OF THE TALK  

How do you make sure your Stateful Workloads remain available when your Kubernetes infrastructure updates? This talk will discuss different strategies of upgrading a Kubernetes cluster, and how you can manage risk for your workload. The talk will showcase demos of each upgrade strategy.

BIO  

Peter is a Senior Software Engineer on GKE at Google. He works on improving Kubernetes for Stateful workloads. His main focus is on enhancing the Kubernetes ecosystem for high availability applications.

KEY TAKE-AWAYS FROM THE TALK  

The mechanics of different upgrade strategies, when to apply a particular upgrade strategy depending on your Stateful workload and how to mitigate risk to your application’s availability.

Aug 18, 202258:45
DoK Talks #144 - Mastering MongoDB on Kubernetes, the power of operators // Arek Borucki

DoK Talks #144 - Mastering MongoDB on Kubernetes, the power of operators // Arek Borucki

https://go.dok.community/slack
https://dok.community/

ABSTRACT OF THE TALK  
During my first talk for DoK community I want to walk you through the world of NoSQL database MongoDB and Kubernetes Operators - Community Edition, Enterprise Edition (MongoDB and Ops Manager on K8s), and Atlas operator, highlight the most important capabilities, talk about use cases and challenges, the theory will be mixed with a live demos!  

BIO  
I'm a SRE / NoSQL / DevOps professional. I hold CKA, CKAD, CKS, also I’m MongoDB Certified DBA and MongoDB Champion. I have experience with multiple cloud providers, Kubernetes, different types of K8s operators (Strimzi, RabbitMQ Cluster Operator), but especially MongoDB K8s Operator. I also work with KEDA. Since 2017, I have been a speaker at MongoDB conferences all around the world (USA, China, Europe).   

KEY TAKE-AWAYS FROM THE TALK  
I would like to share the best practices of running NoSQL database - MongoDB on Kubernetes also I want to show how to manage Atlas (MongoDB cloud) via K8s operator  

https://www.mongodb.com/developer/community-champions/arkadiusz-borucki/

Jul 26, 202201:00:50
DoK Specials - Why are Operators paramount to running stateful workloads on Kubernetes?

DoK Specials - Why are Operators paramount to running stateful workloads on Kubernetes?

In this panel with Sylvain Kalache, Head of Content at the DoK Community, drives a conversation featuring Nic Vermandé- Principal Developer Advocate at Ondat, Julian Fischer- CEO at anynines, and Sergey Pronin- Group Product Manager at Percona.

Jul 20, 202253:37
DoK Talks #141 - Dossier: multi-tenant distributed Jupyter Notebooks // Iacoppo Colonnelli & Dario Tranchitella

DoK Talks #141 - Dossier: multi-tenant distributed Jupyter Notebooks // Iacoppo Colonnelli & Dario Tranchitella

https://go.dok.community/slack
https://dok.community

ABSTRACT OF THE TALK

When providing data analysis as a service, one must tackle several problems. Data privacy and protection by design are crucial when working on sensitive data. Performance and scalability are fundamental for compute-intensive workloads, e.g. training Deep Neural Networks. User-friendly interfaces and fast prototyping tools are essential to allow domain experts to experiment with new techniques. Portability and reproducibility are necessary to assess the actual value of results.

Kubernetes is the best platform to provide reliable, elastic, and maintainable services. However, Kubernetes alone is not enough to achieve large-scale multi-tenant reproducible data analysis. OOTB support for multi-tenancy is too rough, with only two levels of segregation (i.e. the single namespace or the entire cluster). Offloading computation to off-cluster resources is non-trivial and requires the user's manual configuration. Also, Jupyter Notebooks per se cannot provide much scalability (they execute locally and sequentially) and reproducibility (users can run cells in any order and any number of times).

The Dossier platform allows system administrators to manage multi-tenant distributed Jupyter Notebooks at the cluster level in the Kubernetes way, i.e. through CRDs. Namespaces are aggregated in Tenants, and all security and accountability aspects are managed at that level. Each Notebook spawns into a user-dedicated namespace, subject to all Tenant-level constraints. Users can rely on provisioned resources, either in-cluster worker nodes or external resources like HPC facilities. Plus, they can plug their computing nodes in a BYOD fashion. Notebooks are interpreted as distributed workflows, where each cell is a task that one can offload to a different location in charge of its execution.

BIO  

Iacopo Colonnelli is a Computer Science research fellow. He received his Ph.D. with honours in Modeling and Data Science at Università di Torino with a thesis on novel workflow models for heterogeneous distributed systems, and his master’s degree in Computer Engineering from Politecnico di Torino with a thesis on a high-performance parallel tracking algorithm for the ALICE experiment at CERN. His research focuses on both statistical and computational aspects of data analysis at large scale and on workflow modeling and management in heterogeneous distributed architectures.

Dario is an SWE that turned DevOps, and he's regretting this choice day by day. Besides making memes on Twitter that gain more reactions than technical discussions, leading the development of Open Source projects at CLASTIX, an Open Source-based start-up focusing on Multi-Tenancy in Kubernetes.

KEY TAKE-AWAYS FROM THE TALK

From this talk, people will learn:
- The different requirements of Data analysis as a service
- How to configure for multi-tenancy at the cluster level with Capsule
- How to write distributed workflows as Notebooks with Jupyter Workflows
- How to combine all these aspects into a single platform: Dossier

All the software presented in the talk is OpenSource, so attendees can directly play with them and include them in their experiments with no additional restrictions.



Jul 15, 202201:00:10
DoK Talks #140 - Data protection of stateful environment // Timothy Dewin

DoK Talks #140 - Data protection of stateful environment // Timothy Dewin

https://go.dok.community/slack
https://dok.community

ABSTRACT OF THE TALK

More and more we see stateful workloads pop up in Kubernetes clusters. These workloads generate data that is unique and is ephemeral. During this talk we will discuss the challenges of stateful workloads and how you can successfully protect

BIO

Working over a decade in IT as a technical expert for Veeam Software. Specializing in backup for the modern hybrid cloud. Passion for scripting and programming. Husband of Lena, father of Lev.

KEY TAKE-AWAYS FROM THE TALK

Data backup of Kubernetes. DR for Kubernetes

Jun 28, 202242:35
DoK Talks #139 - Private DBaaS on Kubernetes // Sergey Pronin

DoK Talks #139 - Private DBaaS on Kubernetes // Sergey Pronin

https://go.dok.community/slack
https://dok.community

ABSTRACT OF THE TALK
Percona is committed to deliver solutions to run open source databases anywhere without lock in. As part of this commitment, we have created Operators to run MySQL, PostgreSQL and MongoDB on Kubernetes. Learn how Percona Monitoring and Management (PMM) allows you to enable developers to deploy and manage databases anywhere with private Database-as-a-service capability backed by Operators.

BIO

Product and technology leader. Worked in various fields: internet service providers, financial sector and merge & acquisition business. Currently leads product @ Percona focusing on cloud native technologies for open source databases

KEY TAKE-AWAYS FROM THE TALK

Learn how Percona Monitoring and Management (PMM) allows you to enable developers to deploy and manage databases anywhere with private Database-as-a-service capability backed by Operators.

You will get high level overview of Percona's Operators structure and how private DBaaS can boost the productivity of your engineering and IT teams.

Jun 28, 202253:25
DoK Talks #138 - Build your own social media analytics with Apache Kafka // Jakub Scholz

DoK Talks #138 - Build your own social media analytics with Apache Kafka // Jakub Scholz

https://go.dok.community/slack
https://dok.community

ABSTRACT OF THE TALK
Apache Kafka is more than just a messaging broker. It has a rich ecosystem of different components. There are connectors for importing and exporting data, different stream processing libraries, schema registries and a lot more.

The first part of this talk will explain the Apache Kafka ecosystem and how the different components can be used to load data from social networks and use stream processing and machine learning to analyze them.

The second part will show a demo running on Kubernetes which will use Kafka Connect to load data from Twitter and analyze them using the Kafka Streams API.

After this talk, the attendees should be able to better understand the full advantages of the Apache Kafka ecosystem especially with focus on Kafka Connect and Kafka Streams API. And they should be also able to use these components on top of Kubernetes.

BIO

Jakub works at Red Hat as Senior Principal Software Engineer. He has long-term experience with messaging and currently focuses mainly on Apache Kafka and its integration with Kubernetes. He is one of the maintainers of the Strimzi project which provides tooling for running Apache Kafka on Kubernetes. Before joining Red Hat he worked as messaging and solution architect in the financial industry.

KEY TAKE-AWAYS FROM THE TALK

The key takeaway of this talk is that Apache Kafka is more than just a messaging broker. It is a platform and ecosystem of different components which can be used to solve complex tasks when dealing with events or processing data. The talk demonstrates this on loading tweets from Twitter and processing them using the different parts of the Kafka ecosystem. The whole talk and its demos are running on Kubernetes using the Strimzi project. So it also shows how to easily run all the different components on top of Kubernetes with the help of few simple YAML files.

Jun 24, 202256:25
DoK Talks #137 - How to build your own “Doordash” app // Yaniv Ben Hemo

DoK Talks #137 - How to build your own “Doordash” app // Yaniv Ben Hemo

https://go.dok.community/slack

https://dok.community/ 

ABSTRACT OF THE TALK  

The entire app is built in microservices, running on k8s pods and uses k8s-native message broker called memphis  

WORKSHOP

 Learn how to build a cloud-native, distributed, event-driven food delivery application using a message broker, and deploy it to a kubernetes cluster. 

We will also discuss the importance of using a message broker within our app, and what are the benefits it provides to a more resilient application.  

The workshop is based on this article 

- https://medium.com/memphis-dev/how-to-build-your-own-wolt-app-b220d738bb71 

Strongly advise to install before 

- memphis{dev} broker 

- https://github.com/memphisdev/memphis-brokerminikube 

- https://minikube.sigs.k8s.io/docs/start/node.js 

- https://nodejs.org/ 

If any questions arise, please raise a question on our Discord channel 

- https://discord.gg/WZpysvAeTf 

BIO  

Develop since I was 10 years old  

KEY TAKE-AWAYS FROM THE TALK  

Learn the basics of building and deploying cloud-native, distributed, event-driven app

Jun 23, 202257:50
DoK Talks #136 - Building a mesh for databases from scratch and why // Maxwell Miao

DoK Talks #136 - Building a mesh for databases from scratch and why // Maxwell Miao

https://go.dok.community/slack 

https://dok.community/  

ABSTRACT OF THE TALK  

In this talk, Maxwell is going to share his thoughts about Service Mesh and database operations, called Database Mesh, and figure out the design of Pisanix.  

Running databases on Kubernetes is hard,  Database Mesh is building a new standard of databases management on cloud. It tries to make database operation more easier, and provides a framework for different kind of databases.  

Such as using a CRD called VirtualDatabase to describe a virtual database, developers will only need to know about this, meanwhile DBAs will have to bind TrafficStrategy and DatabaseEndpoint to VirtualDatabase, let Pisanix running like a MySQL proxy for applications. More features like SecurityClaim and TrafficQoS will be added to simplify the database operation work.  


BIO  

* Focus on SaaS and Database Mesh 

* Previously worked at 株式会社ネットスターズ、ChuanyangTech、AntFinancial、Yeepay 

* Linuxer & Gopher & Rustacean,open source advocator,cloud native envanglist 

* “CaaSOne” Community Cofounder 

* Contributor of multiple cloud native open source projects 

* Participated in writing Analysis and practice of Tensorflow、Istio Handbook  


KEY TAKE-AWAYS FROM THE TALK  

What's the difference of Service Mesh and Database Mesh.  

Why do people need a mesh for databases.  

What does Pisanix can help DBAs daily work.  

The current status of Pisanix.

Jun 15, 202247:45
DoK Specials - Learn by doing in the DoK Community // Bart Farrell

DoK Specials - Learn by doing in the DoK Community // Bart Farrell

https://go.dok.community/slack 

https://dok.community/  

ABSTRACT OF THE TALK  

As a community we are committed to making learning how to run stateful workloads on Kubernetes as accessible and inclusive as possible.  Using our DoK Landscape which we launched at Kubecon, we will be doing a guided "tour" of the technologies featured on there. We will go one by one, providing tutorials with concrete instructions for you tp get hands on experience. At the end of each sprint we will share our learnings in public.  

BIO  

CNCF Ambassador. Data on Kubernetes (@Dokcommunity) Community Leader, amateur rapper, tech from the human side.

Jun 14, 202215:55
DoK Talks #135 - DoK isn't just Database on Kubernetes // Patrick McFadin

DoK Talks #135 - DoK isn't just Database on Kubernetes // Patrick McFadin

https://go.dok.community/slack
https://dok.community

ABSTRACT OF THE TALK

What about your streaming and analytic workloads? If you are all-in on Kubernetes you can't forget about these important parts of your infrastructure. I'll talk about the current state of the art. Why organizations may hesitate to go beyond deploying databases in Kubernetes and most important, some key things you need to be successful.

BIO

Patrick McFadin is the co-author of the upcoming O’Reilly book “Managing Cloud-Native Data on Kubernetes” He currently works at DataStax in Developer Relations and as a contributor to the Apache Cassandra project. Patrick has worked as Chief Evangelist for Apache Cassandra and as a consultant for DataStax, where he had a great time building some of the largest deployments in production. Previous to DataStax, he held positions as Chief Architect, Engineering Lead and Database DBA/Developer.

KEY TAKE-AWAYS FROM THE TALK

People should walk away with a better understanding of what it takes to deploy streaming and analytic workloads in Kubernetes.

Jun 10, 202246:00