The Backend Engineering Show with Hussein Nasser
By Hussein Nasser
Most of my content in the podcast is an audio version of videos I post on my youtube channel here www.youtube.com/c/HusseinNasser-software-engineering
Buy me a coffee
www.buymeacoffee.com/hnasr
🧑🏫 Courses I Teach
husseinnasser.com/courses
The Backend Engineering Show with Hussein NasserJul 11, 2022
They made Python faster with this compiler option
Fundamentals of Operating Systems Course https://oscourse.win Looks like fedora is compiling cpython with the -o3 flag, which does aggressive function inlining among other optimizations. This seems to improve python benchmarks performance by at most 1.16x at a cost of an extra 3MB in binary size (text segment). Although it does seem to slow down some benchmarks as well though not significantly. O1 - local register allocation, subexpression elimination O2 - Function inlining only small functions O3 - Agressive inlining, SMID 0:00 Intro 1:00 Fedora Linux gets Fast Python 5:40 What is Compiling? 9:00 Compiling with No Optimization 12:10 Compiling with -O1 15:30 Compiling with -O2 20:00 Compiling with -O3 23:20 Showing Numbers Backend Troubleshooting Course https://performance.husseinnasser.com
How Apache Kafka got faster by switching ext4 to XFS
https://oscourse.win
Allegro improved their Kafka produce tail latency by over 80% when they switched from ext4 to xfs. What I enjoyed most about this article is the detailed analysis and tweaking the team made to ext4 before considering switching to xfs. This is a classic case of how a good tech blog looks like in my opinion.
0:00 Intro
0:30 Summary
2:35 How Kafka Works?
5:00 Producers Writes are Slow
7:10 Tracing Kafka Protocol
12:00 Tracing Kernel System Calls
16:00 Journaled File Systems
21:00 Improving ext4
26:00 Switching to XFS
Blog
https://blog.allegro.tech/2024/03/kafka-performance-analysis.html
Google Patches Linux kernel with 40% TCP performance
Get my backend course https://backend.win
Google submitted a patch to Linux Kernel 6.8 to improve TCP performance by 40%, this is done via rearranging the tcp structures for better cpu cache lines, I explore this here. 0:00 Intro 0:30 Google improves Linux Kernel TCP by 40% 1:40 How CPU Cache Line Works 6:45 Reviewing the Google Patch https://www.phoronix.com/news/Linux-6.8-Networking https://lore.kernel.org/netdev/20231129072756.3684495-1-lixiaoyan@google.com/ Discovering Backend Bottlenecks: Unlocking Peak Performance https://performance.husseinnasser.com
Database Torn pages
0:00 Intro
2:00 File System Block vs Database Pages
4:00 Torn pages or partial page
7:40 How Oracle Solves torn pages
8:40 MySQL InnoDB Doublewrite buffer
10:45 Postgres Full page writes
Cloudflare Open sources Pingora (NGINX replacement)
Get my backend course https://backend.win
Cloudflare has announced they are opening sources Pingora as a networking framework! Big news, let us discuss
0:00 Intro
0:30 Reasons why Cloudflare built Pingora?
3:00 It is a framework!
7:30 What in Pingora?
11:50 Security in Pingora
13:45 Multi-threading in Pingora
21:00 Customization vs Configuration
25:00 Summary
The Internals of MongoDB
https://backend.win
https://databases.win
I’m a big believer that database systems share similar core fundamentals at their storage layer and understanding them allows one to compare different DBMS objectively. For example, How documents are stored in MongoDB is no different from how MySQL or PostgreSQL store rows.
Everything goes to pages of fixed size and those pages are flushed to disk.
Each database define page size differently based on their workload, for example MongoDB default page size is 32KB, MySQL InnoDB is 16KB and PostgreSQL is 8KB.
The trick is to fetch what you need from disk efficiently with as fewer I/Os as possible, the rest is API.
In this video I discuss the evolution of MongoDB internal architecture on how documents are stored and retrieved focusing on the index storage representation. I assume the reader is well versed with fundamentals of database engineering such as indexes, B+Trees, data files, WAL etc, you may pick up my database course to learn the skills.
Let us get started.
The Beauty of Programming Languages
In this video I explore the type of languages, compiled, garbage collected, interpreted, JIT and more.
The Danger of Defaults - A PostgreSQL Story
I talk about default values and how PostgreSQL 14 got slower when a default parameter has changed. Mike's blog https://smalldatum.blogspot.com/2024/02/it-wasnt-performance-regression-in.html
Database Background writing
The Cost of Memory Fragmentation
Fragmentation is a very interesting topic to me, especially when it comes to memory. While virtually memory does solve external fragmentation (you can still allocate logically contiguous memory in non-contiguous physical memory) it does however introduce performance delays as we jump all over the physical memory to read what appears to us for example as contiguous array in virtual memory. You see, DDR RAM consists of banks, rows and columns. Each row has around 1024 columns and each column has 64 bits which makes a row around 8kib. The cost of accessing the RAM is the cost of “opening” a row and all its columns (around 50-100 ns) once the row is opened all the columns are opened and the 8 kib is cached in the row buffer in the RAM. The CPU can ask for an address and transfer 64 bytes at a time (called bursts) so if the CPU (or the MMU to be exact) asks for the next 64 bytes next to it, it comes at no cost because the entire row is cached in the RAM. However if the CPU sends a different address in a different row the old row must be closed and a new row should be opened taking an additional 50 ns hit. So spatial access of bytes ensures efficiency, So fragmentation does hurt performance if the data you are accessing are not contiguous in physical memory (of course it doesn’t matter if it is contiguous in virtual memory). This kind of remind me of the old days of HDD and how the disk needle physically travels across the disk to read one file which prompted the need of “defragmentation” , although RAM access (and SSD NAND for that matter) isn’t as bad. Moreover, virtual memory introduces internal fragmentation because of the use of fixed-size blocks (called pages and often 4kib in size), and those are mapped to frames in physical memory. So if you want to allocate a 32bit integer (4 bytes) you get a 4 kib worth of memory, leaving a whopping 4092 allocated for the process but unused, which cannot be used by the OS. These little pockets of memory can add up as many processes. Another reason developers should take care when allocating memory for efficiency.
The Real Hidden Cost of a Request
In this video I explore the hidden costs of sending a request from the frontend to the backend
Heard
https://medium.com/@hnasr/the-journey-of-a-request-to-the-backend-c3de704de223
Why create Index blocks writes
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
https://database.husseinnasser.com
Why create Index blocks writes
In this video I explore how create index, why does it block writes and how create index concurrently work and allow writes.
0:00 Intro
1:28 How Create Index works
4:45 Create Index blocking Writes
5:00 Create Index Concurrently
The Problems of an HTTP/3 Backend
HTTP/3 is getting popular in the cloud scene but before you migrate to HTTP/3 consider its cost. I explore it here.
0:00 Intro HTTP/3 is getting popular
3:40 HTTP/1.1 Cost
5:18 HTTP/2 Cost
6:30 HTTP/3 Cost
https://blog.apnic.net/2023/09/25/why-http-3-is-eating-the-world/
Encrypted Client Hello - The Pros & Cons
The Encrypted Client Hello or ECH is a new RFC that encrypts the TLS client hello to hide sensitive information like the SNI. In this video I go through pros and cons of this new rfc.
0:00 Intro
2:00 SNI
4:00 Client Hello
8:40 Encrypted Client Hello
11:30 Inner Client Hello Encryption
18:00 Client-Facing Outer SNI
21:20 Decrypting Inner Client Hello
23:30 Disadvantages
26:00 Censorship vs Privacy ECH
https://blog.cloudflare.com/announcing-encrypted-client-hello/
https://chromestatus.com/feature/6196703843581952
The Journey of a Request to the Backend
From the frontend through the kernel to the backend process
When we send a request to a backend most of us focus on the processing aspect of the request which is really just the last step.
There is so much more happening before a request is ready to be processed, most of this step happens in the Kernel. I break this into 6 steps, each step can theoretically be executed by a dedicated thread or process. Pretty much all backends, web servers, proxies, frameworks and even databases have to do all these steps and they all do choose to do it differently.
Grab my backend performance course https://performance.husseinnasser.com
0:00 Intro
3:50 What is a Request?
10:14 Step 1 - Accept
21:30 Step 2 - Read
29:30 Step 3 - Decrypt
34:00 Step 4 - Parse
40:36 Step 5 - Decode
43:14 Step 6 - Process
Medium article
https://medium.com/@hnasr/the-journey-of-a-request-to-the-backend-c3de704de223
They Enabled Postgres Partitioning and their Backend fell apart
We discuss this in this video/podcast
www.kylehailey.com/post/postgres-partition-pains-lockmanager-waits
WebTransport - A Backend Game Changer
WebTransport is a cutting-edge protocol framework designed to support multiplexed and secure transport over HTTP/2 and HTTP/3. It brings together the best of web and transport technologies, providing an all-in-one solution for real-time, bidirectional communication on the web.
Watch full episode (subscribers only) https://spotifyanchor-web.app.link/e/cTSGkq5XuAb
WebTransport - A Backend Game Changer
Your SSD lies but that's ok | Postgres fsync
fsync is a linux system call that flushes all pages and metadata for a given file to the disk. It is indeed an expensive operation but required for durability especially for database systems. Regular writes that make it to the disk controller are often placed in the SSD local cache to accumulate more writes before getting flushed to the NAND cells.
However when the disk controller receives this flush command it is required to immediately persist all of the data to the NAND cells.
Some SSDs however don't do that because they don't trust the host and no-op the fsync. In this video I explain this in details and go through details on how postgres provide so many options to fine tune fsync
0:00 Intro
1:00 A Write doesn’t write
2:00 File System Page Cache
6:00 Fsync
7:30 SSD Cache
9:20 SSD ignores the flush
9:30 15 Year old Firefox fsync bug
12:30 What happens if SSD loses power
15:00 What options does Postgres exposes?
15:30 open_sync (O_SYNC)
16:15 open_datasync (O_DSYNC)
17:10 O_DIRECT
19:00 fsync
20:50 fdatasync
21:13 fsync = off
23:30 Don’t make your API simple
26:00 Database on metal?
The problem with software engineering
ego is the main problem to a defective software product. the ego of the engineer or the tech lead seeps into the quality of the product. Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com
2x Faster Reads and Writes with this MongoDB feature | Clustered Collections
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
https://database.husseinnasser.com
In version 5.3, MongoDB introduced a feature called clustered collection which stores documents in the _id index as oppose to the hidden wiredTiger hidden index. This eliminates an entire b+tree seek for reads using the _id index and also removes the additional write to the hidden index speeding both reads and writes.
However like we know in software engineering, everything has a cost. This feature does come with a few that one must be aware of before using it. In this video I discuss the following
- How Original MongoDB Collections Work
- How Clustered Collections Work
- Benefits of Clustered Collections
- Limitations of Clustered Collections
Prime Video Swaps Microservices for Monolith: 90% Cost Reduction
Prime video engineering team has posted a blog detailing how they moved their live stream monitoring service from microservices to a monolith reducing their cost by 90%, let us discuss this 0:00 Intro 2:00 Overview 10:35 Distributed System Overhead 21:30 From Microservices to Monolith 29:00 Scaling the Monolith 32:30 Takeaways https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90 Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com
A Deep Dive in How Slow SELECT * is
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
https://database.husseinnasser.com
In a row-store database engine, rows are stored in units called pages. Each page has a fixed header and contains multiple rows, with each row having a record header followed by its respective columns. When the database fetches a page and places it in the shared buffer pool, we gain access to all rows and columns within that page. So, the question arises: if we have all the columns readily available in memory, why would SELECT * be slow and costly? Is it really as slow as people claim it to be? And if so why is it so? In this post, we will explore these questions and more.
0:00 Intro
1:49 Database Page Layout
5:00 How SELECT Works
10:49 No Index-Only Scans
18:00 Deserialization Cost
21:00 Not All Columns are Inline
28:00 Network Cost
36:00 Client Deserialization
https://medium.com/@hnasr/how-slow-is-select-8d4308ca1f0c
AWS Serverless Lambda Supports Response Streaming
Lambda now supports Response payload streaming, now you can flush changes to the network socket as soon as it is available and it will be written to the client socket. I think this is a game changing feature
0:00 Intro
1:00 Traditional Lambda
3:00 Server Sent Events & Chunk-Encoding
5:00 What happens to clients?
6:00 Supported Regions
7:00 My thoughts
Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)
https://backend.husseinnasser.com
The Cloudflare mTLS vulnerability - A Deep Dive Analysis
Cloudflare released a blog detailing a vulnerability that has been in their system for nearly two years. it is related to mTLS or mutual TLS and specifically client certificate revocation. I explore this in details 0:00 Intro 3:00 The Vulnerability 7:00 What happened? 8:50 Certificate Revocation 12:30 Rejecting certain endpoints 17:00 Certificate Authentication 20:30 Certificate serial number 24:00 Session Resumption (PSK) 35:00 The bug 37:00 How they addressed the problem Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com
The Virgin Media ISP outage - What happened?
BGP (Border gateway protocol) withdrawals caused the Virgin media ISP customers to lose their Internet connection. I go into details on this video. 0:00 Intro 2:00 What happened? 4:11 How BGP works? 11:50 Version media withdrawals 15:00 Deep dive Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com
GitHub SSH key is Leaked - How bad is this?
GitHub Accidentally Exposed their SSH RSA Private key, this is the message you will get .
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
SHA256:uNiVztksCsDhcc0u9e8BujQXVUpKZIDTMczCvj3tD2s.
Please contact your system administrator.
Add correct host key in ~/.ssh/known_hosts to get rid of this message.
Host key for github.com has changed and you have requested strict checking.
Host key verification failed.
In this video I discuss how bad is this,.
0:00 Intro
1:10 What happened?
3:00 SSH vs TLS Authentication
6:00 SSH Connect
7:45 How bad is the github leak?
15:00 What should you do?
18:50 Is ECDSA immune?
https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/
Cookie Hijacking - How Linus Tech Tips got Hacked
How Linus Tech Tips channel got Hacked
In this short video we explain how was it possible for Linux to get hacked with cookies hijacking.
0:00 Intro
0:47 TLDR what happened
5:10 Cookies in Chrome
7:30 Cookies Hijacking
8:46 Session Tokens (Access/Refresh)
10:00 Remedies
All Postgres Locks Explained | A Deep Dive
Get my database engineering course https://database.husseinnasser.com
In this video I do a deep dive in all locks obtained by postgres, I learned a lot while making this video and hope you enjoy it.
0:00 Intro
2:30 What are Locks?
5:30 Overview of Postgres Locks
9:10 Table-Level Locks
11:40 ACCESS EXCLUSIVE
17:40 ACCESS SHARE
19:00 ROW SHARE
20:15 ROW EXCLUSIVE
21:15 SHARE UPDATE EXCLUSIVE
23:30 SHARE
24:50 SHARE ROW EXCLUSIVE
25:18 EXCLUSIVE
25:30 Table Lock Conflict Matrix
28:30 Row-Level Locks
30:00 FOR UPDATE
33:00 FOR NO KEY UPDATE
34:00 FOR SHARE
34:40 FOR KEY SHARE
35:10 Row Lock Conflict Matrix
39:25 Page-Level Locks
42:00 Deadlocks
46:00 Advistory Locks
47:20 Summary
https://www.postgresql.org/docs/current/explicit-locking.html
Pinterest moves to HTTP/3
I cover this on the next episode of the backend engineering course.
0:00 Intro
2:00 Moving h2 to h3 through alt-svc
5:00 Why HTTP/3
6:00 HTTP/1 vs HTTP/2
9:00 TCP Head of Line blocking in HTTP/2
11:00 How HTTP/3 addresses HOL
12:15 Connection Migration
13:30 Stream level congestion control
14:10 1-RTT - 0-RTT
15:41 Pinterest challenges moving HTTP/3
19:00 Migration
21:15 Future work
22:30 Summary
article medium.com/pinterest-engineering/pinterest-is-now-on-http-3-608fb5581094
Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) backend.husseinnasser.com
Why Loom Users got each others’ sessions on March 7th 2023
On March 7 2023, Loom users started seeing each others data as a result of cookies getting leaked from the CDN. This loom security breach is really critical. Let us discuss 0:00 Intro 1:00 Why Cookies 2:00 How this happens 5:50 What caused it? 7:30 How Loom solved it? 8:20 Reading the RCA 10:30 Remedies
How Discord Stores Trillions of Messages - A deep dive
Discord engineering goes into details of how they migrated from Cassandra to ScyllaDB, improved the performance of their reads and writes and rearchitected their backend to support the new load. It is an interesting episode lets get into it
0:00 Intro
1:50 Relational vs Distributed
7:00 The Cassandra Troubles
11:00 SnowFlake vs UUID
14:30 B+Tree
19:20 B+Tree and SSDs
25:30 LSM Trees
31:00 Hot partitions
36:00 Cassandra Garbage Collector Pauses
40:00 Changing the Architecture
45:00 The Data Services
55:00 The Migration
1:02:00 Zoned Named Spaces
1:04:00 Summary
Article here How Discord Stores Trillions of Messages
https://discord.com/blog/how-discord-stores-trillions-of-messages
Postgres Architecture | The Backend Engineering Show
Creating a listener on the backend application that accepts connections is simple. You listen on an address-port pair, connection attempts to that address and port will get added to an accept queue; The application accepts connections from the queue and start reading the data stream sent on the connection.
However, what part of your application does the accepting and what part does the reading and what part does the execution? You can architect your application in many ways based on your use cases. I have a medium post just exploring the different options.
In this video I explore the PostgreSQL process architecture in details. Please note that the information here is derived from both the Postgres doc and code. Discussions about scalability and performance are solely based on my opinions.
0:00 Intro
1:30 Overview
3:30 Postgres MVCC
5:30 Processes vs Threads
7:40 Postmaster Process
8:00 Backend Processes
13:30 Shared Buffers
14:52 Background Workers
17:18 Auxiliary Processes
17:45 Background Writer
22:30 Checkpointer
23:40 Logger
24:06 Autovacuum Launcher and Workers
25:30 WAL Processes
28:53 Startup Process
Read full article
https://medium.com/@hnasr/postgresql-process-architecture-f21e16459907
How Alt-Svc switches HTTP/2 clients to use HTTP/3 | The Backend Engineering Show
The Alt-Svc header/frame is a capability that allows the server to adverse alternative services to the connected application available in protocols, ports or domains. It is available as a response header alt-svc and also as an HTTP/2 frame. Let us discuss this capability.
0:00 Intro
1:38 what is alt-svc?
5:30 uses of h3 in alt-svc
8:00 alt-svc header
10:00 Alt-svc header with 103 early hints
14:48 h2 altsvc frame
18:30 SVCB DNS record
21:20 Summary
Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)
https://backend.husseinnasser.com
Your DNS queries will eventually look like this (0x20 DNS encoding)
Correction: Google is implementing the proposal originally submitted by researchers from Georgia institute of tech. I incorrectly said in the video that google is proposing this .
Google is finally implementing a proposal from 2008 by researchers from Georgia institute of technology to make DNS cache poisoning .
https://astrolavos.gatech.edu/articles/increased_dns_resistance.pdf
https://datatracker.ietf.org/doc/html/draft-vixie-dnsext-dns0x20-00
0:00 Intro
2:00 How DNS Work
5:00 DNS Cache Poisoning
14:00 gOoGLe dot CoM
16:20 ASCII 0x20 casing
18:30 Randomizing the casing with encryption
22:30 limitations of this proposal
24:00 Credits
DropBox Removed their SSDs, got 20% faster writes
In this episode of the backend engineering show I’ll discuss how Dropbox improved their write through put by 20% by removing all their SSDs (yes I was surprised too). DropBox uses an SSD layer as a write-back cache with SMR drives as their backend persistent storage. They changed their model to write directly to the hard drives.
0:00 Intro
2:00 Article Summary
3:00 SMR Drives
6:00 SSD Cache & WriteBack
8:00 Replacing Cache
9:30 Storage Engine Background
14:30 Why did they do it
15:00 The limitation of SSDs & Zoned Namespaces
19:30 Updating the Storage Engine
22:30 Tradeoffs
26:00 Rollout
28:00 Summary
MySQL on HTTP/3 | The Backend Engineering Show
The communication between backend applications and database systems always fascinated me. The protocols keep evolving and we are in constant search for an efficient protocol that best fit the workload of Backend-DB communication.
In this episode of the backend engineering show I go through a blog written by @PlanetScale doing an experimentation of using HTTP/3 and HTTP/2 comparing it with MySQL Binary protocol.
https://planetscale.com/blog/faster-mysql-with-http3
0:00 Intro
7:45 MySQL Binary vs HTTP
10:20 The Tests
15:00 Connection Cost + Select 1
22:00 Parallel Select
26:00 The cost of H2 and H3
How Shopify’s engineering improved writes by 50% with ULID | The Backend Engineering Show
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
https://database.husseinnasser.com
Shopify posted a blog on tips to for scalable payment system, one tip peeked my interest related to switching from UUID to ULID. I explore the reasoning behind this in this video.
https://shopify.engineering/building-resilient-payment-systems
0:00 Intro
1:30 idempotency
6:30 UUID vs ULID
9:50 Clustered Index
13:30 Why UUID4 Inserts are slow
17:15 How ULID helps Shopify
22:00 Problem with tail pages
25:00 Does ULID help in all cases?
Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)
MongoDB Internal Architecture | The Backend Engineering Show
I’m a big believer that database systems share similar core fundamentals at their storage layer and understanding them allows one to compare different DBMS objectively. For example, How documents are stored in MongoDB is no different from how MySQL or PostgreSQL store rows. Everything goes to disk, the trick is to fetch what you need from disk efficiently with as fewer I/Os as possible, the rest is API. In this video I discuss the evolution of MongoDB internal architecture on how documents are stored and retrieved focusing on the index storage representation. I assume the reader is well versed with fundamentals of database engineering such as indexes, B+Trees, data files, WAL etc, you may pick up my database course to learn the skills. Let us get started.
Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon) https://database.husseinnasser.com
How UI/UX can break the backend
The User Interface/User Experience has great impact on the backend architecture and scalability. In this podcast I discuss three UI/UX that affected backend design and scalability.
0:00 Intro
1:40 UI vs UX
4:30 Google Chrome OmniBox
12:30 1 out of X Page
20:00 YouTube Notification
Resources
https://blog.apnic.net/2020/08/21/chromiums-impact-on-root-dns-traffic/
Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)
https://backend.husseinnasser.com
Do DHCP and DNS Servers Communicate?
In this video I explain how DHCP work and how it updates DNS entries for new hosts joining the network. I'll also mention Zero Config
0:00 Intro
1:00 the Network configuration
6:00 Showing DHCP in Wireshark
6:30 DHCP Discover
14:40 DHCP Offer
19:00 DHCP Request
21:30 DHCP ACK
22:00 How DHCP Updates DNS
26:15 Zero Configuration (mDNS, Link-local)
Resources
Dhcp https://datatracker.ietf.org/doc/html/rfc1541
Dynamic updates , dhcp RFC2136
https://datatracker.ietf.org/doc/html/rfc2136
https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/ipaddr_dhcp/configuration/xe-3se/3850/dhcp-xe-3se-3850-book/dhcp-client-option-12.pdf
RFC 1497
https://www.rfc-editor.org/rfc/rfc1497.html
https://www.rfc-editor.org/rfc/rfc6762#ref-Zeroconf
Link-local
https://www.rfc-editor.org/rfc/rfc3927
Compressing Certificates in TLS | The Backend Engineering Show
backend.husseinnasser.com
Certificates provide a way to authenticate both the server and the client and are included as part of the TLS handshake. However, the certificates can be large because the full certificate chain is included in the handshake. The large certificates can go up to 10KB in size and take multiple segments to deliver and assemble. RFC 8879 discusses how TLS compression can be achieved, I discuss that in this podcast. Enjoy.
0:00 Intro
4:15 Certificate Chain
6:00 Faking the chain
8:50 Certificate Stores
10:30 Including ROOT cert in the chain
12:00 The performance penalty of large certificate chain
20:15 RFC 8879 TLS Certificate Compression
23:00 How Compression Works in TLS 1.2 vs TLS 1.3
30:30 What could go wrong?
Resources
datatracker.ietf.org/doc/rfc8879/
www.rfc-editor.org/rfc/rfc5246
www.rfc-editor.org/rfc/rfc6928.html
OpenSSL new vulnerability
Two new vulnerabilities in openssl were discovered, we discuss them in this video
https://www.openssl.org/news/secadv/20221101.txt
TCP Protective Load Balancing coming to Linux Kernel 6.2
Google recent paper on protective load balancing in TCP attempts to improve packet drops and latency by making the host change the flow path using the IPv6 Flow label. The Linux kernels gets the PLB support in Linux 6.2 this December, let us discuss with this is.
When NodeJS I/O Blocks | The Backend Engineering Show
In this episode of the backend engineering show I go through an article I wrote discussing NodeJS Asynchronous I/O
https://medium.com/@hnasr/when-nodejs-i-o-blocks-327f8a36fbd4
Learn the fundamentals of network engineering, get my udemy course
https://network.husseinnasser.com
Buy me a coffee if you liked this
https://www.buymeacoffee.com/hnasr
0:00
3:00 Part 1 Socket/IO
9:48 Part 2 File I/O
12:42 Part 3 DNS
16:22 Part 4 NodeJS Single Threaded
19:10 Part 5 NodeJS Thread Pool
21:23 Part 6 DNS lookup bottleneck in Node
NGINX Internal Architecture - Workers | The Backend Engineering Show
Buy me a coffee if you liked this https://www.buymeacoffee.com/hnasr
In this podcast I explain the NGINX internal process architecture, how NGINX spins up multiple processes per core, how each process is pinned to a CPU core for minimum context switching, how NGINX accepts connections , parses requests and talks to the backend. Get my introduction to NGINX udemy course https://nginx.husseinnasser.com
Cloudflare is moving away from NGINX | The Backend Engineering Show
0:00 Intro
1:53 What NGINX is used for
3:37 NGINX Architecture
7:52 NGINX Limitations
17:12 Cloudflare Pingora
Buy me a coffee if you liked this
www.buymeacoffee.com/hnasr
Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon)
network.husseinnasser.com
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
database.husseinnasser.com
Introduction to NGINX (link redirects to udemy with coupon)
nginx.husseinnasser.com
Python on the Backend (link redirects to udemy with coupon)
python.husseinnasser.com
Become a Member on YouTube
www.youtube.com/channel/UC_ML5xP23TOWKUcc-oAE_Eg/join
Resources Resources:
blog.cloudflare.com/how-we-built-pingora-the-proxy-that-connects-cloudflare-to-the-internet/
www.nginx.com/blog/socket-sharding-nginx-release-1-9-1/amp/
Threads and Connections | The Backend Engineering Show
To learn more about networking fundamentals check out my udemy course Fundamentals of Networking for Effective Backends Head to network.husseinnasser.com for a discount coupon
0:00 Intro
2:00 Single Threading
6:30 Multi-Threading
14:15 Connection Listener
20:15 How Connections are Established
29:00 Single Listener/Worker thread
33:30 Single Listener, Multiple Worker threads
39:00 Single Listener, Multiple Workers with load balancing
42:10 Multiple Listeners on the same port (SO_REUSEPORT)
45:20 Multiple Single Threaded Backend
Buy me a coffee www.buymeacoffee.com/hnasr
Memcached Architecture | The Backend Engineering Show
0:00 Intro
4:40 What is Memcached?
7:45 Memory management
16:00 LRU
25:17 Threading and Connections
30:40 Read Example
34:30 Write Example
36:17 Write and Read collisions
39:40 Locking
40:30 Distributed Cache
43:30 Memcached with Docker/Telnet/NodeJS
45:00 Spin up a Memcached Docker container and telnet
52:17 Memcached and NodeJS
56:15 Four Memached Servers with NodeJS
01:01:00 Summary
Resources
www.cloudflare.com/learning/ddos/memcached-ddos-attack/
holmeshe.me/understanding-memcached-source-code-IV/
github.com/memcached/memcached/blob/master/doc/protocol.txt
docs.oracle.com/cd/E17952_01/mysql-5.6-en/ha-memcached-using-threads.html
holmeshe.me/understanding-memcached-source-code-I/
docs.oracle.com/cd/E17952_01/mysql-5.6-en/ha-memcached-using-memory.html
support-acquia.force.com/s/article/360005256114-Memcached-in-detail
www.alibabacloud.com/blog/redis-vs-memcached-in-memory-data-storage-systems_592091
www.usenix.org/system/files/conference/nsdi13/nsdi13-final197.pdf
memcached.org/blog/persistent-memory-2/
memcached.org/blog/modern-lru/ Buy me a coffee www.buymeacoffee.com/hnasr
Is SmartNIC a game changer for network performance? | The Backend Engineering Show
In this episode of the backend engineering show I go through the main job of the network interface controller (NIC for short) and how the datacenter is pushing it to the limit by allowing it to do more TCP/IP processing, creating what is being popularized as smartNIC.
0:00 Intro
1:20 What is a NIC?
3:40 NIC job
8:00 When does the OS get involved
12:40 Promiscuous mode
14:00 SmartNIC
18:30 Disadvantages
Resources
https://developer.nvidia.com/networking/ethernet-adapters
https://www.theregister.com/2022/08/11/smartnics_network_market/
https://arxiv.org/abs/1803.09615
Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon)
https://network.husseinnasser.com