Skip to main content
The Backend Engineering Show with Hussein Nasser

The Backend Engineering Show with Hussein Nasser

By Hussein Nasser

Welcome to the Backend Engineering Show podcast with your host Hussein Nasser. If you like software engineering you’ve come to the right place. I discuss all sorts of software engineering technologies and news with specific focus on the backend. All opinions are my own.

Most of my content in the podcast is an audio version of videos I post on my youtube channel here www.youtube.com/c/HusseinNasser-software-engineering

Buy me a coffee
www.buymeacoffee.com/hnasr

🧑‍🏫 Courses I Teach
husseinnasser.com/courses
Available on
Apple Podcasts Logo
Castbox Logo
Google Podcasts Logo
Overcast Logo
Pocket Casts Logo
PodBean Logo
RadioPublic Logo
Spotify Logo
Currently playing episode

Episode 58 - The Art of Software Troubleshooting

The Backend Engineering Show with Hussein NasserOct 20, 2018

00:00
13:08
Google Patches Linux kernel with 40% TCP performance

Google Patches Linux kernel with 40% TCP performance

Get my backend course https://backend.win


Google submitted a patch to Linux Kernel 6.8 to improve TCP performance by 40%, this is done via rearranging the tcp structures for better cpu cache lines, I explore this here. 0:00 Intro 0:30 Google improves Linux Kernel TCP by 40% 1:40 How CPU Cache Line Works 6:45 Reviewing the Google Patch https://www.phoronix.com/news/Linux-6.8-Networking https://lore.kernel.org/netdev/20231129072756.3684495-1-lixiaoyan@google.com/ Discovering Backend Bottlenecks: Unlocking Peak Performance https://performance.husseinnasser.com

Mar 05, 202413:35
Database Torn pages

Database Torn pages

0:00 Intro

2:00 File System Block vs Database Pages

4:00 Torn pages or partial page

7:40 How Oracle Solves torn pages

8:40 MySQL InnoDB Doublewrite buffer

10:45 Postgres Full page writes



Feb 29, 202415:33
Cloudflare Open sources Pingora (NGINX replacement)
Feb 28, 202431:06
The Internals of MongoDB

The Internals of MongoDB

https://backend.win

https://databases.win


I’m a big believer that database systems share similar core fundamentals at their storage layer and understanding them allows one to compare different DBMS objectively. For example, How documents are stored in MongoDB is no different from how MySQL or PostgreSQL store rows. 

Everything goes to pages of fixed size and those pages are flushed to disk. 


Each database define page size differently based on their workload, for example MongoDB default page size is 32KB, MySQL InnoDB is 16KB and PostgreSQL is 8KB.


The trick is to fetch what you need from disk efficiently with as fewer I/Os as possible, the rest is API.  


In this video I discuss the evolution of MongoDB internal architecture on how documents are stored and retrieved focusing on the index storage representation. I assume the reader is well versed with fundamentals of database engineering such as indexes, B+Trees, data files, WAL etc, you may pick up my database course to learn the skills.

Let us get started.

Feb 19, 202444:57
The Beauty of Programming Languages

The Beauty of Programming Languages

In this video I explore the type of languages, compiled, garbage collected, interpreted, JIT and more. 


Feb 19, 202418:17
The Danger of Defaults - A PostgreSQL Story

The Danger of Defaults - A PostgreSQL Story

I talk about default values and how PostgreSQL 14 got slower when a default parameter has changed. Mike's blog https://smalldatum.blogspot.com/2024/02/it-wasnt-performance-regression-in.html



Feb 18, 202411:35
Database Background writing

Database Background writing

Background writing is a process that writes dirty pages in shared buffer to the disk (well goes to the OS file cache then get flushed to disk by the OS) I go into this process in this video
Feb 16, 202409:09
The Cost of Memory Fragmentation

The Cost of Memory Fragmentation

Fragmentation is a very interesting topic to me, especially when it comes to memory. While virtually memory does solve external fragmentation (you can still allocate logically contiguous memory in non-contiguous physical memory) it does however introduce performance delays as we jump all over the physical memory to read what appears to us for example as contiguous array in virtual memory. You see, DDR RAM consists of banks, rows and columns. Each row has around 1024 columns and each column has 64 bits which makes a row around 8kib. The cost of accessing the RAM is the cost of “opening” a row and all its columns (around 50-100 ns) once the row is opened all the columns are opened and the 8 kib is cached in the row buffer in the RAM. The CPU can ask for an address and transfer 64 bytes at a time (called bursts) so if the CPU (or the MMU to be exact) asks for the next 64 bytes next to it, it comes at no cost because the entire row is cached in the RAM. However if the CPU sends a different address in a different row the old row must be closed and a new row should be opened taking an additional 50 ns hit. So spatial access of bytes ensures efficiency, So fragmentation does hurt performance if the data you are accessing are not contiguous in physical memory (of course it doesn’t matter if it is contiguous in virtual memory). This kind of remind me of the old days of HDD and how the disk needle physically travels across the disk to read one file which prompted the need of “defragmentation” , although RAM access (and SSD NAND for that matter) isn’t as bad. Moreover, virtual memory introduces internal fragmentation because of the use of fixed-size blocks (called pages and often 4kib in size), and those are mapped to frames in physical memory. So if you want to allocate a 32bit integer (4 bytes) you get a 4 kib worth of memory, leaving a whopping 4092 allocated for the process but unused, which cannot be used by the OS. These little pockets of memory can add up as many processes. Another reason developers should take care when allocating memory for efficiency.

Jan 29, 202439:07
The Real Hidden Cost of a Request
Dec 13, 202313:09
Why create Index blocks writes

Why create Index blocks writes

Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon) https://database.husseinnasser.com Why create Index blocks writes In this video I explore how create index, why does it block writes and how create index concurrently work and allow writes. 0:00 Intro 1:28 How Create Index works 4:45 Create Index blocking Writes 5:00 Create Index Concurrently

Oct 28, 202312:04
The Problems of an HTTP/3 Backend

The Problems of an HTTP/3 Backend

HTTP/3 is getting popular in the cloud scene but before you migrate to HTTP/3 consider its cost. I explore it here. 0:00 Intro HTTP/3 is getting popular 3:40 HTTP/1.1 Cost 5:18 HTTP/2 Cost 6:30 HTTP/3 Cost https://blog.apnic.net/2023/09/25/why-http-3-is-eating-the-world/

Oct 05, 202312:19
Encrypted Client Hello - The Pros & Cons

Encrypted Client Hello - The Pros & Cons


The Encrypted Client Hello or ECH is a new RFC that encrypts the TLS client hello to hide sensitive information like the SNI. In this video I go through pros and cons of this new rfc. 0:00 Intro 2:00 SNI 4:00 Client Hello 8:40 Encrypted Client Hello 11:30 Inner Client Hello Encryption 18:00 Client-Facing Outer SNI 21:20 Decrypting Inner Client Hello 23:30 Disadvantages 26:00 Censorship vs Privacy ECH https://blog.cloudflare.com/announcing-encrypted-client-hello/ https://chromestatus.com/feature/6196703843581952

Sep 29, 202333:18
The Journey of a Request to the Backend
Aug 01, 202352:59
They Enabled Postgres Partitioning and their Backend fell apart
Jun 24, 202332:40
WebTransport - A Backend Game Changer

WebTransport - A Backend Game Changer

WebTransport is a cutting-edge protocol framework designed to support multiplexed and secure transport over HTTP/2 and HTTP/3. It brings together the best of web and transport technologies, providing an all-in-one solution for real-time, bidirectional communication on the web.

Watch full episode (subscribers only) https://spotifyanchor-web.app.link/e/cTSGkq5XuAb


Jun 09, 202315:01
WebTransport - A Backend Game Changer

WebTransport - A Backend Game Changer

WebTransport is a cutting-edge protocol framework designed to support multiplexed and secure transport over HTTP/2 and HTTP/3. It brings together the best of web and transport technologies, providing an all-in-one solution for real-time, bidirectional communication on the web.
Jun 09, 202331:12
Your SSD lies but that's ok | Postgres fsync

Your SSD lies but that's ok | Postgres fsync

fsync is a linux system call that flushes all pages and metadata for a given file to the disk. It is indeed an expensive operation but required for durability especially for database systems. Regular writes that make it to the disk controller are often placed in the SSD local cache to accumulate more writes before getting flushed to the NAND cells. However when the disk controller receives this flush command it is required to immediately persist all of the data to the NAND cells. Some SSDs however don't do that because they don't trust the host and no-op the fsync. In this video I explain this in details and go through details on how postgres provide so many options to fine tune fsync 0:00 Intro 1:00 A Write doesn’t write 2:00 File System Page Cache 6:00 Fsync 7:30 SSD Cache 9:20 SSD ignores the flush 9:30 15 Year old Firefox fsync bug 12:30 What happens if SSD loses power 15:00 What options does Postgres exposes? 15:30 open_sync (O_SYNC) 16:15 open_datasync (O_DSYNC) 17:10 O_DIRECT 19:00 fsync 20:50 fdatasync 21:13 fsync = off 23:30 Don’t make your API simple 26:00 Database on metal?

May 25, 202330:04
The problem with software engineering

The problem with software engineering

ego is the main problem to a defective software product. the ego of the engineer or the tech lead seeps into the quality of the product. Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com

May 21, 202317:39
2x Faster Reads and Writes with this MongoDB feature | Clustered Collections

2x Faster Reads and Writes with this MongoDB feature | Clustered Collections


Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon) https://database.husseinnasser.com


In version 5.3, MongoDB introduced a feature called clustered collection which stores documents in the _id index as oppose to the hidden wiredTiger hidden index. This eliminates an entire b+tree seek for reads using the _id index and also removes the additional write to the hidden index speeding both reads and writes. 


However like we know in software engineering, everything has a cost. This feature does come with a few that one must be aware of before using it. In this video I discuss the following 


  • How Original MongoDB Collections Work
  • How Clustered Collections Work
  • Benefits of Clustered Collections
  • Limitations of Clustered Collections

 



May 11, 202327:02
Prime Video Swaps Microservices for Monolith: 90% Cost Reduction

Prime Video Swaps Microservices for Monolith: 90% Cost Reduction

Prime video engineering team has posted a blog detailing how they moved their live stream monitoring service from microservices to a monolith reducing their cost by 90%, let us discuss this 0:00 Intro 2:00 Overview 10:35 Distributed System Overhead 21:30 From Microservices to Monolith 29:00 Scaling the Monolith 32:30 Takeaways https://www.primevideotech.com/video-streaming/scaling-up-the-prime-video-audio-video-monitoring-service-and-reducing-costs-by-90 Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com

May 06, 202335:58
A Deep Dive in How Slow SELECT * is

A Deep Dive in How Slow SELECT * is

Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon) https://database.husseinnasser.com In a row-store database engine, rows are stored in units called pages. Each page has a fixed header and contains multiple rows, with each row having a record header followed by its respective columns. When the database fetches a page and places it in the shared buffer pool, we gain access to all rows and columns within that page. So, the question arises: if we have all the columns readily available in memory, why would SELECT * be slow and costly? Is it really as slow as people claim it to be? And if so why is it so? In this post, we will explore these questions and more. 0:00 Intro 1:49 Database Page Layout 5:00 How SELECT Works 10:49 No Index-Only Scans 18:00 Deserialization Cost 21:00 Not All Columns are Inline 28:00 Network Cost 36:00 Client Deserialization https://medium.com/@hnasr/how-slow-is-select-8d4308ca1f0c

May 02, 202339:24
AWS Serverless Lambda Supports Response Streaming

AWS Serverless Lambda Supports Response Streaming


Lambda now supports Response payload streaming, now you can flush changes to the network socket as soon as it is available and it will be written to the client socket. I think this is a game changing feature



0:00 Intro

1:00 Traditional Lambda

3:00 Server Sent Events & Chunk-Encoding

5:00 What happens to clients?

6:00 Supported Regions

7:00 My thoughts


Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)

https://backend.husseinnasser.com


Apr 07, 202313:14
The Cloudflare mTLS vulnerability - A Deep Dive Analysis

The Cloudflare mTLS vulnerability - A Deep Dive Analysis

Cloudflare released a blog detailing a vulnerability that has been in their system for nearly two years. it is related to mTLS or mutual TLS and specifically client certificate revocation. I explore this in details 0:00 Intro 3:00 The Vulnerability 7:00 What happened? 8:50 Certificate Revocation 12:30 Rejecting certain endpoints 17:00 Certificate Authentication 20:30 Certificate serial number 24:00 Session Resumption (PSK) 35:00 The bug 37:00 How they addressed the problem Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com

Apr 06, 202343:13
The Virgin Media ISP outage - What happened?

The Virgin Media ISP outage - What happened?

BGP (Border gateway protocol) withdrawals caused the Virgin media ISP customers to lose their Internet connection. I go into details on this video. 0:00 Intro 2:00 What happened? 4:11 How BGP works? 11:50 Version media withdrawals 15:00 Deep dive Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com

Apr 06, 202323:24
GitHub SSH key is Leaked - How bad is this?

GitHub SSH key is Leaked - How bad is this?

GitHub Accidentally Exposed their SSH RSA Private key, this is the message you will get . @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ @ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @ @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY! Someone could be eavesdropping on you right now (man-in-the-middle attack)! It is also possible that a host key has just been changed. The fingerprint for the RSA key sent by the remote host is SHA256:uNiVztksCsDhcc0u9e8BujQXVUpKZIDTMczCvj3tD2s. Please contact your system administrator. Add correct host key in ~/.ssh/known_hosts to get rid of this message. Host key for github.com has changed and you have requested strict checking. Host key verification failed. In this video I discuss how bad is this,. 0:00 Intro 1:10 What happened? 3:00 SSH vs TLS Authentication 6:00 SSH Connect 7:45 How bad is the github leak? 15:00 What should you do? 18:50 Is ECDSA immune? https://github.blog/2023-03-23-we-updated-our-rsa-ssh-host-key/

Mar 30, 202321:57
Cookie Hijacking - How Linus Tech Tips got Hacked

Cookie Hijacking - How Linus Tech Tips got Hacked

How Linus Tech Tips channel got Hacked



In this short video we explain how was it possible for Linux to get hacked with cookies hijacking. 0:00 Intro 0:47 TLDR what happened 5:10 Cookies in Chrome 7:30 Cookies Hijacking 8:46 Session Tokens (Access/Refresh) 10:00 Remedies


Mar 29, 202313:34
All Postgres Locks Explained | A Deep Dive
Mar 19, 202349:11
Pinterest moves to HTTP/3
Mar 16, 202325:54
Why Loom Users got each others’ sessions on March 7th 2023

Why Loom Users got each others’ sessions on March 7th 2023

On March 7 2023, Loom users started seeing each others data as a result of cookies getting leaked from the CDN. This loom security breach is really critical. Let us discuss   0:00 Intro 1:00 Why Cookies 2:00 How this happens 5:50 What caused it? 7:30 How Loom solved it? 8:20 Reading the RCA 10:30 Remedies

Mar 14, 202314:58
How Discord Stores Trillions of Messages - A deep dive
Mar 11, 202301:09:20
Postgres Architecture | The Backend Engineering Show
Feb 16, 202334:04
How Alt-Svc switches HTTP/2 clients to use HTTP/3 | The Backend Engineering Show

How Alt-Svc switches HTTP/2 clients to use HTTP/3 | The Backend Engineering Show

The Alt-Svc header/frame is a capability that allows the server to adverse alternative services to the connected application available in protocols, ports or domains. It is available as a response header alt-svc and also as an HTTP/2 frame. Let us discuss this capability.

0:00 Intro

1:38 what is alt-svc?

5:30 uses of h3 in alt-svc

8:00 alt-svc header

10:00 Alt-svc header with 103 early hints

14:48 h2 altsvc frame

18:30 SVCB DNS record

21:20 Summary

Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)

https://backend.husseinnasser.com

Feb 13, 202323:58
Your DNS queries will eventually look like this (0x20 DNS encoding)

Your DNS queries will eventually look like this (0x20 DNS encoding)

Correction: Google is implementing the proposal originally submitted by researchers from Georgia institute of tech. I incorrectly said in the video that google is proposing this .

Google is finally implementing a proposal from 2008 by researchers from Georgia institute of technology to make DNS cache poisoning .

https://astrolavos.gatech.edu/articles/increased_dns_resistance.pdf

https://datatracker.ietf.org/doc/html/draft-vixie-dnsext-dns0x20-00

0:00 Intro

2:00 How DNS Work

5:00 DNS Cache Poisoning

14:00 gOoGLe dot CoM

16:20 ASCII 0x20 casing

18:30 Randomizing the casing with encryption

22:30 limitations of this proposal

24:00 Credits

Jan 28, 202326:20
DropBox Removed their SSDs, got 20% faster writes

DropBox Removed their SSDs, got 20% faster writes

https://dropbox.tech/infrastructure/increasing-magic-pocket-write-throughput-by-removing-our-ssd-cache-disks

In this episode of the backend engineering show I’ll discuss how Dropbox improved their write through put by 20% by removing all their SSDs (yes I was surprised too). DropBox uses an SSD layer as a write-back cache with SMR drives as their backend persistent storage. They changed their model to write directly to the hard drives.

0:00 Intro

2:00 Article Summary

3:00 SMR Drives

6:00 SSD Cache & WriteBack

8:00 Replacing Cache

9:30 Storage Engine Background

14:30 Why did they do it

15:00 The limitation of SSDs & Zoned Namespaces

19:30 Updating the Storage Engine

22:30 Tradeoffs

26:00 Rollout

28:00 Summary

Jan 24, 202331:18
MySQL on HTTP/3 | The Backend Engineering Show

MySQL on HTTP/3 | The Backend Engineering Show

The communication between backend applications and database systems always fascinated me. The protocols keep evolving and we are in constant search for an efficient protocol that best fit the workload of Backend-DB communication.

In this episode of the backend engineering show I go through a blog written by  @PlanetScale  doing an experimentation of using HTTP/3 and HTTP/2 comparing it with MySQL Binary protocol.

https://planetscale.com/blog/faster-mysql-with-http3

0:00 Intro

7:45 MySQL Binary vs HTTP

10:20 The Tests

15:00 Connection Cost + Select 1

22:00 Parallel Select

26:00 The cost of H2 and H3

Jan 05, 202337:10
How Shopify’s engineering improved writes by 50% with ULID | The Backend Engineering Show
Dec 23, 202232:11
MongoDB Internal Architecture | The Backend Engineering Show

MongoDB Internal Architecture | The Backend Engineering Show

I’m a big believer that database systems share similar core fundamentals at their storage layer and understanding them allows one to compare different DBMS objectively. For example, How documents are stored in MongoDB is no different from how MySQL or PostgreSQL store rows. Everything goes to disk, the trick is to fetch what you need from disk efficiently with as fewer I/Os as possible, the rest is API.  In this video I discuss the evolution of MongoDB internal architecture on how documents are stored and retrieved focusing on the index storage representation. I assume the reader is well versed with fundamentals of database engineering such as indexes, B+Trees, data files, WAL etc, you may pick up my database course to learn the skills. Let us get started.



Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon) https://backend.husseinnasser.com Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon) https://network.husseinnasser.com Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon) https://database.husseinnasser.com



Dec 16, 202244:13
How UI/UX can break the backend

How UI/UX can break the backend

The User Interface/User Experience has great impact on the backend architecture and scalability. In this podcast I discuss three UI/UX that affected backend design and scalability.

0:00 Intro

1:40 UI vs UX

4:30 Google Chrome OmniBox

12:30 1 out of X Page

20:00 YouTube Notification

Resources

https://blog.apnic.net/2020/08/21/chromiums-impact-on-root-dns-traffic/

Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)

https://backend.husseinnasser.com

Dec 01, 202230:35
Do DHCP and DNS Servers Communicate?

Do DHCP and DNS Servers Communicate?

In this video I explain how DHCP work and how it updates DNS entries for new hosts joining the network. I'll also mention Zero Config

0:00 Intro

1:00 the Network configuration

6:00 Showing DHCP in Wireshark

6:30 DHCP Discover

14:40 DHCP Offer

19:00 DHCP Request

21:30 DHCP ACK

22:00 How DHCP Updates DNS

26:15 Zero Configuration (mDNS, Link-local)

Resources

Dhcp https://datatracker.ietf.org/doc/html/rfc1541

Dynamic updates , dhcp RFC2136

https://datatracker.ietf.org/doc/html/rfc2136

https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/ipaddr_dhcp/configuration/xe-3se/3850/dhcp-xe-3se-3850-book/dhcp-client-option-12.pdf

RFC 1497

https://www.rfc-editor.org/rfc/rfc1497.html

https://www.rfc-editor.org/rfc/rfc6762#ref-Zeroconf

Link-local

https://www.rfc-editor.org/rfc/rfc3927

Nov 16, 202230:28
Compressing Certificates in TLS | The Backend Engineering Show

Compressing Certificates in TLS | The Backend Engineering Show

Fundamentals of Backend Engineering Design patterns udemy course (link redirects to udemy with coupon)
backend.husseinnasser.com

Certificates provide a way to authenticate both the server and the client and are included as part of the TLS handshake. However, the certificates can be large because the full certificate chain is included in the handshake. The large certificates can go up to 10KB in size and take multiple segments to deliver and assemble. RFC 8879 discusses how TLS compression can be achieved, I discuss that in this podcast. Enjoy.

0:00 Intro
4:15 Certificate Chain
6:00 Faking the chain
8:50 Certificate Stores
10:30 Including ROOT cert in the chain
12:00 The performance penalty of large certificate chain
20:15 RFC 8879 TLS Certificate Compression
23:00 How Compression Works in TLS 1.2 vs TLS 1.3
30:30 What could go wrong?
Resources
datatracker.ietf.org/doc/rfc8879/
www.rfc-editor.org/rfc/rfc5246
www.rfc-editor.org/rfc/rfc6928.html
Nov 08, 202234:06
OpenSSL new vulnerability

OpenSSL new vulnerability

Two new vulnerabilities in openssl were discovered, we discuss them in this video

https://www.openssl.org/news/secadv/20221101.txt


Nov 06, 202210:27
TCP Protective Load Balancing coming to Linux Kernel 6.2

TCP Protective Load Balancing coming to Linux Kernel 6.2

Google recent paper on protective load balancing in TCP attempts to improve packet drops and latency by making the host change the flow path using the IPv6 Flow label. The Linux kernels gets the PLB support in Linux 6.2 this December, let us discuss with this is.   

Nov 03, 202214:51
When NodeJS I/O Blocks | The Backend Engineering Show

When NodeJS I/O Blocks | The Backend Engineering Show

In this episode of the backend engineering show I go through an article I wrote discussing NodeJS Asynchronous I/O

https://medium.com/@hnasr/when-nodejs-i-o-blocks-327f8a36fbd4

Learn the fundamentals of network engineering, get my udemy course

https://network.husseinnasser.com


Buy me a coffee if you liked this 

https://www.buymeacoffee.com/hnasr



0:00

3:00 Part 1 Socket/IO

9:48 Part 2 File I/O

12:42 Part 3 DNS

16:22 Part 4 NodeJS Single Threaded

19:10 Part 5 NodeJS Thread Pool

21:23 Part 6 DNS lookup bottleneck in Node

Oct 12, 202225:57
NGINX Internal Architecture - Workers | The Backend Engineering Show

NGINX Internal Architecture - Workers | The Backend Engineering Show

Buy me a coffee if you liked this https://www.buymeacoffee.com/hnasr

In this podcast I explain the NGINX internal process architecture, how NGINX spins up multiple processes per core, how each process is pinned to a CPU core for minimum context switching,  how NGINX accepts connections , parses requests and talks to the backend.   Get my introduction to NGINX udemy course  https://nginx.husseinnasser.com

Oct 07, 202215:55
Cloudflare is moving away from NGINX | The Backend Engineering Show
Oct 04, 202226:03
Threads and Connections | The Backend Engineering Show

Threads and Connections | The Backend Engineering Show

In this episode of the backend engineering show I discuss the evolution of multi-threading apps, their pros and cons and then I go through 5 threading model and how they interleave with backend connection management between the threads and requests handlings. Enjoy

To learn more about networking fundamentals check out my udemy course Fundamentals of Networking for Effective Backends Head to
network.husseinnasser.com for a discount coupon

0:00 Intro
2:00 Single Threading
6:30 Multi-Threading
14:15 Connection Listener
20:15 How Connections are Established
29:00 Single Listener/Worker thread
33:30 Single Listener, Multiple Worker threads
39:00 Single Listener, Multiple Workers with load balancing
42:10 Multiple Listeners on the same port (SO_REUSEPORT)
45:20 Multiple Single Threaded Backend
Buy me a coffee www.buymeacoffee.com/hnasr
Sep 01, 202250:18
Memcached Architecture | The Backend Engineering Show

Memcached Architecture | The Backend Engineering Show

Memcached is an in memory cache with one major feature be a transient cache. Memcached has a very simple design. It was originally designed to help with database load by storing the query result in memory to avoid further querying the database. By default it has no authentication, a simple text protocols, servers don’t talk to each other. This video discuss the architecture of the cache, design choices and have some critics of the design choices. I go through a demo at the end using docker, telnet and nodes. Enjoy
0:00 Intro
4:40 What is Memcached?
7:45 Memory management
16:00 LRU
25:17 Threading and Connections
30:40 Read Example
34:30 Write Example
36:17 Write and Read collisions
39:40 Locking
40:30 Distributed Cache
43:30 Memcached with Docker/Telnet/NodeJS
45:00 Spin up a Memcached Docker container and telnet
52:17 Memcached and NodeJS
56:15 Four Memached Servers with NodeJS
01:01:00 Summary
Resources
www.cloudflare.com/learning/ddos/memcached-ddos-attack/
holmeshe.me/understanding-memcached-source-code-IV/
github.com/memcached/memcached/blob/master/doc/protocol.txt
docs.oracle.com/cd/E17952_01/mysql-5.6-en/ha-memcached-using-threads.html
holmeshe.me/understanding-memcached-source-code-I/
docs.oracle.com/cd/E17952_01/mysql-5.6-en/ha-memcached-using-memory.html
support-acquia.force.com/s/article/360005256114-Memcached-in-detail
www.alibabacloud.com/blog/redis-vs-memcached-in-memory-data-storage-systems_592091
www.usenix.org/system/files/conference/nsdi13/nsdi13-final197.pdf
memcached.org/blog/persistent-memory-2/
memcached.org/blog/modern-lru/ Buy me a coffee www.buymeacoffee.com/hnasr
Aug 27, 202250:30
Is SmartNIC a game changer for network performance? | The Backend Engineering Show

Is SmartNIC a game changer for network performance? | The Backend Engineering Show

In this episode of the backend engineering show I go through the main job of the network interface controller (NIC for short) and how the datacenter is pushing it to the limit by allowing it to do more TCP/IP processing, creating what is being popularized as smartNIC.

0:00 Intro

1:20 What is a NIC?

3:40 NIC job

8:00 When does the OS get involved

12:40 Promiscuous mode

14:00 SmartNIC

18:30 Disadvantages

Resources

https://developer.nvidia.com/networking/ethernet-adapters

https://www.theregister.com/2022/08/11/smartnics_network_market/

https://arxiv.org/abs/1803.09615

Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon)

https://network.husseinnasser.com

Aug 15, 202221:23
Consistent Hashing | The Backend Engineering Show
Aug 06, 202224:42
Replacing TCP for the Datacenter - Discussing the Homa paper

Replacing TCP for the Datacenter - Discussing the Homa paper

In this episode of the backend engineering show I go through and discuss the Homa Protocol paper which attempts to replace TCP as a protocol in the data centers. I learned a lot from this paper, I have my criticisms of certain aspects, timestamps for topics discussed below.

It appears there is a path to replace TCP in the datacenter and professor John tries to explain this path.


Referenced materials mentioned in the episode

Overview paper

https://web.stanford.edu/~ouster/cgi-bin/papers/replaceTcp.pdf

Homa 2018 paper (Details)

https://people.csail.mit.edu/alizadeh/papers/homa-sigcomm18.pdf

NIC Offloading in Linux

https://en.wikipedia.org/wiki/TCP_offload_engine#Support_in_Linux

Curl disabling Nigel Algo

https://github.com/curl/curl/commit/4732ca5724072f132876f520c8f02c7c5b654d9

0:00 Intro

3:00 The nature of networking data center

5:30 TCP Segments

7:30 There is no “Request” in TCP

12:00 What so unique about Data centers?

14:00 Message Throughput vs Data throughput

18:25 Congestion Control

22:38 Homa’s Congestion Control

25:00 Server Core Load Balancing

28:30 NIC offloading

30:00 Everything Wrong about TCP

37:00 Why not QUIC?

40:00 Limitation of Streaming

44:10 Load Balancing Stream Reading

47:15 Can we treat Segments as Messages?

51:00 Dispatching Messages is Easier

53:00 Connection Orientation

1:00:00 Sender Driven Congestion Control

1:03:00 In Order Packet Delivery

1:07:00 DCTCP

1:08:30 Homa is Message Based

1:11:00 Home is Connection Less

1:12:00 Receiver Driven Congestion Control

1:15:19 Out of Order Packets

1:16:20 Homa API is not Compatible with TCP

1:17:40 Will Homa come to HTTP?

1:18:45 Conclusion

Aug 01, 202201:23:44