In-depth engineering analysis: Debugging CPU throttling in CFS scheduler with kernel tracepoints. Practical diagnostics, logs and production configuration.
#linux
Engineering writeups and incident post-mortems for #linux.
Articles: 29In-depth engineering analysis: Zero-downtime kernel live patching with kpatch in enterprise Linux environments. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Linux kernel memory fragmentation: compact_memory and memory watermark tuning. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: High-performance IPC in Linux: POSIX shared memory vs Unix domain sockets. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Rust vs C for Linux kernel modules: memory safety guarantees and FFI boundary overhead. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Linux TCP window scaling and socket buffer auto-tuning. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Building deterministic test environments with rootless containers. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Linux network namespace veth pair latency overhead analysis. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Kubernetes kubelet eviction thresholds and cgroups v2: preventing node NotReady cascades. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Profiling Linux kernel lock contention with perf lock and FlameGraphs. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: systemd cgroups v2: practical differences between MemoryHigh and MemoryMax in production. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: PostgreSQL performance cliff caused by Transparent Huge Pages (THP). Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Silent data corruption in legacy C daemon: struct packing alignment and x86-64 ABI mismatch. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Tuning Linux io_uring for async NVMe database storage engine: eliminating syscall overhead. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Migrating crontab spaghetti to systemd timers: memory isolation, logs and restart policies. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: eBPF XDP: dropping 14 Mpps SYN floods at the NIC driver layer. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Linux sysctl tuning for 100k concurrent WebSocket connections on a single node. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: systemd-resolved in isolated VPC: DNS caching gotchas and split-horizon resolution. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Migrating from iptables-legacy to nftables on a 40G edge router. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: NTP clock skew of 1.2s: how clock drift invalidated distributed HMAC signatures. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: glibc malloc arena fragmentation: how 300 MB of payload bloated to 16 GB heap. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Docker devicemapper loop-lvm: legacy production node crashes and migration to overlay2. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: PHP-FPM zombie workers: IPC semaphore exhaustion and pool crashes. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Silent network packet drops: NIC RX ring buffer exhaustion under traffic microbursts. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Redis BGSAVE fork latency: how Copy-on-Write doubled memory footprint and stalled clients. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: Deleted unlinked files holding 200 GB disk space: tracking open fds with lsof. Practical diagnostics, logs and production configuration.
In-depth engineering analysis: OOM killer priority tuning: how a bad oom_score_adj killed a critical monitoring daemon. Practical diagnostics, logs and production configuration.
How short-lived inter-service HTTP calls exhausted ephemeral ports and triggered cascading 500 errors.
A classic war story: why 400 GB of free disk space could not prevent 'No space left on device' when ext4 inode table hit 100% capacity.