Weekend Reading #92
Weekend Reading: A weekly roundup of interesting Software Architecture and Programming articles from tech companies. Find fresh ideas and insights every weekend.
This week: bool.dev explains how X’s For You feed ranks, filters, diversifies, and caches candidates. Netflix compares two Flink autoscaling systems at 30,000-job scale, Meta introduces MetaRoCE for AI-scale Ethernet, and Uber shows how file layout and Hudi column stats reduce the cost of highly selective export workloads.
A Like Is Worth 0.5: How the X Algorithm Actually Ranks Your Feed
👉 For software architects and backend engineers studying feed ranking, recommendation systems, and production ML architecture
This bool.dev article traces the current X For You request path from candidate retrieval through scoring, filtering, diversification, visibility rules, and cached refreshes. The most useful part is the separation between model predictions and ranking policy: the model predicts dozens of probabilities, while configured weights and post-processing rules decide the final ordering. It also corrects common myths, such as treating coefficients as direct counts of likes or reports.
A Tale of Two Flink Autoscalers
👉 For platform engineers and data infrastructure teams running stateful stream processing at scale
Netflix explains why it now runs two Apache Flink autoscalers and how it is converging on the open-source one. The article compares cluster-level autoscaling with per-operator scaling based on true processing rate, then covers the production work needed to make it safe: one durable workflow per job, JobManager metric fixes, forward-chain-aware scaling, sink backpressure detection, and conservative utilization targets. It is a useful build-versus-adopt case study because the hardest part is not the scaling formula, but reliable metrics, state recovery, and operational safety.
MetaRoCE: A New RDMA Transport Built for AI-Scale Ethernet
👉 For infrastructure engineers designing GPU cluster networking and AI training fabrics
Meta introduces MetaRoCE, a clean-sheet RDMA transport designed for AI workloads on commodity Ethernet. The design moves more intelligence to endpoints, supports native out-of-order delivery, packet-level multipathing, loss tolerance without PFC, and receiver-driven rate hints for congestion control. The article is strong because it explains why traditional RoCE assumptions break at million-GPU scale and shows how transport design directly affects GPU utilization, tail latency, and failure recovery.
Running Cost-Efficient Export Workloads at Uber
👉 For data engineers and platform teams optimizing lakehouse workloads and storage costs
Uber describes a common but expensive pattern: narrow export queries that return a few records but scan large historical Hudi tables. These repeated scans keep cold data hot, defeat GCS auto-class tiering, and increase storage, metadata, egress, and latency costs. Uber’s solution combines predicate-column sorting with Hudi column stats so engines can prune files without repeatedly touching Parquet footers, showing why access patterns and physical layout matter as much as storage tiering.