Beyond Absolute Positional Embeddings with Relative and Rotary Methods

This post explores how positional embeddings evolved from absolute to relative to rotary forms, showing how each approach helps transformers capture sequence order and relationships more effectively while balancing flexibility, efficiency, and model complexity.

Inside Transformers: Scaled Dot-Product Attention & the Role of Position

Dive into the heart of transformer layers with a step-by-step look at scaled dot-product attention and discover how adding positional embeddings lets models capture both meaning and order.

BM25 Demystified: A Simple, Robust Baseline for Modern Retrieval

BM25 remains the go-to search baseline: this post shows you how its TF saturation, length normalization, and probabilistic foundation keep it robust.