Caleb Williams grew up in the Washington, D.C. area, specifically in Bowie, Maryland, where he lived in his early years. He later moved to Washington, D.C., where he attended Gonzaga College High School. He lived on the same block as the school in the city's northeast neighborhood. Williams has described Washington, D.C. as one of the best places to grow up, highlighting its diversity, educational opportunities, food, and access to U.S. history as influential in his development. His father, Carl Williams, played a significant role in his upbringing and athletic training.