This study aimed to investigate the effect of STEAM-based education on reducing math anxiety and enhancing academic achievement among ninth-grade middle school students. A quasi-experimental single-group pre-test–post-test design was employed. Twenty-eight ninth-grade male students in Gorgan, Iran, were selected via multi-stage cluster random sampling. Instruments included the Math Anxiety Rating Scale (MARS) and a researcher-developed mathematics achievement test, both validated for content and reliability. The STEAM intervention spanned five sessions incorporating group work, games, problem-solving, technology, and artistic components aligned with the mathematics curriculum. Statistical analysis involved paired t-tests, Wilcoxon signed-rank tests, Spearman correlation, and quantile regression. The STEAM intervention significantly reduced math anxiety in classroom settings (p = 0.001), homework (p = 0.002), and tests (p = 0.001), but not in general anxiety (p = 0.301) or anxiety in social settings (p = 0.251). Students with higher pre-test achievement experienced lower anxiety post-intervention. Academic achievement significantly improved following the intervention (p = 0.000, Z = -3.658). However, the negative correlation between anxiety reduction and achievement gain was not statistically significant (p = 0.606). STEAM-based instruction effectively reduced specific dimensions of math anxiety and significantly enhanced academic performance among ninth-grade students. While it proved beneficial in educational contexts, additional measures may be needed to address generalized and socially driven anxiety.