For a decade, gradient-boosted trees owned tabular machine learning. In the last twelve months tabular foundation models finally — and decisively — beat them on TabArena. But the lesson practitioners are drawing ("stop using trees") is almost the opposite of what the evidence supports, because almost nobody runs the comparison correctly. A field guide to baselines, benchmarking discipline, and the GPU crunch in 2026.
Read more →