XGBoost vs Deep Learning for Tabular Data: Do Trees Still Win in 2026?

If you have been paying attention to ML research over the past three years, you have likely seen a steady stream of papers claiming that some new deep learning architecture finally beats gradient-boosted trees on tabular data. And if you have been running production ML systems over those same three years, you have likely kept shipping XGBoost anyway. Both things are true, and neither is the complete picture. This post is an honest practitioner's head-to-head: when deep learning for tabular data has genuinely arrived, when trees still dominate, and how to make the call on your next project in 2026.

1. The Incumbent Champions: XGBoost, LightGBM, CatBoost

Gradient-boosted decision trees (GBDTs) have dominated competitive tabular ML since roughly 2016, when XGBoost won a disproportionate share of Kaggle competitions and the ML community started taking notice. LightGBM (Microsoft, 2017) pushed training speed dramatically further with leaf-wise tree growth and histogram-based splitting. CatBoost (Yandex, 2017) brought native, zero-leakage categorical encoding — ordered target encoding that eliminates the data leakage that plagued earlier approaches. Together, these three form the toolkit that handles the overwhelming majority of production tabular ML today.

Why trees have been so hard to dislodge

The reasons are structural, not accidental:

• Robustness to feature scale and distribution: GBDTs are invariant to monotonic transformations of input features — you do not need to normalize, log-transform, or standardize. They handle mixed feature types (numeric, ordinal, nominal) natively. This alone eliminates a large category of data preparation bugs.
• Missing value handling: XGBoost and LightGBM learn optimal default directions for missing values during training — a principled approach that outperforms mean/median imputation in most cases. Neural networks, by contrast, require you to decide an imputation strategy before training, and a wrong choice propagates through all subsequent representations.
• Small-data efficiency: A well-regularized XGBoost model can generalize from a few hundred examples. The inductive bias of decision trees — axis-aligned splits that capture interactions through tree structure — matches the piecewise-constant structure that many real-world tabular relationships follow. Deep networks, with millions of parameters and weaker inductive bias, overfit badly on small datasets without heavy regularization and augmentation.
• Training speed and hyperparameter efficiency: A LightGBM model on a 1M-row dataset trains in minutes on a single CPU machine and converges reliably with a modest hyperparameter search. An equivalent deep learning model requires GPU resources, careful learning rate scheduling, and far more tuning to reach comparable accuracy.
• Interpretability: SHAP (SHapley Additive exPlanations) combined with any GBDT gives you feature-level and observation-level explanations that satisfy regulatory requirements in credit, insurance, and healthcare. TreeSHAP is exact and runs in polynomial time. Explaining deep networks requires approximations (LIME, Integrated Gradients) that are slower, noisier, and harder to audit.

2. The Deep Learning Challengers

Neural networks for tabular data are not new — MLPs have been applied to tabular tasks for decades. What is new is a generation of architectures designed specifically to address the reasons trees outperformed them, rather than applying the same image/text architectures to a domain with fundamentally different structure.

TabNet (2019) — Attention-based, interpretable

Google's TabNet was the first serious attempt at an interpretable deep architecture for tabular data. It uses sequential attention to select which features to reason about at each decision step — producing a learned feature mask that provides a form of instance-wise feature importance. In practice, TabNet's interpretability advantage over SHAP-explained GBDTs is less decisive than the paper suggests: the attention masks are not as semantically clean as tree-based importances, and TabNet's training is notoriously finicky. The architecture performs competitively on large datasets (500k+ rows) but rarely beats well-tuned GBDTs on mid-sized tabular benchmarks. It remains a reasonable choice when a deep learning framework is already mandated (e.g., for integration into a TensorFlow serving stack) and interpretability can be demonstrated through the attention mechanism.

FT-Transformer / TabTransformer (2021–2022)

The FT-Transformer (Feature Tokenizer + Transformer, from Gorishniy et al.) takes a different approach: rather than designing a special architecture, it adapts the standard Transformer to tabular data by treating each feature as a token, with a learned embedding per feature value (for categoricals) or per-feature projection (for numerics). This allows the model to learn arbitrary feature interactions via attention — something that decision trees approximate only through higher-order splits. TabTransformer (Google Brain) uses a similar idea but applies attention only to categorical features, passing numeric features directly to the MLP head.

On large tabular datasets with complex feature interactions, FT-Transformer outperforms GBDTs on several benchmark suites. The practical constraint is compute: training FT-Transformer on a 5M-row dataset with 100 features takes GPU hours rather than CPU minutes. For most organizations, this changes the economics of the problem — the question is not just "which model is more accurate" but "how much accuracy gain justifies 10× more training compute and inference latency."

TabPFN — Prior-Fitted Networks and In-Context Learning (2022)

TabPFN (Hollmann et al., 2022, from the AutoML group at Freiburg) is the most practically novel of the tabular DL approaches: a small Transformer pretrained on synthetic datasets drawn from a meta-learned prior over Bayesian networks, capable of doing in-context learning on new tabular tasks at inference time — no finetuning required. You pass your training data and test rows as context, and the model returns predictions in a single forward pass.

The practical implications are striking. On datasets with under 10,000 rows and up to ~100 features, TabPFN matches or outperforms AutoML-optimized XGBoost with zero hyperparameter tuning — it takes seconds rather than hours to "train." For data scientists facing a new tabular task with limited data, TabPFN is now the correct first experiment: the zero-shot baseline. The limitation is scale: the quadratic attention cost of the in-context approach means TabPFN is not viable for large datasets (above ~10k rows the V2 variant handles larger contexts but training still wins at scale). TabPFN v2 (2024) extended coverage to regression, more features, and better calibration while keeping the zero-shot promise.

Tabular Foundation Models (TabFM)

Tabular foundation models, often shortened to TabFM, are the most ambitious entry: networks pretrained across a very large number of tables with the goal of learning representations that generalize zero-shot or few-shot to new tasks. TabPFN is pretrained on synthetic tables drawn from a prior. Other research lines pretrain on real heterogeneous tables and encode column names and descriptions as well as the values, so the model can use what a feature means and not only its statistics.

The appeal is strongest where labeled data is scarce and column names carry meaning, as in customer churn or medical risk scoring. These models are newer and less proven in production than gradient boosting, but the direction is clear: the foundation model paradigm is arriving for tabular data, and it will change the calculus for organizations with limited labeled data.

SAINT, NODE, and other approaches

SAINT (Self-Attention and Intersample Attention Transformer) applies attention both across features and across training samples — the inter-sample attention mechanism allows the model to learn from the relationship between training examples rather than just feature-target relationships. It performs well on classification but adds considerable memory overhead. NODE (Neural Oblivious Decision Ensembles) differentiably implements an oblivious decision tree ensemble — trees where each level uses the same split feature across all branches — allowing gradient-based optimization while preserving tree-like inductive bias. NODE is particularly effective on mid-sized datasets and often outperforms GBDTs in the 50k–500k row regime.

Tabular Foundation Models vs XGBoost

A tabular foundation model is a network pretrained on a very large number of tables, synthetic or real, so that it can predict on a table it has never seen without being trained on it. “TabFM” is often used as shorthand for any tabular foundation model, and this section uses it in that broad sense. TabPFN is the best-known example. Most of these models work by in-context learning: the labeled rows are passed to the model as context together with the rows to predict, and the answer comes back in a forward pass. No gradient steps or hyperparameter search are run on your data.

That changes the comparison with XGBoost in three ways.

• Effort. XGBoost needs a tuning loop and usually some feature work to reach its best. A foundation model gives a strong result out of the box, which makes it a useful first baseline on a new problem.
• Scale. In-context models read the training rows at prediction time, so memory and latency grow with the size of the table. Gradient boosting pays that cost once, in training, and then predicts in microseconds to milliseconds on a CPU. On large tables the trees are cheaper to run and remain hard to beat on accuracy.
• Operations. A boosted model is a small file with exact TreeSHAP explanations and mature monitoring. A foundation model usually needs a GPU, is harder to explain to a regulator, and comes with its own license terms for the weights, which need checking before production use.

So a tabular foundation model is worth trying when the table is small, when there is no time to tune, or when you need a quick read on whether a problem is predictable at all. XGBoost, LightGBM or CatBoost remain the safer choice for large tables, tight latency budgets, CPU-only serving and decisions that must be explained feature by feature.

The two can also work together. A foundation model tends to make different errors from a tree ensemble, so averaging or stacking the two can beat either one alone, at the price of running both. Whatever you choose, compare them fairly: the same splits, a properly tuned gradient-boosted baseline, and a record of training time, prediction latency and serving cost next to the accuracy metric. We did exactly that on three public tables; the results are in our own run below.

3. Head-to-Head: When Does Deep Learning Actually Win?

The honest summary from the benchmarks: deep learning closes the gap on large datasets and wins decisively when the problem involves modalities or representations that trees handle poorly. The TabBench benchmark suite (2023–2024) shows FT-Transformer and SAINT outperforming XGBoost on about 30% of datasets overall — concentrated in the large-data and embedding-heavy categories. On the remaining 70%, tuned GBDTs win or tie within noise.

4. When Trees Still Win — and Why It Matters

The cases where gradient-boosted trees remain the right call in 2026 are not edge cases — they describe the majority of real production tabular ML workloads:

Small-to-medium datasets with pure numeric/categorical features

If your dataset has under 500k rows and features that are direct numeric measurements or clean categoricals (not free-text, not image-derived embeddings), a well-tuned LightGBM or CatBoost model will almost certainly outperform any deep tabular architecture. The inductive bias of decision trees — axis-aligned splits that handle interaction effects through tree depth — is well-matched to the piecewise structure of most business data (pricing tiers, age brackets, geographic categories, time-of-day patterns). You can prove this empirically in a day with a cross-validated comparison; you do not need to take it on faith.

Interpretability requirements

In regulated industries — consumer credit, insurance underwriting, clinical decision support, HR analytics — explainability is not optional. SHAP values on a GBDT are exact, fast to compute (TreeSHAP runs in O(TLD) time), and produce the feature-attribution format that regulators and compliance teams recognize. More practically: when a model makes a decision that a customer or clinician disputes, you need to explain it in minutes, not run a 20-minute Integrated Gradients computation. The interpretability tooling around GBDTs in 2026 (SHAP, EBMs/GA²Ms, monotone constraints in XGBoost/LightGBM) is mature and production-tested in ways that DL interpretability tools are not.

Training speed, iteration velocity, and infrastructure

A data scientist can train, evaluate, and iterate on a LightGBM model on a laptop. The same workflow with an FT-Transformer requires a GPU, CUDA dependencies, a larger codebase, and training times that are 5–50× longer for similar dataset sizes. In a consulting or startup context where model iteration velocity matters more than the last 0.5% AUC, this is a decisive consideration. Production serving of a GBDT is also dramatically simpler: a pickled model or ONNX export runs inference in microseconds on CPU, with no GPU serving infrastructure required.

Kaggle as ground truth

Kaggle tabular competitions remain the closest thing to a reproducible benchmark for real-world tabular ML. In 2026, the picture is consistent: GBDT ensembles (usually XGBoost + LightGBM + CatBoost stacked) win or reach the top-3 in approximately 70% of tabular competitions. Deep learning solutions appear in winners' write-ups primarily on competitions with text features, image-derived features, or very large datasets. This is not because Kagglers are conservative — it is because they run exhaustive experiments and the numbers bear it out.

5. Practical Benchmarks: What the Research Actually Says

The benchmark landscape for tabular DL has matured significantly. Three studies are worth citing for practitioners:

• Gorishniy et al. (2022) — "Revisiting Deep Learning Models for Tabular Data": Systematic comparison across 11 datasets showing FT-Transformer is competitive with tuned GBDTs on large datasets but is rarely better on small-to-mid ones. Honest evaluation, no cherry-picking — a useful reference.
• Grinsztajn et al. (2022) — "Why tree-based models still outperform deep learning on tabular data": Identifies the structural reasons trees win: better handling of irregular target functions, no need for augmentation, robustness to uninformative features. Synthetic ablations confirm the mechanism, not just the observation.
• TabBench (2023–2024) — large-scale benchmark across 300+ datasets: The largest systematic comparison to date. GBDTs win on most datasets under 100k rows; FT-Transformer and SAINT are competitive above 500k rows. TabPFN V2 achieves remarkable results in the zero-shot small-data regime. No single model dominates across all conditions — which is the correct and production-relevant conclusion.

Our own run: TabPFN vs XGBoost on three public tables

Published benchmarks are useful, but they are run by the people who built the models. So in November 2026 we ran a small comparison of our own: TabPFN v2 (the openly licensed weights, no tuning) against XGBoost with default settings and XGBoost tuned by a 30-configuration random search, on three public OpenML tables of different sizes. One Apple Silicon laptop, one random seed, TabPFN on the built-in GPU, XGBoost on the CPU. The small and medium tables use 5-fold cross-validation; the large one a single stratified 80/20 split. The metric is ROC AUC.

• Small table: TabPFN with no tuning matched the tuned XGBoost, which needed an 80-second search. Both beat default XGBoost by about 0.02, which is smaller than the spread between folds (about 0.03).
• Medium table: all three are equal, and tuning XGBoost gained nothing.
• Large table: XGBoost wins, both on the same 10,000 rows and by using all the data, which TabPFN v2 cannot.
• Prediction time is the real gap. Scoring the 9,769 test rows of the adult table took XGBoost 7 milliseconds and TabPFN 50 seconds. On the 1,000-row test fold of the churn table, 2 milliseconds against 8 seconds.

The decision threshold mattered more than the model

Cost is per applicant throughout this table. A reader pointed out that the German credit data ships with a cost matrix: approving a bad borrower costs 5, rejecting a good one costs 1. Scored on that cost instead of AUC, the picture changes. We repeated the small-table run with 5-fold cross-validation three times (15 test folds), added a plain random forest and the same forest wrapped in the reader's pairwise difference learning library (pdll), and chose each model's threshold on out-of-fold predictions inside the training fold, never on the test fold.

At the usual 0.5 threshold every model is worse than rejecting every applicant. With the threshold chosen for the 5:1 cost, all five fall to between 0.53 and 0.58, and the top four are within 0.015 of each other, well inside the fold-to-fold spread of about 0.05. Four of the five chosen thresholds sit near the 1/6 that theory predicts for calibrated probabilities; default XGBoost needed 0.03, a sign of how overconfident its scores are. Moving the threshold cut the cost by a third to a half. Switching between the top four models moved it by under 3%. The pairwise difference wrapper matched its own base forest on this table and took about 25 times longer to fit; one dataset says little about a method built for many small ones.

6. Production Considerations: The Gap That Papers Don't Cover

Most benchmark papers optimize for accuracy on held-out test sets. Production deployments optimize for accuracy + latency + serving cost + monitoring complexity + team maintainability. These additional dimensions strongly favor GBDTs in most organizations:

The production gap also manifests in feature engineering. GBDTs benefit enormously from hand-crafted features: ratio features, interaction terms, temporal lag features, binned aggregations. A skilled feature engineer can add 3–5% AUC to a GBDT model that a DL architecture achieves automatically through representation learning — but the DL model then requires more data to generalize and more infrastructure to serve. Neither approach is free.

7. The Hybrid Approach: DL Embeddings + Tree Prediction

The most pragmatic answer to the GBDT-vs-DL question in production is often: both. The hybrid architecture — use a neural network to learn embeddings, then feed those embeddings to a GBDT for the final prediction — combines the representation power of DL with the reliability and interpretability of trees.

How it works in practice

The canonical pattern has three stages:

1. Entity embeddings: Train a neural network (a shallow MLP, a pretrained language model for text features, or a dedicated entity embedding layer) to produce dense representations of high-cardinality categorical features (user IDs, product SKUs, geographic entities, free-text columns). These embeddings encode semantic similarity that one-hot or ordinal encoding cannot capture.
2. Feature augmentation: Concatenate the learned embeddings with the original numeric features to produce an enriched feature matrix. For text-containing columns (product descriptions, support tickets, customer reviews), use a language model encoder to produce a fixed-size representation and include it as additional features.
3. GBDT on augmented features: Train XGBoost or LightGBM on the combined feature matrix. The GBDT now benefits from the semantic structure captured in the embeddings while retaining all its production advantages: CPU inference, interpretability via SHAP, fast retraining, and robustness to missing values in the original numeric features.

This approach consistently outperforms pure GBDT on datasets with high-cardinality categoricals or text features, while matching or exceeding end-to-end DL on everything else. It is particularly effective for recommendation-adjacent tasks (click-through prediction, next-purchase forecasting) where user and item embeddings from collaborative filtering or pre-trained language models encode information that no hand-crafted feature can match.

When the hybrid is overkill

The hybrid architecture adds engineering complexity — you now have two training pipelines, two artifacts to version and monitor, and an embedding layer that can drift independently of the GBDT. If your categorical features are low-to-mid cardinality (<1,000 categories) and you have no text columns, CatBoost's native ordered target encoding almost always delivers equivalent or better results without the added complexity. Reserve the hybrid for genuinely high-cardinality or multimodal cases.

8. The Israeli Ecosystem: Tabular ML in Production

Israel's enterprise AI landscape is heavily tabular. The sectors that drive MLAIA's work — financial services, insurance, telecom, healthcare analytics, AdTech — are all fundamentally tabular data problems. In our experience working with Israeli enterprises in 2025–2026, the dominant production stack remains XGBoost or LightGBM with CatBoost as a categorical specialist, with TabPFN increasingly adopted as a fast zero-shot baseline for new problem scoping. FT-Transformer appears in production primarily at companies with existing GPU infrastructure (generally, R&D arms of larger tech companies) and where the dataset exceeds 200k rows of training data.

The Israeli financial services sector faces an interesting constraint: the Bank of Israel's model risk management guidelines (aligned with the Basel Committee's SR 11-7 equivalent) require explainability for credit decisions. This effectively mandates GBDTs — or at minimum, SHAP-explainable models — for any decision that affects credit access. The same constraint applies to insurance pricing models regulated by the Commissioner of Capital Markets. Healthcare analytics teams at health funds and hospitals operate under similar explainability expectations.

Where we see deep learning for tabular data gaining genuine production traction in Israel: e-commerce recommendation (embedding-augmented GBDTs for user-item matching), fraud detection at scale (FT-Transformer for large telco and payment transaction datasets), and clinical NLP tasks where patient records combine structured tabular fields with free-text physician notes — a genuinely multimodal problem where the hybrid architecture delivers meaningful gains over trees alone.

The 2026 Decision Framework

The answer to "do gradient-boosted trees still beat deep learning on tabular data?" in 2026 is: yes, in the majority of cases — but the exceptions are real, well-understood, and increasingly common. The right framework is not algorithm loyalty but problem diagnosis:

• Under 10k rows, no GPU budget: Start with TabPFN V2. It will match a tuned XGBoost in seconds and give you a performance ceiling to beat.
• 10k–500k rows, pure numeric/categorical, production simplicity required: LightGBM or CatBoost with Optuna hyperparameter tuning. Add SHAP. Ship.
• High-cardinality categoricals or text columns: Hybrid — entity embeddings or language model features + GBDT. The complexity pays off.
• Over 500k rows, GPU available, multimodal inputs: Evaluate FT-Transformer against a tuned GBDT. On >1M rows with text or embedding-heavy features, FT-Transformer or SAINT often wins.
• Zero labeled data, fast scoping required: A tabular foundation model such as TabPFN V2. The zero-shot result tells you whether the problem is worth investing in.
• Regulatory explainability required: GBDTs + SHAP. Full stop. Not a tradeoff — a constraint.

Gradient-boosted trees are not losing their throne anytime soon. But the kingdom is being productively challenged, and the challengers are improving fast. The practitioners who win in 2026 are the ones who know exactly when to call in the new guard — and when to keep shipping XGBoost.

Frequently asked questions

Do gradient-boosted trees still beat deep learning on tabular data in 2026?

Gradient-boosted trees still win most tabular problems in 2026. Deep learning wins in specific cases: TabPFN on small datasets, FT-Transformer or SAINT on very large ones with text features, and tabular foundation models (TabFM) when labels are scarce.

How do TabNet, TabPFN and TabFM compare with XGBoost?

XGBoost, LightGBM and CatBoost remain the default for most tabular problems. TabPFN v2 matches a tuned XGBoost on datasets under about 10k rows with no tuning but doesn't scale to large data. TabNet rarely beats tuned trees on mid-sized data. Tabular foundation models (TabFM) target cases with little labeled data and little time to tune, but they are newer and less proven in production.

Related: how we approach forecasting, churn and causal analysis, and the interactive prediction explorer with SHAP explanations.