Tabular Data in 2026: Do Gradient-Boosted Trees Still Beat Deep Learning?

If you have been paying attention to ML research over the past three years, you have likely seen a steady stream of papers claiming that some new deep learning architecture finally beats gradient-boosted trees on tabular data. And if you have been running production ML systems over those same three years, you have likely kept shipping XGBoost anyway. Both things are true, and neither is the complete picture. This post is an honest practitioner's head-to-head: when deep learning for tabular data has genuinely arrived, when trees still dominate, and how to make the call on your next project in 2026.

1. The Incumbent Champions: XGBoost, LightGBM, CatBoost

Gradient-boosted decision trees (GBDTs) have dominated competitive tabular ML since roughly 2016, when XGBoost won a disproportionate share of Kaggle competitions and the ML community started taking notice. LightGBM (Microsoft, 2017) pushed training speed dramatically further with leaf-wise tree growth and histogram-based splitting. CatBoost (Yandex, 2017) brought native, zero-leakage categorical encoding — ordered target encoding that eliminates the data leakage that plagued earlier approaches. Together, these three form the toolkit that handles the overwhelming majority of production tabular ML today.

Why trees have been so hard to dislodge

The reasons are structural, not accidental:

• Robustness to feature scale and distribution: GBDTs are invariant to monotonic transformations of input features — you do not need to normalize, log-transform, or standardize. They handle mixed feature types (numeric, ordinal, nominal) natively. This alone eliminates a large category of data preparation bugs.
• Missing value handling: XGBoost and LightGBM learn optimal default directions for missing values during training — a principled approach that outperforms mean/median imputation in most cases. Neural networks, by contrast, require you to decide an imputation strategy before training, and a wrong choice propagates through all subsequent representations.
• Small-data efficiency: A well-regularized XGBoost model can generalize from a few hundred examples. The inductive bias of decision trees — axis-aligned splits that capture interactions through tree structure — matches the piecewise-constant structure that many real-world tabular relationships follow. Deep networks, with millions of parameters and weaker inductive bias, overfit badly on small datasets without heavy regularization and augmentation.
• Training speed and hyperparameter efficiency: A LightGBM model on a 1M-row dataset trains in minutes on a single CPU machine and converges reliably with a modest hyperparameter search. An equivalent deep learning model requires GPU resources, careful learning rate scheduling, and far more tuning to reach comparable accuracy.
• Interpretability: SHAP (SHapley Additive exPlanations) combined with any GBDT gives you feature-level and observation-level explanations that satisfy regulatory requirements in credit, insurance, and healthcare. TreeSHAP is exact and runs in polynomial time. Explaining deep networks requires approximations (LIME, Integrated Gradients) that are slower, noisier, and harder to audit.

2. The Deep Learning Challengers

Neural networks for tabular data are not new — MLPs have been applied to tabular tasks for decades. What is new is a generation of architectures designed specifically to address the reasons trees outperformed them, rather than applying the same image/text architectures to a domain with fundamentally different structure.

TabNet (2019) — Attention-based, interpretable

Google's TabNet was the first serious attempt at an interpretable deep architecture for tabular data. It uses sequential attention to select which features to reason about at each decision step — producing a learned feature mask that provides a form of instance-wise feature importance. In practice, TabNet's interpretability advantage over SHAP-explained GBDTs is less decisive than the paper suggests: the attention masks are not as semantically clean as tree-based importances, and TabNet's training is notoriously finicky. The architecture performs competitively on large datasets (500k+ rows) but rarely beats well-tuned GBDTs on mid-sized tabular benchmarks. It remains a reasonable choice when a deep learning framework is already mandated (e.g., for integration into a TensorFlow serving stack) and interpretability can be demonstrated through the attention mechanism.

FT-Transformer / TabTransformer (2021–2022)

The FT-Transformer (Feature Tokenizer + Transformer, from Gorishniy et al.) takes a different approach: rather than designing a special architecture, it adapts the standard Transformer to tabular data by treating each feature as a token, with a learned embedding per feature value (for categoricals) or per-feature projection (for numerics). This allows the model to learn arbitrary feature interactions via attention — something that decision trees approximate only through higher-order splits. TabTransformer (Google Brain) uses a similar idea but applies attention only to categorical features, passing numeric features directly to the MLP head.

On large tabular datasets with complex feature interactions, FT-Transformer outperforms GBDTs on several benchmark suites. The practical constraint is compute: training FT-Transformer on a 5M-row dataset with 100 features takes GPU hours rather than CPU minutes. For most organizations, this changes the economics of the problem — the question is not just "which model is more accurate" but "how much accuracy gain justifies 10× more training compute and inference latency."

TabPFN — Prior-Fitted Networks and In-Context Learning (2022)

TabPFN (Hollmann et al., 2022, from the AutoML group at Freiburg) is the most practically novel of the tabular DL approaches: a small Transformer pretrained on synthetic datasets drawn from a meta-learned prior over Bayesian networks, capable of doing in-context learning on new tabular tasks at inference time — no finetuning required. You pass your training data and test rows as context, and the model returns predictions in a single forward pass.

The practical implications are striking. On datasets with under 10,000 rows and up to ~100 features, TabPFN matches or outperforms AutoML-optimized XGBoost with zero hyperparameter tuning — it takes seconds rather than hours to "train." For data scientists facing a new tabular task with limited data, TabPFN is now the correct first experiment: the zero-shot baseline. The limitation is scale: the quadratic attention cost of the in-context approach means TabPFN is not viable for large datasets (above ~10k rows the V2 variant handles larger contexts but training still wins at scale). TabPFN v2 (2024) extended coverage to regression, more features, and better calibration while keeping the zero-shot promise.

Google's TabFM — Tabular Foundation Model (2024–2025)

Google's TabFM is the most ambitious entry: a true tabular foundation model trained across thousands of diverse tabular datasets with the goal of learning transferable representations that generalize zero-shot or few-shot to new tasks. Unlike TabPFN (which uses synthetic prior data), TabFM trains on real heterogeneous tabular corpora — creating embeddings that encode both column semantics (via column name and description) and cross-dataset statistical patterns.

In-house benchmarks show TabFM achieving competitive performance on new datasets with minimal or no labeled data — particularly on tasks where column names carry semantic meaning (customer churn, medical risk scoring) that a pretrained language backbone can leverage. The architecture combines a column-name encoder (effectively a text embedding model applied to feature names) with a tabular transformer that processes the actual values, enabling it to benefit from semantic priors about what features likely mean.

TabFM is not yet publicly available at the scale reported in the research preview, but the direction is clear: the foundation model paradigm is arriving for tabular data, and it will change the calculus for organizations with limited labeled data.

SAINT, NODE, and other approaches

SAINT (Self-Attention and Intersample Attention Transformer) applies attention both across features and across training samples — the inter-sample attention mechanism allows the model to learn from the relationship between training examples rather than just feature-target relationships. It performs well on classification but adds considerable memory overhead. NODE (Neural Oblivious Decision Ensembles) differentiably implements an oblivious decision tree ensemble — trees where each level uses the same split feature across all branches — allowing gradient-based optimization while preserving tree-like inductive bias. NODE is particularly effective on mid-sized datasets and often outperforms GBDTs in the 50k–500k row regime.

3. Head-to-Head: When Does Deep Learning Actually Win?

The honest summary from the benchmarks: deep learning closes the gap on large datasets and wins decisively when the problem involves modalities or representations that trees handle poorly. The TabBench benchmark suite (2023–2024) shows FT-Transformer and SAINT outperforming XGBoost on about 30% of datasets overall — concentrated in the large-data and embedding-heavy categories. On the remaining 70%, tuned GBDTs win or tie within noise.

4. When Trees Still Win — and Why It Matters

The cases where gradient-boosted trees remain the right call in 2026 are not edge cases — they describe the majority of real production tabular ML workloads:

Small-to-medium datasets with pure numeric/categorical features

If your dataset has under 500k rows and features that are direct numeric measurements or clean categoricals (not free-text, not image-derived embeddings), a well-tuned LightGBM or CatBoost model will almost certainly outperform any deep tabular architecture. The inductive bias of decision trees — axis-aligned splits that handle interaction effects through tree depth — is well-matched to the piecewise structure of most business data (pricing tiers, age brackets, geographic categories, time-of-day patterns). You can prove this empirically in a day with a cross-validated comparison; you do not need to take it on faith.

Interpretability requirements

In regulated industries — consumer credit, insurance underwriting, clinical decision support, HR analytics — explainability is not optional. SHAP values on a GBDT are exact, fast to compute (TreeSHAP runs in O(TLD) time), and produce the feature-attribution format that regulators and compliance teams recognize. More practically: when a model makes a decision that a customer or clinician disputes, you need to explain it in minutes, not run a 20-minute Integrated Gradients computation. The interpretability tooling around GBDTs in 2026 (SHAP, EBMs/GA²Ms, monotone constraints in XGBoost/LightGBM) is mature and production-tested in ways that DL interpretability tools are not.

Training speed, iteration velocity, and infrastructure

A data scientist can train, evaluate, and iterate on a LightGBM model on a laptop. The same workflow with an FT-Transformer requires a GPU, CUDA dependencies, a larger codebase, and training times that are 5–50× longer for similar dataset sizes. In a consulting or startup context where model iteration velocity matters more than the last 0.5% AUC, this is a decisive consideration. Production serving of a GBDT is also dramatically simpler: a pickled model or ONNX export runs inference in microseconds on CPU, with no GPU serving infrastructure required.

Kaggle as ground truth

Kaggle tabular competitions remain the closest thing to a reproducible benchmark for real-world tabular ML. In 2026, the picture is consistent: GBDT ensembles (usually XGBoost + LightGBM + CatBoost stacked) win or reach the top-3 in approximately 70% of tabular competitions. Deep learning solutions appear in winners' write-ups primarily on competitions with text features, image-derived features, or very large datasets. This is not because Kagglers are conservative — it is because they run exhaustive experiments and the numbers bear it out.

5. Practical Benchmarks: What the Research Actually Says

The benchmark landscape for tabular DL has matured significantly. Three studies are worth citing for practitioners:

• Gorishniy et al. (2022) — "Revisiting Deep Learning Models for Tabular Data": Systematic comparison across 11 datasets showing FT-Transformer is competitive with tuned GBDTs on large datasets but is rarely better on small-to-mid ones. Honest evaluation, no cherry-picking — a useful reference.
• Grinsztajn et al. (2022) — "Why tree-based models still outperform deep learning on tabular data": Identifies the structural reasons trees win: better handling of irregular target functions, no need for augmentation, robustness to uninformative features. Synthetic ablations confirm the mechanism, not just the observation.
• TabBench (2023–2024) — large-scale benchmark across 300+ datasets: The largest systematic comparison to date. GBDTs win on most datasets under 100k rows; FT-Transformer and SAINT are competitive above 500k rows. TabPFN V2 achieves remarkable results in the zero-shot small-data regime. No single model dominates across all conditions — which is the correct and production-relevant conclusion.

6. Production Considerations: The Gap That Papers Don't Cover

Most benchmark papers optimize for accuracy on held-out test sets. Production deployments optimize for accuracy + latency + serving cost + monitoring complexity + team maintainability. These additional dimensions strongly favor GBDTs in most organizations:

The production gap also manifests in feature engineering. GBDTs benefit enormously from hand-crafted features: ratio features, interaction terms, temporal lag features, binned aggregations. A skilled feature engineer can add 3–5% AUC to a GBDT model that a DL architecture achieves automatically through representation learning — but the DL model then requires more data to generalize and more infrastructure to serve. Neither approach is free.

7. The Hybrid Approach: DL Embeddings + Tree Prediction

The most pragmatic answer to the GBDT-vs-DL question in production is often: both. The hybrid architecture — use a neural network to learn embeddings, then feed those embeddings to a GBDT for the final prediction — combines the representation power of DL with the reliability and interpretability of trees.

How it works in practice

The canonical pattern has three stages:

1. Entity embeddings: Train a neural network (a shallow MLP, a pretrained language model for text features, or a dedicated entity embedding layer) to produce dense representations of high-cardinality categorical features (user IDs, product SKUs, geographic entities, free-text columns). These embeddings encode semantic similarity that one-hot or ordinal encoding cannot capture.
2. Feature augmentation: Concatenate the learned embeddings with the original numeric features to produce an enriched feature matrix. For text-containing columns (product descriptions, support tickets, customer reviews), use a language model encoder to produce a fixed-size representation and include it as additional features.
3. GBDT on augmented features: Train XGBoost or LightGBM on the combined feature matrix. The GBDT now benefits from the semantic structure captured in the embeddings while retaining all its production advantages: CPU inference, interpretability via SHAP, fast retraining, and robustness to missing values in the original numeric features.

This approach consistently outperforms pure GBDT on datasets with high-cardinality categoricals or text features, while matching or exceeding end-to-end DL on everything else. It is particularly effective for recommendation-adjacent tasks (click-through prediction, next-purchase forecasting) where user and item embeddings from collaborative filtering or pre-trained language models encode information that no hand-crafted feature can match.

When the hybrid is overkill

The hybrid architecture adds engineering complexity — you now have two training pipelines, two artifacts to version and monitor, and an embedding layer that can drift independently of the GBDT. If your categorical features are low-to-mid cardinality (<1,000 categories) and you have no text columns, CatBoost's native ordered target encoding almost always delivers equivalent or better results without the added complexity. Reserve the hybrid for genuinely high-cardinality or multimodal cases.

8. The Israeli Ecosystem: Tabular ML in Production

Israel's enterprise AI landscape is heavily tabular. The sectors that drive MLAIA's work — financial services, insurance, telecom, healthcare analytics, AdTech — are all fundamentally tabular data problems. In our experience working with Israeli enterprises in 2025–2026, the dominant production stack remains XGBoost or LightGBM with CatBoost as a categorical specialist, with TabPFN increasingly adopted as a fast zero-shot baseline for new problem scoping. FT-Transformer appears in production primarily at companies with existing GPU infrastructure (generally, R&D arms of larger tech companies) and where the dataset exceeds 200k rows of training data.

The Israeli financial services sector faces an interesting constraint: the Bank of Israel's model risk management guidelines (aligned with the Basel Committee's SR 11-7 equivalent) require explainability for credit decisions. This effectively mandates GBDTs — or at minimum, SHAP-explainable models — for any decision that affects credit access. The same constraint applies to insurance pricing models regulated by the Commissioner of Capital Markets. Healthcare analytics (working with Clalit, Maccabi, Sheba) operate under similar explainability expectations.

Where we see deep learning for tabular data gaining genuine production traction in Israel: e-commerce recommendation (embedding-augmented GBDTs for user-item matching), fraud detection at scale (FT-Transformer for large telco and payment transaction datasets), and clinical NLP tasks where patient records combine structured tabular fields with free-text physician notes — a genuinely multimodal problem where the hybrid architecture delivers meaningful gains over trees alone.

The 2026 Decision Framework

The answer to "do gradient-boosted trees still beat deep learning on tabular data?" in 2026 is: yes, in the majority of cases — but the exceptions are real, well-understood, and increasingly common. The right framework is not algorithm loyalty but problem diagnosis:

• Under 10k rows, no GPU budget: Start with TabPFN V2. It will match a tuned XGBoost in seconds and give you a performance ceiling to beat.
• 10k–500k rows, pure numeric/categorical, production simplicity required: LightGBM or CatBoost with Optuna hyperparameter tuning. Add SHAP. Ship.
• High-cardinality categoricals or text columns: Hybrid — entity embeddings or language model features + GBDT. The complexity pays off.
• Over 500k rows, GPU available, multimodal inputs: Evaluate FT-Transformer against a tuned GBDT. On >1M rows with text or embedding-heavy features, FT-Transformer or SAINT often wins.
• Zero labeled data, fast scoping required: TabFM (when publicly available at scale) or TabPFN V2. The zero-shot result tells you whether the problem is worth investing in.
• Regulatory explainability required: GBDTs + SHAP. Full stop. Not a tradeoff — a constraint.

Gradient-boosted trees are not losing their throne anytime soon. But the kingdom is being productively challenged, and the challengers are improving fast. The practitioners who win in 2026 are the ones who know exactly when to call in the new guard — and when to keep shipping XGBoost.