NVIDIA Kumo Relational Overview
NVIDIA has introduced Kumo Relational, a pre-trained relational foundation model that excels in making predictions directly on enterprise databases without the need for task-specific training, manual feature engineering, or table flattening. This model is notable for being the first few-shot foundation model to outperform supervised machine learning on common relational benchmarks.
How Kumo Relational Works
Kumo Relational operates on relational schemas rather than a single flat table. It utilizes an architecture that includes:
- Extracting entity-centric subgraphs up to an anchor timestamp to prevent data leakage.
- Applying column and row attention within tables to enhance prediction accuracy.
- Utilizing graph attention over primary-key/foreign-key relationships combined with cross-sample attention across context examples.
This design allows the model to work on row embeddings instead of every cell, significantly reducing computational costs and enabling it to scale to large contexts and deeper subgraphs.
Training and Performance Benchmarks
Kumo Relational is pre-trained on synthetic relational data generated using a PluRel-based generator, which simulates schemas, clustered foreign-key links, and temporal column values. NVIDIA evaluated the model on 41 tasks across 15 relational databases, including RelBench V1 and V2, SAP SALT, and 4DBInfer, covering various domains such as e-commerce, medical, enterprise, social, and academic. Importantly, none of these datasets were included in the pre-training process.
The results indicate that Kumo Relational outperforms supervised relational deep learning models and expert-engineered gradient-boosted tree pipelines. Specifically, on RelBench V1 classification tasks, it exceeds gradient-boosted trees by nearly 4 AUROC points on average. Additionally, on the SAP SALT benchmark, Kumo Relational shows an 8% improvement in mean reciprocal rank (MRR) over large AutoGluon ensembles, with fine-tuning further increasing this margin to up to 16%.
The model also demonstrates robustness under various conditions, maintaining performance with very small context fractions, partial foreign-key removal, and the addition of noisy columns.
Limitations and Deployment Options
While Kumo Relational excels in scenarios where relationships and temporal dynamics are significant, it may not always outperform task-specific supervised models, particularly when large labeled training sets are available. For these cases, fine-tuning of the foundation model is supported.
Deployment options for Kumo Relational include the open-source GPU-native structured-data-models library, which allows for weights to be downloaded from the Hugging Face Hub, or as an NVIDIA NIM. The code is released under the Apache 2.0 license, and the model weights are available under the OpenMDW License Agreement v1.1.
Original source: Hugging Face Blog