Jacob Si

PhD Student at Imperial College London

Hi! I am a Computer Science PhD student at Imperial College London co-advised by Yingzhen Li and Yves-Alexandre de Montjoye. My recent research interests have been in deep generative models and LLMs within structured data such as tables.

Before joining Imperial, I earned an MEng from the University of California, Los Angeles, and an HBSc at the University of Toronto. I was fortunate to be advised by Rahul Krishnan at the Vector Institute, and Jonathan Kao at UCLA SEAS. Lastly, I am grateful to my instructors, Jimmy Ba and David Duvenaud, for sparking my initial interest in machine learning.

If you are interested in research or would like to collaborate, please feel free email me!

Email:

‘y.si23‘ @ ‘imperial.ac.uk‘

‘jacobyhsi‘ @ ‘ucla.edu‘

‘jacobyhsi‘ @ ‘cs.toronto.edu‘

Selected Publications [full list]

(*) denotes equal contribution

Under Review

TabGen: Training Tabular Diffusion Models with a Simple and Effective Continuous Representation

Jacob Si, Zijing Ou, Mike Qu, Zhengrui Xiang, and Yingzhen Li

In Submission.

Abstract

Diffusion models for tabular data generation face the conundrum of modeling under a separate versus a unified data representation. The former encounters the challenge of jointly modeling all multi-modal distributions of tabular data in one model. While the latter alleviates this by learning a single representation for all features, it currently leverages sparse suboptimal encoding heuristics and necessitates additional computation costs. In this work, we address the latter by presenting TabGen, a tabular diffusion architecture trained with a unified continuous representation, the CatConverter. To motivate the design, we provide geometric insights into how the data manifold affects diffusion models. The key attributes of the CatConverter are composed of its dense representation, flexibility to provide ample separability for nominal features, and ability to preserve intrinsic relationships. Ultimately, TabGen provides a simple yet effective approach for training tabular diffusion models under a continuous data manifold. Our results showcase that TabGen achieves superior performance across a broad suite of evaluations. It is the first to synthesize tabular data that exceeds the downstream quality of the original datasets while preserving privacy and remaining computationally efficient.
ICMLSpotlight

InterpreTabNet: Distilling Predictive Signals from Tabular Data by Salient Feature Interpretation

Jacob Si, Wendy Yusi Cheng, Michael Cooper, and Rahul Krishnan

In the 41st International Conference on Machine Learning, 2024.
Spotlight Presentation [top 3.5%]
Abstract PDF Code

Tabular data are omnipresent in various sectors of industries. Neural networks for tabular data such as TabNet have been proposed to make predictions while leveraging the attention mechanism for interpretability. However, the inferred attention masks are often dense, making it challenging to come up with rationales about the predictive signal. To remedy this, we propose InterpreTabNet, a variant of the TabNet model that models the attention mechanism as a latent variable sampled from a Gumbel-Softmax distribution. This enables us to regularize the model to learn distinct concepts in the attention masks via a KL Divergence regularizer. It prevents overlapping feature selection by promoting sparsity which maximizes the model’s efficacy and improves interpretability to determine the important features when predicting the outcome. To assist in the interpretation of feature interdependencies from our model, we employ a large language model (GPT-4) and use prompt engineering to map from the learned feature mask onto natural language text describing the learned signal. Through comprehensive experiments on real-world datasets, we demonstrate that InterpreTabNet outperforms previous methods for interpreting tabular data while attaining competitive accuracy.
Book Chapter

Assessing Infant Mortality Rate: Problems stemming from Household Living Conditions, Women’s Education and Health

Jacob Si, and Rohan Alexander

In "Telling Stories with Data: With Applications in R" by Rohan Alexander

Abstract PDF

What areas can be improved in order to promote the well-being of women in India and hence, reduce the infant mortality rate? Utilizing the data from the 1998-1999 India National Family Health Survey provided by the Demographic and Health Survey (DHS) program, we look to depict the demographics of Indian women and infants in different states of India. We have found that the root causes of poor infant mortality rates stem from having poor living conditions that affect the likelihood of women to attain education and understand the importance of antenatal care and birth delivery assistance. We also explore other factors such as potentially inheritable traits (unhealthy body weight and anaemia disease) as well as an infant’s diet. These factors are crucial in the development of an infant and the reduction of the infant mortality rate.