Siyu Lou

Shanghai Jiao Tong University

800 Dongchuan Rd.

Shanghai, China

Siyu Lou is currently a Ph.D. student at Shanghai Jiao Tong University and Eastern Institute for Advanced Study, supervised by Prof. Quanshi Zhang and Prof. Yuntian Chen. Her research interest lies at the intersection of data-driven machine learning models and the broader fields of science, with the aim of developing transparent and interpretable models that can provide insights into complex systems.

Prior to her Ph.D. study, Siyu worked as a research assistant at the Cross Media Language Intelligence Lab for one year, under the supervision of Prof. Mengyue Wu. Her work there primarily focused on cross modal learning.

Siyu received her bachelor’s degree from Shanghai Jiao Tong university, and dual-master’s degrees from Shanghai Jiao Tong University and Institut polytechnique de Grenoble in France.

selected publications

Defining and extracting generalizable interaction primitives from DNNs

Lu Chen*, Siyu Lou*, Benhao Huang, and Quanshi Zhang

In International Conference on Learning Representations, 2024

Abs PDF

Faithfully summarizing the knowledge encoded by a deep neural network (DNN) into a few symbolic primitive patterns without losing much information represents a core challenge in explainable AI. To this end, Ren et al. (2023) have derived a series of theorems to prove that the inference score of a DNN can be explained as a small set of interactions between input variables. However, the lack of generalization power makes it still hard to consider such interactions as faithful primitive patterns encoded by the DNN. Therefore, given different DNNs trained for the same task, we develop a new method to extract interactions that are shared by these DNNs. Experiments show that the extracted interactions can better reflect common knowledge shared by different DNNs.
HarsanyiNet: Computing Accurate Shapley Values in a Single Forward Propagation

Lu Chen*, Siyu Lou*, Keyan Zhang, Jin Huang, and Quanshi Zhang

In International Conference on Machine Learning, 2023

Abs PDF Code

The Shapley value is widely regarded as a trustworthy attribution metric. However, when people use Shapley values to explain the attribution of input variables of a deep neural network (DNN), it usually requires a very high computational cost to approximate relatively accurate Shapley values in real-world applications. Therefore, we propose a novel network architecture, the HarsanyiNet, which makes inferences on the input sample and simultaneously computes the exact Shapley values of the input variables in a single forward propagation. The HarsanyiNet is designed on the theoretical foundation that the Shapley value can be reformulated as the redistribution of Harsanyi interactions encoded by the network.
Audio-text retrieval in context

Siyu Lou, Xuenan Xu, Mengyue Wu, and Kai Yu

In IEEE International Conference on Acoustics, Speech and Signal Processing, 2022

Abs PDF Code

Audio-text retrieval based on natural language descriptions is a challenging task. It involves learning cross-modality alignments between long sequences under inadequate data conditions. In this work, we investigate several audio features as well as sequence aggregation methods for better audio-text alignment. Moreover, through a qualitative analysis we observe that semantic mapping is more important than temporal relations in contextual retrieval. Using pre-trained audio features and a descriptor-based aggregation method, we build our contextual audio-text retrieval system. Specifically, we utilize PANNs features pre-trained on a large sound event dataset and NetRVLAD pooling, which directly works with averaged descriptors. Experiments are conducted on the AudioCaps and CLOTHO datasets, and results are compared with the previous state-of-the-art system. With our proposed system, a significant improvement has been achieved on bidirectional audio-text retrieval, on all metrics including recall, median and mean rank