Generalization and MIA

This repository contains the code used to generate the experimental results Generalization and Membership Inference Attack: A Practical Perspective — Fateme Rahmani, Mahdi Jafari Siavoshani, Mohammad Hossein Rohban

The code is organized as research notebooks plus a small reusable Backbone package for model definitions, training helpers, augmentations, attack scoring, and plotting.

Background: Membership Inference and LIRA

Membership inference attacks (MIA) try to decide whether a particular data point was part of a model's training set. The Likelihood Ratio Attack (LIRA) of Carlini et al. formalizes this as a hypothesis test: for a given data point, the attacker compares how likely the target model's loss is under the "in" distribution (models that trained on the point) versus the "out" distribution (models that did not). A large likelihood ratio is evidence of membership.

The paper revisits the longstanding debate over whether better generalization translates to lower MIA vulnerability. Using LIRA and the modern TPR @ 0.1% FPR metric, it shows that combining augmentation techniques can reduce attack success by up to ~100×, that knowledge of the training procedure does not always help the attacker once augmentations are combined, and that across 1K+ trained models the train/test accuracy gap predicts vulnerability more reliably than test accuracy alone.

LIRA Road Map

The high-level idea behind LIRA is to reduce a hard "did this point train the target model?" question to a tractable distributional one. We train many shadow models, each on a random half of the dataset, then characterize the per-sample loss distributions for the "in" vs "out" populations and run a likelihood ratio test against the target model's loss for each query point.

The four panels above walk through the full reasoning: (1) the original membership question on a member/non-member split feeding the target model, (2) generating per-point in/out shadow training sets, (3) reformulating the attack as a comparison between "in" and "out" model populations for each data point, and (4) the final solution — reduce each shadow model to a scalar score (e.g. loss) and run a likelihood ratio test against the target model's score.

Evaluation Steps

Our evaluation protocol has two training phases. First, a target ResNet-18 is trained on a 50/50 member/non-member split of CIFAR-10 so that a known ground-truth membership label exists for every sample. Second, 128 shadow ResNet-18 models are trained, each on a random half of the full dataset so that every data point ends up "in" for roughly 64 shadow models and "out" for the other 64.

With the shadow models in hand, the next phases compute per-sample losses across all 128 shadow models, fit per-sample in/out loss distributions, and finally apply the likelihood ratio test using the target model's loss for that sample.

The histograms illustrate why the attack works: for many samples, the in-distribution (blue) of losses is visibly separated from the out-distribution (orange), so the target model's loss carries enough information to bias the likelihood ratio. The paper evaluates attack success with TPR @ 0.1% FPR, a low-FPR operating point that captures the privacy-relevant question of how confidently an attacker can identify some members rather than how often they are right on average.

Dependencies

The main Python dependencies are listed in requirements.txt. The project example setup:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

Install the PyTorch build that matches your machine and CUDA version if you plan to train on GPU.

Running the Notebooks

1. Start Jupyter from the repository root:

   jupyter notebook

2. Review config.py before running experiments. It defines the shared paths used by the notebooks:

   PROJECT_ROOT = Path(__file__).resolve().parent
   DATASET_DIR = PROJECT_ROOT.parent / "Dataset"
   DEFAULT_LIRA_DATA_DIR = LIRA_DATA_DIR / DEFAULT_VERSION
   DEFAULT_LIRA_MODEL_DIR = LIRA_MODELS_DIR / DEFAULT_VERSION

   Change DATASET_DIR, DEFAULT_VERSION, or the model/result directory constants there if your local layout is different.

3. Run the training notebooks first:

   • ResNet-Train.ipynb trains a target ResNet-18 model.
   • ResNet-Train-Shadow.ipynb trains shadow models and writes intermediate model/data outputs.

4. Run the LIRA notebooks:

   • LIRA/Result.ipynb
   • LIRA/Result-Augmented.ipynb
   • LIRA/Result_Visualization.ipynb

Data and Outputs

The notebooks use CIFAR-10 through torchvision.datasets.CIFAR10. By default, config.py expects the dataset directory at ../Dataset/ relative to the repository root.

Generated artifacts such as trained models, membership CSV files, per-sample scores, and result CSV files are expected under paths like LIRA/data/, models/, and data/. These generated files are not included in the repository.

The figures embedded in this README are drawn from the May 2023 result report (Fateme_Rahmani_Report_2023-05-03.pdf) and live in the images/ folder alongside this file.

References

[1] F. Rahmani, M. Jafari Siavoshani, and M. H. Rohban, "Generalization and Membership Inference Attack: A Practical Perspective" doi:10.48550/arXiv.2604.19936.

[2] N. Carlini, S. Chien, M. Nasr, S. Song, A. Terzis and F. Tramèr, "Membership Inference Attacks From First Principles," 2022 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA, 2022, pp. 1897-1914, doi: 10.1109/SP46214.2022.9833649.