DP NTK Regression

1 Environment Setup

You should install Pytorch by the following code

# if you have a gpu, use the following command
pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu117
# if you only have cpu on your machine, use the following command
pip install torch==1.13.1+cpu torchvision==0.14.1+cpu torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cpu

Then, install other Python dependencies, using the following command

pip install -r requirements.txt

2 Run DP NTK Regression on CIFAR10

If you want to run the two-class classification task, please refer to run_dp_ntk_regression_2_cls.ipynb.

If you want to run the ten-class classifciation task, please refer to run_dp_ntk_regression_10_cls.ipynb.

Here are some information for other python sciprts.

• ntk_utils.py contains some useful function for computing $H^{\mathrm{dis}}$ and $\alpha$ in NTK Regression.
• truncated_laplace_utils.py contains some useful functions for generating the truncate Laplace noise.

3 Draw Figures

Please refer to draw_figs_2_cls.ipynb for drawing figures for the two-class classification task.

Please refer to draw_figs_10_cls.ipynb for drawing figures for the ten-class classification task.

You should find the following code block in draw_figs_*_cls.ipynb

x_data = [0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 1.1, 1.2, 1.3, 1.4, 1.5]
test_acc_list = [0.7705000638961792, 0.8025, 0.8630, 0.9180000424385071, 0.9084998965263367, 0.9545000195503235, 0.9790000319480896, 0.9689998626708984, 0.9794999957084656, 0.9805000424385071, 0.981499969959259]
train_acc_list = [0.7839500308036804, 0.8103, 0.8726, 0.9182001352310181, 0.9191500544548035, 0.9565500617027283, 0.9762499928474426, 0.9739500284194946, 0.9793000221252441, 0.9796000719070435, 0.9799000024795532]

You should substitue the x_data, test_acc_list and train_acc_list with eps_eponent_list, test_acc_list, and train_acc_list, respectively. Then run the block, you can get the figure.

4 Results

The following figure is the result on the two-class classification task.

The following figure is the result on the ten-class classification task.