Discussion of ‘Network cross-validation by edge sampling’

Extract

1. Edge cross-validation for network model selection

We thank the authors for their new contribution to network modelling. Data reuse, encompassing methods such as bootstrapping and cross-validation, is an area that to date has largely resisted obvious and rapid development in the network context. One of the major reasons is that mimicking the original sampling mechanisms is challenging if not impossible. To avoid deleting edges and destroying some of the network structure, the resampling strategy proposed in Li et al. (2020) based on splitting node pairs rather than nodes is therefore insightful and effective. Matrix completion is the key technique involved, with its use here providing a new perspective for network analysis.

The proposed edge cross-validation procedure operates effectively on an adjacency matrix |$A=(A_{ij})_{n\times n}$| instead of on the original network, as described in the following algorithm.

 

Algorithm 1.

The general edge cross-validation procedure.

Step 1. Give a loss function |$L$| and select the rank |$\hat{K}$| for matrix completion.

Step 2. For |$m=1$| to |$m=N$|⁠:

|$\qquad$|(a) Assign to each |$(i,j)$| in the learning set |$\Omega$| a prespecified probability |$p$|⁠.

|$\qquad$|(b) Obtain |$\skew6\hat{A}$| from |$(A, \Omega)$| by a low-rank matrix completion algorithm with rank |$\hat{K}$|⁠.

|$\qquad$|(c) For each candidate model |$q=1,\ldots,Q$|⁠, fit the model on data |$\skew6\hat{A}$| and evaluate its loss |$L_q^{(m)}$| by comparing the resulting estimated parameters over the held-out set |$\{A_{ij}:(i,j) \not \in \Omega\}$|⁠.

Step 3. Let |$L_q=N^{-1}\sum_{m=1}^NL_q^{(m)}$| and select the candidate model |$\hat{q}=\arg\min_{1\leq q\leq Q}L_q$|⁠.

The authors proposed using |$p=0.9$| and replicated the validation |$N=3$| times. Therefore only about 30% of the edges are used for cross-validation. Borrowing from the noisy network setting of Chang et al. (2018), the proposal below will use all of the edges for validation. It is in the spirit of jittered bootstrapping or resampling via jittering. Here jittering means that a small amount of noise is added to every single data point; see, for example, Henning (2007, § 3.3). Interestingly, the low-rank assumption does not appear to be necessary in this approach.