Threshold Guidelines for the INTREPPPID Model

Knowing how to interpret the scores generated by PPI inference models can sometimes be challenging. What follows is some guidance how to use interpret protein interaction scores computed by the INTREPPPID model.

When benchmarked on strict H. sapiens protein-interaction datasets, INTREPPPID scores an average Brier Score of 0.200 (±0.004). This indicates that, while not perfect, INTREPPPID is fairly well calibrated.

Knowing at what threshold to label two proteins as "interacting" given its INTREPPPID score can be challenging, and primarily depends on your needs and the analysis you are conducting. The Youden J's statistic can be a useful method for determining the optimal cut-off point. We measured an average J statistic of 0.479 (±0.080).

Sometimes, precision is more important than recall. In these cases, taking the top e.g. 95th percentile can be a useful technique.