PMC

Expectation pooling performs better and is more robust to random seeds than are global max and average pooling (a), and expectation pooling suffers less from overfitting than global max pooling (b). Here (a) shows the AUCs of models with different pooling methods on the simulated datasets 1 (short motif), 2 (long motif) and 3 (mixed motifs). The mean AUCs on these datasets are 0.795774, 0.866507 and 0.720751 for models with global max pooling; 0.81092, 0.870577 and 0.801181 for expectation pooling; and 0.545254, 0.636738 and 0.53726 for global average pooling, respectively. (b) The learning curves for models with different pooling methods is shown. The difference between training and testing loss for the model with expectation pooling is still moderate after 20–40 epochs; in contrast, the difference for the model with max pooling becomes very large immediately after 5 epochs. In addition, the model with global average pooling is difficult to fit and leads to low performance, while the methods with expectation pooling require more than 20 epoch to fit, and a slight overfitting occurs after 40 epochs

(a) Sum pooling; (b) Max pooling; (c) Modified pooling.

The performance of our model utilizing expectation pooling on real datasets, where the window size = 10, m = 1, kernel number = 8, kernel size = 15 and the optional structures are utilized. Expectation pooling increases the AUC on real datasets. The x-axis shows the AUC difference under expectation pooling and max pooling. The models with expectation pooling are better than are the models with max pooling on 653 datasets but worse than the models with max pooling on 37 datasets. This figure clearly shows that expectation pooling achieves better performances

Contributions of SF pooling and spatial pooling are estimated for each neuron. (a) A matrix of predictions of binocular interaction maps is shown for four cases depending on the presence/absence of SF pooling and spatial pooling. Gabor function was fit to each binocular interaction map. Then the degree of shrinkage and elongation was estimated. The extent of spatial pooling was estimated based on the length of the binocular interaction map along the 45° diagonal (compare a and b). The extent of SF pooling was estimated by the width of the binocular interaction map along the −45° diagonal (compare a and c). When a neuron collects inputs over space and SF domains, the binocular interaction becomes elongated along the 45° diagonal and shrunken along the −45° diagonal (d). (b) Spatial pooling and SF pooling were estimated simultaneously for binocular disparity-selective complex cells (n = 39). Red open circles correspond to a–d in (a). Triangles with red, blue, green and magenta indicate the example neurons shown in a,b,c,d, respectively. Grey circles indicate the neurons which were fitted by narrower SF bandwidth than 1.3 octaves, without which the SF pooling would have been negative. Coloured curves are hyperbolic functions that depict various degrees of spatial and SF pooling estimated as elongation index (EI). (c) and (d) Histograms of SF pooling and spatial pooling, respectively.