Efficient parallel learning algorithms are proposed for training a powerful modular neural
network, the hierarchical mixture of experts (HME). Parallelizations are based on the
concept of modular parallelism, ie parallel execution of network modules. From modeling the
speedup as a function of the number of processors and the number of training examples,
several improvements are derived, such as pipelining the training examples by packets.
Compared to experimental measurements, theoretical models are accurate. For regular …