In the last decade, there has been a tremendous improvement in healthcare delivery. Several healthcare organisations are taking advantage of machine-learning techniques to provide healthcare at an affordable rate. Artificial Neural Networks (ANN) is a machine-learning technique which has been widely used in the public healthcare in managing decisions. From prescription using Electronic Healthcare Record (EHR) to the use of demographic data to predict the propensity of a disease, the combination of ANN and deep learning techniques has been used to improve the reliability of predictions in healthcare. In this paper, a review of artificial neural network architectures is presented. We explore the benefits of ANN in healthcare and discuss some of the architectures of ANN. Different from under reviews, we look at the challenges and possible solutions in the deployment of ANN architectures in healthcare delivery.