The Internet of Things (IoT) has embraced a 'vertical' off-loading model, where avalanches of raw data generated by numerous edge devices are continuously pushed through the network to a remote processing location, such as a datacenter or a cloud. In this rather unbalanced architecture, edge devices are typically not expected to perform sophisticated data processing and analytics, and data fusion takes place remotely from the original source of data. As a result, the underlying network and the remote datacenter have to handle increased amounts of unstructured raw data, which, in turn, may affect the overall performance and decrease reaction times. As a potential solution to these shortcomings, this paper introduces a distributed hierarchical data fusion architecture for the IoT networks, consisting of edge devices, network and communications units, and cloud platforms. According to the proposed approach, different data sources are combined at each level of the IoT hierarchy to produce timely and accurate results by utilising computational capabilities of intermediate nodes. This way, mission-critical decisions, as demonstrated by the presented smart healthcare scenario, are taken with minimum time delay, as soon as necessary information is generated and collected. The initial evaluation suggests that the proposed approach enables fine-grained decision taking at different data fusion levels, and, as a result, improves the overall performance and reaction time.