Device-to-device (D2D) communication has emerged as a promising concept to improve resource utilization in fifth generation cellular networks. D2D network’s architectural capability to offload traffic from the backhaul network to direct links enables it to be used for internet of things (IoT) services. In a densely deployed setting of IoT devices, D2D network may experience critical interferences due to a limited number of spectral resources. To increase the overall signal-to-interference-plus-noise ratio (SINR) of the network while reducing the computational load on a macro base station, a novel decentralized interference management methodology is proposed for dense in-band D2D underlay LTE-A network. The proposed interference management scheme can decouple interference in a network into cross-cluster and intra-cluster interference and tackle with them separately. To mitigate the cross-cluster interference in a dense D2D network we propose dividing the densely deployed D2D user equipments (UEs) network into well-separated clusters using spectral clustering with modified kernel weights. The proposed spectral clustering scheme obtains well-separated clusters with regards to cross-cluster interference, that is, the UEs that offer maximum interference to each other are grouped into the same cluster. Thereafter, a dynamic resource allocation algorithm is proposed within each cluster to reduce the intra-cluster interference. The proposed dynamic resource allocation algorithm uses graph coloring to allocate resources in such a manner that after each spectrum allocation, a small cell base station updates the interference graph and assigns the next largest interference affected UE a spectrum resource that minimizes the overall intra-cluster interference the most. In conventional graph coloring, the adjacent UEs are allocated different spectrum resources without taking into consideration if the allocated spectrum resource might result in increased interference in the cluster. The simulation results show that the proposed clustering strategy considerably reduces the average cross-cluster interference as compared to other benchmark clustering algorithms such as K-means and KPCA. Moreover, the proposed resource allocation algorithm decreases the intra-cluster interference in the network resulting in the overall SINR maximization of the network.