research-article

Structural Deep Clustering Network

Authors:

Peng CuiAuthors Info & Claims

WWW '20: Proceedings of The Web Conference 2020

Pages 1400 - 1410

https://doi.org/10.1145/3366423.3380214

Published: 20 April 2020 Publication History

Abstract

Clustering is a fundamental task in data analysis. Recently, deep clustering, which derives inspiration primarily from deep learning approaches, achieves state-of-the-art performance and has attracted considerable attention. Current deep clustering methods usually boost the clustering results by means of the powerful representation ability of deep learning, e.g., autoencoder, suggesting that learning an effective representation for clustering is a crucial requirement. The strength of deep clustering methods is to extract the useful representations from the data itself, rather than the structure of data, which receives scarce attention in representation learning. Motivated by the great success of Graph Convolutional Network (GCN) in encoding the graph structure, we propose a Structural Deep Clustering Network (SDCN) to integrate the structural information into deep clustering. Specifically, we design a delivery operator to transfer the representations learned by autoencoder to the corresponding GCN layer, and a dual self-supervised mechanism to unify these two different deep neural architectures and guide the update of the whole model. In this way, the multiple structures of data, from low-order to high-order, are naturally combined with the multiple representations learned by autoencoder. Furthermore, we theoretically analyze the delivery operator, i.e., with the delivery operator, GCN improves the autoencoder-specific representation as a high-order graph regularization constraint and autoencoder helps alleviate the over-smoothing problem in GCN. Through comprehensive experiments, we demonstrate that our propose model can consistently perform better over the state-of-the-art techniques.

References

[1]

Charu C Aggarwal and ChengXiang Zhai. 2012. A survey of text clustering algorithms. In Mining text data. Springer, 77–128.

[2]

Mikhail Belkin and Partha Niyogi. 2003. Laplacian eigenmaps for dimensionality reduction and data representation. Neural computation 15, 6 (2003), 1373–1396.

Digital Library

[3]

Mathilde Caron, Piotr Bojanowski, Armand Joulin, and Matthijs Douze. 2018. Deep clustering for unsupervised learning of visual features. In ECCV. 132–149.

[4]

Xifeng Guo, Long Gao, Xinwang Liu, and Jianping Yin. 2017. Improved deep embedded clustering with local structure preservation. In IJCAI. 1753–1759.

[5]

John A Hartigan and Manchek A Wong. 1979. Algorithm AS 136: A k-means clustering algorithm. Journal of the Royal Statistical Society. Series C (Applied Statistics) 28, 1(1979), 100–108.

Digital Library

[6]

William Grant Hatcher and Wei Yu. 2018. A survey of deep learning: platforms, applications and emerging research trends. IEEE Access 6(2018), 24411–24432.

[7]

Geoffrey E Hinton and Ruslan R Salakhutdinov. 2006. Reducing the dimensionality of data with neural networks. Science 313, 5786 (2006), 504–507.

[8]

Pan Ji, Tong Zhang, Hongdong Li, Mathieu Salzmann, and Ian Reid. 2017. Deep subspace clustering networks. In NIPS. 24–33.

[9]

Zhuxi Jiang, Yin Zheng, Huachun Tan, Bangsheng Tang, and Hanning Zhou. 2017. Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering. In IJCAI. 1965–1972.

[10]

Thomas N Kipf and Max Welling. 2016. Variational graph auto-encoders. arXiv preprint arXiv:1611.07308(2016).

[11]

Thomas N. Kipf and Max Welling. 2017. Semi-Supervised Classification with Graph Convolutional Networks. In ICLR.

[12]

Johannes Klicpera, Aleksandar Bojchevski, and Stephan Günnemann. 2019. Predict then Propagate: Graph Neural Networks meet Personalized PageRank. In ICLR.

[13]

Yann Le Cun, Ofer Matan, Bernhard Boser, John S Denker, Don Henderson, Richard E Howard, Wayne Hubbard, LD Jacket, and Henry S Baird. 1990. Handwritten zip code recognition with multilayer networks. In ICPR, Vol. 2. IEEE, 35–40.

[14]

David D Lewis, Yiming Yang, Tony G Rose, and Fan Li. 2004. Rcv1: A new benchmark collection for text categorization research. Journal of machine learning research 5, Apr (2004), 361–397.

Digital Library

[15]

Qimai Li, Zhichao Han, and Xiao-Ming Wu. 2018. Deeper insights into graph convolutional networks for semi-supervised learning. In AAAI.

[16]

Weiwei Liu and Ivor W. Tsang. 2017. Making Decision Trees Feasible in Ultrahigh Feature and Label Dimensions. Journal of Machine Learning Research 18 (2017), 81:1–81:36.

Digital Library

[17]

Weiwei Liu, Donna Xu, Ivor W. Tsang, and Wenjie Zhang. 2019. Metric Learning for Multi-Output Tasks. IEEE Transactions on Pattern Analysis and Machine Intelligence 41, 2(2019), 408–422.

Digital Library

[18]

Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, Nov (2008), 2579–2605.

[19]

Alireza Makhzani, Jonathon Shlens, Navdeep Jaitly, Ian Goodfellow, and Brendan Frey. 2015. Adversarial autoencoders. arXiv preprint arXiv:1511.05644(2015).

[20]

Pankaj Malhotra, Anusha Ramakrishnan, Gaurangi Anand, Lovekesh Vig, Puneet Agarwal, and Gautam Shroff. 2016. LSTM-based encoder-decoder for multi-sensor anomaly detection. arXiv preprint arXiv:1607.00148(2016).

[21]

Jonathan Masci, Ueli Meier, Dan Cireşan, and Jürgen Schmidhuber. 2011. Stacked convolutional auto-encoders for hierarchical feature extraction. In ICANN. Springer, 52–59.

Digital Library

[22]

Vinod Nair and Geoffrey E Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In ICML. 807–814.

[23]

Andrew Y Ng, Michael I Jordan, and Yair Weiss. 2002. On spectral clustering: Analysis and an algorithm. In NIPS. 849–856.

[24]

Peter J Rousseeuw. 1987. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of computational and applied mathematics 20 (1987), 53–65.

Digital Library

[25]

Allan Stisen, Henrik Blunck, Sourav Bhattacharya, Thor Siiger Prentow, Mikkel Baun Kjærgaard, Anind Dey, Tobias Sonne, and Mads Møller Jensen. 2015. Smart devices are different: Assessing and mitigatingmobile sensing heterogeneities for activity recognition. In SenSys. 127–140.

[26]

Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008. Extracting and composing robust features with denoising autoencoders. In ICML. 1096–1103.

[27]

Chun Wang, Shirui Pan, Ruiqi Hu, Guodong Long, Jing Jiang, and Chengqi Zhang. 2019. Attributed Graph Clustering: A Deep Attentional Embedding Approach. In IJCAI. 3670–3676.

[28]

Xiao Wang, Houye Ji, Chuan Shi, Bai Wang, Yanfang Ye, Peng Cui, and Philip S. Yu. 2019. Heterogeneous Graph Attention Network. In WWW. 2022–2032.

[29]

Junyuan Xie, Ross Girshick, and Ali Farhadi. 2016. Unsupervised deep embedding for clustering analysis. In ICML. 478–487.

[30]

Bo Yang, Xiao Fu, Nicholas D Sidiropoulos, and Mingyi Hong. 2017. Towards k-means-friendly spaces: Simultaneous deep learning and clustering. In ICML. 3861–3870.

[31]

Yi Yang, Dong Xu, Feiping Nie, Shuicheng Yan, and Yueting Zhuang. 2010. Image clustering using local discriminant models and global integration. IEEE Transactions on Image Processing 19, 10 (2010), 2761–2773.

Digital Library

Cited By

Yun KYun HLee SOh JKim MLim MLee JKim CSeo JChoi J(2024)A Study on Machine Learning-Enhanced Roadside Unit-Based Detection of Abnormal Driving in Autonomous VehiclesElectronics10.3390/electronics1302028813:2(288)Online publication date: 8-Jan-2024
https://doi.org/10.3390/electronics13020288
Zhu YBai LNing ZFu WLiu JJiang LFei SGong SLu LDeng MYi M(2024)Deep Learning for Clustering Single-cell RNA-seq DataCurrent Bioinformatics10.2174/157489361866622113009405019:3(193-210)Online publication date: Mar-2024
https://doi.org/10.2174/1574893618666221130094050
Yang ZZhang HYang CLi BZhao XLong Y(2024)Scale Fairness on Spectral ClusteringProceedings of the 36th International Conference on Scientific and Statistical Database Management10.1145/3676288.3676301(1-9)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3676288.3676301
Show More Cited By

Index Terms

Structural Deep Clustering Network
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Unsupervised learning
    2. Machine learning approaches
      1. Neural networks
2. Information systems
  1. Information systems applications

Index terms have been assigned to the content through auto-classification.

Recommendations

Deep image clustering with contrastive learning and multi-scale graph convolutional networks
Abstract
Deep clustering has shown its promising capability in joint representation learning and clustering via deep neural networks. Despite the significant progress, the existing deep clustering works mostly utilize some distribution-based clustering ...
Highlights
- This paper for the first time enables multi-scale structure learning for image clustering via GCNs.
- Multi-scale structure learning and two levels of contrastive learning are jointly enforced.
- A novel deep image clustering approach ...
Deep structural enhanced network for document clustering
Abstract
Recently, deep document clustering, which employs deep neural networks to learn semantic document representation for clustering purpose, has attracted increasing research interests. Traditional deep document clustering models rely only the ...
Attention-based hierarchical denoised deep clustering network
Abstract
Clustering is a basic task of data analysis and decision making. Recently, graph convolution network (GCN) based deep clustering frameworks have produced the state-of-the-art performance. However, the traditional GCN has not fully learnt the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '20: Proceedings of The Web Conference 2020

April 2020

3143 pages

ISBN:9781450370233

DOI:10.1145/3366423

Editors:
Yennun Huang
Acadmica sinica, Taiwan
,
Irwin King
The Chinese University of Hong Kong, Hong Kong
,
Tie-Yan Liu
Microsoft Research Asia, China
,
Maarten van Steen
University of Twente, Netherlands

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '20

Sponsor:

SIGWEB

WWW '20: The Web Conference 2020

April 20 - 24, 2020

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

273
Total Citations
View Citations
2,952
Total Downloads

Downloads (Last 12 months)643
Downloads (Last 6 weeks)43

Reflects downloads up to 28 Aug 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yun KYun HLee SOh JKim MLim MLee JKim CSeo JChoi J(2024)A Study on Machine Learning-Enhanced Roadside Unit-Based Detection of Abnormal Driving in Autonomous VehiclesElectronics10.3390/electronics1302028813:2(288)Online publication date: 8-Jan-2024
https://doi.org/10.3390/electronics13020288
Zhu YBai LNing ZFu WLiu JJiang LFei SGong SLu LDeng MYi M(2024)Deep Learning for Clustering Single-cell RNA-seq DataCurrent Bioinformatics10.2174/157489361866622113009405019:3(193-210)Online publication date: Mar-2024
https://doi.org/10.2174/1574893618666221130094050
Yang ZZhang HYang CLi BZhao XLong Y(2024)Scale Fairness on Spectral ClusteringProceedings of the 36th International Conference on Scientific and Statistical Database Management10.1145/3676288.3676301(1-9)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3676288.3676301
Ding SWu BDing LXu XGuo LLiao HWu X(2024)Towards Faster Deep Graph Clustering via Efficient Graph Auto-EncoderACM Transactions on Knowledge Discovery from Data10.1145/367498318:8(1-23)Online publication date: 16-Aug-2024
https://dl.acm.org/doi/10.1145/3674983
Chen JJi YZou RZhang YCheung YBaeza-Yates RBonchi F(2024)QGRL: Quaternion Graph Representation Learning for Heterogeneous Feature Data ClusteringProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3637528.3671839(297-306)Online publication date: 25-Aug-2024
https://dl.acm.org/doi/10.1145/3637528.3671839
Sun LHu JZhou SHuang ZYe JPeng HYu ZYu PChua TNgo CKa-Wei Lee RKumar RLauw H(2024)RicciNet: Deep Clustering via A Riemannian Generative ModelProceedings of the ACM Web Conference 202410.1145/3589334.3645428(4071-4082)Online publication date: 13-May-2024
https://dl.acm.org/doi/10.1145/3589334.3645428
Wang PWu DChen CLiu KFu YHuang JZhou YZhan JHua X(2024)Deep Adaptive Graph Clustering via von Mises-Fisher DistributionsACM Transactions on the Web10.1145/358052118:2(1-21)Online publication date: 8-Jan-2024
https://dl.acm.org/doi/10.1145/3580521
Huang BWang D(2024)Structured Deep Graph Clustering Network Based on Consistency ConstraintInternational Journal of Pattern Recognition and Artificial Intelligence10.1142/S021800142452018938:10Online publication date: 16-Jul-2024
https://doi.org/10.1142/S0218001424520189
Pei XDeng XXiong NMumtaz SWu J(2024)Complex Graph Analysis and Representation Learning: Problems, Techniques, and ApplicationsIEEE Transactions on Network Science and Engineering10.1109/TNSE.2024.341785011:5(4990-5007)Online publication date: Sep-2024
https://doi.org/10.1109/TNSE.2024.3417850
Tan DHuang ZPeng XZhong WMahalec V(2024)Deep Adaptive Fuzzy Clustering for Evolutionary Unsupervised Representation LearningIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.324366635:5(6103-6117)Online publication date: May-2024
https://doi.org/10.1109/TNNLS.2023.3243666
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents