Summary and Contributions: (1) A partially view-aligned problem (PVP) is studied in this paper. This is an interesting and novel problem which has been ignored by existing multi-view learning methods. (2) A partially view-aligned clustering method is proposed to solve the PVP, which contains a differentiable data alignment neural network.
Strengths: The PVP is a new setting with broad application and it is not enough investigated in the community. In addition, I think that it is with some practical values. For example, when one accesses to the data from multiple cameras, different cameras would provide a partial view of the same place due to difference receptive field, and aligning them and reasoning about it seems difficult and interesting. The proposed representation learning module and the differentiable alignment model are reasonable to improve the feature representation capability and solve the problem of partially aligned views. Experimental results also demonstrate the effectiveness of the model.
Weaknesses: The discussion on the partially view-aligned problem is not comprehensive enough to me. Besides the multi-view community, do the other communities such multi-modality will face this issue? Has it studied in these communities? If did, what the difference with them? This problem seems quite important and it is unclear why previous works ignored it. More literature review is needed. Some important details of the network setting is missing. What is the memory cost and parameter complexity of the proposed model? How about the compared AE2-Nets?
Correctness: Yes
Clarity: It is easy to read, but further revision is needed.
Relation to Prior Work: More discussion is needed.
Reproducibility: Yes
Additional Feedback: 1. Besides the multi-view community, do the other communities such multi-modality will face this issue? Has it studied in these communities? If did, what the difference with them? 2. What specific characteristics of clustering are utilized in the proposed PVC method? 3. Is it ok to plug the proposed differentiable aligning module into other neural network? How about the convergence of the module?
Summary and Contributions: The paper studies a special Multi-view clustering problem in which the data in different views might be partially aligned, i.e., the columns across different views might come from different categories. In the proposed approach, the neural network learns the permutation matrix in an end-to-end manner.
Strengths: 1. This paper investigates the special Muti-view clustering problem in which the data in different views might be partially aligned, i.e., the columns across different views might come from different categories. Such a so-called PVP is novel to the community. 2. To solve the view alignment, PVP is reformulated as an integer linear programming problem which is further solved in a differentiable way so that the learning of the permutation matrix could jointly work with representation learning of deep neural networks.
Weaknesses: 1. Why use ReLu in eq.8; 2. Is the differentiable module (eq8-10, Fig2.b) work in a recurrent manner? If does, how many layers (iteration) in the module. How about the convergence of this module, as well as its influence on the entire network. 3. Why in some tests, your method (with partially aligned data) could be better than the methods with fully aligned data?
Correctness: Yes
Clarity: Yes
Relation to Prior Work: Yes
Reproducibility: Yes
Additional Feedback: 1.Why use ReLu in eq.8; 2.Is the differentiable module (eq8-10, Fig2.b) work in a recurrent manner? If does, how many layers (iteration) in the module. How about the convergence of this module, as well as its influence on the entire network. 3.Why in some tests, your method (with partially aligned data) could be better than the methods with fully aligned data?
Summary and Contributions: The work studies partially view-aligned problem, i.e., for two data matrices X1 and X2 corresponding to two views, they do not assume that X1 and X2 are fully aligned in row-wise. Instead, only a small portion of the matrices has established the correspondence in advance. To solve this problem, a novel multi-view clustering method termed as partially view-aligned clustering is proposed, which consists of a differentiable alignment module and a representation learning module. Extensive experiments verify the effectiveness of our learning paradigm.
Strengths: 1. The partially view-aligned problem is novel as far as I known; 2. The proposed solution achieves promising result, and it seems also novel and effective to learn the data permutation matrix in a differentiable neural module. 3. The paper is well-written and easily followed, and extensive experiments are conducted.
Weaknesses: 1. I like the solution of learning the permutation matrix in a neural network, however, some details are expected to this module, esp, could the module be trained in an end-to-end manner? If does, as this module seems like a recurrent network, which one optimizer is used to train your whole network? The convergence analysis should also be conducted in an ablation studies. 2. Why adopt the relu in your module? I do not see the detailed derivation, any specific reason? Are there other choices?
Correctness: Yes
Clarity: Yes
Relation to Prior Work: Yes
Reproducibility: Yes
Additional Feedback: 1. I like the solution of learning the permutation matrix in a neural network, however, some details are expected to this module, esp, could the module be trained in an end-to-end manner? If does, as this module seems like a recurrent network, which one optimizer is used to train your whole network? The convergence analysis should also be conducted in an ablation studies. 2. Why adopt the relu in your module? I do not see the detailed derivation, any specific reason? Are there other choices?
Summary and Contributions: This paper studies partially view-aligned clustering, which is a new problem and has not been explored before. Giving two datasets from two modalities that are not fully aligned, the proposed method reconstructs the data in a latent space and perform an alternate of Hungarian algorithm to generate alignment. Comprehensive evaluations are conducted on multiple benchmark datasets.
Strengths: 1. This paper presents a new paradigm of multi-view learning, i.e., partially view-aligned clustering. Considering the difficulty of full data alignment in practice, this paper would inspire many other research topics in machine learning. 2. The proposed model could effectively utilize the partially-aligned data for model training. 3. The proposed differentiable alignment module is a surrogate of the non-differentiable Hungarian algorithm. This new module could be incorporated into many other machine learning models.
Weaknesses: 1. In experiments, Table 1 shows that partially aligned PVC obtains even better than baselines with full alignment. It seems that, in the partially aligned setting, A and U have the same size. The authors may add discussions on the performance of PVC when U is much larger than A. 2. I wonder if the model could handle multi-view data. As the proposed method matches two graphs corresponding to two views, it seems difficult to extend the method to multiple graphs. 3. In Fig.3, why the convergence curve is fluctuant with the training epoch? 4. Some typos in the current version should be addressed.
Correctness: The claims and proposed method in this paper are correct. The empirical methodology is also correct.
Clarity: The paper is very well written. Technical details are easy to follow.
Relation to Prior Work: This paper presents a new problem setting. The authors provide comprehensive comparisons between this paper and prior works.
Reproducibility: Yes
Additional Feedback: Update: the authors' response has addressed my previous concerns. Thus, I'd like to keep my original score.