This paper adds contrastive learning loss into traditional adversarial training and obtained competitive results. Under a vanilla auto-encoder framework, they add two novel losses: a Cooperative Contrastive Learning loss to bring similar constrained sentences together directly and a Cooperative Classification loss to indirectly achieve this by predicting the constraints using hidden states from encoders or the critic. Constraints are defined as sentence length, syntax tree height etc.
Comments
- Wow, the way they turning a linguistic intuition into training algorithm is amazing.
- A couple of questions:
- In line 12 of Algorithm 1, l_{crc} seems to be backpropated twice. Is it necessary?
- I’m not fully clear how \mathcal{L}_{clf} works. Which sentences are you comparing by “predicts the different constraints”? In my understanding, you create pos/neg samples from your mini-batch.
- How do you create mini-batch exactly? What if there were no pos or neg samples? Will you try to balance their ratio?
- Funny that the work by a Chinese group leaded by Lenovo was not able to be reproduced and “Repeated attempts to obtain the original source code failed”. Made my day dudes.
Rating
- 5: Transformative: This paper is likely to change our field. It should be considered for a best paper award.
- 4.5: Exciting: It changed my thinking on this topic. I would fight for it to be accepted.
- 4: Strong: I learned a lot from it. I would like to see it accepted.
- 3.5: Leaning positive: It can be accepted more or less in its current form. However, the work it describes is not particularly exciting and/or inspiring, so it will not be a big loss if people don’t see it in this conference.
- 3: Ambivalent: It has merits (e.g., it reports state-of-the-art results, the idea is nice), but there are key weaknesses (e.g., I didn’t learn much from it, evaluation is not convincing, it describes incremental work). I believe it can significantly benefit from another round of revision, but I won’t object to accepting it if my co-reviewers are willing to champion it.
- 2.5: Leaning negative: I am leaning towards rejection, but I can be persuaded if my co-reviewers think otherwise.
- 2: Mediocre: I would rather not see it in the conference.
- 1.5: Weak: I am pretty confident that it should be rejected.
- 1: Poor: I would fight to have it rejected.
0 voters