Learning to Tag OOV Tokens by Integrating Contextual Representation and...

OOV words often degrades the performance of slot taggers, which could be solved with extern knowledge like WordNet. This paper proposes a simple method to incorporate knowledge triples into BiLSTM taggers for tagging OOVs.


WordNet provides triples like a those in a Knowledge Graph which are expanded in two hops to create input triples. The subject and object embeddings are learnt by max-margin ranking. Given the hidden state of a token, these two levels of concepts are attended and concatenated to provide knowledge aware representations (the right part of the following figure).



  • The methods are trivial and not very interesting. The relation embeddings are not learnt or used.
  • The definitions of S_q and S_{q^\prime} are missing.
  • The baselines are too weak. How about a BERT tagger with WordNet as gazetteers?
  • The application of this method is quite limited as WordNet has not been updated for a long time. What happens if a token is OOV of WordNet?
  • 5: Transformative: This paper is likely to change our field. It should be considered for a best paper award.
  • 4.5: Exciting: It changed my thinking on this topic. I would fight for it to be accepted.
  • 4: Strong: I learned a lot from it. I would like to see it accepted.
  • 3.5: Leaning positive: It can be accepted more or less in its current form. However, the work it describes is not particularly exciting and/or inspiring, so it will not be a big loss if people don’t see it in this conference.
  • 3: Ambivalent: It has merits (e.g., it reports state-of-the-art results, the idea is nice), but there are key weaknesses (e.g., I didn’t learn much from it, evaluation is not convincing, it describes incremental work). I believe it can significantly benefit from another round of revision, but I won’t object to accepting it if my co-reviewers are willing to champion it.
  • 2.5: Leaning negative: I am leaning towards rejection, but I can be persuaded if my co-reviewers think otherwise.
  • 2: Mediocre: I would rather not see it in the conference.
  • 1.5: Weak: I am pretty confident that it should be rejected.
  • 1: Poor: I would fight to have it rejected.

0 投票者