Plain text tagging

[nikhilranjan7]

Hi, I am trying to use this model for inference but the plain text tagging comes out to be very weird.

Input:
"Hello, today is Monday but not Tuesday, maybe tomorrow"

Output:
'Hello, today </time x3> is </ timex 3>, maybe this AFTER now type + - today like $ ',

Is this expected or am I missing something ?

[dennlinger]

Hi @nikhilranjan7,
presumably you are using one of the seq2seq-based models, correct? If you are directly using them through a pipeline (and without our provided scripts), then there is a good chance that they do not produce the output that you expected. As you might find in our scripts, we have to perform extensive post-processing to combat model hallucinations.
In addition, I want to point to #9, where I also mention that we found some issues in the evaluation setup of our seq2seq-based models that overestimate their performance.

As a "temporary fix", I can highly recommend to use the tagging-based models, which (at a slightly lower performance) perform more consistently. Let me know if you have any further questions or I got some of the assumptions wrong.

Best,
Dennis