I actually came to the same conclusion recently! Sometimes simple solutions outperform the complex ones…yet, BERT won this one. Thanks for sharing your experience, that gives hope on getting even better results on NLP tasks!