The spread of fake news on online media is very dangerous and can lead to casualties, effects on psychology, character assassination, elections for political parties, and state chaos. Fake news that concerning Covid-19 massively spread during the pandemic. Detecting misinformation on the Internet is an essential and challenging task since humans face difficulty detecting fake news. We applied BERT and GPT2 as pre-trained using the BiGRU-Att-CapsuleNet model and BiGRU-CRF features augmentation to solve Fake News detection in Constraint @ AAAI2021 - COVID19 Fake News Detection in English Dataset. This research proved that our hybrid model with augmentation got better accuracy compared to our baseline model. It also showed that BERT gave a better result than GPT2 in all models; the highest accuracy we achieved for BERT is 0.9196, and GPT2 is 0.8986.