Recent years have seen significant improvement in gate-level Hardware Trojan (HT) detection using machine learning (ML) techniques. The majority of ML approaches for gate-level Hardware Trojan detection have relied exclusively on extracting numerical features from circuit netlists, such as fan-in/fan-out counts, path delays, and structural metrics, to train classification models. This work introduces BERT-HIT which leverages BERT (Bidirectional Encoder Representations from Transformers) for HT detection. Our method learns to understand the structural patterns and connectivity relationships in gate-level netlists, capturing complex contextual dependencies between circuit elements that hand-crafted features cannot represent. This direct text-processing approach eliminates the feature engineering bottleneck while achieving superior detection performance. Furthermore, we incorporate attention visualization techniques to provide explainability, revealing which parts of the netlist text contribute most to HT detection decisions. Our experiments on Trust-HUB benchmarks demonstrate that BERT-HIT achieves superior detection capabilities with an average 94.9\% True Positive Rate, 96.6\% True Negative Rate, and 0.829 Matthews Correlation Coefficient, significantly outperforming existing feature-oriented detection methods.