Semantic communications, which aim to effectively convey the meaning of messages (such as text and images) rather than transmitting the exact messages themselves, have garnered widespread attention from industry and academia. A suitable joint source-channel coding (JSCC) scheme is crucial for semantic communication systems, as it can significantly improve system performance, such as communication reliability. Current research efforts primarily focus on employing various deep neural network (DNN) models, particularly the Transformer model, to design JSCC schemes. However, existing Transformer-based JSCC schemes usually exhibit a considerable number of model parameters and computational demands, limiting their real-world applicability. To address this challenge, we propose a novel DNN model based on DeLighT, a deep and lightweight variant of the standard Transformer, using a text semantic communication system (TSC) as an example. This proposed model enables a lightweight JSCC scheme for the TSC system. Through simulation results, we demonstrate that the proposed JSCC scheme achieves comparable or better communication reliability than the Transformer-based JSCC scheme while requiring significantly fewer parameters and smaller runtime.