Construction of a mental health risk model for college students with long and short-term memory networks and early warning indicators

3.1.1 Input layer

In the MH risk model for college students in this article, the input data are time series data [18,19], which include a series of regularly collected measurements of MH related indicators.

The shape of the data input is the number of samples, time step, and number of features.

Sample size refers to the number of data points or samples owned, representing the observation points of different college students, The time step represents the continuity in time, corresponding to the time point of data collection. Data were recorded once a day, with a time step corresponding to the number of days. The number of features represents the number of features included in each time step, which are usually measurement indicators related to MH. Each feature can be a variable, such as anxiety level, depression level, social interaction, etc.

3.1.2 LSTM neural unit

LSTM is also a recursive structured neural network [20,21], with the most significant feature being the inclusion of more complex units. The LSTM unit has three gates [22] (input gate, forgetting gate, and output gate), as well as an internal cell state. These gates control the flow of information, allowing LSTM to better handle long sequences and capture long-term dependencies.

3.1.3 Output layer

In neural networks, the output layer is the last layer of the model, responsible for generating the final prediction or output of the model. At the output layer, there is only one neuron, as this is a binary task used to represent whether the sample is at risk (1 represents risk, 0 represents no risk).

When LSTM proceeds to the output layer, the model determines the final output through a sigmoid activation function. The calculation formula is as follows:

where x is the input value. The sigmoid function can convert any input between 0 and 1. To obtain the final output, the experiment requires the Cross Entropy Loss function [25] to measure the difference between the predicted results of the model and the actual results. The calculation formula is as follows:

where y is the true label (0 or 1), and p is the probability predicted by the model (between 0 and 1). formula (7) can calculate the loss of each sample and take the average value as the loss of the entire training set.

3.2.1 Dataset partitioning

In this experiment, the obtained datasets were divided into two parts: the training set and the test set, which ensured that the validation set and test set data came from the same distribution to evaluate the performance of the model on unprecedented data. The experimental random sampling method was used to divide two different parts, with a ratio of 0.8 for the training set and 0.2 for the test set, respectively. This enabled the partitioning of the dataset to prevent overfitting and underfitting of the model.

The validation set was mainly used to adjust the model parameters trained in the training set. The test set was used as a dataset that had never existed before. After determining the model parameters, the test set was used for model performance evaluation to avoid overfitting of the model.

3.2.2 Model hyperparameter setting

In the LSTM training process, hyperparameters [26] can be divided into two categories: mandatory input parameters and optional parameters. In this experiment, in addition to the necessary input parameters such as the dimension of input features, the dimension of hidden layer states, and the number of layers stacked by LSTM, the optional parameters include learning rate, batch size, and epochs. By adjusting different combinations of these hyperparameters, optimal model performance can be achieved.

During the training process of the LSTM model, optimizing hyperparameters is crucial to ensure optimal performance. In this study, hyperparameters include learning rate, batch size, and epochs. To find the best parameter combination, we employed the grid search method, systematically traversing various parameter combinations.

The learning rate, as a critical parameter in optimization algorithms, directly impacts the speed and quality of model training. Starting from 0.01, we gradually reduced the learning rate and determined the optimal value by comparing the performance of the model under different learning rates. The choice of batch size strikes a balance between computational resource constraints and model performance. After multiple experiments, we determined an optimal batch size that ensures both training efficiency and avoids overfitting. The setting of the number of epochs is based on when the model’s loss value on the training set stabilizes. Furthermore, to prevent overfitting and enhance the model’s generalization ability, we introduced L2 regularization. By adding a regularization term during training, we limited the size of model weights, making the model more robust. The selection of the regularization strength λ is based on cross-validation results, ensuring optimal performance on an independent validation set. Through multiple experiments, we meticulously adjusted the λ value until finding the optimal regularization parameter. In the initial training, the dimension of the input features was determined to be 16, and the dimension of the hidden layer state was 240; the number of LSTM layers was 2, and the number of neurons in each layer was 64. Other hyperparameters of the model, such as learning rate, batch size, and epoch number, were set to 0.01, 32, and 50 in the initial experiment, respectively.

3.2.3 Model training process

In the model architecture, the LSTM model architecture in the experiment was clarified. Next during the training process, it is necessary to determine the number of LSTM layers, the number of neurons in each layer, and the settings of each hyperparameter. The model was tested using the training set.

Each LSTM layer contains 64 neurons. This parameter determines the number and complexity of memory units in each layer. 64 neurons are a medium-sized choice for this experiment and many other tasks, which can be used to effectively learn features and temporal information.

The learning rate determines the magnitude of parameter updates in each epoch of the model. A smaller learning rate can help the model converge more stably, but it may require more epochs to achieve optimal performance. 0.01 is a common starting learning rate.

The batch size determines the number of samples used to update parameters in each training step. A smaller batch size can improve the training speed of the model, but may introduce noise. 32 is a common batch size choice that balances performance and training speed.

Choosing 50 epochs is a reasonable starting point, but the actual number of epochs in the future needs to be adjusted based on the performance of the model after initial training.

For the setting of these hyperparameters, each parameter was tested in the experiment, as shown in Table 1.

Table 1

Model hyperparameter settings

The situation of different epoch trainings is shown in Figure 1. Under different epochs, the loss value of the model varied, with epochs set to 50 (black). Due to insufficient epochs, the model cannot reach convergence and the loss value was still in a decreasing state. The epoch of 100 (red) finally converged at 0.0519, while the epoch of 150 (blue) also reached convergence at 0.0674, but the overall loss value was lower than the red curve.

Figure 1

Loss values under different epochs.

Next the accuracy and loss values of the model under different learning rates were analyzed in the experiment. When the learning rate was 0.001, the accuracy of the model in the entire epoch was the highest, reaching 0.9046. However, when the learning rate was 0.005, the final calculated loss value of the model was the lowest, and finally converged around 0.0462.

From Figure 2(a), it can be seen that the accuracy and training duration of parameter batch size of the model were plotted in 16, 32, 64, 128, and 256, respectively. Figure 2(b) shows the memory consumption and central processing unit (CPU) consumption of model batch size in 16, 32, 64, 128, and 256, respectively. For batch size, at 16:00 , the accuracy, training duration, memory consumption, and CPU consumption were 80.2%, 2.5 h, 3.4 GB (Gigabyte), and 32%, respectively. At 32:00, the accuracy, training duration, memory consumption, and CPU consumption were 82.5%, 4 h, 4.2 GB, and 43%, respectively. At 64 h, the accuracy, training duration, memory consumption, and CPU consumption were 84.8%, 6.5 h, 6.2 GB, and 49%, respectively. At 128 h, the accuracy, training duration, memory consumption, and CPU consumption were 90.3%, 9 h, 9.2 GB, and 52%, respectively. At 256, the accuracy, training duration, memory consumption, and CPU consumption were 91.6%, 19 h, 16.9 GB, and 83%, respectively. Upon careful analysis of the data in Figure 2, it can be observed that as batch size increased from 16 to 128, the accuracy increased while all resource consumption indicators steadily increased. When batch size increased to 256, the accuracy only improved by 1.3% (91.6–90.3%), but the training time increased from 9 to 19 h; memory consumption increased from 9.2 to 16.9; the CPU utilization rate ranged from 52 to 83%. There was a huge difference between resource investment and the final effect, so in the subsequent variable testing, batch size selected 128 as the best choice.

Figure 2

Resource consumption of different batch size models. (a) Training duration and accuracy of models. (b) Memory CPU resource consumption.

The impact of different numbers of hidden layers on the four performance indicators of the final model, R2 (R-squared), MAE (Mean Absolute Error), MSE (Mean Squared Error), and precision are analyzed. Analysis showed that after 240 hidden layers, R2 and precision indicators could not increase with the increase of hidden layers, resulting in saturation and a slight decrease. However, MSE and MAE error values actually increased with the increase of hidden layers.

Based on the continuous experimentation with various parameters, the final training of the model was conducted using the parameters listed in Table 2 on the training set.

3.2.4 Regularization and avoidance of overfitting

In this experiment, L2 regularization [27,28] was used to regularize the model, which can effectively control the complexity of the model and improve its generalization ability. By adding regularization terms to limit the weight of the model to prevent overfitting, the calculation method is the square root of the sum of squares of ownership weights. Adding L2 regularization term can cause the model to have a trend of shrinking towards the original weight value when updating weights during the training process. The relevant formula is as follows:

where L reg is the loss value of the regularization term and λ is a regularization parameter, also known as regularization intensity, which controls the degree of influence of regularization. A larger λ leads to stronger regularization, leading to model parameters tending towards smaller values, N is the number of parameters in the model, and θ i represents the ith parameter of the model.

In the L2 regularization process, the regularization parameters need to be manually adjusted. The function of this parameter is to control the degree to which the regularization term punishes the loss function. During the adjustment process, a partitioned dataset needs to be used. The model was trained on the training set and evaluated for performance under different λ values on the validation set. The λ value with the best performance on the validation set was selected and used for the final test set. This article adjusted the final evaluation results of the model by inputting different λ parameters multiple times. Based on the evaluation results, the regularization parameter λ with the best neutral performance in all cross validation [29] was selected.

The experiment used the divided validation set for model regularization. Table 3 lists the regularization loss results under different λ values.

By adding L2 regularization terms, the model tends to shrink the weight values to a smaller range during training, thereby preventing overfitting caused by excessive weight. This makes the model’s generalization ability better and performs better on unknown data.

Finally, the four evaluation indicators accuracy, precision, MSE, and MAE in Table 4 were obtained by predicting the test set using the parameters selected in Table 3.

According to Table 4, after adjusting the parameters, the model maintained an average precision of 0.906 for the four psychological indicators. The average accuracy was 0.852, and the average values of MSE and MAE reached 0.32 and 0.37 respectively, indicating that the model had a good performance in prediction.

In order to comprehensively evaluate the performance of the model, in addition to the existing metrics of accuracy and precision, this study introduced recall and F1 score as additional evaluation metrics. These two metrics help us gain a more comprehensive understanding of the model’s performance in identifying MH risks. Recall refers to the proportion of correctly identified MH risk cases by the model among all actual risk cases, while the F1 score is the harmonic mean of precision and recall, used to balance precision and recall.

By adding these two evaluation metrics, we gain a more comprehensive understanding of the model’s performance, ensuring the reliability and generalizability of the research results. Specific evaluation results indicate that the model’s performance in recall and F1 score is also excellent, achieving 0.97 (recall) and 0.96 (F1 score), respectively. Together with the previously mentioned accuracy and precision, the model’s efficiency in predicting MH risks among college students is demonstrated.

4.1.1 Selection of early warning indicators

In the output of the model constructed above, this article conducts graded warning and intervention of MH risks for college students [30], helping them take appropriate measures before their psychological problems gradually worsen. In the selection of warning indicators, the experiment added corresponding weights to the indicators set in each data, as shown in Table 5.

Finally, the comprehensive psychological abnormality index was obtained from formula (9), and the five psychological indicators correspond to a weight. The results calculated through weight constraints can reflect the psychological situation of students to a certain extent.

where P is the final index result, W is the weight, and X is the numerical value of each psychology.

4.1.2 Threshold setting

Setting a flexible threshold to increase the sensitivity of early warning is an effective method, especially for early detection of MH problems. The experiment was divided into three levels: low level, medium level, and high level. Each level corresponds to a different range of comprehensive indices and triggers different levels of warning measures.

4.2.1 Low level warning effect

The goal of low-level early warning is to identify MH issues in the early stages and encourage students to actively seek psychological support. Through the analysis of research data, it was found that low-level early warning has a certain effect in detecting potential problems in the early stage. Students usually have a positive response to these low-level warnings and demonstrate a higher willingness to engage with MH professionals.

4.2.2 Medium level warning effect

The goal of medium level warning is to provide more specific recommendations to support students’ MH. According to the data, medium level warnings also have significant effects within a higher comprehensive index range. Students tend to actively respond to mid-level warnings and are more willing to take proactive actions, such as consulting with MH experts or receiving evaluations.

4.2.3 High level warning effect

The goal of high-level warning is to take urgent action immediately when the composite index reaches a dangerous level. According to the data, high-level warnings provide timely and effective warnings in very serious situations. This warning triggered a rapid response. It was observed that in the context of high-level warnings, students received urgently needed support and intervention to help them deal with MH issues.

Overall, the multi-level warning threshold scheme had shown good results in this study. It not only improved the early identification of MH issues, but also ensured that emergency measures are taken in dangerous situations. This early warning system has had a positive impact on the MH management of college students. Based on experimental data and student feedback, it can be concluded that this early warning system provides more timely and targeted support for students, which helps to improve their MH.