Spatial layout optimization model integrating layered attention mechanism in the development of smart tourism management

Attention mechanism

In the field of tourism recommendation, attention mechanisms have also received widespread attention and successful applications. Similar to applications in natural language processing and machine translation, attention mechanisms in tourism recommendations mimic human visual principles, selectively focusing on various factors related to tourism and extracting important information, while ignoring factors that have a relatively small impact on the final recommendation results. By learning the importance of various elements in the data for the final tourism recommendation, the attention mechanism assigns different weights to each element. This means that in tourism recommendation, factors with higher weights will have a greater impact on the final tourism recommendation, while factors with lower weights will have a smaller impact on the final recommendation. The attention mechanism model is shown in Fig. 1.

In Fig. 1, Q represents Query, K represents Key, and V represents Value. The attention mechanism describes the problem as a given target element query, calculates the similarity between the query and the key, and finally multiplies the value to obtain the final result. The formula for calculating attention a is as follows:

(1) a=∑i=1lf(Q,K)Vwhere l represents the number of samples, f represents the function for calculating query and key similarity. Common methods include dot product, cosine similarity, and multi-layer neural networks.

Self attention network is a further improved model of attention mechanism, which determines the user’s focus on information by calculating the interdependence between different elements in a sequence, and calculates the user’s interest preference score based on this. The core operation of self attention mechanism is the scaling dot product operation, as shown in Fig. 2.

The self attention network embeds queries (Q), keys (K), and values (V) into the same dimensional space, then performs dot multiplication on the queries and keys, and performs scaling operations to obtain the similarity between the queries and keys through the softmax function. Finally, the similarity is weighted and multiplied with the value to obtain the attention score. The specific formula is as follows:

(2) Attention(Q,K,V)=softmax(QKTdk)Vwhere dk is the dimension of the query and key.

In addition to scaling dot product operations, self attention networks also apply a series of techniques to ensure smooth training, including:

Feed Forward Network (FFN): Although the self attention mechanism can use adaptive weights to aggregate embeddings of all previous attractions, it ultimately remains a linear model. In order to endow the model with nonlinear features and consider the interactions between different potential attractions, the self attention mechanism will process the obtained attention through a double-layer feedforward network. It is worth noting that these multi-layer feedforward networks share parameters in the tourism recommendation model. The specific formula is as follows:

(3) Output=ReLU(W2⋅ReLU(W1⋅Attention(Q,K,V)+b1)+b2)where Attention(Q,K,V) represents the attention output obtained by the self attention mechanism, W1 and W2 are the weight parameters of the double-layer feedforward network, b1 and b2 are bias parameters, respectively, and ReLU is the modified linear unit activation function. In this way, by introducing a dual-layer feedforward neural network, the tourism recommendation model has gained more complex nonlinear modeling capabilities to better capture the intricate relationships between users and different tourist attractions.

Residual connections (RC): In some cases, multi-layer neural networks have been proven effective in learning travel recommendation features. However, simply adding more layers does not always lead to better performance. The core idea of residual connection is to solve this problem by transmitting low-level features to higher levels. Therefore, if the underlying features are useful, the model can more easily pass them on to the final layer. In the context of tourism recommendations, we also assume that residual connections are useful. The specific formula is as follows:

(4) RC(o)=0+ewhere Among them, o is the output of the network before residual connection, and e is the embedding vector of the last sample in the input sequence.

Layer normalization (LN): Adjusting it to zero mean and unit variance can help improve the stability of the neural network and accelerate the training process. In layer normalization, the statistical data used is based on the features in the current batch rather than the entire dataset. Specifically, for the input vector x, which contains an example of all features, the layer normalization operation is defined as:

(5) LayerNorm(x)=α⋅x−μσ2+ε+βwhere ⋅ represents dot product operation, μ is the mean of input vector x, σ is the variance of input vector x, α and β are scaling factors and bias terms.

Dropout: During the training process, neurons are randomly closed with a certain probability, and all neurons are used during testing to alleviate overfitting problems and improve the model’s generalization rate.

Multilayer self attention mechanism recommendation algorithm based on dynamic spatial perception (MSAMR-DSP)

The dynamic characteristics of tourist attractions are crucial in recommendation tasks. Researchers typically focus on how to capture users’ changing interests over time from historical interaction sequences, while ignoring the constantly evolving hidden features of attractions. Firstly, the user’s historical interaction sequence not only includes the dynamic changes in user interests over time, but also contains information on the evolution of potential features of scenic spots over time. If the potential features of tourist attractions are considered invariant, this information will be lost when constructing the association between users and items, resulting in incomplete predictions. Therefore, it is not appropriate to only explore the dynamic changes in the sequence from the user’s perspective. Secondly, most sequence models only focus on the implicit feedback sequence of individual users, while ignoring the collaborative relationships between other users. Analyzing the dynamic changes in the sequence from the perspective of scenic spots can enhance the collaborative effect between users in the model to a certain extent, thereby more accurately analyzing users’ intentions.

Therefore, the method proposed in this article uses a self attention network module to obtain patterns of item feature transformation between different users. Then, the obtained dynamic feature representation of the scenic spot and the traditional user dynamic feature representation are input into the neural network to obtain its complex interaction pattern, achieving a more complete prediction. Therefore, we propose a multilayer self attention mechanism recommendation algorithm based on dynamic spatial perception (MSAMR-DSP), which overcomes the problem of previous recommendation systems ignoring the dynamic characteristics of items. It models personalized sequence patterns between users and items from two levels: user dynamic interest features and item dynamic features; Secondly, it solves the problem of model dynamic sensitivity caused by modeling both item and user dynamic features simultaneously. By balancing the dynamic and static features of the model, it makes the training more stable.

The application level of attention mechanism in current recommendation systems is relatively single, and the mining depth of multiple factors is shallow, failing to fully consider the impact of multi factor attention mechanism and the fusion weight of various indicators on the system. On the basis of the above research, we introduced multi-dimensional and multi-level attention modeling, mining more comprehensive and accurate weights of various types of information in the model. The following text will introduce attention mechanism models and their applications at different levels. The network structure is shown in Fig. 3.

This study models attention mechanisms through multi-layer perceptrons, and incorporates the representation vector q of user u’s historical interaction items q into the attribute preferences of the recommended item r, which can be obtained through Eq. (1):

(6) qj=∑t=1t⁡μj,tqj,twhere μj,t is the preference of the recommended project qr for the t-th attribute qj of the historical interaction project, which is defined as:

(7) μj,t=exp(score(qr,t,qj,t))[ ∑t=1n∑ j,tr,tλexp ]

In recommendation systems, there is a significant difference in the historical interaction data for different items. The traditional softmax function may excessively penalize users with longer historical interaction information. Therefore, this article introduces a smoothing exponent λ with 0≤λ≤1. The attention on overly active parts of the system will not be excessively penalized. In this case, q is the alignment score learned by the model. In this study, an additive (Concat) method was used to calculate the alignment score score(qr,t,qj,t), and a hyperbolic tangent function (tanh) was used as the nonlinear activation function. The calculation method for score(qr,t,qj,t) is as follows:

(8) score(qr,t,qj,t)=va1Ttan⁡h(Wa1[qr,t;qj,t])where va1 and Wa1 are learnable parameters that participate in training as part of the overall model. By Eq. (6), we obtain the representation vector qj of user u’s historical interaction item qj integrated with the attribute preference of the recommended item qr. Furthermore, by introducing an additional layer of attention mechanism, we capture the interaction vector q¯j of the recommended item with respect to the attention weight αr,j obtained from the model above, where qj∈Iu and Iu represents the set of historical interaction items of user u. The structure of the model is similar to the aforementioned process, and the calculation of the new layer of attention weight αr,j is as shown in equation:

(9) ηr,j=exp(score(q¯r,qj))[∑t=1n∑¯rjλexp]

The alignment score a still uses a multi-layer perceptron model for learning, using tanh as the nonlinear activation function, which is defined as shown in equation:

(10) score(q¯r,qj)=va2Ttanh(Wa2[q¯r;qj])

Similarly, va2 and Wa2 are learnable parameters that can be obtained through model training.

Next, we constructed an attention model using user and social information, aiming to explore the attention relationship between user u and their trusted user v. The network structure is similar to that shown in Fig. 4.

pu is the embedding vector of user u, and the trusted user set of user u is su={s1,s2,...,sm}. In this model, a multi-layer perceptron network is introduced to learn the attention relationship between users and their trusted friends. Although most MLPs use the sigmoid function as the activation function, due to the high risk of gradient vanishing and the large amount of derivative calculation during backpropagation, ReLU is used as the nonlinear activation function of the model in this study. The alignment score score(pu,sm) of user u for its trusted user m is obtained by the following formula:

(11) score(pu,sm)=RELU(...RELUW2(RELU(W1[pu,pm]+b1)+b2)...)where W=[W1,W2] with W1∈Rd×h and W2∈Rd×1 (d is the dimension of input features, and h is the number of neurons in the hidden layer) is the weight matrix of the attention model. and b1∈Rh and b2∈R are the bias vectors of the system. Next, normalize the alignment score a as follows:

(12) λu,m=exp⁡(score(pu,sm))[∑m∈suexp(score(pu,sm))]λ

Similar to the above model, in order to fully integrate the user’s preference for trusting users while considering the user’s original information, the definition of the user’s comprehensive feature representation P~u is as follows:

(13) P~u=αpu+(1−α)∑m∈suλu,msmwhere α is a balance parameter used to balance between the user’s original features and the trusted user embedding. The parameter α controls the proportion of the weighted sum of the original feature pu of user u and its trusted user embedding sm in the comprehensive feature representation. Specifically, when α is large, the comprehensive feature representation of user u relies more on its own original features; When α is small, the comprehensive feature representation relies more on the embedding of its trusted users. By adjusting the value of α, the model can find an optimal balance between utilizing the user’s own characteristics and trustworthy user characteristics, thereby enhancing the expressive power of the user’s comprehensive feature representation.

In recommendation systems, the inner product of user and item implicit feature vectors is usually used for rating prediction. In this study, after using a multi-layer attention model, the user comprehensive feature representation P~u and the item comprehensive feature representation P~r were obtained, respectively. Therefore, the specific form of rating prediction for the recommended item r by user u is as follows:

(14) r~ur=P~u⋅P~r

Model fusion

This study has fully considered the multi-layer attention information in recommendation problems, including: using a double-layer attention network to simultaneously mine the preferences of the recommended items for historical items and their item attributes, and mining the attention weights of the target users towards their trusted users under a new social trust relationship, which brings rich auxiliary information to this study and alleviates the problem of data sparsity in the system. Excessive mining of basic information may result in the loss of some key features. Therefore, based on the above attention model, this study uses a multi-layer perceptron to mine the original implicit features of users and items. A simple pooling layer is used to concatenate the user embedding vector and item embedding vector, as shown in Eq. (15):

(15) Con(pu,qr)=[pu∙qr¯puqr].

Similar to previous studies, multilayer perceptrons still use the ReLU function as the activation function, as shown in Formula (16):

(16) C~(pu,qr)=RELU(...RELUW2(RELU(W1Con(pu,qr)+b1)+b2)...)

After adjusting the vector dimension, the obtained implicit features of the original information and the score prediction of the fused attention obtained in “Multilayer Self Attention Mechanism Recommendation Algorithm Based on Dynamic Spatial Perception (MSAMR-DSP)” are element fused to obtain a comprehensive variable, as shown in Eq. (17):

(17) h=r~ur⊙C~ur.

At the top of the model, there is also a multi-layer perceptron used for the final rating prediction of the system. To prevent overfitting, the Dropout mechanism is introduced, which reduces the homogenization problem in the multi-layer perceptron by causing some neurons to lose contact with probability p. After processing by the Dropout mechanism, the comprehensive variable is obtained. During the training process, the backpropagation algorithm BPTT is used for model optimization, and its cross entropy loss function is as follows:

(18) Loss=−∑u=1m∑i=1nrurlog⁡h~+(1−rur)log⁡(1−h~).

This article has constructed attention models for project attributes and project interaction levels, as well as user social attention models and basic information mining models. The various parts have been integrated to obtain rating predictions that incorporate richer information. This model adjusts weights by introducing temporal and spatial feature embeddings to adapt to dynamic changes in users and travel destinations. Specifically, temporal feature embedding includes seasonal changes, time period changes, and holiday effects, while spatial feature embedding covers geographic location and real-time weather conditions. The model utilizes an embedding layer to transform these features into high-dimensional vectors and fuse them with other features to form a joint feature vector for dynamically adjusting weights. The overall framework of this study is shown in Fig. 4.

In this article, we choose ReLU as the activation function in the MLP instead of the traditional activation functions sigmoid or tanh, mainly for the following reasons: First, ReLU’s computation is very simple, requiring only a threshold operation, which makes its computational efficiency high during both forward and backward propagation. This helps speed up model training and prediction. Sigmoid and tanh activation functions require the computation of exponential functions, which is computationally expensive, especially in deep networks, significantly increasing training time. Second, ReLU can effectively handle nonlinear relationships because it is linear in the positive region and zero in the negative region, allowing it to introduce nonlinear transformations and thus enhance the model’s expressive power. While sigmoid and tanh can also introduce nonlinearity, their gradient saturation properties limit their effectiveness in deep networks. Finally, for deeper networks, the choice of ReLU activation function is particularly critical because it can better alleviate the vanishing gradient problem, thus supporting deeper neural network structures. In shallow networks, sigmoid and tanh may still be usable, but in deep networks, the advantages of ReLU are more pronounced.

Experimental preparation

This study uses mobile signal data from the popular tourist destination of Beijing as a sample for an empirical investigation of the spatiotemporal grid of tourism demand, which aims to validate the model’s efficacy. The spatiotemporal location data of all users, including locals and visitors as well as other groups, is included in the mobile signaling data that is provided by China Mobile (https://www.statista.com/statistics/272097/customer-base-of-china-mobile/, doi: 10.18653/v1/2022.seretod-1.7) and China Unicom (https://zenodo.org/records/11827345, doi: 10.5281/zenodo.11827345). Online data collection also includes geographic information maps of cities and the locations of key attractions. Furthermore, we used the K-fold cross validation method to keep each sub sample as the test set for model validation and the remaining nine sub samples as the training set for model training in order to fully validate the performance of various models. Each dataset was randomly divided into 10 equal sub samples. Table 1 displays comprehensive dataset details.

In the meanwhile, we collected statistics on passenger volume from six scenic Beijing grids, as shown in Fig. 5. The tourism demand time series for many picturesque locations show nonlinear and cyclical patterns.

The experimental hardware platform is Intel (R) Core (TM) i7-6700 [email protected] GHz (8-core CPU), 16 GB of memory, Intel (R) HD Graphics 530. The experimental software platform is Win10 operating system, and the development environment is Python 3.7 programming language. The Tensorflow library and Scikit learn library in Python are used to build suggested sentiment analysis methods and comparative experiments. According to the experiment, different hyperparameters and settings have been tested. By comparison, the following parameters were ultimately selected: the dimension of the word vector is 300, the learning rate is 0.00001, the dropout value is 0.75, the batch size is 30, the lstm_size is 32, and the iteration count is 1 K.

Performance evaluations

Mean absolute error (MAE), root mean square error (RMSE), mean absolute percentage error (MAPE), and coefficient of determination (R2) are the four performance measures that are used. Among these, runtime is the amount of time needed for the model to finish executing on the same server without requiring special computations. These are the remaining computations:

(19) MAE=1n∑i=1n|xi−x~i|

(20) RMSE=1n∑i=1n(xi−x~i)2

(21) MAPE=1n∑i=1n|xi−x~i|xiand

(22) R2=∑i=1n(x~i−x¯i)2∑i=1n(xi−x¯i)2where xi, x~i, and x¯i stand for the average of real tourist demand observations, anticipated tourism demand observations, and actual tourism demand observations, respectively. n indicates the number of predicted samples.

In addition, we use two widely used evaluation metrics in the recommendation field, namely Area Under Curve (AUC) and Normalized discounted Cumulative Gain (NDCG). The definition of AUC is as follows:

(23) AUC=∑i∈U+,j∈U−I(Pi,Pj)|U+|×|U−|where U+ and U− are sets of positive and negative examples in the system test set, respectively. Pi is the predicted probability of project i. I(Pi,Pj)={1,Pi>Pj0.5,Pi=Pj0,Pi<Pj. Generally speaking, ACU∈[0,1], and the closer its value is to 1, the higher the probability of positive examples ranking before negative examples, that is, the higher the accuracy of recommendations.

NDCG measures the ranking quality of recommendation lists in TOP-N tasks, defined as:

(24) NDCG@N=DCGiDCG,DCG@N=∑i=1k2r(l)−1log2(i+1).

NDCG has a log2(i+1) discount on the lower ranked items in the recommendation list, where i is the ranking of the items in the list, and iDCG is the ideal DCG value, where the high rated items are exactly at the forefront of the low rated items. The larger the NDCG value, the better the sorting effect of the list obtained by the algorithm in actual testing.

Experimental result

Firstly, we use the algorithm proposed in this article for training, and during the training process, the weight results are shown in Fig. 6.

Next, we tested the predictive performance and recommendation performance of the algorithm proposed in this article separately. The compared prediction algorithms include TVPM, ARDL, and VARM. The recommended algorithms to be compared are Probabilistic Matrix Factorization (PMF), Neural Collaborative Filtering (NCF), Social-based Rating Prediction (SBRP), and Attention-based Enhanced Adaptive Social Recommendation (AEASR). The experimental results are shown in Figs. 7 and 8, Tables 2 and 3.

Finally, we added different types of noise to the data to validate the robustness of the proposed algorithm, as shown in Tables 4 and 5.

By conducting stability analysis on different datasets and practical application scenarios, the performance and robustness of the algorithm are evaluated to ensure that it can maintain good performance and universality in various situations. This includes comparing the performance of models on different datasets, analyzing their stability in extreme situations, and evaluating their adaptability in different environments.

We conducted ablation experiments on the dataset to validate the effectiveness of our algorithm. This experiment modified the original model by removing only the self attention module, labeled as MSAMR-DSP1, and only the dynamic perception module, labeled as MSAMR-DSP2. The experimental results are shown in Table 6. Through ablation experiments, it was found that the method proposed in this article exhibited better performance.

Experimental discussion

From the above experimental results, we can draw the following conclusion.

From Fig. 6, it can be seen that the algorithm proposed in this article has adaptive weight parameters. During the training process, the parameters are constantly changing, which can help the algorithm converge better.

From Fig. 7, it can be seen that the algorithm proposed in this article has significant advantages in MAE, RMSE, MAPE, and R metrics. This indicates that the proposed algorithm can better predict the number of tourists to tourist attractions, as shown in Fig. 8. In Fig. 8, we used the proposed algorithm to predict the future tourism volume of different scenic spots, and the results showed that the predicted value can well predict the actual value.

Finally, from Tables 3 and 4, it can be seen that the algorithm proposed in this article has a high advantage in recommendation performance. The spatial optimization algorithm proposed in this article, which integrates multi-layer attention mechanisms, outperforms other baseline algorithms, especially traditional models such as PMF, SocialMF, and SBPR, in terms of various evaluation metrics on both datasets. This further proves that the fusion of attention mechanisms and recommendation systems has brought significant performance improvements.