mathematics

Article
Dynamics Behavior of a Predator-Prey Diffusion Model
Incorporating Hunting Cooperation and Predator-Taxis
Huisen Zhang

College of Mathematics and Statistics, Northwest Normal University, Lanzhou 730070, China;
[email protected]

Abstract: In this paper, we consider a predator-prey diffusion model incorporating hunting coopera-
tion and predator-taxis. Firstly, we establish the global existence of a classical solution for the model
in any spatial dimension. Secondly, we analyze the stability/instability caused by predator-taxis, and
we observe that predator-taxis play a key role in inducing stability changes. Specifically, if the positive
equilibrium is stable for the corresponding reaction-diffusion model, the attractive predator-taxis
can further stabilize the system, while the repulsive predator-taxis may lead to a change in spatial
stability, if the positive equilibrium is unstable for the corresponding reaction-diffusion model, the
attractive predator-taxis makes the model remain unstable, while the repulsive predator-taxis has a
stabilizing effect. Finally, numerical simulations are employed to validate the obtained results.

Keywords: predator-prey; hunting cooperation; predator-taxis; global solution; instability

MSC: 35A09; 35K59

1. Introduction
The predator-prey relationship is a common phenomenon in nature and has gar-
nered significant attention in the field of biomathematics research. Through conducting a
investigation into the predator-prey model, we can reveal the evolution law of populations.
Hunting cooperation is a prevalent form of species interaction, which refers to the co-
operative behavior exhibited by certain predators in the process of capturing prey. Hunting
Citation: Zhang, H. Dynamics
cooperation plays an important role in maintaining ecological diversity and regulating pop-
Behavior of a Predator-Prey Diffusion ulation density. Hunting cooperation can occur between individuals of the same species,
Model Incorporating Hunting such as lions, wild chimpanzees, fish, ants, birds, and so on [1–5]. Additionally, hunting
Cooperation and Predator-Taxis. cooperation behavior may also exist between different species, such as cheetahs and jackals
Mathematics 2024, 12, 1474. https:// preying on deer or badgers and coyotes targeting ground squirrels [6].
doi.org/10.3390/math12101474 To reveal the impact of hunting cooperation by predators on species extinction, Duarte
et al. [7] investigated a tri-trophic food chain model incorporative cooperative hunting,
Received: 12 April 2024
Revised: 7 May 2024
and they found that predators were more sensitive to extinction when hunting cooperation
Accepted: 7 May 2024
increased. Berec [8] considered hunting cooperation in the predator-prey model where the
Published: 9 May 2024 Holling II functional response is employed. Berec found that hunting cooperation exerts a
destabilizing influence on predator-prey dynamics. Alves and Hilker [9] proposed that the
attack rate of a predator species would increase with the increase in predator density due
to foraging promotion. Therefore, they established a series of predator-prey models with
Copyright: © 2024 by the authors. hunting cooperation and Holling type functional responses, and concluded that cooperative
Licensee MDPI, Basel, Switzerland. hunting can improve the persistence of predator populations. Fu and Zhang [10] studied a
This article is an open access article Holling II type predator-prey diffusion model incorporative hunting cooperation
distributed under the terms and
u (1 + αv)uv
 
conditions of the Creative Commons  ut = d1 ∆u + σu 1 − κ − 1 + h(1 + αv)u , x ∈ Ω, t > 0,


Attribution (CC BY) license (https:// (1)
(1 + αv)uv
 vt = d2 ∆v + x ∈ Ω, t > 0,

creativecommons.org/licenses/by/
 − v,
1 + h(1 + αv)u
4.0/).

Mathematics 2024, 12, 1474. https://doi.org/10.3390/math12101474 https://www.mdpi.com/journal/mathematics

Mathematics 2024, 12, 1474 2 of 12

where u, v denote prey and predator population densities, respectively. σ and κ are the per
capita birth rate and carrying capacity for prey species, h designates the average hunting
time of a predator for prey, α represents the rate of predator cooperation in hunting. Fu and
Zhang [10] investigated linear stability and Turing instability of positive constant steady
states, and analyzed the existence and stability for the Hopf bifurcated periodic solutions
of this model.
In predator-prey models, the phenomenon of predators moving towards areas with
higher prey density, known as prey-taxis, has been extensively studied. In 1987, Kareiva
and Odell [11] first proposed an ecological model with prey-taxis to elucidate the directional
movement of species. In the following decades, scholars have established numerous models
with prey-taxis to investigate the dynamic relationship between predators and prey [12–14].
Recently, a novel form of predator-taxis has been developed and distinguished from prey-
taxis, in which prey escapes from areas with a high predator density to avoid being captured.
This means that the prey is moving in the direction of the predator. Several researchers
have also explored and constructed several ecological models involving predator-taxis,
see [15–17]. These models assumed that prey would move toward a lower density of
predators to avoid dangerous situations.
Hunting cooperation and predator-taxis are prevalent in real ecosystems, so to better
describe biological phenomena, we discuss the following predator-prey diffusion model
that incorporates hunting cooperation and predator-taxis
  u (1 + αv)uv
u = d ∆u + ∇ · ( ∇ v ) + 1 − − , x ∈ Ω, t > 0,

 t 1 χu σu
1 + h(1 + αv)u



 κ

(1 + αv)uv

vt = d2 ∆v + x ∈ Ω, t > 0,

− v,

1 + h(1 + αv)u (2)

x ∈ ∂Ω, t > 0,




 ∂ν u = ∂ν v = 0,


x ∈ Ω,

u( x, 0) = u0 ( x ), v( x, 0) = v0 ( x ),


where Ω is a bounded domain in Rn (n ≥ 1), ν represents the outward unit normal vector
to ∂Ω, the homogeneous zero-flux condition ensures that (2) is self-contained for x ∈ ∂Ω.
Additionally, the initial data u0 ( x ), v0 ( x ) are smooth non-negative functions that are not
identically zero. The term −∇ · (χu∇v) denotes the directional movement of prey, and χ
represent the sensitivity coefficient of predator-taxis. In particular, when χ < 0, the prey
will move in the direction of the gradient of predators because of group defense when
sensing the risk of predation, and then the predator–taxi is attracted; when χ > 0, it means
that the prey exhibits an opposite movement direction for the gradient of predator, aiming
to evade predation, and the predator-taxi is called repulsive. The model (2) well describes
the effects of predator-taxis and hunting cooperation on a predator-prey diffusion model.
From the perspective of mathematical research, by studying the dynamic behavior of the
model (2), we can understand the relationship between species in the ecosystem more
clearly so as to provide some theoretical support for maintaining a balance of the ecosystem
and preventing species extinction.
The organization of this paper is as follows. Section 2 presents the global existence of
a classical solution to model (2). In Section 3, we analyze the instability caused by predator-
taxis for positive equilibrium in the model (2). Section 4 uses numerical simulations to
verify the theoretical results. Finally, concise conclusions are provided in Section 5.

2. Global Existence of Classical Solution

In this section, we prove the global existence of a solution for the model (2) in any
spatial dimensions. Moreover, throughout this section, we denote Ci (i = 1, 2, 3, ·R· · ) by
generic positive
R constants that may differ from line to line. For simplicity, write Ω wdx
instead of Ω w.
Mathematics 2024, 12, 1474 3 of 12

2.1. Local Existence and Preliminary

We firstly employ the abstract theorem of the quasilinear parabolic system [18,19] to
establish the local existence of a solution for model (2).

Lemma 1. Assume the initial data (u0 ( x ), v0 ( x )) ∈ [W 1,p (Ω)]2 with p > n and u0 ( x ), v0 ( x ) ≥
0(̸≡ 0). Then, we have the following:
(i) There exists a positive constant Tmax such that model (2) admits a unique local-in-time classical
solution (u( x, t), v( x, t)), which satisfies;
 2
(u( x, t), v( x, t)) ∈ C (Ω × [0, Tmax )) ∩ C2,1 (Ω × (0, Tmax )) .

(ii) There exists a constant M1 > 0 such that;

∥u(·, t)∥ L1 (Ω) + ∥v(·, t)∥ L1 (Ω) ≤ M1 f or all t ∈ (0, Tmax ). (3)

(iii) There exists a constant M2 > 0 such that;

0 < v( x, t) ≤ M2 , u( x, t) > 0 f or all ( x, t) ∈ Ω̄ × (0, Tmax ). (4)

(iv) If Tmax < ∞, we have.

 
lim sup ∥u(·, t)∥ L∞ (Ω) + ∥v(·, t)∥W 1,∞ (Ω) = ∞. (5)
t↗ Tmax

Proof. The local existence and uniqueness of the solution to (2) follows from Theorem 7.3
and Corollary 9.3 in [18] or Theorem 14.4 and 14.6 in [19], and the extensibility criterion (5)
can be obtained directly from Theorem 15.5 in [19]. By utilizing the strong maximum
principle on model (2), it can be find that u( x, t) > 0 and v( x, t) > 0 in Ω̄ × (0, Tmax ).
To prove (ii), we integrate the first two equations of (2) over Ω, which implies that
d u
Z Z Z Z  Z
 σ 
(u + v) =σ u 1− − v= ( σ + 1) u − u2 − ( u + v )
dt Ω Ω κ Ω Ω κ Ω
(6)
κ ( σ + 1)2 κ ( σ + 1)2 | Ω |
Z Z Z
≤ − (u + v) ≤ − ( u + v ).
Ω 4σ Ω 4σ Ω

By employing the Gronwall inequality, one obtains (3).

Next, we establish the upper bound of v( x, t), by the second equation in (2) and
u, v > 0, we have
(1 + αv)uv 1


 vt − d2 ∆v = − v < v, x ∈ Ω, t > 0,
1 + h(1 + αv)u h



∂ ν v = 0, x ∈ ∂Ω, t > 0, (7)




v( x, 0) = v0 ( x ), x ∈ Ω.


Hence, we easily obtain that there exists a constant M2 > 0 such that v( x, t) ≤ M2
from Theorem 3.1 in [20] and the comparison principle.

To establish the global-in-time of the solution, we next recall some preliminary esti-
mates. For p ∈ (1, ∞), we represent the sectorial operator by A and define it as
 
∂z
Az := −∆z for z ∈ D ( A) := z ∈ W 2,p (Ω) : = 0, z ∈ ∂Ω , (8)
∂ν

similarly, we denote Ad1 u := −d1 ∆u, Ad2 v := −d2 ∆v, where Ad1 and Ad2 possess the same
properties as A with a scaling. Here, we will give some properties for A, but applying the
properties of Ad1 and Ad2 in the subsequent discussion.

Lemma 2 ([21]). Let k ∈ {0, 1}, p ∈ [1, ∞], and q ∈ (1, ∞). Then, for any z ∈ D (( A + 1)ϑ ),
there exists C1 > 0 such that
Mathematics 2024, 12, 1474 4 of 12

∥z∥W k,p (Ω) ≤ C1 ∥( A + 1)ϑ z∥ Lq (Ω) , (9)

where ϑ ∈ (0, 1) and satisfies

n n
k− < 2ϑ − .
p q
If, in addition q ≥ p, then for any u ∈ L p (Ω), there exist C2 > 0 and β > 0 such that

−ϑ− n2 ( 1p − 1q ) − βt
∥( A + 1)ϑ e−t( A+1) z∥ Lq (Ω) ≤ C2 t e ∥ z ∥ L p (Ω) , (10)

with the associated heat semigroup {e−t( A+1) }t≥0 is known to map L p (Ω) into D (( A + 1)ϑ ).
Additionally, for all p ∈ (1, ∞) and η > 0, there exist C3 > 0 and ζ > 0 such that
1
∥( A + 1)ϑ e−tA ∇ · z∥ L p (Ω) ≤ C3 t−ϑ− 2 −η e−ζt ∥z∥ L p (Ω) (11)

is valid for any Rn –valued z ∈ L p (Ω).

Lemma 3 (Gagliardo-Nirenberg interpolation inequality [22]). Suppose p, q ∈ [1, ∞], and

qn
ς ∈ (0, p), where p < ∞ for q = n, and p ≤ n−q for q < n. Then, for ϑ ∈ (0, 1] given by:
− np = (1 − nq )ϑ − nς (1 − ϑ) and some C4 > 0, we have

∥z∥ L p (Ω) ≤ C4 (∥∇z∥ϑLq (Ω) ∥z∥1L−

ς (Ω) + ∥ z ∥ Lς (Ω) )
ϑ
(12)

for any z ∈ W 1,q (Ω) ∩ Lς (Ω).

2.2. Global Existence of Solution

In this section, we study the solution to model (2) and its global existence. To begin
with, we provide a bound for ∥v(·, t)∥W 1,∞ (Ω) .

Lemma 4. Assume (u( x, t), v( x, t)) to be a classical solution of (2). Then there exists a constant
M3 > 0 such that
∥v(·, t)∥W 1,∞ (Ω) ≤ M3 f or all t ∈ (0, Tmax ).

Proof. Let τ ∈ (0, Tmax ) be chosen, satisfying τ < 1, fix q > n and ϑ ∈ ( 12 (1 + nq ), 1).
Rewrite the equation of v in (2) as follows
∂v (1 + αv)uv
− d2 ∆v + v = , (13)
∂t 1 + h(1 + αv)u

in view of the variation-of-constants formula, one yields

Z t
(1 + αv)uv
v(·, t) = e−t( Ad2 +1) v0 + e−(t−s)( Ad2 +1) ds.
0 1 + h(1 + αv)u
From (9) and (10), we find that for any t ∈ (τ, Tmax ),

∥v(·, t)∥W 1,∞ (Ω)

≤ C1 ∥( Ad2 + 1)ϑ v∥ Lq (Ω)

Z t
(1 + αv)uv
≤ C1 C2 t−ϑ e− βt ∥v0 ∥ Lq (Ω) + C1 C2 (t − s )−ϑ e− β(t−s) ds
0 1 + h(1 + αv)u Lq (Ω)
Z t
1
≤ C1 C2 t−ϑ e− βt ∥v0 ∥ Lq (Ω) + C C (t − s)−ϑ e− β(t−s) ∥v∥ Lq (Ω) ds
h 1 2 0
Z t
≤ C3 t−ϑ + C3 (t − s)−ϑ e− β(t−s) ds
0
Z ∞
≤ C3 t−ϑ + C3 ζ −ϑ e− βζ dζ
0
≤ C3 τ −ϑ + C3 Γ(1 − ϑ ),
Mathematics 2024, 12, 1474 5 of 12

where Γ(·) is a Gamma function, then Γ(1 − ϑ) > 0 due to 0 < ϑ < 1. Therefore, we can obtain

∥v(·, t)∥W 1,∞ (Ω) ≤ M3 .

Next, we state u( x, t) is uniformly bounded in L p (Ω) for any p ≥ 2

Lemma 5. Assume (u( x, t), v( x, t)) to be a classical solution of (2). Then for any p ∈ [2, ∞), there
exists a constant M4 > 0 such that

∥u(·, t)∥ L p (Ω) ≤ M4 f or all t ∈ (0, Tmax ).

Proof. For p ≥ 2, the equation of u in (2) is multiplied by u p−1 and then integrated over Ω,
one has
1 d
Z
up
p dt Ω
 
u (1 + αv)uv
Z 
= u p−1 d1 ∆u + ∇ · (χu∇v) + σu 1 − −
Ω κ 1 + h(1 + αv)u
Z Z Z
≤ d1 u p−1 ∆u + u p−1 ∇ · (χu∇v) + σ up
Ω Ω Ω
4d1 ( p − 1) p
Z Z Z
≤ − |∇u 2 |2 + M3 |χ|( p − 1) u p −1 ∇ u + σ up
p2 Ω Ω Ω
4d ( p − 1) 2M3 |χ|( p − 1)
Z p
Z p p
Z
≤ − 1 2 |∇u 2 |2 + u 2 |∇u 2 | + σ up
p Ω p Ω Ω
4d1 ( p − 1)
Z p
Z
≤ − 2
|∇u | + σ
2 up
p2 Ω Ω
M |χ|( p − 1) M |χ| p
 
2d1
Z p
Z
+ 3 |∇u 2 |2 + 3 up
p M3 |χ| p Ω 2d1 Ω
!Z
2 2
M3 χ ( p − 1)
2d ( p − 1)
Z p
= − 1 2 |∇u 2 |2 + +σ up,
p Ω 2d1 Ω

this implies
!Z
d 2d ( p − 1) M32 χ2 p( p − 1)
Z Z Z p
p
u + u ≤− 1 p
|∇u | +
2 2
+ σp + 1 up. (14)
dt Ω Ω p Ω 2d1 Ω

From (12), we have

Z
p 2
up = u2 L2 ( Ω )
Ω
 2
p ϑ p 1− ϑ p
≤ C1 ∇u 2
L2 ( Ω )
u 2 2 + u 2 2
L p (Ω) L p (Ω)
p p 2(1− ϑ ) p
 
2ϑ 2
≤ C2 ∇u 2 L2 ( Ω )
u2 2 + u2 2
Lp (Ω) Lp (Ω) (15)
 
p 2 ϑ p 2 p 2
≤ C2 ϵϑ ∇u + ϵ (1 − ϑ ) u
2 2
L2 ( Ω )
+ u ϑ −1 2 2 2
L p (Ω) L p (Ω)
p 2
p p
 ϑ

= C2 ϵϑ ∇u 2 L2 (Ω) + ϵ ϑ−1 (1 − ϑ) + 1 u 2 L1 (Ω)
p 2
 
≤ C3 ∇u 2 L2 (Ω) + 1 ,
Mathematics 2024, 12, 1474 6 of 12

np−n
where 0 < ϵ < 1 and ϑ = 2+np−n ∈ (0, 1). Fix C3 with
!
M32 χ2 p( p − 1) 2d ( p − 1)
+ σp + 1 C3 = 1 . (16)
2d1 p

Combining (14)–(16), we find that

d
Z Z
up + u p ≤ C4 .
dt Ω Ω

Therefore, we have Z  Z 
p
u p ≤ max C4 , u0 .
Ω Ω

Now we prove that ∥u(·, t)∥ L∞ (Ω) is uniformly bounded for any t ∈ (0, Tmax ).

Lemma 6. Assume (u( x, t), v( x, t)) to be a classical solution for (2). Then there exists a constant
M5 > 0 such that
∥u(·, t)∥ L∞ (Ω) ≤ M5 f or all t ∈ (0, Tmax ).

Proof. The equation of u in (2) is written as

 u (1 + αv)uv
ut − d1 ∆u + u = χ∇ · (u∇v) + u + σu 1 − − . (17)
κ 1 + h(1 + αv)u

The use of the variation-of-constants formula in (17) to obtain

Z t
u(·, t) = e − t ( A d1 +1) u 0 + χ e−(t−s)( Ad1 +1) ∇ · (u∇v)ds
0
Z t  
−(t−s)( Ad1 +1)
 u (1 + αv)uv (18)
+ e u + σu 1 − − ds
0 κ 1 + h(1 + αv)u
:= w1 (·, t) + w2 (·, t) + w3 (·, t).

Then, we discuss the L∞ (Ω)-bounded of w1 , w2 and w3 , respectively.

From the parabolic maximum principle, we have

∥w1 (·, t)∥ L∞ (Ω) = ∥ e − t ( A d1 +1) u 0 ∥ L ∞ ( Ω )

≤ C1 ∥( Ad1 + 1)ϑ e−t( Ad1 +1) u0 ∥ Lq (Ω) (19)

≤ C2 t−ϑ e− βt ∥u0 ∥ Lq (Ω) ≤ C3 ∥u0 ∥ L∞ (Ω) ,

n
where τ ∈ (0, 1), ϑ ∈ ( 2q , 1), q > n, β > 0.
In Lemma 2, letting k = 0, q := n + 2 and p = ∞, taking ϑ ∈ ( 2q
n 1
, 2 ), we can establish
that η ∈ (0, 12 − ϑ ). Consequently, there exists C2 > 0 and ζ > 0 such that
Z t
∥w2 (·, t)∥ L∞ (Ω) =χ e−(t−s)( Ad1 +1) ∇ · (u∇v)ds
0 L∞ (Ω)
Z t
≤ C1 χ ( Ad1 + 1)ϑ e−(t−s)( Ad1 +1) ∇ · (u∇v)ds
0 Lq (Ω)
Z t (20)
ϑ −(t−s)( Ad1 +1)
≤ C1 χ ( A d1 + 1 ) e ∇ · (u∇v)ds
0 Lq (Ω)
Z t
−ϑ− 21 −η
≤ C2 χ (t − s) e−(ζ +1)(t−s) ∥u∇v∥ Lq (Ω) ds,
0
Mathematics 2024, 12, 1474 7 of 12

from Lemmas 4 and 5, we can obtain that

∥u∇v∥ Lq (Ω) ≤ C3 .

Then Z t
1
∥w2 (·, t)∥ L∞ (Ω) ≤ C4 (t − s)−ϑ− 2 −η e−(ζ +1)(t−s) ds
0
Z ∞
≤ C4
1
δ−ϑ− 2 −η e−(ζ +1)δ dδ (21)
0
≤ C4 Γ( 21 − ϑ − η ),
where C4 > 0 and 12 − ϑ − ε > 0, then we find that Γ( 12 − ϑ − η ) is positive.
Similarly, for w3 (·, t), one has

∥w3 (·, t)∥ L∞ (Ω)

Z t  
 u (1 + αv)uv
= e−(t−s)( Ad1 +1) u + σu 1 − − ds
0 κ 1 + h(1 + αv)u ∞
Z t   L (Ω)
σ 1
≤ C1 ( Ad1 + 1)ϑ e−(t−s)( Ad1 +1) − u2 + (1 + σ )u − v ds
0 κ h Lq (Ω)
Z t  
σ 1
≤ C1 ( Ad1 + 1)ϑ e−(t−s)( Ad1 +1) − u2 + (1 + σ)u − v ds
0 κ h Lq (Ω)
Z t
σ 1 (22)
≤ C2 (t − s)−ϑ e−(t−s) β − u2 + (1 + σ )u − v ds
0 κ h Lq (Ω)
Z t  
κσ 1
≤ C2 (t − s)−ϑ e−(t−s) β ∥u∥ Lq (Ω) + + ∥ v ∥ L∞ (Ω)
0 4 h
Z t
≤ C3 (t − s)−ϑ e−(t−s) β ds
0
Z ∞
≤ C3 ρ−ϑ e−ρβ dρ
0
≤ C4 Γ(1 − ϑ),

where Γ(1 − ϑ ) > 0 holds from 0 < ϑ < 1. Hence, by (19), (21) and (22), we get that for any
t ∈ (0, Tmax ), u(·, t) is bounded on the L∞ (Ω).

Assume that 0 < Tmax < ∞, where Tmax is the maximum existence time. From
Lemmas 4 and 6 with some p > 2, we have

∥u(·, t)∥ L∞ (Ω) + ∥v(·, t)∥W 1,∞ (Ω) < ∞ f or all t ∈ (0, Tmax ),

this is contrary to Lemma 1 (iv), it can be inferred that Tmax = ∞ and, consequently, the
solution (u( x, t), v( x, t)) remains bounded for all ( x, t) ∈ Ω × (0, ∞). On the basis of
Lemmas 1, 4 and 6, we have the following conclusion.

Theorem 7. Let the initial data (u0 ( x ), v0 ( x )) ∈ [W 1,p (Ω)]2 with p > n and u0 ( x ), v0 ( x ) ≥
0(̸≡ 0). Then, model (2) admits a unique global-in-time classical solution (u( x, t), v( x, t)) ∈

2
C (Ω × [0, ∞)) ∩ C2,1 (Ω × (0, ∞)) satisfying

∥u(·, t)∥ L∞ (Ω) + ∥v(·, t)∥W 1,∞ (Ω) ≤ M f or all t ∈ [0, ∞),

where M is a positive constant that depends on the initial value, but not on t.
Mathematics 2024, 12, 1474 8 of 12

3. Stability/Instability Caused by Predator-Taxis

In this section, the instability of the positive constant steady state for model (2) caused
by predator-taxis behaviour is investigated. For convenience, we consider the case of (2) in
one-dimensional space and assume Ω = (0, lπ ), then model (2) can be rewritten as

u (1 + αv)uv
ut = d1 u xx + χ(uv x ) x + σu(1 − ) − , x ∈ (0, lπ ), t > 0,


1 + h(1 + αv)u



 κ

(1 + αv)uv


vt = d2 v xx + − v, x ∈ (0, lπ ), t > 0,

1 + h(1 + αv)u (23)





 u x ( x ) = v x ( x ) = 0, x = 0, lπ, t > 0,



u( x, 0) = u0 ( x ), v( x, 0) = u0 ( x ), x ∈ (0, lπ ),


We denote the positive equilibrium of (23) by E∗ = (u∗ , v∗ ), and it can be easily to show
that u∗ = (1−h)(11+αv∗ ) , while v∗ corresponds the positive root of the following equation

f (v) := ρ3 v3 + ρ2 v2 + ρ1 v + ρ0 = 0,

where ρ3 = (1 − h)2 α2 κ, ρ2 = 2(1 − h)2 ακ, ρ1 = (1 − h)(1 − h − σα)κ, ρ0 = σ(1 − (1 − h)κ).

First, let us assume
1
( H0 ) h < 1, κ > .
1−h
Observing the facts ρ3 > 0 and ρ2 > 0, if ( H0 ) is true, then ρ0 < 0 can be obtained,
where the sign of the coefficients of f (v) is + + + − or + + − −. In both cases, the
symbol changed only once. According to Descartes’ rule of signs [23], f (v) = 0 has a
unique positive root, which means that model (23) exhibits a unique positive equilibrium
when condition ( H0 ) holds true. To simplify matters, we will assume condition ( H0 ) to
hold throughout the remainder of the discussion.
Model (23) linearization at E∗ has the form
     
ut ∆u u
  = D  + J(u∗ ,v∗ )  , (24)
vt ∆v v

where
χu∗
   
d1 a11 a12
D= , J(u∗ ,v∗ ) =  ,
0 d2 a21 a22
and
σ 1 + (2 − h)αv∗
a11 = − + h(1 − h)(1 + αv∗ )v∗ , a12 = − < 0,
κ (1 − h)(1 + αv∗ ) 1 + αv∗
(1 − h)αv∗
a21 = (1 − h)2 (1 + αv∗ )v∗ > 0, a22 = > 0.
1 + αv∗
By simple calculation, we know that a11 a22 − a12 a21 > 0. The characteristic equation
of (23) is
µ2 − Tk µ + Dk (χ) = 0, (25)
where

Tk = −(d1 + d2 )( kl )2 + a11 + a22 ,

Dk (χ) = d1 d2 ( kl )4 − ( a11 d2 + a22 d1 − χu∗ a21 )( kl )2 + a11 a22 − a12 a21 ,

Mathematics 2024, 12, 1474 9 of 12

and ( kl )2 , k ∈ N = {0, 1, 2, · · · } be the eigenvalues of −∆ on Ω subject to the zero-flow

boundary condition.
According to the standard linear stability theory, for any non-negative integer k, the
positive equilibrium E∗ is locally asymptotically stable if and only if all the roots of the
characteristic equation have negative real parts, E∗ is unstable if the characteristic equation
has at least one root with a positive real part for some non-negative integer k.
Since we are interested in the predator-taxis-driven instability, we take the assumption

( H1 ) a11 + a22 < 0,

It can be easily shown that the condition ( H1 ) guarantees E∗ is locally asymptotically

stable in the corresponding ordinary differential equation model (d1 = d2 = χ = 0).
Let

Dk ( 0 ) d1 d2 ( kl )4 − ( a11 d2 + a22 d1 )( kl )2 + a11 a22 − a12 a21

χ(k) := − =− , k ∈ N.
u∗ a21 ( kl )2 u∗ a21 ( kl )2

Then, we can deduce the following theorem.

Theorem 8. Suppose ( H0 ) and ( H1 ) hold. For model (23), then we have

(i) If Dk (0) > 0, then for all χ > 0, E∗ is always locally asymptotically stable;
(ii) If Dk (0) > 0 and χ < 0, then E∗ is locally asymptotically stable when χ > maxk∈N χ(k)
and it is unstable when χ < maxk∈N χ(k );
(iii) If Dk (0) < 0, then E∗ is locally asymptotically stable when χ > maxk∈N χ(k ) > 0 and it is
unstable when χ < 0 or 0 < χ < maxk∈N χ(k).

Proof. From the assumption ( H1 ), we know that Tk < 0 for all k ∈ N, thus, we only discuss
the sign of Dk (χ).
(i) If Dk (0) > 0 and χ > 0, it is evident that Dk (χ) > 0 for any k ∈ N, then E∗ exhibits
locally asymptotically stability;
(ii) If Dk (0) > 0, together with a21 > 0, one has χ(k ) < 0 for any k ∈ N. When
maxk∈N χ(k) < χ < 0, then Dk (χ) > 0 for any k ∈ N, thus E∗ is locally asymptotically
stability. When χ < maxk∈N χ(k ) < 0, we have Dk (χ) < 0 for some k ∈ N, which
implies E∗ is unstable;
(iii) If Dk (0) < 0, then E∗ is unstable for the corresponding reaction–diffusion model.
From a21 > 0, it follows that χ(k) > 0 for any k ∈ N. When χ > maxk∈N χ(k), we
observe that Dk (χ) > 0 for any k ∈ N, and then E∗ is locally asymptotically stability.
When χ < 0 and 0 < χ < maxk∈N χ(k ), we have Dk (χ) < 0 for some k ∈ N, then E∗
is unstable.
Next, we show that χ(k ) can reach its maximum on k ∈ N. Set

d1 d2 m2 − ( a11 d2 + a22 d1 )m + a11 a22 − a12 a21

Υ(m) = − , m > 0.
u∗ a21 m

A simple calculation gives that Υ′ (m) > 0 for m < m0 and Υ′ (m) < 0 for m > m0 with
r
a11 a22 − a12 a21
m0 = .
d1 d2
√ √
Then, there exists k0 ∈ N such that when k0 = max{[ m0 l ], [ m0 l ] + 1}, we have
maxk∈N χ(k) = χ(k0 ).

Remark 9. Theorems 8 (i) and (ii) show that when the positive equilibrium is stable with respect to
the reaction-diffusion model, the attractive predator-taxis can further stabilize the system, while the
repulsive predator-taxis may lead to a change in spatial stability, which means the formation of the
Mathematics 2024, 12, 1474 10 of 12

pattern caused by the predator-taxis. Theorem 8 (iii) shows that when the positive equilibrium is
unstable with respect to the reaction-diffusion model, the attractive predator-taxis make the model
remain unstable, while the repulsive predator-taxis have a stabilizing effect.

Remark 10. From an ecological point of view, a stable equilibrium means that over time, predator
and prey species will eventually converge at this point, and their numbers will remain constant, thus
achieving permanent coexistence. The conclusion of Theorem 8 shows that attractive predator-taxis
are beneficial to species coexistence.

4. Numerical Simulations
In this part, we give some numerical simulations in order to verify the above theoretical
analysis by utilizing the MATLAB software. We fixed Ω = (0, 20), dx = 0.25, dt = 0.01,
and set parameter

σ = 1, κ = 3, α = 0.6, h = 0.5, d1 = 1. (26)

It is evident that model (23) possesses a trivial equilibrium E0 = (0, 0), a boundary
equilibrium E1 = (3, 0), and a unique positive equilibrium E∗ = (1.382, 0.745). Note that
under the given parameters (26), both conditions ( H0 ) and ( H1 ) are satisfied, rendering
E∗ stable with respect to the corresponding ODE model (i.e., d1 = d2 = χ = 0). When
we choose d2 = 0.01, the equilibrium E∗ is unstable with respect to the corresponding
reaction-diffusion model (see Figure 1), by performing calculations χ(k) = 0.13, taking
χ = 0.2 > 0.13, E∗ is unstable with respect to model (23), which shows that positive
predator-taxis have stabilizing effects (see Figure 2). When we choose d2 = 0.1, E∗ is
locally asymptotically stable with respect to the corresponding reaction-diffusion model
(see Figure 3), the calculation gives that χ(k ) = −0.537, taking χ = −0.59 < −0.537, E∗
is stable with respect to the model (23), indicating that the negative predator-taxis have a
destabilizing effect (see Figure 4).

Figure 1. When d2 = 0.01, χ = 0, E∗ is unstable for model (23) with (26).

Figure 2. When d2 = 0.01, χ = 0.2 > 0.13, E∗ is local asymptotically stable, which is caused by the
predator-taxis for model (23) with (26).
Mathematics 2024, 12, 1474 11 of 12

Figure 3. When d2 = 0.1, χ = 0, E∗ is local asymptotically stable for model (23) with (26).

Figure 4. When d2 = 0.1, χ = −0.59 < −0.537, E∗ is unstable caused by the predator-taxis for
model (23) with (26).

5. Concluding Remarks
This paper mainly discusses the dynamics of the predator-prey diffusion model with
hunting cooperation and predator-taxis, where hunting cooperation refers to the coop-
erative behavior of some predator populations in the process of capturing prey. The
predator-taxis effect refers to the response of prey to different predator distributions, i.e.,
the prey may move in the direction of a low predator gradient to avoid being captured
by predators, or it may move in the direction of a high density of prey to defend against
predator attacks. Hunting cooperation and predator-taxis have far-reaching implications
for protecting endangered species and maintaining sustainable ecosystems.
Firstly, the global existence and uniform boundedness of a classical solution of the
model are proven strictly. Secondly, the local stability of a positive equilibrium is discussed
through linearization analysis. Finally, we use numerical simulations to visualize the
results. In addition, it can be seen from [10] that the stability of the reaction-diffusion model
depends on the ratio d1 /d2 of diffusion coefficients. Combined with the discussion results
of this paper, we found that in the same region, if this ratio is less than a critical value, it
means that the predator occupies an advantageous position, which indicates that due to
strong predation pressure, the prey should stay away from the predator’s habitat to avoid
being predation. On the contrary, if this ratio is greater than a critical value, the situation is
more favorable for the prey, which means that in the face of an attack by predators, the prey
must collectively gather together and resist to obtain more survival opportunities, which is
more conducive to the long-term development of the population.

Funding: This work is supported by the National Natural Science Foundation of China (No.
12161080).
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Mathematics 2024, 12, 1474 12 of 12

Data Availability Statement: Data are contained within the article.

Acknowledgments: The author would like to thank the referees for providing very helpful comments
and suggestions.
Conflicts of Interest: The author declares no conflicts of interest.

References
1. Scheel, D.; Packer, C. Group hunting behaviour of lions: A search for cooperation. Anim. Behav. 1991, 41, 697–709. [CrossRef]
2. Boesch, C.; Boesch, H.; Vigilant, L. Cooperation in Primates and Humans: Mechanisms of Evolution; Kappeler, P.M., van Schaik, C.P.,
Eds.; Springer: New York, NY, USA, 2006.
3. Cook, W.L.; Streams, F.A. Fish predation on Notonecta (Hemiptera): Relationship between prey risk and habitat utilization.
Oecologia 1984, 64, 177–183. [CrossRef] [PubMed]
4. Hector, D.P. Cooperative hunting and its relationship to foraging success and prey size in an avian predator. Ethology 1986, 73,
247–257. [CrossRef]
5. Moffett, M.W. Foraging dynamics in the group-hunting myrmicine ant, pheidologeton diversus. J. Insect. Behav. 1988, 1, 309–331.
[CrossRef]
6. Goodale, E.; Beauchamp, G.; Ruxton, G.D. Mixed-Species Groups of Animals: Behavior, Community Structure, and Conservation;
Academic Press: Cambridge, MA, USA, 2017.
7. Duarte, J.; Januario, C.; Martins N.; Sardanyes, J. Chaos and crises in a model for cooperative hunting: A symbolic dynamics
approach. Chaos 2009, 19, 043102. [CrossRef]
8. Berec, L. Impacts of foraging facilitation among predators on predator-prey dynamics. Bull. Math. Biol. 2010, 72, 94–121.
[CrossRef] [PubMed]
9. Alves, M.T.; Hilker, F.M. Hunting cooperation and Allee effects in predators. J. Theo. Biol. 2017, 419, 13–22. [CrossRef] [PubMed]
10. Fu, S.M.; Zhang, H.S. Effect of hunting cooperation on the dynamic behavior for a diffusive Holling type II predator-prey model.
Commun. Nonlinear Sci. Numer. Simulat. 2021, 99, 105807. [CrossRef]
11. Kareiva, P.; Odell, G. Swarms of predators exhibit “preytaxis” if individual predators use area-restricted search. Am. Nat. 1987,
130, 233–270. [CrossRef]
12. Jin, H.Y.; Wang, Z.A. Global stability of prey-taxis systems. J. Differ. Equ. 2017, 262, 1257–1290. [CrossRef]
13. Choi, W.; Ahn, I. Predator invasion in predator-prey model with prey-taxis in spatially heterogeneous environment. Nonlinear
Anal. Real World Appl. 2022, 65, 103495. [CrossRef]
14. Zuo, W.J.; Song, Y.L. Stability and double-Hopf bifurcations of a Gause-Kolmogorov-type predator-prey system with indirect
prey-taxis. J. Dyn. Diff. Equat. 2021, 33, 1917–1957. [CrossRef]
15. Banda, H.; Chapwanya, M.; Dummani, P. Pattern formation in the Holling-Tanner predator-prey model with predator-taxis. A
nonstandard finite difference approach. Math. Compu. Simul. 2022, 196, 336–353. [CrossRef]
16. Chen, M.X.; Zheng, Q.Q. Predator-taxis creates spatial pattern of a predator-prey model. Chaos Solitons Fractals 2022, 161, 112332.
[CrossRef]
17. Gao, X.Y. Global solution and spatial patterns for a ratio-dependent predator-prey model with predator-taxis. Results Math. 2022,
77, 66. [CrossRef]
18. Amann, H. Dynamic theory of quasilinear parabolic equations. II. Reaction-diffusion systems. Differ. Integral Equ. 1990, 3, 13–75.
[CrossRef]
19. Amann, H. Nonhomogeneous linear and quasilinear elliptic and parabolic boundary value problems. In Function Spaces,
Differential Operators and Nonlinear Analysis; Teubner–Taxte Math: Stuttgart, Germany, 1993; Volume 133, pp. 9–126.
20. Alikakos, N.D. L p bounds of solutions of reaction-diffusion equations. Commun. Part. Diff. Equ. 1979, 4, 827–868. [CrossRef]
21. Horstmann, D.; Winkler, M. Boundedness vs. blow-up in a chemotaxis system. J. Differ. Equ. 2005, 215, 52–107. [CrossRef]
22. Friedman, A. Patial Differential Equations; Holt Rinehart Winston: New York, NY, USA, 1969.
23. Burnside, W.S.; Panton, A.W. The Theory of Equations: With an Introduction to the Theory of Binary Algebraic Forms, 2nd ed.; Dublin
University Press Series: Dublin, Ireland, 1886.

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual
author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to
people or property resulting from any ideas, methods, instructions or products referred to in the content.