Lecture 7: Multivariate Linear Regression

A. The Basic Principle

We consider the multivariate extension of multiple
linear regression – modeling the relationship
between m responses Y1,…,Ym and a single set of r
predictor variables z1,…,zr. Each of the m responses
is assumed to follow its own regression model, i.e.,
Y1 = B01 + B11z1 + B21z2 +  + Br1zr
Y2 = B02 + B12z1 + B22z2 +  + Br2zr
  
Y1 = B01 + B11z1 + B21z2 +  + Br1zr
 ε1  
 
ε
where E ε = E   2   = 0, Var ε = Σ
   
 
 εm  
 
Conceptually, we can let

[zj0, zj1, …, zjr]

denote the values of the predictor variables for the jth

trial and
 Yj1  εj1 
Y  ε 
Yj =  j2 
,ε =  j2 

      
   
 Yjm  εjm 
be the responses and errors for the jth trial. Thus we
have an n x (r + 1) design matrix
z10 z11  z1r 
z z21  z2r 
Z = 
20

     
 
zn0 zn1  znr 
 If we now set

 Y11 Y12  Y1m 

Y Y22  Y2m 
Y =  =  Y1 | Y2 |  | Y m  
21

       
 
 Yn1 Yn2  Ynm 

β01 β02  β0m 

β β12  β1m 
β =  = β1 | β2 |  | β m  
11

       
  ε'1 
βr1 βr2  βrm 
 
  
ε11 ε12  ε1m  ε' 
ε ε22  ε2m   2 
ε =   = ε |  | ε m   =   
21
| ε2
       1   
   
εn1 εn2  εnm 
  
 ' 
ε m  
the multivariate linear regression model is

Y = Zβ + ε
  
with
E εi  = 0
  
and
Cov εi,εk  = σikI, i, k = 1,  , m
 
Note also that the m observed responses on the jth
trial have covariance matrix
σ11 σ12  σ1m 
σ σ  σ 
Σ = 
21 22 2m 

      
 
σ m1 σ m2  σ mm 
 ^
The ordinary least squares estimates b are found in a
~
manner analogous to the univariate case – we begin
by taking

 
-1
ˆ
β '
Z'Yi
i  = ZZ
 
collecting the univariate least squares estimates yields

βˆ =    
-1 -1
βˆ | βˆ |  | βˆ  = ZZ '
Z' 
Y | Y |  | Y  = '
ZZ Z'
Y
 
1  2  m      1  2  m  
 
Now for any choice of parameters
B =  b1 | b2 |  | b m  
   
the resulting matrix of errors is
Y - Zβ

 The resulting Error Sums of Squares and Crossproducts
is

Y- Zb  Y  Y  Y  
' '

 1 1 1 - Zb1  1 - Zb1 m  - Zb m  

 
Y - ZB   
'
Y - ZB =     
 

Y- Zb  Y  Y  Y  
' '
 m m 1 - Zb1  m  - Zb m  m  - Zb m  
^
We can show that the selection b~(i) = b~(i) minimizes the
ith diagonal sum of squares

Y - Zb  Y 
'

i  i  i  - Zbi
generalized
i.e., variance

tr   
Y - ZB Y - ZB  and  Y - ZB  Y - ZB 
' '

 
 
are both minimized.
so we have matrices of predicted values

Yˆ - Zβˆ = ZZ Z Y
-1
' '

 
and we have a resulting matrices of residuals

Y - Yˆ  
= I - Z ZZ Z' 
 -1
ˆ
ε =
 
'

Y
 
Note that the orthogonality conditions among
residuals, predicted values, and columns of the
design matrix which hold in the univariate case are
also true in the multivariate case because

 
Z I - Z ZZ Z'  = Z' - Z' = 0
 -1
' '

  
 … which means the residuals are perpendicular to the
columns of the design matrix

 
ˆ = Z I - Z ZZ Z'  = Z' - Z' = 0
 -1
' ' '
Zε

  

and to the predicted values

Y ε = β Z I - Z ZZ Z'  Y
ˆ ˆ' ' 
-1
'
ˆ '
= 0

   

Furthermore, because
Y = Yˆ + ˆε
we have  

Y'Y = Yˆ'Yˆ + ˆˆ
'
εε

total sums of predicted sums

residual (error)
squares and of squares and sums of squares
crossproducts crossproducts and crossproducts
Example – suppose we had the following six sample
observations on two independent variables (palatability
and texture) and two dependent variables (purchase
intent and overall quality):
Overall Purchase
Palatability Texture
Quality Intent
65 71 63 67
72 77 70 70
77 73 72 70
68 78 75 72
81 76 89 88
73 87 76 77

Use these data to estimate the multivariate linear

regression model for which palatability and texture are
independent variables while purchase intent and
overall quality are the dependent variables
We wish to estimate
Y1 = B01 + B11z1 + B21z2

and

Y2 = B02 + B12z1 + B22z2

jointly.

The design matrix is

1 65 71
1 72 77

1 77 73
Z =  
 1 68 78
1 81 76
 
1 73 87
 so
1 65 71
1 72 77
 1 1 1 1 1 1   6 436 462
'   1 77 73  
ZZ = 65 72 77 68 81 73   =  436 31852 33591 
 1 68 78
71 77 73 78 76 87 462 33591 35728
1 81 76
 
1 73 87
and
-1
 6 436 462
 
ZZ
-1
'
= 436 31852 33591
462 33591 35728
62.560597030 -0.378268027 -0.453330568
 
= -0.378268027 0.005988412 -0.000738830
-0.453330568 -0.000738830 0.006584661
 and 63
70
 1 1 1 1 1 1    445
  72  
Z'y1 = 65 72 77 68 81 73   =  32536 
 75
71 77 73 78 76 87 34345
89
 
so 76

 
-1
ˆ
β1 = ZZ Z'y 1
'

 
62.560597030 -0.378268027 -0.453330568  445
  
= -0.378268027 0.005988412 -0.000738830 32536
-0.453330568 -0.000738830 0.006584661 34345
-37.501205460
 
=  1.134583728
 0.379499410
 and 67
70
 1 1 1 1 1 1    444
  70  
Z'y2 = 65 72 77 68 81 73   =  32430 
 72
71 77 73 78 76 87 34260
88
 
so 77

 
-1
ˆ
β2 = ZZ Z'y 2
'

 
62.560597030 -0.378268027 -0.453330568  444
  
= -0.378268027 0.005988412 -0.000738830 32430
-0.453330568 -0.000738830 0.006584661 34260
-21.432293350
 
=  0.940880634
 0.351449792
 so
-37.501205460 -21.432293350
ˆ
β = β
ˆ |β ˆ  =  1.134583728 0.940880634

 1 2   
  0.379499410 0.351449792

This gives us estimated values matrix

1 65 71 63.19119 64.67788

1 72 77 73.41028 73.37275
 -37.501205460 -21.432293350 
1 77 73   77.56520 76.67135
Yˆ = Zβ =    1.134583728 0.940880634 =  
 
 1 68 78
 0.379499410 0.351449792  69.25144 69.96067
1 81 76 83.24203 81.48922
   
1 73 87 78.33986 77.82812
 and residuals matrix

63 67 63.19119 64.67788  0.191194960 -2.322116943

70 70 73.41028 73.37275  3.410277515 3.372746244

72 70 77.56520 76.67135  5.565198512 6.671350244
ˆε = Y - Yˆ =  -  =  
  75 72 69.25144 69.96067 -5.748557985 -2.039326498
89 88 83.24203 81.48922 -5.757968347 -6.510777845
     
76 77 78.33986 77.82812  2.339855345 0.828124797

Note that each

column sums to zero!
B. Inference in Multivariate Regression
The least squares estimators

b = ~[b(1) | ~b(2) ||b~(m)]

~
of the multivariate regression model have the
following properties

- E β
ˆ  = β i.e., E βˆ = βˆ
 i 

i

 ˆ
- Cov β , β ˆ
i k  = σik

'
ZZ
-1


 1,  , m
, i, k =

- E ˆ  1 ' 
ε = 0 and E  ˆˆ
εε = Σ-1
 n - r - 1  
if the model is of full rank, i.e., rank(Z)= r + 1 < n.
~
Note that e and b are also uncorrelated.
~ ~
 This means that, for any observation z0
~
ˆ = z'0 βˆ | βˆ |  | βˆ  = z'0βˆ | z'0βˆ |  | z'0βˆ 
z'0β
  
1  2 m    1 2  m  

is an unbiased estimator, i.e.,

E z'0βˆ = z'0β
  

We can also determine from these properties that the
estimation errors
ˆ
z'0βi - z'0βi 
 
have covariances
'
   
'
E z0 βi - β ˆ β - ˆ
β z
i  i  i  0
   
' 
  
' 
 
 
-1
ˆ ˆ ' '
= z0 E  βi - βi βi - βi   z0 = σikz0 ZZ z0
  
 
 Furthermore, we can easily ascertain that

ˆ = Yˆ0
z'0β


^
i.e., the forecasted vector Y0 associated with the values
~
of the predictor variables z~0 is an unbiased estimator
of Y~ 0.

The forecast errors have covariance


E  Y0i - z'0βˆ
  i  '
Y0k - z β
0 
k   ik 0  
ˆ  = σ 1 + z' ZZ
'
-1
z0 
Thus, for the multivariate regression model with full
rank (Z) = r + 1, n  r + 1 + m, and normally
~
distributed errors ~e,

βˆ =  
-1
'
ZZ Z'Y
  
is the maximum likelihood estimator of b and
~
βˆ ~

N  
βˆ, Σ

where the elements of S are

~

Cov β
ˆ ,β
 

i
ˆ
k   = σ ZZ
ik
'
-1
, i, k = 1,  , m
 ^
Also, the maximum likelihood estimator of b is
~
independent of the maximum likelihood estimator of
the positive definite matrix S
~
given by

  
'
ˆΣ = 1 εε
'
=
1
Y - Zβˆ Y - Zβˆ
 n  n  
and
ˆ ~ Wp,n-r-1  Σ
nΣ

all of which provide additional support for using the
least squares estimate – when the errors are normally
distributed
ˆ and n-1εε
β '

 
are the maximum likelihood estimators of
β and Σ

 These results can be used to develop likelihood ratio
tests for the multivariate regression parameters.

The hypothesis that the responses do not depend on

predictor variables zq+1, zq+2,…, zr is
β1 
  (q + 1) x m
= 
Big Beta (2)
H0 : β2 = 0 where β
  β  (r - q) x m
 2 

If we partition Z
~
in a similar manner

Z = Z1 | Z2 

m x (q + 1) m x (r - q)
 we can write the general model as

β1 
 
E Y = Zβ = Z1 | Z2     = Z1β1 + Z2β2
 
 β 
 2 
 ^
The extra sum of squares associated with b(2) are
~

      = n ˆΣ - ˆΣ
' '
ˆ Y - Z1β
Y - Z1β ˆ - ˆ Y - Zβ
Y - Zβ ˆ
1 1 1
 

where
ˆ  
-1
β 1 = ZZ '
1 1 Z'1Y
and   

  
'
ˆ
Σ1 = n-1 ˆ
Y - Z1β ˆ
Y - Z1β
1 1

The likelihood ratio for the test of the hypothesis

H0:b(2) = 0
~ ~

is given by the ratio of generalized variances

max L β  =   =  ˆΣ 
1,Σ

Lˆ , ˆΣ1
β1
n2

 
β ,Σ

   
1
Λ =
 
 

max L β, Σ ˆΣ ˆ, ˆΣ
L β 1
β,Σ 
    
which is often converted to Wilks’ Lambda statistic

ˆ
Σ
Λ2 n = 
ˆ
Σ1

Finally, for the multivariate regression model with
full rank (Z)~ = r + 1, n  r + 1 + m, normally
distributed errors ~e, and the null hypothesis is true
^ ^
(so n(S~ 1 – S)
~
~ Wq,r-q(S))
~

 ˆ
Σ 
 1 
- n - r - 1 -  m - r + q + 1 ln    ~ χ2m r-q
 
 2   ˆ
Σ1 
  
when n – r and n – m are both large.
If we again refer to the Error Sum of Squares and
Crossproducts as
^
E
~
= nS
~

and the Hypothesis Sum of Squares and Crossproducts

as
H = n(S1 - S)
~ ~ ~

then we can define Wilks’ lambda as

ˆ
Σ E s
1
Λ 2n
=  =
ˆ
Σ1 E+H
 = 
i =1 1 + ηi
 
where h1  h2    hs are the ordered eigienvalues of
HE
~~
-1
where s = min(p, r - q).
There are other similar tests (as we have seen in our
discussion of MANOVA):
s
ηi
 = tr H H + E  
 -1
Pillai’s Trace   
i=1 1 + ηi
s
Hotelling-Lawley Trace 
i=1
ηi = tr 
  
HE-1

Roy’s Greatest Root

η1
1 + η1

Each of these statistics is an alternative to Wilks’

lambda and perform in a very similar manner
(particularly for large sample sizes).
Example – For our previous data (the following six
sample observations on two independent variables -
palatability and texture - and two dependent variables -
purchase intent and overall quality
Overall Purchase
Palatability Texture
Quality Intent
65 71 63 67
72 77 70 70
77 73 72 70
68 78 75 72
81 76 89 88
73 87 76 77

to test the hypotheses that i) palatability has no joint

relationship with purchase intent and overall quality
and ii) texture has no joint relationship with purchase
intent and overall quality.
We first test the hypothesis that palatability has no
joint relationship with purchase intent and overall
quality, i.e.,
H0:b(1) = 0
~
The likelihood ratio for the test of this hypothesis is
given by the ratio of generalized variances

max L β  =   =  ˆΣ 
 2,Σ

L ˆ , ˆΣ2
β2
n2

 
β ,Σ

   
2
Λ =
 
 

max L β, Σ ˆ, ˆΣ
ˆ
L βΣ  2
β,Σ 
    
For ease of computation, we’ll use the Wilks’ lambda
statistic
ˆ
Σ E
Λ2 n =  = 
ˆ
Σ2 E+H

The error sum of squares and crossproducts matrix is

114.31302415 99.335143683
E =  
 99.335143683 108.5094298 

and the hypothesis sum of squares and crossproducts

matrix for this null hypothesis is

214.96186763 178.26225891
H =  
  178.26225891 147.82823253 
 so the calculated value of the Wilks’ lambda statistic is

2n
E
Λ = 
E+H

114.31302415 99.335143683
99.335143683 108.5094298 
 
=
114.31302415 99.335143683 214.96186763 178.26225891
99.335143683 108.5094298  + 178.26225891 147.82823253
   
2536.570299
= = 0.34533534
7345.238098
 The transformation to a Chi-square distributed
statistic (which is actually valid only when n – r and n
– m are both large) is

 ˆ
Σ 
 1 
- n - r - 1 -  m - r + q + 1 ln   
 2  ˆΣ1 
  
 1 
= - 6 - 2 - 1 - 2 - 2 + 1 + 1 ln 0.34533534
 2 
= 0.92351795

at a = 0.01 and m(r - q) = 1 degrees of freedom, the

critical value is 9.210351 - we have a strong non-
rejection. Also, the approximate p-value of this chi-
square test is 0.630174 – note that this is an extremely
gross approximation (since n – r = 4 and n – m = 4).
We next test the hypothesis that texture has no joint
relationship with purchase intent and overall quality,
i.e.,
H0:b(2) = 0
~
The likelihood ratio for the test of this hypothesis is
given by the ratio of generalized variances

max L β  =   =  ˆΣ 
1,Σ

L ˆ , ˆΣ1
β1
n2

 
β ,Σ

   
1
Λ =
 
 

max L β, Σ ˆ, ˆΣ
ˆ
L βΣ  1
β,Σ 
    
For ease of computation, we’ll use the Wilks’ lambda
statistic
ˆ
Σ E
Λ2 n =  = 
ˆ
Σ1 E+H

The error sum of squares and crossproducts matrix is

114.31302415 99.335143683
E =  
 99.335143683 108.5094298 

and the hypothesis sum of squares and crossproducts

matrix for this null hypothesis is

21.872015222 20.255407498
H =  
  20.255407498 18.758286731
 so the calculated value of the Wilks’ lambda statistic is

2n
E
Λ = 
E+H

114.31302415 99.335143683
99.335143683 108.5094298 
 
=
114.31302415 99.335143683 21.872015222 20.255407498
99.335143683 108.5094298  + 20.255407498 18.758286731
   
2536.570299
= = 0.837135598
3030.059055
 The transformation to a Chi-square distributed
statistic (which is actually valid only when n – r and n
– m are both large) is

 ˆ
Σ 
 1 
- n - r - 1 -  m - r + q + 1 ln   
 2  ˆΣ1 
  
 1 
= - 6 - 2 - 1 - 2 - 2 + 1 + 1 ln 0.837135598
 2 
= 0.15440838

at a = 0.01 and m(r - q) = 1 degrees of freedom, the

critical value is 9.210351 - we have a strong non-
rejection. Also, the approximate p-value of this chi-
square test is 0.925701 - note that this is an extremely
gross approximation (since n – r = 4 and n – m = 4).
SAS code for a Multivariate Linear Regression Analysis:
DATA;
INPUT z1 z2 y1 y2;
LABEL z1='Palatability Rating'
z2='Texture Rating'
y1='Overall Quality Rating'
y2='Purchase Intent';
CARDS;
65 71 63 67
72 77 70 70
77 73 72 70
68 78 75 72
81 76 89 88
73 87 76 77
;
PROC GLM;
MODEL y1 y2 = z1 z2/;
MANOVA H=z1 z2/PRINTE PRINTH;
TITLE4 'Using PROC GLM for Multivariate Linear Regression';
RUN;
 SAS output for a Multivariate Linear Regression Analysis:
Dependent Variable: y1 Overall Quality Rating

Sum of
Source DF Squares Mean Square F Value Pr > F
Model 2 256.5203092 128.2601546 3.37 0.1711
Error 3 114.3130241 38.1043414
Corrected Total 5 370.8333333

R-Square Coeff Var Root MSE y1 Mean

0.691740 8.322973 6.172871 74.16667

Source DF Type I SS Mean Square F Value Pr > F

z1 1 234.6482940 234.6482940 6.16 0.0891
z2 1 21.8720152 21.8720152 0.57 0.5037

Source DF Type III SS Mean Square F Value Pr > F

z1 1 214.9618676 214.9618676 5.64 0.0980
z2 1 21.8720152 21.8720152 0.57 0.5037

Dependent Variable: y1 Overall Quality Rating

Standard
Parameter Estimate Error t Value Pr > |t|
Intercept -37.50120546 48.82448511 -0.77 0.4984
z1 1.13458373 0.47768661 2.38 0.0980
z2 0.37949941 0.50090335 0.76 0.5037
 SAS output for a Multivariate Linear Regression Analysis:
Dependent Variable: y2 Purchase Intent

Sum of
Source DF Squares Mean Square F Value Pr > F
Model 2 181.4905702 90.7452851 2.51 0.2289
Error 3 108.5094298 36.1698099
Corrected Total 5 290.0000000

R-Square Coeff Var Root MSE y2 Mean

0.625830 8.127208 6.014134 74.00000

Source DF Type I SS Mean Square F Value Pr > F

z1 1 162.7322835 162.7322835 4.50 0.1241
z2 1 18.7582867 18.7582867 0.52 0.5235

Source DF Type III SS Mean Square F Value Pr > F

z1 1 147.8282325 147.8282325 4.09 0.1364
z2 1 18.7582867 18.7582867 0.52 0.5235

Dependent Variable: y2 Purchase Intent

Standard
Parameter Estimate Error t Value Pr > |t|
Intercept -21.43229335 47.56894895 -0.45 0.6829
z1 0.94088063 0.46540276 2.02 0.1364
z2 0.35144979 0.48802247 0.72 0.5235
 SAS output for a Multivariate Linear Regression Analysis:
The GLM Procedure
Multivariate Analysis of Variance

E = Error SSCP Matrix

y1 y2
y1 114.31302415 99.335143683
y2 99.335143683 108.5094298

Partial Correlation Coefficients from the Error SSCP Matrix / Prob > |r|

DF = 3 y1 y2
y1 1.000000 0.891911
0.1081
y2 0.891911 1.000000
0.1081
 SAS output for a Multivariate Linear Regression Analysis:
The GLM Procedure
Multivariate Analysis of Variance

H = Type III SSCP Matrix for z1

y1 y2
y1 214.96186763 178.26225891
y2 178.26225891 147.82823253

Characteristic Roots and Vectors of: E Inverse * H, where

H = Type III SSCP Matrix for z1
E = Error SSCP Matrix
Characteristic Characteristic Vector V'EV=1
Root Percent y1 y2
1.89573606 100.00 0.10970859 -0.01905206
0.00000000 0.00 -0.17533407 0.21143084

MANOVA Test Criteria and Exact F Statistics

for the Hypothesis of No Overall z1 Effect
H = Type III SSCP Matrix for z1
E = Error SSCP Matrix

S=1 M=0 N=0

Statistic Value F Value Num DF Den DF Pr > F
Wilks' Lambda 0.34533534 1.90 2 2 0.3453
Pillai's Trace 0.65466466 1.90 2 2 0.3453
Hotelling-Lawley Trace 1.89573606 1.90 2 2 0.3453
Roy's Greatest Root 1.89573606 1.90 2 2 0.3453
 SAS output for a Multivariate Linear Regression Analysis:
The GLM Procedure
Multivariate Analysis of Variance

H = Type III SSCP Matrix for z2

y1 y2
y1 21.872015222 20.255407498
y2 20.255407498 18.758286731

Characteristic Roots and Vectors of: E Inverse * H, where

H = Type III SSCP Matrix for z2
E = Error SSCP Matrix
Characteristic Characteristic Vector V'EV=1
Root Percent y1 y2
0.19454961 100.00 0.06903935 0.02729059
0.00000000 0.00 -0.19496558 0.21052601

MANOVA Test Criteria and Exact F Statistics

for the Hypothesis of No Overall z2 Effect
H = Type III SSCP Matrix for z2
E = Error SSCP Matrix

S=1 M=0 N=0

Statistic Value F Value Num DF Den DF Pr > F
Wilks' Lambda 0.83713560 0.19 2 2 0.8371
Pillai's Trace 0.16286440 0.19 2 2 0.8371
Hotelling-Lawley Trace 0.19454961 0.19 2 2 0.8371
Roy's Greatest Root 0.19454961 0.19 2 2 0.8371
We can also build confidence intervals for the
predicted mean value of Y0 associated with z~ 0 - if the
~
model
Y = Zβˆ+ε

has normal errors, then

 



β z0 ~ N m β z0, z0 ZZ z0Σ
ˆ' ' ' '
-1

and independent

ˆ ~ Wn-r-1 Σ
nΣ

so '
 ˆ' 'z   ˆ' 'z 
 β z0 - β 0   n
-1
  β z0 - β 0 
2
T =   ˆ
Σ 
  
   n - r - 1  
 
-1 -1
 z0 ZZ z0
' '
  z0 ZZ z0
' '

    
 Thus the 100(1 – a)% confidence interval for the
predicted mean value of Y0 associated with z~ 0 (b’z0) is
~ ~~
given by

     z ZZ  m  n - r - 1 
' -1
 
βˆ'z0 - β'z0 n
βˆ'z0 - β'z0 Fm,n-r- m α  
-1
 ˆ
Σ 
'
0
'
z0 
 n - r - 1   n-r-m
 

and the 100(1 – a)% simultaneous confidence intervals

~ with z0 (z’0 b(i) ) are
for the mean value of Yi associated ~ ~
~

m  n - r - 1
Fm,n-r- m α  z ZZ   n
-1
'
zβˆ  ' '
z0 σˆii
0 i  0

 n-r-m n-r-1

i = 1,…,m
Finally, we can build prediction intervals for the
predicted value of Y~ 0 associated with ~z0 – here the
prediction error
Y = Zβˆ+ε

has normal errors, then

 



β z0 ~ N m β z0, z0 ZZ z0Σ
ˆ' ' ' '
-1

and independent

ˆ ~ Wn-r-1 Σ
nΣ

so '
 ˆ' 'z   ˆ' 'z 
 β z0 - β 0   n
-1
  β z0 - β 0 
2
T =   ˆ
Σ 
  
   n - r - 1  
 
-1 -1
 z0 ZZ z0
' '
  z0 ZZ z0
' '

    
 the prediction intervals the 100(1 – a)% prediction
interval associated with ~z0 is given by

     1 + z ZZ   m  n - r - 1 
' -1
ˆ'z0  n  ˆ'z0 -1
Y0 - β  ˆ
Σ  Y0 - β '
0
'
z0  Fm,n-r- m α  
 n - r - 1   n-r-m
 

and the 100(1 – a)% simultaneous prediction intervals

with ~z0 are

m  n - r - 1 n
 
-1
zβˆ 
'
0i  Fm,n-r- m α  1 + z0 ZZ z0
' '
σˆii

 n-r-m n-r-1

i = 1,…,m