Coding With Mata in Stata
Coding With Mata in Stata
31 31
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
5 Controlling the ow 5.1 Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Conditional statements . . . . . . . . . . . . . . . . . . . 6 Coding Mata functions 6.1 Dening functions . . 6.2 Declarations . . . . . 6.3 Passing arguments . 6.4 Returning values . . 6.5 .mo and .mlib les .
Version: 31-10-2008, 18:48
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
Introduction
3
3.1
Mata is a matrix algebra language which is available in Stata since version 9. Stata and mata commands are set in Courier. Options in [brackets] are optional.
Mata is directly accessed from the Stata command window. Type the stata command
mata
The two Stata manuals Mata Matrix Programming provide systematic information about MATA commands. It also discusses the implemented numerical methods. The online help in Stata describes the use of all Mata commands with its options. However, it does not explain the numerical properties as in the Reference manual. We can start the online query in the Command window by
help mata command
to return to the normal Stata command prompt. Note: All matrix and function denitions are kept in the Mata workspace when you end mata and accessible when you re-enter mata. 3.2 Use in .do les
Mata commands can also be used in do les. A sequence of Mata commands is preceded by the Stata command mata and terminated by end. There can be several such sequences which access the same Mata workspace. 3.3 Use in .ado les
If you dont know the exact expression for the command, you can search for Mata commands in the Stata documentation by issuing the command
search mata word
Search commands are answered in the result window. Alternatively, you can display the result in the Viewer window by issuing the command
view search mata word
You can also use the Stata online help in the menu bar: Help/Search....
4
4.1
4.1.2
You can build matrices out of several submatrices. Suppose you have submatrices A to D.
A=(1, B=(5, C=(3, D=(1, 2 \ 3, 6, 7 \ 4 \ 5, 2, 3 \ 4) 8, 9, 10) 6) 4, 5, 6)
4.1.1
In order to build a matrix you have two options. You can declare the whole matrix separating columns by a comma and rows by a backslash. For example a 2x2 matrix
A=(1, 2 \ 3, 4)
A B is built using the column join operation , C D and the row-join operator \: The matrix E =
E = (A, B \ C, D)
Or you can rst declare an empty matrix with e.g. 2 rows and 3 columns
B = J(2, 3, .)
or
E = A, B \ C, D
which results in
1 2 3 4 5 +--------------------------+ | 1 2 5 6 7 | | 3 4 8 9 10 | | 3 4 1 2 3 | | 5 6 4 5 6 | +--------------------------+
You can give a matrix whatever name you want except for reserved words (see help m2 reswords). Names consist of any combination of letters, numbers and underscores but cannot start with a number. Mata is sensitive with respect to the upper and lower case. 4.1.3 If a variable has previously been assigned a value, the new value overrides value and dimensions of the predecessor. You can display the value of variable B by simply typing its name
B 1 2 3 +----------------+ 1 | 5 6 7 | 2 | 8 9 10 | +----------------+
1 2 3 4
Mata can also deal with string matrices which are matrices with string elements. To create a string matrix, we enclose its elements in quotation marks.
S=("Thijs", "Kurt" \ "Ghazala", "Gabrielle")
which results in
4.1.5
Creating a scalar
4.1.4
Creating a vector
Note: standard Stata matrix commands distinguish between scalars and 1 1 matrices. 4.2 Accessing data from Stata
Consider the example dataset auto.dta which you can download from the Stata (not Mata) command prompt.
webuse auto.dta
4.2.1
You can declare a column vector with a series of number, e.g. from 3 to 5 as
g = (3::5) g 1 +-----+ 1 | 3 | 2 | 4 | 3 | 5 | +-----+
st data(colvector, rowvector ) copies observations of numeric variables from the Stata dataset into a Mata matrix. For example,
X = st_data(.,("mpg", "rep78", "weight"))
creates a new matrix X with all observations in rows and the variables mpg, rep78, and weight in columns. A subsample of observations, e.g. observations 1 to 5 and 7 to 9, is created by
X = st_data((1::5\7::9),("mpg", "rep78", "weight")) X 1 2 3 +----------------------+ 1 | 22 3 2930 | 2 | 17 3 3350 | 3 | 22 . 2640 | 4 | 20 3 3250 | 5 | 15 4 4080 | 6 | 26 . 2230 | 7 | 20 3 3280 | 8 | 16 3 3880 |
10
creates Y as rows 1 - 5 and 7 - 9 of columns 3,4, and 7 of matrix X. st subview(Y, X, matrix, matrix ) allows to choose sequences of observations and variables by providing a r x 2 matrix as 3rd argument and a 2 x c matrix as 4th argument. For example
st_subview(Y, X, (1, 5\7, 9), (3,7\4,7))
st data(matrix, rowvector ) allows to choose sequences of observations by providing a r x 2 matrix as rst argument. For example
X = st_data((1,5\7,9),("mpg", "weight", "rep78"))
picks observations 1 to 5 and 7 to 9. Rows with missing values are omitted if a third argument, set to zero, is provided
X = st_data((1::5\7::9),("mpg", "weight", "rep78"), 0)
creates Y as rows 1 - 5 and 7 - 9 of columns 3 - 4, and 7 - 7 of matrix V. See help m5 st_subview() for the various rules to pick the submatrices. A memory ecient way to access data for use in e.g. linear regressions is
M = X = y = . st_view(M, ., ("price", "mpg", "weight", "rep78"), 0) st_subview(y, M, ., 1) st_subview(X, M, ., (2\.))
Observations can also be selected based on a Stata dummy variable, e.g. touse
X = st_data(.,("mpg", "weight", "rep78"), "touse")
4.2.3
4.2.2
Both st_data and st_view ll in matrix X with the same data. st_data() is generally faster than st_view but uses less memory. st_view also allows (or risks) changing the value of the underlying data. For example,
X[2,1] = 123
st view(X, colvector, rowvector ) creates a matrix X that is a view onto the current Stata dataset. For example
X = . st_view(X, ., ("mpg", "weight", "rep78"))
creates a new matrix X which is a view on all observations of the variables mpg, weight, and rep78. Again the 3rd argument indicates whether missing values or particular observations are to be used:
st_view(X, ., ("mpg", "weight", "rep78"), 0) st_view(X, ., ("mpg", "weight", "rep78"), "touse")
after st_data(), changes the value in the matrix X, but the Stata dataset remains unchanged; after st_view(), it would cause the value of mpg in the second observation to change to 123. Note: There are two functions st_sdata() and st_sview() for use with string variables. 4.3 Managing the Mata workspace
st subview(Y, X, colvector, rowvector ) creates a new view matrix Y from an existing view matrix X. For example
st_view(X, ., .) st_subview(Y, X, (1::5\7::9), (3,4,7))
To see all existing variables in the Mata workspace, issue the command
mata describe
11
12
The command
mata clear
E 1 2 3 4 5 +--------------------------+ | 1 2 5 6 7 | | 3 4 8 9 10 | | 3 4 1 2 3 | | 5 6 4 5 6 | +--------------------------+
deletes all variables from the Mata workspace without disturbing Stata. Specic matrices can be deleted by mata drop namelist For example, matrix A and vector f are deleted with
mata drop A f
1 2 3 4
Scalars from individual elements are extracted by using subscripts in square brackets e.g.
E[2, 3] 8
4.4 4.4.1
1 2 3 4
A contiguous submatrix e.g. consisting of rows 2 to 4 and columns 2 to 5 is extracted by range subscripts which mark the top-left and the bottom-right element of the submatrix
E[|2, 2\ 3, 4|] 1 2 3 +-------------+ 1 | 4 8 9 | 2 | 4 1 2 | +-------------+
4.4.2
One of the most basic operation is to extract partitions of a matrix. Consider matrix E
13
14
4.4.3
The Mata function vec() transforms a matrix into a column vector with one column stacked onto the next
B 1 2 3 +----------------+ 1 | 5 6 7 | 2 | 8 9 10 | +----------------+ b = vec(B) b 1 +------+ 1 | 5 | 2 | 8 | 3 | 6 | 4 | 9 | 5 | 7 | 6 | 10 | +------+
Note: the rst alternative is performed faster than the latter. A submatrix consisting of individual rows and columns e.g. rows 1 and 4 and columns 3, 5 and 2 can be formed as
E[(1\ 4), (3, 5, 2)] 1 2 3 +-------------+ 1 | 5 7 2 | 2 | 4 6 6 | +-------------+
More generally, the two arguments can be predened vectors. For example,
r = (1\ 4) c = (3, 5, 2) E[r,c] 1 2 3 +-------------+ 1 | 5 7 2 | 2 | 4 6 6 | +-------------+
Note: Stata considers it good style to specify the rst subscript vector as column vector and the second subscript vector as row vector. Though this is not necessary. More partitioning rules can be found under help m2 subscripts.
15
16
4.4.4
4.4.6
Sorting a matrix
diag(b), creates a square matrix with the elements of vector b on its diagonal.
diag(b) [symmetric] 1 2 3 +-------------+ 1 | 1 | 2 | 0 5 | 3 | 0 0 9 | +-------------+
sort(X, idx) returns X with rows in ascending or descending order of the columns specied by idx. For instance, sort(X, 1) sorts X on its rst column:
X = (2, 3, 1\ 2, 2, 2\ 1, 1, 3) X 1 2 3 +-------------+ 1 | 2 3 1 | 2 | 2 2 2 | 3 | 1 1 3 | +-------------+ sort(X,1) 1 2 3 +-------------+ 1 | 1 1 3 | 2 | 2 3 1 | 3 | 2 2 2 | +-------------+
4.4.5
lowertriangle(A) returns the lower triangle of A. uppertriangleA) returns the upper triangle of A:
sort(X, (1,2)) sorts X on its rst and second columns (meaning rows with equal values in their rst column are ordered on their second col-
17
18
umn):
sort(X,(1,2)) 1 2 3 +-------------+ 1 | 1 1 3 | 2 | 2 2 2 | 3 | 2 3 1 | +-------------+
1 2 3 4 5
See help mata sort() for functions to randomly permutate matrices. 4.5 Basic matrix operators
If the matrices involved in the operation are of incompatible sizes, Mata will return an error message:
F*d *: <istmt>: r(3200); 3200 conformability error function returned error
Mata provides the following basic operators matrix operations: + * / ^ Addition (of matrix or scalar) Subtraction (of matrix or scalar) Multiplication (with matrix or scalar) Divison (by scalar) Power (of scalar)
4.6
Matrix multiplication
is an alternative way to calculate X Z. cross() has the following advantages over the standard matrix-notation approach: cross() omits the rows in X and Z that contain missing values, which amounts to dropping observations with missing values. cross() uses less memory, especially when used with views. cross() makes calculations in special cases more eciently. For instance, if you code cross(X, X), calculation is made for a symmetric matrix result. In the four-argument version, cross(X, xc, Z, zc), X is augmented on the right with a column of ones if xc is dierent from 0 and Z is similarly augmented if zc is dierent from 0. For example,
19
20
cross(X, 1, Z, 0)
adds a constant vector to X and nothing to Z. cross(X, w, Z) returns X diag(w)Z. cross(X, xc, w, Z, zc) is augmented as in the four-argument case. cross(X, 0, 1, Z, 0) is equivalent to cross(X,Z). 4.6.1 Element-by-element operators
scalar. == returns false if the variables have dierent dimensions. For example,
G = (1, 2 \ 3, 4) H = (1, 5 \ 6, 4) G == T 0 G[1,2] == 2 1 G == 1 0 G != 1 1
To indicate an element-by-element operation, precede a standard operator with a colon :. Mata colon operators are: :* Multiplication (with matrix of same dimensions) :/ Division (by matrix of same dimensions) :^ Power (of matrix of same dimensions) For example,
x = (1, 2, 3) y = (4, 5, 6) x:*y 1 2 3 +----------------+ 1 | 4 10 18 | +----------------+
Mata relational operators <, > Less, greater than <=, >= Less, greater than or equal to can only be used with variables of identical dimensions and returns always a scalar. It evaluates the relation for all elements and returns true if it holds for all elements. For example
G < H 0 G <= H 1
Mata colon relational operators :== :!= :<, :> :<=, :>= Equal to Not equal to Less, greater than Less, greater than or equal to
Note: Element-by-element addition and subtraction is equal to the matrix operation. 4.6.2 Relational and logical operators
can be used with scalars, vectors and matrices and returns the dimension of the highest dimension variable. For example,
can be used with scalars, vectors and matrices and returns always a
21
22
G :== H [symmetric] 1 2 +---------+ 1 | 1 | 2 | 0 1 | +---------+ G :<= 2 1 2 +---------+ 1 | 1 1 | 2 | 0 0 | +---------+ G :== (1\4) [symmetric] 1 2 +---------+ 1 | 1 | 2 | 0 1 | +---------+
4.7 4.7.1
4.7.2
23
24
4.7.3
4.8
The function e(i, n) returns a vector with all n elements equal to zero except for the ith, which is set to one.
e(2, 5) 1 2 3 4 5 +---------------------+ 1 | 0 1 0 0 0 | +---------------------+
Mata oers several functions to calculate the inverse of a matrix: luinv(A) cholinv(A) invsym(A) inverse of full rank, square matrix A inverse of positive denite, symmetric matrix A generalized inverse of positive-denite, symmetric matrix A
Mata oers several functions to solve the linear equation system AX = B: lusolve(A,B) A is full rank, square matrix cholinv(A) A is positive denite, symmetric matrix Note: it is always numerically more accurate to solve AX = B directly than to calculate X = A1 B. Type help m4 solvers for more functions. An example: estimate a linear regression using the dataset auto.dta:
webuse auto.dta mata y = st_data(.,"price") X = st_data(.,("mpg", "weight")) X = X, J(rows(X),1,1) b = invsym(X*X)*X*y
4.7.4
The uniform() function generates a matrix of random numbers whose elements are uniformly distributed in the interval (0,1). For example,
uniform(2, 3) 1 2 3 +-------------------------------------------+ 1 | .1369840784 .643220668 .5578016951 | 2 | .6047949435 .684175977 .1086679425 | +-------------------------------------------+
uniformseed(newseed) sets the seed: a string previously obtained from uniformseed() may be specied for the argument, or an integer number may be specied. uniformseed() has the same eect as Statas set seed command. Mata has no built-in function to draw from the normal distribution. Independent draws from standard normal can be generated by applying the inverse cumulative normal distribution to uniform draws. For example,
invnormal(uniform(2,3)) 1 2 3 +----------------------------------------------+ 1 | .3014337824 -1.545904789 .1389086436 | 2 | 1.133267712 -.6583710099 -1.700496348 | +----------------------------------------------+
There is a better version that saves memory, addresses missing values, elegantly adds a constant and eciently solves for the parameter vector
M = X = y = . st_view(M, ., ("price", "mpg", "weight"), 0) st_subview(y, M, ., 1) st_subview(X, M, ., (2\.)) XX = cross(X,1,X,1) Xy = cross(X,1,y,0) b = cholsolve(XX,Xy)
25
26
5
5.1
Controlling the ow
Loops
The do-loop
do { stmt } while (exp)
The while-loop
while (expr) { stmt }
executes stmt one or more times, until expr is zero. For example
n = 5 i = 1 do { printf("i=%g\n", i) i++ } while (i<=n) printf("done\n")
executes stmt zero or more times as long as expr is not equal to zero. For example
n = 5 i = 1 while (i<=n) { printf("i=%g\n", i) i++ } printf("done\n")
break exits the innermost for, while, or do loop. continue restarts the innermost for, while, or do loop. 5.2 Conditional statements
is equivalent to
expr1 while (expr2) { stmt expr3 }
evaluates expr, and if it is true (evaluates to a nonzero number), stmt is executed. Several conditions can be nested by else if ; else introduces a statement that is executed when all conditions are false:
if (expr1) { stmt1 } else if (expr2) { stmt2 } else { stmt3 }
For example,
n = 5 for (i=1; i<=n; i++) { printf("i=%g\n", i) } printf("done\n")
For example,
27
28
6.2
Declarations
Variables (scalar, vectors and matrices) need not be declared in Mata functions. However, careful programmers use declarations in Mata. See help m2 declarations for a discussion. In the above example we would add declarations at three places: in front of function denitions, inside the parentheses dening the functions arguments, and at the top of the body of the function, dening private variables the function will use:
real colvector zeros(real scalar c) { real colvector a a = J(c,1,0) return(a) }
6
6.1
In this case we tell Mata what input to expect (a scalar) and, if someone attempts to use our function incorrectly, Mata will stop execution. Declarations consist of an element type (transmorphic, numeric, real, complex, string, pointer) and an organizational type (matrix, vector, rowvector, colvector, scalar). By default, declarations are optional in Mata. However, you can ask in the Stata command prompt (not Mata) to be strict on declarations set matastrict on
returns a cx1 column vector of zeros. You can call the function as any Mata function:
b = zeros(3) b 1 +-----+ 1 | 0 | 2 | 0 | 3 | 0 | +-----+
6.3
Passing arguments
Functions may require several arguments separated by a ,. Suppose we extend the above function zeros() to create matrices of zeros:
real matrix zeros(real scalar c, real scalar r) { real matrix A
29
30
A = J(c, r, 0) return(A) }
Functions may be coded to allow receiving a variable number of arguments. This is done by placing a vertical or bar (|) in front of the rst argument that is optional. For instance, we could allow the function zeros() to return a matrix if both arguments are specied and a column vector if only the rst one is specied:
real matrix zeros(real scalar c,| real scalar r) { real matrix A if (args()==1) r = 1 A = J(c, r, 0) return(A) }
The function args() is used to determine the number of arguments. Important to note: Mata passes arguments to functions by address not by value. When you code
f(n)
returns nothing but changes the value of the matrix A which is passed as rst argument:
A = . zeros(A, 2, 3) A 1 2 3 +-------------+ 1 | 0 0 0 | 2 | 0 0 0 | +-------------+
it is the address of n that is passed to f(), not a copy of the values in n. f(n) can therefore modify n. See help m2 syntax for more details. 6.5 6.4 Returning values
return(expr) causes the function to stop execution and return to the caller, returning the evaluation of expr. return(expr) may appear multiple times in the program. For example,
real matrix zeros(real scalar c,| real scalar r) { if (args()==1) {
Once dened, new Mata functions can be stored as object code (i.e. in compiled form) in .mo les using the mata mosave command. The function can then be accessed in subsequent Mata sessions without being dened again. For example, running the following .do le:
31
32
version 10 mata: real matrix zeros(real scalar c,| real scalar r) { real matrix A if (args()==1) r = 1 A = J(c, r, 0) return(A) } mata mosave zeros(), replace end end
= r(N) b indepvar _cons, c(.) V indepvar _cons post b V, depname(depvar) obs(N) esample(touse) local cmd = "ols" display
will produce the le zeros.mo which is stored in the working directory. If you close Stata and run it again, the function zeros() will be readily available. Several Mata functions can be simultaneously stored in a .mlib le. See help mata_mlib for more details.
capture mata mata drop m_ols() version 10 mata: void m_ols(string scalar varlist, string scalar touse) { real matrix M, X, V real colvector y, b real scalar n, k, s2 M = X = y = . st_view(M, ., tokens(varlist), touse) st_subview(y, M, ., 1) st_subview(X, M, ., (2\.)) n = rows(X) k = cols(X) XX = cross(X,1,X,1) if (rank(XX) < k+1) { errprintf("near singular matrix\n") exit(499) } Xy = cross(X,1,y,0) b = cholsolve(XX,Xy) e = y - (X, J(n,1,1))*b s2 = (ee)/(n-k) V = s2*cholinv(XX) st_eclear() st_matrix("r(b)", b) st_matrix("r(V)", V) st_numscalar("r(N)", n) } end
7
7.1
The following Stata and Mata commands are stored in the le ols.ado:
program define ols, eclass version 10.0 syntax varlist(numeric) [if] [in] gettoken depvar indepvar: varlist marksample touse mata: m_ols("varlist", "touse") tempname b V matrix b = r(b) matrix V = r(V)