CRSPSift USER GUIDE

Updated June 15, 2012

105 West Adams, Suite 1700
Chicago, IL 60603
Tel: 312.263.6400
Fax: 312.263.6430
Email: [email protected]
CONTENTS

CHAPTER 1: INTRODUCTION........................................................... 1

CHAPTER 2: INSTALLATION AND SETUP.. ......................................... 2

CHAPTER 3: TSQUERY: TIME SERIES ACCESS................................... 6

ENTITIES.............................................................................................................6

DATA ITEMS.......................................................................................................9

DATE.................................................................................................................. 14

LAYOUT............................................................................................................ 16

DIRECT EDIT.................................................................................................... 18

EXECUTING A QUERY.................................................................................... 18

CHAPTER 4: STKQUERY: STOCK DATA ACCESS............................... 19

SECURITY SELECTION.................................................................................... 19

HEADER IDENTIFICATION............................................................................ 20

DATA ITEMS..................................................................................................... 20

GLOBAL OPTIONS........................................................................................... 21

DATA GROUPS................................................................................................. 22

PORTFOLIOS.................................................................................................... 23

OUTPUT OPTIONS.......................................................................................... 23
CHAPTER 5: INDQUERY: INDEX DATA ACCESS................................ 26

INDEX SELECTION.......................................................................................... 26

HEADER IDENTIFICATION............................................................................ 27

TIME SERIES DATA.......................................................................................... 27

PORTFOLIO GROUP OPTIONS...................................................................... 27

OUTPUT OPTIONS.......................................................................................... 28

CHAPTER 6: CCMQUERY:
CRSP/COMPUSTAT MERGED DATABASE ACCESS............................. 30

THE CCMQUERY TOOLBAR........................................................................... 30

COMPANY SELECTION.................................................................................... 30

DATA ITEMS AND GROUPS............................................................................ 31

CHAPTER 7: TRSQUERY: TREASURIES DATA ACCESS...................... 36

QUERY DATE RANGE SPECIFICATION......................................................... 36

TREASURY SELECTION................................................................................... 36

DATA ITEM SELECTION.................................................................................. 38

DATA GROUPS................................................................................................. 39

OUTPUT OPTIONS.......................................................................................... 40

CHAPTER 8: MUTQUERY:
US SURVIVOR-BIAS-FREE MUTUAL FUND DATABASE ACCESS......... 42

SECURITY SELECTION.................................................................................... 42

DATA ITEMS..................................................................................................... 43

DATA GROUPS................................................................................................. 44
OUTPUT OPTIONS.......................................................................................... 45

OUTPUT FILE SUFFIXES................................................................................. 46

CHAPTER 9: TOOLS . . ..................................................................... 47

CALENDAR FILE WIZARD............................................................................... 47

REFERENCE DATA........................................................................................... 48

APPENDIX A: AVAILABLE CURRENCY CODES............................................ 50

CHAPTER 10: CRSPSIFT OPTIONS................................................. 53

CRSPSIFT OPTIONS - ENVIRONMENT SCANNER ....................................... 53

CRSPSIFT OPTIONS - SERVICES ..................................................................... 55

CRSPSIFT OPTIONS - QUERY QUEUE .......................................................... 55

 Chapter 1: INTRODUCTION

CRSPSift, CRSP’s Securities Information Filtering Tool, IndQuery is an access tool for CRSP Indexes included in
lets you extract data from your Stock & Index, CRSP/ the CRSP US Stock or Stock & Index Databases.
Compustat Merged, US Treasury, Mutual Fund, and
new Research Index databases. In addition to an updated CCMQuery enables you to access all Compustat data from
interface, CRSPSift offers many features designed to make the CRSP/Compustat Merged Database link history data by
accessing CRSP databases easier. Some of these features using Compustat’s GVKEY and IID identifiers.
include:
TrsQuery is a tool for accessing CRSP Daily and Monthly
‚‚ Data Environments: Create a data environment for each US Treasury Databases.
set of data you use. To use a different data set, simply
select its environment from a menu and rerun your query. MutQuery provides access to the CRSP Mutual Funds
database containing fund and holdings data items.
‚‚ Multiple Queries: Edit and run up to ten queries
simultaneously. RIQuery introduces CRSP’s Research Index database built
upon CRSP’s new investable index series and containing
‚‚ Direct Edit: Type or paste an existing request file from a new intra-day data items.
command-line utility into a window and then save it as
a CRSPSift query. CRSPSift automatically updates its
graphical interface to reflect your query options. CRSPSIFT MINIMUM SYSTEM REQUIREMENTS
FOR THE PROFESSIONAL EDITION
‚‚ Output Layout Preview: TsQuery features an animated
report preview that suggests how your output will look. ‚‚ Microsoft Windows XP Professional SP3 or Windows 7
This saves you the time of running a query multiple times 32- and 64-bit
to find the right format.
‚‚ Display resolution of 1024 x 768 or higher
‚‚ Company Selection: A company lookup utility. Search
and sort results by Company Name, PERMNO, ticker ‚‚ Powershell 1.0 for Windows XP. Windows 7 comes
symbol, or other identifier. prepackaged with Powershell.

‚‚ Filtering Options: A point-and-click interface to simplify ‚‚ Adobe Flash Player 11

filtering query results by exchange, share type, NASDAQ
Market Tiers, portfolio assignments, SIC Code, or group Note: CRSP recommends using CRSPSift Professional with
membership. a local system account, on databases installed locally.

CRSPSIFT UTILITIES

TsQuery is a tool for extracting time-series data – based in

fixed and relative ranges – for the CRSP security universe,
CRSP Indexes, and CRSP/Compustat linked companies (for
Compustat subscribers).

StkQuery is an access tool for CRSP security data. Use it

to extract event histories for name and identifier changes,
distributions, delisting. It can also perform time-series
extractions.

CRSPSift User Guide • PAGE 1

 Chapter 2: INSTALLATION AND SETUP

This chapter shows you how to install the CRSPSift software

on your computer and describes the configuration options
available when you launch it for the first time.

INSTALLING CRSPSIFT

To install CRSPSift, insert the installation disc into your

computer’s DVD drive and navigate to it in Windows
Explorer. Double click the file setup.html to open it in your
web browser

To begin installation, click the Next button.

After reading the installation terms contained in the

document, proceed to the section titled Go To An Installer
and click the Install Professional Edition - XP SP3 and
Windows 7 Only link to launch the installer program.

Note: Installers for prerequisite software are included on the

CRSPSift installation disc.
Read the License Agreement. To continue installation, click
See the Introduction to this guide for a complete list of
the radio button labeled “I accept the terms in the license
required software.
agreement,” and then click Next.
When all prerequisites are in place, the CRSPSift
InstallShield Wizard starts. The beginning of the process is
automated.

CRSPSift User Guide • PAGE 2

3 Enter your name and the name of your organization on Click the Finish button to exit the InstallShield Wizard.
the Customer Information screen, and then click Next.
Chapter 2: Introduction

Now choose the destination folder where CRSPSift

will be installed on your computer. If you would like LAUNCHING CRSPSIFT FOR THE FIRST TIME
to install it to a location other than the default, click
the Change button and navigate to the folder of your The InstallShield Wizard creates a shortcut to the
choice. When you have selected a destination folder, CRSPSift application on your Desktop. Double-click
click the Next button.
CRSPSift User Guide

the shortcut icon to launch CRSPSift.

Before you can use CRSPSift you must tell it the

location of your installed CRSP databases. The first
time you run the application you are presented with
the Environment Startup Scanner.

The InstallShield Wizard is now ready to install the

program. Click the Install button to begin.

Using this tool you create what are known as data

environments. A data environment is a collection of
one or more compatible database products.

FINDING ENVIRONMENTS
Upon launch, the Environment Startup Scanner
automatically searches the default CRSP database
installation directory (C:\crspdata). It builds and displays
environments for the most recent databases found.
4 a name, and check the Set as Default box if you want
this to the the default environment for new queries.
All Stock Databases and CRSP/Compustat Merged
Databases, for Compustat subscribers, are detected by
If your CRSP databases are installed in a directory CRSPSift and will appear in the lists below.
other than the default, click the disclosure triangle next
Chapter 2: Introduction

to Scan Folder Hint Paths under Scan Options. When you select a Stock Database, The Select CRSP/
Compustat table below displays Compustat databases
compatible with it, if any are installed. Select a
database from the list if you wish to include CRSP/
Compustat data in your environment. Finally, click
the Browse button to select a working directory where
input and output files for this environment will
A folder hint path serves as a guide telling the be directed. Click the Save button to save the new
Environment Scanner where to look for CRSP environment.
databases. The default locations are already listed. To
Only the environments that reflect the newest data
add your own, click the Add button and navigate to
cuts will automatically be displayed. Additional
the directory where your CRSP databases are installed.
combinations can be created.
Then click the Start Scan button to scan again with the
CRSPSift User Guide

new directory included.

EDITING AN EXISTING ENVIRONMENT
The Environments screen is where you can manage You can also edit an existing environment to change
the environments detected by CRSPSift on its its name and working directory, add or remove
initial launch. To manage your environments, select Compustat data, or modify the environment’s
Options... from the Tools menu to open the CRSPSift combination of daily and monthly stock databases.
Options window. Then click Environments in the list
To modify an environment, select it in the
of options categories on the left.
environments list and then click the Edit Environment
A list of all your saved environments appears in the button. The procedure for editing an environment is
main area of the screen. the same as that for creating a new one. Refer to the
discussion above for instructions.

ENVIRONMENT DETAILS
The Environment Details area displays the properties
of the environment currently selected in the New
And at the bottom of the screen are controls for Environments Found list. Some of these properties
managing them. You can edit or delete existing may be changed, including an environment’s name, its
environments, as well as create new ones. default status, and the working directory where query
output files are saved.

To make an environment the default for new

queries, first select its row in the list. Next, in the
Environment Details area, click the False value next to
the Default option.

ADDING A NEW ENVIRONMENT

To add a new environment, click the Load
Environment button. Provide the environment with
5 the Close button to close the Startup Environment
Scanner window. The CRSPSift Welcome screen
appears, and you are now ready to begin using
CRSPSift query tools.
Chapter 2: Introduction

Select True to make this environment the default for

new queries.

Likewise for the Name and Working Directory

properties. Click the value next to the property name
and change it to the desired setting.
CRSPSift User Guide

Note: CRSP provides default environment names

that we believe to be intuitive. We encourage you to
edit names so that they are meaningful to you for
your purposes.

LOADING ENVIRONMENTS
Once installed databases have been scanned and
combined into environments, and each environment’s
properties are set appropriately, you can select which
environments you wish to load into CRSPSift.

For each environment you wish to load, click the

checkbox in the Load column to the left of its name.

Click the Add button to load the selected

environments. When the progress message below the
environments list displays “Finished uploading,” click
 Chapter 3: TSQUERY: TIME SERIES ACCESS

TsQuery is designed to generate output from a combination LIST

of database frequencies, and varying types of data into Once the Entity Type Selection is made, a series of options
different formats and layouts that serve a range of purposes. become available. Under List, a menu of identifier keys
is available from the drop down menu. Next, identify the
Click the New Query button on the CRSPSift toolbar.
issue/issues of interest. If you know the identifier, you may
Select the Data Environment you wish to use and highlight
type it in, or you may click on Find... to do a search.
the TsQuery icon. Optionally give your query a name in
the Query Name field, and then click the OK button. You
may now fill out the screens required to define your query:
entities, data items, date information, and report layout.

ENTITIES

In the Entities screen, you select which securities, indexes, or

portfolios your query will use. To add an entity, select an entity Use the search form to locate a particular company in the
type from the drop-down menu and click the New button. CRSP database. Companies matching your search criteria
appear in a table, initially sorted by PERMNO. If your
‚‚ List, the default, is meant for entering a single security, or search yields many matches, click the column labels to resort
a small number of securities, manually. the table in the order most useful to you.
‚‚ List All calls the full universe of securities in the database
selected.
‚‚ List File accesses securities specified in a user-created
input file.
‚‚ Portfolio All allows the user to create an equal- or value-
weighted portfolio from the full universe.
‚‚ Portfolio File accepts a user-created input file of securities
in a portfolio.

When you have found the company/companies you want,

click its row in the table, and then click the Select button. To
select more than one company, hold down the Ctrl key while
making selections. The PERMNO of that company will
appear in the identifier field.

CRSPSift User Guide • PAGE 6

7 DATE OPTIONS The Entity Event Date specifies a single date in
There are two options on the Entities screen for YYYYMMDD. An Entity Event Date must be paired with
restricting query output by date: Date Range and a Relative Date Range on the Date tab. The Relative
Event Date. The Date Options allow you to target date Date Range controls the number of calendar periods
ranges or single dates specific to an entity. before and after the Event Date to report in output.
Event Date does not work with indexes or portfolios.
Chapter 3: Introduction

Date Options selected on the Entities screen are

further tied to the Date Range option on the Date For example, suppose you enter an Entity Event Date
tab. If Date Range is selected on the Entites tab, a of 19991231. Next, in the Date tab, you select the
Fixed Date Range must be specified on the Date tab. Daily calendar and give a Relative Date Range of 2
Likewise, an Event Date must be paired with a Relative days before and 2 days after the Entity Event Date.
Date Range on the Date tab. The query output will include data for trading days
19911229 though 19920104, including the event date.
Date Range
For more information about calendars, Fixed Date
Ranges, and Relative Date Ranges, refer to the section
on the Date tab below.

ENTITY FILTERS
Several entity filters are available in TsQuery, allowing
CRSPSift User Guide

you to refine your selection of securities by such

If given, a Date Range must include both beginning
criteria as stock exchange, share type, and SIC code.
and ending dates. Dates may be in YYYYMMDD,
These options are available for list and portfolio entity
YYYYMM, or YYYY format. For formats that do not
types but will not work with indexes.
specify months or days, the beginning date in the range
will start with the first period within the specified range.
Exchange
The ending date will be the last period in the range.
The Exchange Filter allows you to filter the trading
For example, the output of a query with Entity Date
history of issues on the basis of stock exchange. This
Range 200605 - 200703 would include daily data from
option is available only when using the List and
May 1, 2006 through March 31, 2007, or monthly data
Portfolio Entity Types.
from May 31, 2006 through March 31, 2007.

The valid query output is the union of the security’s

trading history, the Date Range specified on the
Entities tab, and the Fixed Date Range given on the
Date tab. The Entity Date Range must fall within
the Fixed Date Range. Note that the Entity Date
Range must also exceed the frequency duration of the
calendar selected for your output. For example, if your
Share Type
calendar is set to report annually, the Date Range must
span more than 12 months.
The Share Type Filter allows you to restrict the output
on the basis of share type for individual securities.
Event Date
The selection is based on the CRSP Share Type Code
variable.
8 Portfolio Assignment

Portfolio Assignments are only available for use with

a Stock and Index Database. A Stock and Index
Database subscriber has access to nine sets of daily
and eight sets of monthly portfolios. Each entity in
Chapter 3: Introduction

the CRSP universe may be a member of a number of

CRSP portfolios, which are based on combinations
of market capitalization and exchange membership or
group membership.

The Port Assign Filter allows you to restrict your query

Select at least one item from both Group A and Group to a subset of the CRSP universe, composed of your
B. Hold the Control button while clicking to select choice of portfolio sets and the portfolios within them.
multiple items from a group.
To filter by portfolio assignment, select a Database
NASDAQ Market Tiers Type, Daily or Monthly, and then select a portfolio
set from the list of portfolio sets. Next enter a list
When the NMSIND option is used, only NASDAQ or range of portfolios from the set. For example, to
issue ranges are restricted. The filter has no effect on choose portfolios 1, 3, 4, 5, and 10, enter “1,3-5,10”
CRSPSift User Guide

data from issues trading on other exchanges. in the Portfolio List field. Refer to Chapter 4 of the
Stock and Index Data Description Guide for a list of
all portfolios and sets of portfolios.

SIC

SIC issue range restriction is applicable to List and

Portfolio Entity Types. Each number represents a single
SIC Code. You can filter the data to output a range of
SIC values or an individual SIC value
Note: Be sure to match the database type of the items
For example, 1000-2000,3725 would extract all you select on the Data Items tab to the database type
securities with SIC Codes between 1000 and 2000, of the portfolio you filter by here. For example, if
and all with and SIC code of 3725. you filter by portfolios in the Monthly NYSE Cap
Assignment, select items only from the Monthly Stock
& Index group on the Data Items tab.

Groups

Currently the only group defined is the S&P 500

Constituents. Selecting this filter limits output to
those issues listed in the S&P 500 during the time
9 period covered by the query. Use of Groups is limited Clicking an entity’s row in the hopper displays the
to Stock and Index Database subscribers. options and values you have set for it.
Chapter 3: Introduction

Using this method, add as many entities as your query

requires. If you wish to change the order in which
entities appear in the hopper, click on a row and use
the arrow buttons to move it up or down.

To go back and edit an entity click its row in the

hopper, and then click the Edit button.
Filter Tuning
When you have finished editing the entity, click the
Each entity filter can be further refined by selecting
Save button to save your changes. To delete an entity
one of three tuning flags. The flags are:
from the hopper, select its row and click the Remove
CRSPSift User Guide

‚‚ Keep what is valid - Keep only the trading history of button. The Remove All button deletes all entities that
an issue where the selected filter applies. you have added.
‚‚ Keep none if ever invalid - Discard all data on an
issue if the selected filter fails to match at any point DATA ITEMS
in the time period of the query.
‚‚ Keep all if ever valid - If the selected filter matches In the Data Items screen you can select the CRSP data
an issue at any point during the time period of the items relevant to your query.
query, all data on that issue is kept.

THE ENTITY HOPPER SEARCHING AND NAVIGATING DATA ITEMS

After setting appropriate options, click the Add button Data items are grouped in the Navigate tab according
to add this entity to the hopper in the lower area of to the database to which they belong: Daily Stock,
the screen. Monthly Stock, Compustat (available to CCM
Database subscribers), and Favorites. Click the plus
sign next to a database to open a categorized list of
data items. When you select a category, a list of the
data items from that category populates the Search
Results area in the central area of the screen.
10 In addition to navigating items by category, you can After selecting a data item, if you want to keep its
search the available data items. To find items this way, default settings, click the Add button to add that item
click the Search tab. to the hopper below. Otherwise, click the value of a
setting you wish to change, modify the settings, and
then click Add. Values that appear in bold type are
editable. Changing default settings affects the item
Chapter 3: Introduction

report formatting.

Data Length

CRSP recommends keeping the Data Length

field consistent with the Fixed Width so that the
item is not truncated. Data Length is based on
the values in the underlying data items.

Search is limited to the database selected in the Search Data Type

By menu. Additional criteria can be supplied as well,
CRSP recommends that Data Type not be edited.
such as whether results must be an exact match for
What appears in the field is what CRSP has
the search term, begin or end with the term, or simply
identified to be appropriate for the underlying
contain the term.
CRSPSift User Guide

data item.
Both mnemonic codes and full item names are
Fixed Width
searched and returned for each item. Click the
column names—Item ID, Variant, Data Item Name, or The allotted number of characters that an item
Header—to change the order in which items are sorted. will be given in the output file. Fixed Width must
be greater than or equal to the Data Length for the
requested item. Note that expanding Fixed Width
does not increase a data item’s numeric precision.

Header

If column headers are used, more descriptive

names may replace the default itemids.

Item Lag
DATA ITEM PROPERTIES
Used to either lead or lag a data item. A positive
Further details about an individual item are available
number of periods will lag the data by the
on the far right of the screen.
number of periods designated in the output
Data Item Settings Data Item Description calendar, while a negative number will lead the
data item. For example, given a December 2006
date in an output file, prices requested with a two
month lag will return the October 2006 values,
and prices requested with a two month lead will
return prices for February 2007.
11 INDNO/SUBNO

Data items related to an index or portfolio require more specific information before they can be added to
the item hopper - a particular INDNO value for an index, or a particular SUBNO value for a portfolio.
When these items are selected, red text instructing you how to refine your selection appears above the Data
Item Settings area.
Chapter 3: Introduction

Click on the INDNO or SUBNO to open the list of available indexes or portfolios to which the item may
be related.
CRSPSift User Guide
12 Repeat this process until all the data items of interest to you appear in the hopper at the bottom of the screen. As in
the Entities screen, use the arrow buttons to the right of the hopper to change the order in which data items appear.
Chapter 3: Introduction

To remove a single item, select its row and click the single document button. To remove all items from the
hopper, click the multiple document button.

Edit. Click this to edit an item’s attributes.

Remove the selected item from your query.

CRSPSift User Guide

Remove all items from the Query Items list.

Click the OK button to confirm the addition of the selected items to your query. Click cancel to exit the
StkQuery Data Items window without making any changes.

To modify the settings for a data item in the hopper, select its row and click the Edit button. Change the data
item settings to your liking, and then click the Save Changes button. When you change an item’s settings from
the default, its entry in the hopper displays a Yes in the “Customized?” column. To cancel changes, click the
Cancel button.
13 GLOBAL ITEM OPTIONS
When selecting Compustat data items using the new CCM databases, several global option settings are available.
The default currency conversion option is to report data items in the currency in which they are reported. By
clicking on the drop down menu, you may change this option to convert all items to US Dollars.
Chapter 3: Introduction

You can also choose to include footnotes and/or data codes for the Compustat items in your query that have
them. The Footnote and Data Code items can also be extracted automatically for all items that have them by
choosing Yes for the settings. This can also be done manually by selecting the individual data codes or footnotes
for select data items in the Data Item Settings.

NOTE: Global options must be selected before choosing data items in order for them to be applied.
CRSPSift User Guide

It is important to note that the Global Item Options apply only to Compustat data accessed from the CRSP/
Compustat Merged Database.

KEYSETS
Keysets are CRSP-created groups of Compustat secondary keys that are used to further define Compustat data
items. CRSP has identified a default keyset for each data item. Sift allows you to override the default keysets, or
select multiple keysets in Sift.
14 On the Data Items screen, clicking on the Keyset box opens a table of all existing keysets for the items with
statistics about each. The default keyset is highlighted upon opening the box and is typically STD, or Standard.
The Keyset number is provided, the number of GVKEYs, the number of data points, the date range populated by
the data, and a description of the Keyset.
Chapter 3: Introduction

Choose different or additional keysets and click the Select button to close the window.
CRSPSift User Guide

DATE

The Date screen allows you to control what dates will be used in the output for the data items selected for
each entity.

CALENDAR OPTIONS
You can define your own set of dates for a query, or you can select one of several predefined calendars. To use
a predefined calendar, click the “Calendar Name” radio button and select a calendar—Daily, Weekly, Monthly,
Quarterly, or Annually—from the menu.

You may want to create a custom calendar with dates for output that are not available in one of the standard
Calendar Name options. Click on the Calendar File radio button to select a user-generated calendar.

Refer to the Calendar File Wizard chapter in this guide for information on creating a custom calendar file.

DATE RANGE
Date usage is linked to any date information entered in the Entities screen. If an event date is assigned to an
issue, either in the Entities screen, or included in the list entity input file, Relative Date Range should be selected.
15 A relative date range specifies a window of time, in days, based on the event date chosen in the Entities screen.
It is tied to the calendar selected for the output. Take, for example, an event date of October 18, 2005, with the
Daily calendar selected for output. We would like to limit the results of our query to information from the five
trading days preceding, but not including, the event date. To accomplish this, we first set the date range type to
Relative. Then, in the Relative Event Date Range controls, we set the first parameter to five trading days before
and the second parameter to one trading day before.
Chapter 3: Introduction

The query will now return information only from October 13, 2005 to October 17, 2005. Similarly, depending
on your needs, you could set the range to a window of days either surrounding or after a selected event date.

Helpful hint:

When using relative dates, include Date, caldt, as an output item in your query. Without caldt, all that returns is
an item’s position relative to the event date, with the event date always equal to zero.

For all queries that do not employ event dates, the default Fixed Date Range should be selected in the Date screen.
CRSPSift User Guide

If a date range is specified in the Entities tab, Fixed Date Range acts as an umbrella to the beginning and ending
dates selected, and must cover all dates requested. For example, if you used the following list entity input file, the
Fixed Date Range set in the Date screen would need to span 19751230-19961231.

12490 19751230 19861001 IBM

10107 19900930 19961231 Microsoft

Optional Parameters

The Report On menu allows you to select either a Calendar Basis or Fiscal Basis for reporting. This option
applies only to Compustat data items, controlling whether to report based on a security’s fiscal year or the
December calendar year. Queries accessing only CRSP data report on the December calendar basis.

The Display field allows you to select specific dates or a range that falls within a fixed date range that will be
displayed in the output.
16 LAYOUT

On the Layout screen you specify the file type, save location, and physical layout of a query’s output.
Chapter 3: Introduction
CRSPSift User Guide

OUTPUT OPTIONS
TsQuery can save report output in a variety of file formats. By default TsQuery saves reports as Formatted Text
files with no field delimiter, which is intended to be human-readable.

Choosing a Field Delimiter

If you plan to import query output into another program for further processing, select a field delimiter from the
Field Delimiter menu. A field delimiter is a marker that tells the importing program where to split a data record
into its constituent data fields. Available options are Pipe (|), Comma (,), and Space ( ). You can enter your
own preferred field delimiter by selecting Custom and typing a value in the text field next to the menu.

Supported File Formats

In addition to Formatted Text, TsQuery can save query output directly to any of the following file formats:

‚‚ Excel 97-2003 Workbook (*.xls) - Excel version 8.

NOTE: Excel version 8 has a row limit of 65536 total rows. If a query generates more than that number of
rows, the output will be truncated at the row limit.

‚‚ Excel 2007 Workbook (*.xlsx) - Excel XML

‚‚ MAT-files (*.mat) - Matlab version 7

‚‚ SAS Dataset (*.sas7bdat) - SAS version 7

‚‚ Stata File (*.dta)

‚‚ SPSS File (*.sav)

17 BASIC FORMATTING Report Name
In the default Formatted Text format, X, Y, and
A text description that will appear at the top of the
Z correspond to the Data Item, Date, and Entity
information added to the query. The preview pane of report. (Formatted Text output only)
the Layout screen shows an animated preview of how
Calendar Format
your report will appear, allowing you to visualize it
Chapter 3: Introduction

without first having to execute the query. The preview Controls formatting of the dates appearing in the
is best used to visualize the appearance of Formatted
output when date headers are chosen. Options include:
Text (.txt) output. The appearance of output in other
formats, such as Excel or Matlab, will vary according FORMAT EXAMPLE
to the target application. Previewing saves time 20071231
YYYYMMDD (default)
and enhances the readability of your output. If, for
YYMMDD 071231
instance, your query uses only a few entities, you may
MM/DD/YY 12/31/07
wish to place Entity on the report’s X axis. If it uses
MM/DD/YYYY 12/31/2007
many, you may prefer to place it along the Y or Z axis
DD-mmm-YYYY 31-Dec-2007
to limit the output’s physical width.
Cal-Based 2007.4 (for the
The nature of your query will determine the proper quarterly calendar)
report layout. Use the controls to the left of the
preview pane to test various scenarios. To see how a Row Delimiter
CRSPSift User Guide

report will look given particular settings, modify the

Controls the number of rows between output lines.
layout controls, and then click the Preview button.
The first integer is the number of blank lines between
The contents of the pane will reorganize to reflect
rows when the Z-axis value changes and impacts Z-flag
your settings, and the axes will briefly flash with color
options 1 and 3. The second integer is the number of
to identify themselves: X is purple, Y is green, and Z
blank lines between all data rows. The default is 0,0.
is red. It may be helpful to think of X as “rows,” Y as
“columns,” and Z as “page”.
Character Delimiter
When you have found a suitable layout, click the
A character string placed before and after all character
Browse... button next to the Location and File Name
string fields—Company Name, for example—in report
field. Navigate to where you want to save your report,
records. The default is no character string delimiter.
give the file a descriptive name, and then click the Save
For example, a value of * would cause the character
button. When you execute the query, CRSPSift will
string field Company Name to be surrounded by
save the report in this file.
asterisks in output.
ADVANCED LAYOUT SETTINGS
Partial Period Data
Click the Advanced... button to access the following
advanced layout settings. Includes partial-period data in the output. If not
selected, ts_print will not include the last month of data
for a company that stopped trading mid-month, because
only months with end-of-month data are normally
included. This option applies to monthly data.

No Fill

The No Fill option, enabled by default, causes rows

outside an issue’s date range or the user’s date
specification not to print to the output file. No Fill is
applicable under the following conditions:
18 • Item is chosen for the X-axis input file, the default missing delisting return, -55.0,
would be used instead.
• Date is the Y-axis
DIRECT EDIT
• Entity is the Z-axis
Chapter 3: Introduction

• Z-Axis Data Flag is set to 1 or 3 To use this feature, click the Direct Edit tab. Click the
Edit button above the text area there, and then type
• Fixed Date Range is selected on the Date tab. or paste your query code, overwriting the template
query code. When you are finished, click Save. If the
No Fill does not work with relative dates. syntax of your query is correct, CRSPSift automatically
populates the appropriate fields in the other screens to
Compact
reflect your query code.
Compresses output by removing all spaces and trailing
decimal zeros in numbers. Compact is ideal for EXECUTING A QUERY
producing output to be loaded into another program.
After executing a query, click the View Output button
Delisting Returns on the CRSPSift toolbar.

Default Missing: Outputs the default value (-88.0)

CRSPSift User Guide

for missing delisting returns for entities that have

delisted during the selected dates. You must have a
Returns data item selected in the Data Items tab in The File Download window opens, prompting you to
order to include Delisting Returns in your output. open the output file in the target application or to save
it in a different location.
Custom Missing: Outputs user-specified missing
delisting return codes. You may assign missing values
for a range of delisting codes for select beginning and
ending exchanges. To do this, create a text input file
containing the following fields in the following order:
begin delist code, end delist code, begin exchange
code, end exchange code, alternate delisting return
value, alternate delisting return without dividends
value.

For example:
Click Open to open the output file in the target
200 299 1 3 -0.50 -0.55 application (Excel 2007, in the example shown below).

500 570 3 3 -0.40 -0.45

571 600 3 3 -0.30 -0.35

Note that in this example, the first row would assign

a -0.50 value to missing delisting returns for
securities with delisting codes 200-299 that initially
traded on NYSE and ended up trading on NASDAQ,
and -0.55 for missing delisting returns without
dividends. If your query includes a security with a
missing delisting return that was not included in your
 Chapter 4: STKQUERY: STOCK DATA ACCESS

StkQuery is used to extract CRSP stock header, event, and ticker symbol of Apple Computer is AAPL. Given that
time-series data items. Designed primarily to access data on knowledge, you can set the key to Active Ticker and type
an individual security, it can also access a list of securities AAPL as the identifier.
supplied in an input file, or all securities in the database.
If you do not know an identifier, click the Find… button to
Locate the StkQuery toolbar below the main CRSPSift toolbar. open the Company Search window.

In the Database field, select which CRSP database your

query will use, either Daily or Monthly. Next, specify a
date range. Enter dates using one of the following formats:
YYYYMMDD, YYYYMM, or YYYY. The following table
provides examples of how to use the various formats.

START END RESULT

199609 199612 all data from the beginning of September through December of 1996
1990 (empty) all data in the year 1990
1994 19940615 all data from the beginning of 1994 until June 15, 1994
19961231 (empty) data only on the date December 31, 1996
Use the search form to locate a particular company in the
CRSP database. Companies matching your search criteria
By default, StkQuery returns data from the last three months
appear in a table, initially sorted by PERMNO. If your
of the CRSP database being accessed.
search yields many matches, click the column labels to resort
the table in the order most useful to you. When you have
SECURITY SELECTION found the company you want, click its row in the table, and
then click the Select button. The PERMNO of that company
From the Choose Selection Type drop-down menu in the will appear in the identifier field.
Security Selection area, specify the key, such as PERMNO or
You may also choose to access all issues in the database, or
Active Ticker symbol, your query will use.
issues you specify in an input file. To search all issues, click
All Issues in the Security Selection area. To use an input file,
first create a plain text file (using the Microsoft Windows
Notepad accessory, for instance) with an entry for each issue,
one per row, using the active key of your choice.

For example, to search for Apple and Microsoft by

PERMNO using an input file, create and save the following
two line text file:

14593
10107

Next, identify the issue or issues of interest. If you know

Next, select PERMNO from the Active Key menu. Click
an identifier, you can enter it directly. For example, the

CRSPSift User Guide • PAGE 19

20 Input File in the Security Selection area, and then click the Browse... button to locate the file you created.

HEADER IDENTIFICATION

Click the Header tab if it is not the active tab. In this screen, select the stock header information that you want in
Chapter 4: Introduction

your report and how it should appear.

DATA ITEMS

Use the Data Items tab to control what individual event and time series data items your query will use.
CRSPSift User Guide

SELECT/EDIT DATA ITEMS

Clicking this button launches the interface for selecting individual event and time series items in StkQuery.

The StkQuery Data Items window is divided into several areas.

21 Items are divided into logical groups in the Navigate/ StkQuery allows users to modify output data items by
Search area. changing default parameters for certain types of items.
Defaults can be modified for all items selected in a
query by defining parameters in the Global Options
table on the Data Items screen.
Chapter 4: Introduction

PARAMETER DEFAULT DEFINITION

AdjDate Default is to adjust The adjustment date is the
values through the latest anchor date on which all data are
date of the database. presented as they were reported
and are adjusted back in time by
selected or default factors from
that point.
AdjTyp Default is to apply all The adjustment type value
factors represents the factors that are
applied to price adjustments.
Choices are 1 = Apply All Factors
Clicking the name of a group causes its data items to
or 2 = Apply Stock Splits and
appear in the Search Results area. Or, rather than Dividends
browsing by category, click the Search tab. There you AdjTypes Default is to apply stock The adjustment types values
can enter a descriptive keyword (e.g., “NASDAQ”) to splits and dividends represents the factors that are
find items of interest. Browsing and searching both applied to shares and volume
CRSPSift User Guide

adjustments. Choices are 1 = Apply

cause the data items found to appear in the Search
All Factors or 2 = Apply Stock Splits
Results area of the window. and Dividends
BaseAmount Default is 100 The base amount is the amount to
be reported on a specified base. For
use with Time Series items Total
Return Index Level and Price Index
Level.
BaseDate If specifying a base amount, a base
date must be provided for the date
to apply the amount. Values before
and after the specified date will be
indexed to the amount.
GapRule Default is to set values Gap rule defines when and if to
to missing upon introduce missing values when a
exceeding the specified specified number of periods passed
gap with no new data. Choices are 0 =
When you highlight an item in the Search Results area, Continue data after the gap and 1 =
its attributes and a short text description appear in the set missing values after the gap
window areas to the right. GapWindow Default is 10 The gap window defines the
maximum number of dates allowed
GLOBAL OPTIONS to pass between two dates where
the previous value of an item
remains valid. For example, with
the default of 10, a price from up
to ten trading dates ago is eligible
for use in the returns calculation
if there are no more current prices.
Once that 10th day has passed,
using the default gap rule, the price
is considered missing and will no
longer be available for use in the
returns calculation.
22
PARAMETER DEFAULT DEFINITION position of an item, select it and click the blue up and
RightsRule Default is to use Share The rights rule is used to apply down arrows to move it up or down the list. The other
History share factors from rights buttons are:
distributions. Choices are 0 = Use
shares history, 1 = Recreate shares
Edit. Click this to edit an item’s attributes.
history.
Chapter 4: Introduction

ValidExch Default is all exchanges Valid exchange identifies the

Remove the selected item from your query.
exchanges of interest to be
used. Choices can be modified
by clicking on the text box and
Remove all items from the Query Items list.
checking/unchecking available
exchanges. Valid exchanges is used
in conjunction with the calculated Click the OK button to confirm the addition of the
returns items, cret and cretx.
selected items to your query. Click cancel to exit the
StkQuery Data Items window without making any
Fields that can be edited are parameters in bold, black
changes.
text associated with adjusted values. These are set to
default values and may be modified. Import Data Item File...

Clicking this button allows you to import a list of data

items into your query from a text file. The file contains
a single column of item mnemonics item IDs.
CRSPSift User Guide

Export to File...

Use this option to output the Selected Data Items list

to a text file containing each item’s mnemonic itemid.
This text file may then be imported into other queries
using the Import Data Item File... button.

DATA GROUPS
Select each item you want in your query, adjust its
parameters if necessary, and then click the Add to The Data Groups tab allows you to select combinations
Query button to add it to the Query Items area below of related event or time series data items.
the Search Results.
Available groups include:

Event Data

‚‚ Adjusted Delistings
‚‚ Adjusted Distributions
‚‚ Adjusted Shares
‚‚ Delisting History
‚‚ Distribution History
‚‚ Group Inclusion
‚‚ NASDAQ History
‚‚ Name History
‚‚ Name History - Short
‚‚ Portfolio History
Items in the Query Items area appear in the order that
‚‚ Raw Shares History
they will appear in your query’s output. To adjust the
‚‚ Shares History
23 Time Series Data Screen output appears in the output area at the bottom
of the StkQuery window.
‚‚ Adjusted Market Summary
‚‚ Adjustted Price, Shares
‚‚ Market Summary
‚‚ Price and Index Levels
Chapter 4: Introduction

‚‚ Price and Returns

‚‚ Price and Shares

PORTFOLIOS

By default, the output of the last query execution

The Portfolios screen is applicable for Stock and
is shown in the Current tab. The Save Output and
Index Databases only. It provides a list of predefined
Clear Output buttons at the top of the output area
portfolios. Choose one or more of them to view the
allow you to save output to a text file or to clear it
performance of an issue or issues over time, using
away. Clicking the History tab shows you the output
the selected portfolios as benchmarks. The data
of every query execution since the StkQuery session
frequency — daily or monthly — will determine the list
was launched. When you switch to the query history
of available portfolios.
view, the buttons above the output area change to Save
History and Clear History. Save History allows you to
CRSPSift User Guide

capture the output of an entire session. You can select

a new security, or different data or options related to
the same security. Each time you click Execute the new
output is saved in the History tab. By clicking Save
History, you can save all the output to one text file.

OUTPUT TO FILE
You can also save query output in a number of external
file formats. When you choose a non-screen output
format, the Output File Name field becomes active. In
order to successfully execute the query, you must select
a location to store the output file. Click the Browse
OUTPUT OPTIONS button to do so.

You can save the output of an StkQuery in one of

several formats. Click the Output tab to show the
Output Options panel.

OUTPUT TO SCREEN
The default option is text-based output, formatted to
fit the screen width specified in the Width control. 80
characters is the default output width.
24 Supported File Formats Clicking one of the output files links opens the File
Download dialog box.
‚‚ Pipe(|) Delimited, For Import - pipe-deliimited
ASCII text, suitable for importing into databases
or software packages not directly supported by
CRSPSift. This option generates a single output file.
Chapter 4: Introduction

‚‚ Excel 97-2003 Workbook (*.xls) - Excel version 8.

NOTE: Excel version 8 has a row limit of 65536
total rows. If a query generates more than that
number of rows, the output will be truncated at the
row limit.

‚‚ Excel 2007 Workbook (*.xlsx) - Excel XML

‚‚ MAT-files (*.mat) - Matlab version 7

Click Open to open the output file in the appropriate
‚‚ SAS Dataset (*.sas8bdat) - SAS version 7
application (Excel 2007, in the example shown below).
‚‚ Stata File (*.dta)
CRSPSift User Guide

‚‚ SPSS File (*.sav)

VIEWING OUTPUT FILES Alternatively, you can click the View Output button,
With the exception of the Pipe Delimited option, if located in the CRSPSift toolbar, to view the output
you select a non-screen output format, StkQuery will files generated by your query.
generate one or more output files, one for each unique
output of the query. When the query completes
execution, the output area displays links to the output
files generated. Each file will have the name specified
in the Output File Name field, with the addition of an
extension identifying the type of data it contains.
If a query generated only one file, the target
application will open the file directly, as shown above.
If the query resulted in multiple output files instead
of opening the files, CRSPSift will open a Windows
Explorer window showing the folder where you chose
to save output.
25 OUTPUT FILE SUFFIXES
The following table lists output file suffixes and the
type of data they indicate is contained in a file.

OUTPUT FILE SUFFIX

DATA GROUP CATEGORY
DAILY MONTHLY
Chapter 4: Introduction

Adjusted Delistings adel madel

Adjusted Distributions adis madis
Adjusted Shares ashr mashr
Delisting History del mdel
Distribution History dis mdis
Group Inclusion grp mgrp
Name History nam mnam
Name History – Short nams mnams
NASDAQ History ndi mndi
Portfolio History port mport
Shares History shr mshr
Shares, Escl Imputed rshr mrshr
Adjusted Market Summary adj madj
CRSPSift User Guide

Adjusted Price, Shares adjwgt madjwgt

Market Summary dd md
Price and Index Levels ds ms
Price and Returns dr mr
Price and Shares dx mx
 Chapter 5: INDQUERY: INDEX DATA ACCESS

IndQuery is used to access index header and time-series data. button to open the Find INDNO window. If your query will
access an individual daily index, make sure you have selected
Locate the IndQuery toolbar below the main CRSPSift toolbar. Single Series from the INDNO Type menu.

The Find INDNO window will appear as shown below.

First select whether your query will use the Daily or Monthly
database from the Database menu.

Specify a date range for your query. Enter dates using one of
the following formats: YYYYMMDD, YYYYMM, or YYYY.
The following table shows some examples of how to use the
various formats.

START END RESULT

199609 199612 all data from the beginning of September through
December of 1996
1990 (empty) all data in the year 1990
1994 19940615 all data from the beginning of 1994 until June 15, 1994
19961231 (empty) data only on the date December 31, 1996

INDEX SELECTION
From the Available Index Groups menu, select the index
group that contains the index you want. Click the table
In the Choose Indno Type drop-down menu, specify
row containing your target index, and then click the Select
whether your query will access a Single Series (an individual
button. That index’s INDNO will now appear in the Enter
index) or a Group of Portfolios. Note that Single Series is for
INDNO box.
use only with a Stock and Index Database.
If you are using the Daily database, and you selected Group
of Portfolios from the INDNO Type menu, the Find
INDNO window will instead look like this:

If you know the INDNO of an index, you can type it directly

in the Enter INDNO box. Otherwise, click the Find…
Simply click to choose the index group you want, and then

CRSPSift User Guide • PAGE 26

27 click the Select button. The index group’s INDNO will then click the right arrow to move it to the Selected
appear in the Enter INDNO box. Items list.

You may also choose to access all indexes in the

database, or those that you specify in an input file.
To search all indexes, click All Indexes in the Index
Chapter 5: Introduction

Selection area. To use an input file, create a text file and

enter the INDNO of each index your query will use,
one per line. For example, to specify that your query
will access the CRSP NYSE Value-Weighted Market
Index (INDNO 1000000) and the CRSP NYSE MKT
Value-Weighted Market Index (INDNO 1000020),
create and save the following two line text file: PRE-DEFINED ITEM GROUPS
Alternatively, you can select pre-defined groups of
1000000 data items.
1000020

Click Input File in the Index Selection area, and

then click the Browse… button to locate the input
file you created.
CRSPSift User Guide

HEADER IDENTIFICATION

In the Header tab, choose what header information

you want at the top of your query report. Options
include standard index header information, header PORTFOLIO GROUP OPTIONS
information with date ranges in YYYYMMDD format,
or no header information. The Portfolio Group Options is for use only with a
Stock and Index Database. Its tab is enabled only
if you have selected Group of Portfolios from the
INDNO Type menu. Here you can select portfolio
range and rebalancing options for your query.

TIME SERIES DATA

In the Time Series tab, you can select the time series
data items that are relevant to your query. You can
select individual data items, choose pre-defined
groups of items, or supply a list of data items in an
input file.

INDIVIDUAL ITEMS
Select an item from the Available Items list at left, and
28 OUTPUT OPTIONS OUTPUT TO FILE
You can also save data in a number of external file
You can save the output of an IndQuery in one of formats. When you choose a non-screen output format,
several formats. Click the Output tab to show the the Output File Name field becomes active. In order
Output Options panel. to successfully execute the query, you must select a
location to store the output file. Click the Browse
Chapter 5: Introduction

button to select a location to save the file.

OUTPUT TO SCREEN
The default option is text-based, formatted to fit
the screen width specified in the Width control. 80
characters is the default width. Screen output appears
in the hopper at the bottom of the IndQuery window.
CRSPSift User Guide

Supported File Formats

• Pipe(|) Delimited, For Import - pipe-deliimited

ASCII text, suitable for importing into databases
or software packages not directly supported by
CRSPSift. This option generates a single output file.

• Excel 97-2003 Workbook (*.xls) - Excel version 8.

Note: Excel version 8 has a row limit of 65536
total rows. If a query generates more than that
number of rows, the output will be truncated at
the row limit.
By default, the output of the last query execution is
shown in the Current tab. The Save Output and Clear
• Excel 2007 Workbook (*.xlsx) - Excel XML
Output buttons at the top of the output area allow
you to save to a text file or to clear it away. Clicking • MAT-files (*.mat) - Matlab version 7
the History tab shows you the results of every query
execution since the session was opened. When you • SAS Dataset (*.sas8bdat) - SAS version 7
switch to the query history view, the buttons above
the output area change to Save History and Clear • Stata File (*.dta)
History. Save History allows you to capture the entire
• SPSS File (*.sav)
session. You can select a new index, or different data or
options related to the same index. Each time you click
Viewing Output Files
Execute the new output is saved in the history. Then,
by clicking Save History, you can save all the output to If you select a non-screen output format, IndQuery will
one text file. generate one or more output files, one for each unique
output of the query. When the query completes
execution, the output area displays links to the output
29 files generated. Each file will have the name specified If a query generated only one file, the target
in the Output File Name field, with the addition of an application will open the file directly, as shown above.
extension identifying the type of output it contains. If the query resulted in multiple output files, however,
instead of opening the files, CRSPSift will open a
Windows Explorer window showing the folder where
you chose to save output.
Chapter 5: Introduction

Clicking one of the output files links opens the File

Download dialog box.

OUTPUT FILE SUFFIXES

The following table lists output file suffixes and the

type of data they indicate is contained in a file.
CRSPSift User Guide

OUTPUT FILE SUFFIX

INDIVIDUAL TIME SERIES DAILY MONTHLY
Dind mind
PRE-DEFINED TIME SERIES DAILY MONTHLY
Click Open to open the output file in the appropriate Index Levels, used counts and values dlvl mlvl
application (Excel 2007, in the example shown below).
Index returns, used counts, and values dret mret
Index counts and values dval mval

Alternatively, you can click the View Output button,

located in the CRSPSift toolbar, to view the output
files generated by your query.
 Chapter 6: CCMQUERY: CRSP/COMPUSTAT MERGED DATABASE ACCESS

CCMQuery provides access to Compustat data for all using GVKEY with no IID, CCMQuery will automatically
securities in the Compustat universe as well as linking attach the IID of the primary isssue, most often .01.
information for those securities also in the CRSP Universe.
From the drop-down menu in the Company Selection area,
Click the New Query button on the CRSPSift toolbar. specify the key type you wish to use.
Select a data environment that includes a CRSP/Compustat
database. Highlight the CCMQuery icon, enter a descriptive
name, and then click the OK button.

THE CCMQUERY TOOLBAR

First specify a date range for your query. Enter dates using
one of the following formats: YYYYMM, or YYYY. The
following table shows some examples of how to use the
various formats. Next, identify the issue or companies of interest. If you know
an identifier, enter it directly. For example, the GVKEY of
START END RESULT Apple Computer is 1690. Given that knowledge, you can set
199609 199612 All data from the beginning of Septemper through December the key to GVKEY and enter 1690 as the identifier.
of 1996
1990 (empty) All data in the year 1990 If you do not know an identifier, click the Find… button to
open the Company Lookup window.
Next, from the Date Display menu, choose either Fiscal
Period (the default) or Calendar.

COMPANY SELECTION

Entities in the CCM Database are accessed by:

‚‚ GVKEY - company identifier Use the search form to locate a particular company in the
‚‚ GVKEY.IID - security identifier CCM database. Companies matching your search criteria
‚‚ GVKEYX - index appear in a table, initially sorted by GVKEY, then by IID. If
your search yields many matches, click the column labels to
Securities linked with CRSP can be accessed by CRSP re-sort the table in the order most useful to you. When you
identifiers also (PERMNO and variations, PERMCO). have found the company you want, click its row in the table,
and then click the Select button. The GVKEY of that
Each GVKEY has associated with it a primary issue. When company will appear in the identifier field.

CRSPSift User Guide • PAGE 30

31 Queries can be run against the full universe by Expand the Compustat data items category (CCM-All
selecting All Issues, or by creating a list of keys and Available) to see categories of Compustat data items.
selecting Input File. To use an input file, create a text
file with an entry for each company, one per row, using Select Subgroups within the categories and view
the active key you selected. For example, to search for items within each of the subgroups. When you select
Apple and Microsoft by GVKEY using an input file, a subgroup, a list of its items appears in the Search
select GVKEY from the Active Key menu. Results area. Further details about the selected item are
shown in the area to the right of the Search Results.
Next create and save the following two line text file in Highlight items to be added to your query and click
the format GVKEY.IID: the Add to Query button. Added items appear in the
Chapter 6: Introduction

Query Items section of the screen.

1690.01
12141.01 KEYSETS

For company and financial statement data, if the IID Keysets are CRSP-created groups of Compustat
is not explicitly provided, the primary issue default will secondary keys that are used to further define
be provided. If security-level data is requested, both Compustat data items. CRSP has identified a default
GVKEY and IID are necessary. Accessing security data keyset for each data item. Sift allows you to override
using an input file without IID will result in an error the default keysets, or select multiple keysets in Sift.
message and the query will return no data.

Once your input file is properly created, Click Input

File in the Company Selection area.

DATA ITEMS AND GROUPS

On the Data Items screen, clicking on the Keyset box
opens a table of all existing keysets for the items with
statistics about each. The default keyset is highlighted
CRSPSift User Guide

upon opening the box and is typically STD, or

Standard. The Keyset number is provided, the number
of GVKEYs, the number of data points, the date range
populated by the data, and a description of the Keyset.

ditional Keysets.
Compustat data may be accessed either by selecting
and combining individual data items in a query, or by
selecting pre-defined groups of data items.

DATA ITEMS
Click on Select/Edit Data Items to open a screen that
offers two categories: Compustat data items and Favorites. Choose different or additional keysets and click the
Select button to close the window.

Keyset Display

Compustat data items can be qualified by a set of

secondary keys. For example, the data item SALE
has secondary keys for industry format, data format,
32 population source, and consolidation level. A different Compustat Expanded Keyset
value of company sales can be available for different
combinations of these keys, such as a combination Each component of the item’s keyset is broken out into
that represents the originally reported sales or the final its own column in output.
restated sales from a later filing.
KYGVKEY CONSOL DATAFMT INDFMT POPSRC FYYYY FYRA SALE

CRSP calls these collections of keys and values a keyset 1690 C STD INDL D 2005 9 13931

and assigns a numeric code and mnemonic tag to each 1690 C STD INDL D 2006 9 19315

unique collection. Each of these represents different 1690 C STD INDL D 2007 9 24006

output series. When multiple keysets are available

FAVORITES
Chapter 6: Introduction

the user can specify both the item and keyset to

identify the series of interest, or use the default preset If you find that you commonly use a certain subset of
combination most commonly used. items, at any time, you may highlight them and click
the Add to Favorites button. Favorites is a user-created
The Keyset Display drop-down menu in CCMQuery’s category designed to house these commonly accessed
Output tab affects how a data item’s keyset is data items. It allows storage of Daily and Monthly
represented in query output. CRSP stock items as well as Compustat data items.

Options include CRSP Mnemonic Keyset Tag, Keyset

Number, and Compustat Expanded Keyset. The
following examples illustrate these options using
Microsoft Excel output.

CRSP Mnemonic Keyset Tag

CRSPSift User Guide

Output represents the item’s keyset using a mnemonic

tag assigned by CRSP.
To control the order in which your report will display
data items, use the up and down arrows to the right of
KYGVKEY KEYSET_TAG FYYYY FYRA SALE
the Query Items list. Click an item and then click the
1690 STD 2005 9 13931
up or down arrow to reorganize the list. To remove an
1690 STD 2006 9 19315
item or group of items, select them from the list and
1690 STD 2007 9 24006
click the X button, located above the arrow buttons.
Keyset Number
Select Global Item Options to include Footnotes or
Output represents the item’s keyset using a keyset Data Code items for those items that have them, and
number assigned by CRSP. to change the default from providing output in its
Reported Currency to converting it to US Dollars.
KYGVKEY KEYSET FYYYY FYRA SALE
1690 1 2005 9 13931
When items and options are selected, click on OK to
1690 1 2006 9 19315
return to the main CCMQuery screen. Selected items
1690 1 2007 9 24006
appear in the Screen.

Note: Global options must be set prior to selecting

data items in order for them to be applied.
33 Additional Descriptions category in the data items
selection screen. There you can add each of the five
items to your query, one by one. That method is best
if you want to modify properties (such as keyset or
output header) of some but not all items.

Alternatively, you could add all five items at once by

selecting Filing Dates under the Groups tab. This is an
IMPORTING DATA ITEMS FROM A FILE efficient way of selecting multiple items if you intend
to apply the same properties to all items in the group.
Instead of choosing data items using the item selection
Chapter 6: Introduction

screen, you can import a set of data items from a text OUTPUT TO SCREEN
file. To do so, click “Import Data Item File…” The file
The default option is text-based, formatted to fit
should contain one data item per line, in the format
the screen width specified in the Width control. 80
itemid.<keyset>. Keyset number is optional; the
characters is the default width. Screen output appears in
item’s default keyset will be used if it is omitted, as for
the hopper at the bottom of the CCMQuery window.
the data item filedate in the following example.

ofid.0
ofcd.0
ofnm.0
filedate
sale.5

Imported items will be added below any items already

present in the Selected Data Items list.
By default, the output of the last query execution is
EXPORTING DATA ITEMS TO A FILE shown in the Current tab. The Save Output and Clear
Click “Export To File…” to export items in the Selected Output buttons at the top of the output area allow you
CRSPSift User Guide

Data Items list to a text file in the format described above. to save output to a text file or to clear it away. Clicking
the History tab shows you the output of every query
GROUPS execution since the session was opened.
Click on the Groups tab to view choices of pre-defined
When you switch to the query history view, the
groups of Compustat data.
buttons above the output area change to Save History
and Clear History. Save History allows you to capture
the output of an entire session. You can select a
different key (GVKEY or PERMNO, for example), or
different data or options related to the same key. Each
time you click Execute the new output is saved in the
history. Then, by clicking Save History, you can save all
the output to one text file.

OUTPUT TO FILE

You can also save data in a number of external file

Groups provide a convenient way to select multiple
formats. When you choose a non-screen format, the
related data items at once. For example, there are
Output File Name field becomes active and is required.
five items related to Filing Dates data. These items
are found under the Filing Dates sub-category of the
34 Click the Browse button to select a location to save
the file.

Clicking one of the output files links opens the File

Download dialog box.
Chapter 6: Introduction

SUPPORTED FILE FORMATS

Pipe(|) Delimited, For Import - pipe-delimited ASCII
text, suitable for importing into databases or software
packages not directly supported by CRSPSift. This
option generates a single output file.

Excel 97-2003 Workbook (*.xls) - Excel version 8.

Note: Excel version 8 has a row limit of 65536 total Click Open to open the application and output file to
rows. If a query generates more than that number of which the data items were written.
rows, the output will be truncated at the row limit.

‚‚ Excel 2007 Workbook (*.xlsx) - Excel XML

‚‚ MAT-files (*.mat) - Matlab version 7

CRSPSift User Guide

‚‚ SAS Dataset (*.sas7bdat) - SAS version 7

‚‚ Stata File (*.dta)

‚‚ SPSS File (*.sav)

VIEWING OUTPUT FILES

If you select a non-screen output format, CCMQuery
will generate one or more output files, one for
each unique output of the query. When the query
completes execution, the output area displays links
to the generated output files. Each file will have the
name specified in the Output File Name field, with the
addition of an extension identifying the type of output
it contains (for example, “fd” for Filing Dates data, or
“adjf” for Adjustment Factors data).
35 OUTPUT FILE SUFFIXES
The following table lists output file suffixes and the
type of data they indicate is contained in a file.

DATA CATEGORY DATA GROUP NAME OUTPUT FILE SUFFIX

Additional Descriptions Adjustment Factors adjf
Additional Descriptions Company History hcom
Additional Descriptions Company History – CST hcst
Additional Descriptions Company Industry Pres ipcd
Additional Descriptions Company Officer Titles off
Chapter 6: Introduction

Additional Descriptions Filing Dates fd

Additional Descriptions GICS History hgic
Annual Data Index Per Desc – Annl xades
Audit Data Audit Data – Annl aaud
Audit Data Audit Data –Qtr iaud
CCM – Common Subsets Company Summary cos
CCM – Common Subsets Period Summary – Annl pa
CCM – Common Subsets Period Summary – Qtr pq
Company Market Data Fortune 500 Data fo
Company Market Data Market Data – Annl amkt
Company Market Data Market Data – Qtr imkt
Description and Links CCM Header and Ranges mstr
Description and Links Company Description com
Description and Links Company Security List secl
Description and Links Index Header in
Description and Links Index Header – pre GICS spind
Description and Links Link History link
CRSPSift User Guide

Description and Links Link Used ulink

Description and Links Link Used Ranges lnkrng
Operating Segment Segment Currency test_sr
Operating Segment Segment Customer test_sc
Operating Segment Segment Detail test_sd
Operating Segment Segment Geographic Area test_sg
Operating Segment Segment Item test_sm
Operating Segment Segment NAICS test_sy
Operating Segment Segment Product test_sp
Operating Segment Segment Source test_ss
Quarterly Data Index Per Desc – Qtr test_xqdes
Security Data Security – Constituents test_hidx
Security Data Security – Dividend FN test_divfn
Security Data Security – Header test_sec
Security Data Security – Header Hist test_hsec
Security Data Security – S&P test_sind
Security Data Security – S&P Constit test_idx
Security Data Security – Split Ev FN test_sptfn
Security Data Security – Split Events test_spt
 Chapter 7: TRSQUERY: TREASURIES DATA ACCESS

CRSPSift’s TrsQuery tool is used to access CRSP Daily and to 1968 are assigned the value OXX. The earliest maturity in
Monthly US Treasuries Data. Individual treasury issues and the file with a CUSIP is February 15, 1969.
a variety of treasury series are available.
CRSPID - The CRSP Issue Identification Number in the
format YYYYMMDD. TCCCCE, where:
QUERY DATE RANGE SPECIFICATION
YYYY Maturity Year
Locate the TrsQuery toolbar below the main CRSPSift toolbar. MM Maturity Month
DD Maturity Day
T Type of Issue (TYPE)
CCCC Integer part of Coupon Rate
Specify a date range. Enter dates using one of the following (COUPRT * 100)
formats: YYYYMMDD, YYYYMM, or YYYY. The following
E Uniqueness Number (UNIQ)
table provides examples of how to use the various formats.
For example, 19850515.504250 identifies a 4¼% callable
START END RESULT
bond which matures May 15, 1985.
199609 199612 all data from the beginning of September through December
of 1996
TREASNOX - CRSP’s Unique US Treasury Index Number
1990 (empty) all data in the year 1990
Identifier. Index TREASNOX’s are 7-digit numbers between
1994 19940615 all data from the beginning of 1994 until June 15, 1994
2000001 and 2000069. The complete list of treasuries
19961231 (empty) data only on the date December 31, 1996
indexes is below:
By default, TrsQuery returns data from the last three
TREASNOX TERM TYPE DESCRIPTION
months of the CRSP database included in your query’s
INDEX FAMILY: RISKFREE
Environment. For more information on Environments, see
RISKFREE
Chapter 8: Environments. 2000001 Risk-Free Rates – 1 Month
2000002 Risk-Free Rates – 3 Month
RISKFREE2
TREASURY SELECTION 2000061 Risk Free 4-Week
2000062 Risk Free 13-Week
SELECTING A KEY TYPE 2000063 Risk Free 26-Week

Select what type of key your query will use from the Choose INDEX FAMILY: FIXEDTERM
FIXED TERM
Key Type drop-down menu in the Treasury Selection area.
2000003 1 Year Bonds, Start Date 1/31/1941
Available keys are:
2000004 2 Year Bonds, Start Date 1/31/1941
2000005 5 Year Bonds, Start Date 4/30/1941
TREASNO - (default) CRSP’s Unique US Treasury Number
2000006 7 Year Bonds, Start Date 4/30/1941
Identifier can be used to access individual issues. TREASNOs
2000007 10 Year Bonds, Start Date 5/31/1941
are 6-digit numbers between 200001 and 206341.
2000008 20 Year Bonds, Start Date 1/31/1942
2000009 30 Year Bonds, Start Date 11/29/1941
CUSIP - Treasury CUSIP. The Committee on Uniform
INDEX FAMILY: TERMSTRUCT
Security Identification Procedures began assigning CUSIP FAMA TERM STRUCTURE
identification numbers in 1968. Issues that matured prior 2000010 Fama 12 Month T-Bills – 1 Month

CRSPSift User Guide • PAGE 36

 TREASNOX TERM TYPE DESCRIPTION TREASNOX TERM TYPE DESCRIPTION
37

2000011 Fama 12 Month T-Bills – 2 Month 2000054 6-Month Certificate of Deposit Rate
2000012 Fama 12 Month T-Bills – 3 Month 2000055 30-Day Commercial Paper Rate
2000013 Fama 12 Month T-Bills – 4 Month 2000056 60-Day Commercial Paper Rate
2000014 Fama 12 Month T-Bills – 5 Month 2000057 90-Day Commercial Paper Rate
2000015 Fama 12 Month T-Bills – 6 Month 2000058 Federal Funds Effective Rate
2000016 Fama 12 Month T-Bills – 7 Month 2000059 Federal Funds Minimum Trading Range
Chapter 7: Introduction

2000017 Fama 12 Month T-Bills – 8 Month 2000060 Federal Funds Maximum Trading Range
2000018 Fama 12 Month T-Bills – 9 Month
2000019 Fama 12 Month T-Bills – 10 Month ENTERING A KEY VALUE
2000020 Fama 12 Month T-Bills – 11 Month Identify the treasury or treasuries of interest. There are
2000021 Fama 12 Month T-Bills – 12 Month
three ways to specify a key value. First, if you know
2000022 Fama 6 Month T-Bills – 1 Month
the key value you want, you can type it directly in the
2000023 Fama 6 Month T-Bills – 2 Month
text field. Another option is to browse the database
2000024 Fama 6 Month T-Bills – 3 Month
sequentially. Use the arrow buttons to the right of
2000025 Fama 6 Month T-Bills – 4 Month
the Find... button to step through the database in the
2000026 Fama 6 Month T-Bills – 5 Month
order of the key type you have chosen. Finally, the
2000027 Fama 6 Month T-Bills – 6 Month
Find… button opens the Treasuries Search window,
INDEX FAMILY: BONDMAPORT
FAMA BOND PORTFOLIO RETURN where you can use a search form to locate a particular
2000028 Fama BondPort Return – 6Mo Range <=6Month treasury in the CRSP database.
2000029 Fama BondPort Return -6Mo Range <=12Month
CRSPSift User Guide

2000030 Fama BondPort Return -6Mo Range <=18Month USING THE TREASURIES SEARCH WINDOW
2000031 Fama BondPort Return -6Mo Range <=24Month
2000032 Fama BondPort Return -6Mo Range <=30Month
2000033 Fama BondPort Return -6Mo Range <=36Month
2000034 Fama BondPort Return -6Mo Range <=42Month
2000035 Fama BondPort Return -6Mo Range <=48Month
2000036 Fama BondPort Return -6Mo Range <=54Month
2000037 Fama BondPort Return -6Mo Range <=60Month
2000038 Fama BondPort Return > 60 <= 120 Month
2000039 Fama BondPort Return > 120 Month
2000040 Fama BondPort Return -12Mo Range <=12Month
2000041 Fama BondPort Return -12Mo Range <=24Month
2000042 Fama BondPort Return -12Mo Range <= 36Month
2000043 Fama BondPort Return -12Mo Range <= 48Month
2000044 Fama BondPort Return -12Mo Range <= 60Month
If you have selected TREASNOX as your query’s
2000045 Reserved for future use – Duplicate of 2000038
key type, you can search by partial or complete
2000046 Reserved for future use – Duplicate of 2000039 TREASNOX, Index Family, or Term Type Description.
INDEX FAMILY: DISCBOND
FAMA-BLISS DISCOUNT BONDS
If you have chosen any of the other three key types,
2000047 Fama Bilss Discount Bonds 1 Year search the Treasuries Search window by one of the
2000048 Fama Bilss Discount Bonds 2 Year following criteria: TREASNO, CRSPID, CUSIP, Date
2000049 Fama Bilss Discount Bonds 3 Year Issued, or Maturity Date.
2000050 Fama Bilss Discount Bonds 4 Year
2000051 Fama Bilss Discount Bonds 5 Year Treasuries matching your search criteria appear in a
INDEX FAMILY: RATES table in the main area of the window. If your search
RATES yields many matches, click the column labels to re-sort
2000052 1-Month Certificate of Deposit Rate the table in the order most useful to you. When you
2000053 3-Month Certificate of Deposit Rate have found the treasury you want, click its row in the
38 table, and then click the Select button. Its TREASNO Note: For complete definitions of data items and
will appear in the identifier field of the Input tab. calculations employed by the CRSP US Treasury
Databases, refer to the Treasury Guide at http://www.
ACCESSING ALL TREASURIES crsp.chicagobooth.edu/documentation.
To access all treasuries in the database, select the All
Treasuries radio button.
Chapter 7: Introduction

USING AN INPUT FILE

Finally, you may choose to access only certain treasuries
specified in an input file. To use an input file, first
create a plain text file (using the Microsoft Windows
Notepad accessory, for instance) with an entry for each
treasury, one per row, using the key type of your choice.
For example, to access four particular treasuries by
TREASNO, create the following four line file:

200007
200014
200158
200161 The TrsQuery Data Items window is divided into
several areas. Items are divided into logical groups in
Save the file in your data environment’s working the Navigate/Search area.
CRSPSift User Guide

directory. Next, Click the Input File option in the

Treasury Selection area, and then click the Browse…
button to locate the input file you created. The Browse
window opens automatically to the working directory.
Now select the preferred key type from the Choose Key
Type menu.

DATA ITEM SELECTION

Use the Data Items tab to control what individual data

items your query will use. Any items you have chosen
appear in the Selected Data Items table.

SELECT/EDIT DATA ITEMS

Clicking this button launches the interface for
selecting individual items in TrsQuery.
39 Clicking the name of a group causes its constituent down arrows to move it up or down the list.
data items to appear in the Search Results area. Or,
rather than browsing by category, click the Search The other functions available are:
tab. There you can enter a descriptive keyword (e.g.,
Edit. Click this to edit an item’s attributes.
“return”) to find items of interest. Navigate and Search
both result in found data items appearing in the Search
Remove the selected item from your query.
Results area of the window.
Chapter 7: Introduction

Remove all items from the Query Items list.

Click the OK button to confirm the addition of the

selected items to your query. Click cancel to exit the
TrsQuery Data Items window without making any
changes.

Import Data Item File...

Clicking this button allows you to import a list of data

items into your query from a user-created text file. A
When you highlight an item in the Search data item file is a plain text file containing a list of
Results area, its attributes and a short text TrsQuery data ItemIDs, one per line. For example, in
description appear in the right panel of the screen. Notepad or another text editor, create a file with the
CRSPSift User Guide

following four lines:

treasno
tname
tdbid
tdask

Save the file as trsimport.txt in your CRSPSift working

directory. Now click the Import Data Item File...
button and select trsimport.txt. The data items listed
in the file will be added your query.

Export to File...
Select each item you want in your query, and click the
Add to Query button to add it to the Query Items area Use this option to output the Selected Data Items list
below the Search Results. to a text file containing each item’s mnemonic ItemID.
This text file may then be imported into other queries
using the Import Data Item File... button.

DATA GROUPS

The Data Groups tab allows you to select clusters of

related data items, saving you the time of selecting and
Items in the Query Items area appear in the order that adjusting a group’s items individually.
they will appear in your query’s output. To adjust the
position of an item, select it and click the blue up and The following data groups are available in TrsQuery.
40 Groups correspond to the item groupings of the tree
navigation structure in the TrsQuery Data Items
window. Some data groups apply to individual
treasury issues, while others apply to treasury series.
The table indicates to which category a group applies.

DATA GROUP AGGREGATE ISSUE

Chapter 7: Introduction

SERIES
DAILY TREASURY ITEMS
By default, the output of the last query execution is
Index Daily x
shown. The Save Output and Clear Output buttons at
Quote Daily x the top of the output area allow you to save output to
Yield Daily x a text file or to clear it away. Clicking the History tab
Fixed Term shows you the output of every query execution since
Fixed Term Header x the query was opened.
Fixed Term Timeseries x
MONTHLY TREASURY ITEMS When you switch to the query history view, the
Fama Bond x buttons above the output area change to Save History
Fama-Bliss Discount x and Clear History. Save History allows you to capture
Fama-Bliss Riskfree x the output of an entire session. You can select a new
Index Monthly x treasury, or different data or options related to the
Quote Monthly x same treasury. Each time you click Execute the new
Term Struct x output is saved in the history. Then, by clicking Save
CRSPSift User Guide

Yield Monthly x History, you can save all the output to one text file.
RATES
Rates Header x OUTPUT TO FILE
Rates Timeseries x
You can also save query output in a number of external
TREASURY DESCRIPTIONS
file formats. When you choose a non-screen output
Daily Debt x
format, the Output File Name field becomes active. In
Debt x
order to successfully execute the query, you must select
Fixed Term x
a location to store the output file. Click the Browse
Rates x
Treasury Index x
button to select a location to save the file.
Treasury Issue x
Treasury Master x

Treasury Payments x

OUTPUT OPTIONS

You can save the output of a TrsQuery in one of

several formats. Click the Output tab to show the
Output Options panel.

OUTPUT TO SCREEN
The default option is text-based output, formatted to
fit the screen width specified in the Width control. SUPPORTED FILE FORMATS
80 characters is the default output width. Screen
‚‚ Pipe(|) Delimited, For Import - pipe-deliimited
output appears in the output area at the bottom of the
ASCII text, suitable for importing into databases
TrsQuery window.
or software packages not directly supported by
41 CRSPSift. This option generates a single output file. Click Open to open the output file in the appropriate
application (Excel, in the example shown below).
‚‚ Excel 97-2003 Workbook (*.xls) - Excel version 8.
NOTE: Excel version 8 has a row limit of 65536 total
rows. If a query generates more than that number of
rows, the output will be truncated at the row limit.
Chapter 7: Introduction

‚‚ Excel 2007 Workbook (*.xlsx) - Excel XML

Alternatively, you can click the View Output button,
‚‚ MAT-files (*.mat) - Matlab version 7 located in the CRSPSift toolbar, to view the output
files generated by your query in Windows Explorer.
‚‚ SAS Dataset (*.sas8bdat) - SAS version 7

‚‚ Stata File (*.dta)

‚‚ SPSS File (*.sav)

VIEWING OUTPUT FILES

If you select a non-screen output format, TrsQuery will
generate one or more output files, one for each unique
output format produced by the query. When the query
completes execution, the output area displays links to
the output files generated. Each file will have the name OUTPUT FILE SUFFIXES
CRSPSift User Guide

specified in the Output File Name field, with the addition The following table lists output file extensions for each
of an extension identifying the type of output it contains. type of treasury data.

DATA GROUP OUTPUT FILE SUFFIX

Treasury Master tmstr
Treasury Issue treas
Treasury Index tidx
Treasury Payments tpaymt
Yield Daily yieldd
Quote Daily quoted
Index Daily indexd
Yield Monthly yieldm
Clicking one of the output files links opens the File Quote Monthly quotem
Download dialog box. Index Monthly indexm
Fama-Bliss Discount fbdisc
Fama-Bliss Riskfree fbrfree
Term Struct term
Fama Bond fbond
Debt debt
Daily Debt ddebt
Fixed Term fterm
Fixed Term Header fh
Fixed Term Timeseries ds
Rates rate
Rates Header rh
Rates Timeseries rs
 Chapter 8: MUTQUERY: US SURVIVOR-BIAS-FREE MUTUAL FUND
DATABASE ACCESS

MutQuery is used to extract data from the CRSP US Survivor-Bias-Free Mutual Funds Database. Access is designed for Fund
data, cross-referencing securities with funds, and for holdings company information. A subsequent release of Sift will also
allow for getting to fund information through a security. Locate the MutQuery toolbar below the main CRSPSift toolbar.

The menu bar for MutQuery is like that of most other Sift tools. The default date range is set for the most recent three
months up through the end date of the database. Specify a date range by entering any one of the following formats:
YYYYMMDD, YYYYMM, or YYYY. The following table provides examples of how to use the various formats.

START END RESULT

199609 199612 all data from the beginning of September through December
of 1996
1990 (empty) all data in the year 1990
1994 19940615 all data from the beginning of 1994 until June 15, 1994
19961231 (empty) data only on the date December 31, 1996

SECURITY SELECTION

From the Choose Selection Type drop-down menu in the Mutual Fund Selection area,
specify the key, such as CRSP FUNDNO or CRSP COMPANY KEY, for use in your
query. In the next version of Sift, the key type that you select will limit your data item
selection to those items that are relevant to the key.

Next, identify the issue or issues of interest. If you know an identifier, you can enter it
directly. If you do not know an identifier, click the Find… button to open the Search
window. Searches for data in the Mutual Funds Database are limited to CRSP FUNDNO,
CRSP PORTNO, and CRSP COMPANY KEY for use as primary key selections.

Use the search form to locate a particular fund, portfolio, or

company in the database. Results of your search criteria appear in
a table, initially sorted by key – FUNDNO for funds and
portfolios, and CRSP COMPANY KEY for securities. If your
search yields many matches, click the column labels to resort the
table in the order most useful to you. When you have found the
entity that you want, click its row in the table, and then click the
Select button. The key of that entity will appear in the identifier
field.

CRSPSift User Guide • PAGE 42

43 You may also choose to access all issues in the database, Items are divided into logical groups in the Navigate/
or issues you specify in an input file. To search all Search area.
issues, click All Issues in the Mutual Fund Selection
area. To use an input file, create a plain text file with
an entry for each issue, one per row, using the active
key of your choice. Select the key type that from the
Active Key menu that corresponds to that used in your
input file and click Input on File. Click the Browse…
button to locate the file you created.

DATA ITEMS
Chapter 8: Introduction

Clicking the name of a group causes its data items to

appear in the Search Results area. Or, rather than
browsing by category, click the Search tab. There you
Data can be selected as either individual items going can enter a descriptive keyword (e.g., “fee”) to find items
through the Data Items tab, or as preselected groups of of interest. Browsing and searching result in data items
logically categorized data by using the Data Groups tab. appearing in the Search Results area of the window.

SELECT/EDIT DATA ITEMS

Clicking on the Select/Edit Data Items tab launches the
interface for selecting individual items in MutQuery.

The MutQuery Data Items window is divided into

several areas.
CRSPSift User Guide

Note:
A future version of the Mutual Funds database will
allow for context when selecting items. Depending on the
access key that is selected – FUNDNO, PORTNO, or
COMPANY KEY, only those categories of data items that
are applicable to the key will be available for selection.

Important to note for this release, COMPANY KEY

access is limited to the Company Info Category located
under Holdings. Through PORTNO, access is available
for Holdings detail data. FUNDNO allows acces to
all mutual funds items, summary data, time series, and
Portno mappings.
44 When you highlight an item in the Search Results area, DATA GROUPS
its attributes and a short text description appear in the
window areas to the right. The Data Groups tab allows you to select clusters of
related event and time series data items.
Select each item you want in your query, and click the
Add to Query button to add it to the Query Items area
below the Search Results.

Available groups include:

Chapter 8: Introduction

Mutual Fund Data

‚‚ Contact Information – Management Company

contact information
‚‚ Dividends – Distribution type and amount,
reinvestment NAV history
Items in the Query Items area appear in the order that ‚‚ Front Load – Load amount, percentage, relevant
they will appear in your query’s output. To adjust the date ranges
position of an item, select it and click the blue up and ‚‚ Fund Fee – Fees and ratios history
down arrows to move it up or down the list. The other ‚‚ Fund Header – Descriptors and flags, Manager,
buttons are: Advisor
‚‚ Fund Header History – History of Descriptors and
Edit. Click this to edit an item’s attributes. flags, Manager, Advisor
‚‚ Fund Style – Flags and classification codes
Remove the selected item from your query. ‚‚ Master Items – Identifier codes, date ranges for daily
and monthly data
Remove all items from the Query Items list.
‚‚ Rear Load – Load type, dollar amount, withdrawal
Click the OK button to confirm the addition of fees data
the selected items to your query. Click cancel to ‚‚ Annual and Quarterly Summary – Snapshots of
exit the MutQuery Data Items window without fund performance-related data, Fund makeup by
CRSPSift User Guide

making any changes. percentage

‚‚ Daily and Monthly Time Series – Daily NAVs and
Import Data Item File... returns, Monthly NAVs, TNA, returns
‚‚ CRSP Portno Map – Identifier mapping of portfolios
Clicking this button allows you to import a list of data associated with a fund with begin and end dates
items into your query from a text file.
Portfolio Data
Export to File...
‚‚ Holdings Detail – Listing of fund holdings, their
Use this option to output the Selected Data Items list
ranks within the fund, market value, shares held, %
to a text file containing each item’s mnemonic itemid.
of fund TNA
This text file may then be imported into other queries
using the Import Data Item File... button.
45 Company Data all the output to one text file.

‚‚ Company Information - Security identifiers, non- OUTPUT TO FILE

equity coupon rates and maturity dates You can also save query output in a number of external
file formats. When you choose a non-screen output
OUTPUT OPTIONS format, the Output File Name field becomes active. In
order to successfully execute the query, you must select
You can save the output of a MutQuery in one of a location to store the output file. Click the Browse
several formats. Click the Output tab to show the button to do so.
Output Options panel.

OUTPUT TO SCREEN
Chapter 8: Introduction

The default option is text-based output, formatted to

fit the screen width specified in the Width control. 80
characters is the default output width.

Helpful Hint:
Many queries on the mutual funds database can result in
large files. Consider output format when extracting data.
The screen default is appropriate for individual funds. Notes about supported Formats
Excel 2007 will be acceptable for many queries, but any
query that is extracting a large universe of funds over a Many queries on the mutual funds database result in
long date range will quickly exceed Excel row limits. SAS is large files. Consider output format when extracting data.
CRSP’s recommended output format for large query results.
The screen default is appropriate for individual funds.

Screen output appears in the output area at the bottom Pipe-deliimited ASCII text is suitable for importing
of the MutQuery window. into databases or software packages not directly
supported by CRSPSift. This option generates a single
output file.

Excel 97-2003 (version 8) has a row limit of 65536 total

CRSPSift User Guide

rows. If a query generates more than that number of

rows, the output will be truncated at the row limit.

Excel 2007 will be acceptable for many queries, but

By default, the output of the last query execution is any query that is extracting a large universe of funds
shown in the Current tab. The Save Output and Clear over a long date range will quickly exceed Excel row
Output buttons at the top of the output area allow you limits and will truncate the results. SAS is CRSP’s
to save output to a text file or to clear it away. Clicking recommended output format for large query results.
the History tab shows you the output of every query
execution since the query was opened. When you VIEWING OUTPUT FILES
switch to the query history view, the buttons above
With the exception of the Pipe Delimited option,
the output area change to Save History and Clear
if you select a non-screen output format, MutQuery
History. Save History allows you to capture the output
will generate one or more output files, one for each
of an entire session. You can select a new security, or
unique data type layout. When the query completes
different data or options related to the same security.
execution, the hopper displays links to the files that
Each time you click Execute the new output is saved in
were generated. Each file will have the name specified
the History tab. By clicking Save History, you can save
46 in the Output File Name field, with the addition of an OUTPUT FILE SUFFIXES
extension identifying the type of output it contains.
The following table lists output file suffixes and the
type of data they indicate is contained in a file.

DATA TYPE SUFFIX

Master Items mast
Header hdr
Header History hist
Contact Information cntct
Clicking one of the output files links opens the File Fund Style sty
Download dialog box. Fund Fee fee
Chapter 8: Introduction

Front Load fload

Rear Load rload
Dividends div
Quarterly Summary qtr_sm
Annual Summary ann_sm
Daily Times Series dly_ts
Monthly Time Series mth_ts
CRSP Portno Map portmap
Holdings Information hld
Company Detail cominfo

Click Open to open the output file in the appropriate

application (Excel 2007, in the example shown below).
CRSPSift User Guide

Alternatively, you can click the View Output button,

located in the CRSPSift toolbar, to view the output
files generated by your query.

If a query generated only one file, the target

application will open the file directly, as shown above.
If the query resulted in multiple output files, however,
instead of opening the files, CRSPSift will open a
Windows Explorer window showing the folder where
you chose to save output.
 Chapter 9: TOOLS

CALENDAR FILE WIZARD Then either specify an end date in the same manner, or choose
to end after a certain number of occurrences. “Occurrences”
When using TsQuery, you can define your own set of dates, refers to the frequency you define in the Frequency area below.
or you can select one of several predefined calendars. To For example, say you define a weekly frequency recurring on
create a custom calendar with only those dates of interest to Wednesdays, and choose to end after twelve occurrences.
you, select “Calendar File...” from the Tools menu. The query report will include data only from the twelve
Wednesdays subsequent to the start date you specified.

FREQUENCY
With the date range in place, define the frequency of dates your
query will access. In the Frequency area, select a frequency from
the list at the left. If your query will use only individual dates
with no regular interval, select None. You can then select those
dates in the Custom Dates area below the Frequency area. If your
query’s calendar will have some regular frequency, select one of
the other options: Daily, Weekly, Monthly, or Yearly. When you
select one, controls appear to the right allowing you to customize
the frequency. Customization options are as follows.

Daily

Specify the frequency of days in the calendar, e.g., every day,

every five days, etc.

CALENDAR OUTPUT FILE

In the Calendar File window, first specify a location to save
the file. Type a path into the Output File text box, or click
Browse... to navigate to the desired directory.

DATE RANGE FOR FREQUENCY Weekly

Next, define the custom calendar’s frequency and the date
range for the frequency. In the Date Range for Frequency Specify the frequency of weeks in your calendar, and select
area, click the start date menu and select a date from the which days of the week will be included in a query report.
popup calendar. For example, a calendar might include only the Monday and
Friday of every third week between the start and end dates.

CRSPSift User Guide • PAGE 47

48 Monthly To use your custom calendar with TsQuery, click on the
Calendar File radio button on the Date tab, browse to
A monthly frequency provides two options. One is the location of your calendar file, and enter the full
Chapter 9: Introduction

to select the frequency of months and specify which path and filename. Other date options on this tab are
day of the month to include, for instance, the tenth superceded by the Calendar File option. To dismiss the
day of every other month. The other also allows you Calendar File window, click the Close button.
specify the frequency of months, the type of day to
include (e.g., Weekday, Weekend Day, or Thursday),
and when that day occurs in the month (e.g., Third REFERENCE DATA
or Last). For example, a calendar might specify the
second weekend day of every month, or the last CRSP STOCK AND INDEX DATA
Thursday of every third month. CRSPSift features built-in reference tables describing
data codes and their values. References can be found
CRSPSift User Guide

under References in the Tools menu. When selected,

tables appear as floating panels. These panels can be
positioned to the side of the main CRSPSift window
or docked to the window. See the Docking section
below for more information.
Yearly
The following reference tables are available in
The yearly frequency offers options similar to those of the
CRSPSift for CRSP Stock and Index data:
monthly. A calendar might include March 3rd of every
year, or the first weekend day in January of every year. ‚‚ Delisting Codes
‚‚ Distribution Codes
‚‚ Missing Return Codes
‚‚ NASD Index Code
‚‚ NASDAQ National Market Indicator
‚‚ NASDAQ Status Code
CUSTOM DATES ‚‚ North American Security Exchange and Index
Codes
In addition to, or in place of, a specified frequency,
‚‚ Security Status and Valid Exchanges
a calendar can include specific custom dates. To add
specific dates, click the menu on the left in the Custom
Dates area. Use the popup calendar control to find The information in these tables can also be found in
and select a date, and then click the Add Date button Chapter 5 of the Data Description Guide: Data Coding
to add it to the list on the right. To remove a date from Schemes.
the list, click the Remove Date button.
COMPUSTAT XPRESSFEED DATA
The following references are available in CRSPSift
for CCM:

‚‚ Reference Codes
‚‚ Reference Numbers
When you are finished defining a custom calendar, ‚‚ Currency & Exchange Rates
click the Run button at the bottom of the Calendar File ‚‚ Economic Indicators
window to create the file. If you would like to view the
generated file, click the View File button in the dialog Reference Codes and Reference Numbers explain codes
box that appears. If not, click the Done button. The found in program output. Currency & Exchange Rates
calendar file is now saved in the location you specified. and Economic Indicator data can be displayed within
49 the reference panel or output to a variety of file formats, DOCKING SUB-WINDOWS
including Excel, SAS, and Matlab. Sub-windows in CRSPSift, such as the Batch Queries
Chapter 9: Introduction

window and the new Stock and CCM References, can

CCM Link References include tables identifying
be “docked” within the main application window. To
codes found in the tables generated by CRSP in the
dock a window click and drag it by its title bar. For
process of linking CRSP and Compustat company and
instance, the Stock Reference title bar:
security-level identifiers. All Reference tables associated
with CCM require a valid data environment that
includes a full CCM database.
As you drag the window, the following five icons
CRSP TREASURIES DATA appear:
The following reference tables are available in
CRSPSift for CRSP Treasuries data:
CRSPSift User Guide

‚‚ First Coupon Payment Date

‚‚ Name of Government Security
The triangle on each outer icon indicates where in the
‚‚ Number of Interest Payments per Year
CRSPSift window the window will be docked with
‚‚ Payment of Estate Tax Code
respect to the query area, left, right, above, or below.
‚‚ Source Daily
Drag the window’s title bar over the desired icon to
‚‚ Source Monthly
dock it in the desired location. Dragging a window
over the central icon will cause the window to appear
CRSP MUTUAL FUNDS
as a tab in the queries area.
CRSP Mutual Funds reference tables are available in
CRSPSift for:

‚‚ Delisting Codes
‚‚ Lipper Asset Codes
‚‚ Rear Load Type
‚‚ Policy
‚‚ Wisenberger Objective Codes

Additional tables and reference information can be

found in the Appendices at the end of the CRSP US
Survivor-Bias-Free Mutual Fund Database Guide.

RESEARCH INDEX DATABASE

The following reference tables are available in
CRSPSift for the Research Index Database:

‚‚ Holding Type
‚‚ Organization Type
‚‚ Price Source
‚‚ Primary Exchange
‚‚ Security Status
‚‚ Share Change Type
‚‚ Share Type
‚‚ Size Band Label
‚‚ Trading Status
50 APPENDIX A: AVAILABLE CURRENCY CODES CURRENCY CODE CURRENCY DESCRIPTION
CHF Swiss Franc
The following is a list of the currency codes (and
Chapter 9: Introduction

CLF Chilean Unidades De Fomento

descripions) available for all currency data (“/ky CLP Chilean Peso
currency”). CNY Chinese Yuan Renminbi
COP COLOMBIAN PESO
CURRENCY CODE CURRENCY DESCRIPTION
CRC Costa Rica Colon
AED United Arab Dirham
CUP Cuban Peso
AFA Afghanistan Afghani
CVE Cape Verde Escudo
ALL Albanian Lek
CYP Cyprus Pound
AMD Armenian Dram
CZK Czech Republic Koruna
ANG Neth. Antillian Guilder
DEM German Deutsche Mark
AOA ANGOLAN NEW KWANZA
CRSPSift User Guide

DJF Djibouti Franc

AON INACTIVE-ANGOLAN NEW KWANZA
DKK DANISH KRONE
AOR INACTIVE-Angolan Kwanza Rejustado
DOP Dominican Peso
ARA INACTIVE-Argentine Austral
DZD Algerian Dinar
ARS Argentine Peso
ECS Ecuador Sucre
ATS Austrian Schilling
EEK Estonian Kroon
AUD Australian Dollar
EGP Egyptian Pound
AWG ARUBAN GUILDERS
ESP Spanish Peseta
AZM Azerbaijan Manat
ETB Ethiopian Birr
AZN AZERBAIJAN MANAT
EUR EURO
BAM BOSNIA & HERZEGOVINA CV MARK
FIM Finnish Markka
BBD Barbados Dollar
FJD Fiji Dollar
BDT Bangladesh Taka
FKP INACTIVE-FALKLAND ISLAND POUND
BEF Belgian Franc
FRF French Franc
BEL INACTIVE-Belgium Financial Franc
GBP POUNDS STERLING
BGL INACTIVE-Bulgarian Lev (Old)
GEL GEORGIA LARI
BGN Bulgarian Lev
GHC Ghana Cedi
BHD Bahraini Dinar
GHS GHANA CEDI (NEW)
BIF Burundi Franc
GIP INACTIVE-GIBRALTER POUND
BMD Bermuda Dollar
GMD Gambia Dalasi
BND Brunei Dollar
GNF Guinea Franc
BOB Bolivian Boliviano
GRD Greek Drachma
BOV INACTIVE-Bolivia Mvdol
GTQ Guatemala Quetzal
BRC INACTIVE-Brazilian Cruzado
GWP INACTIVE-GUINEA-BISSAU FRANC
BRE INACTIVE-Brazilian Cruzeiro
GYD Guyana Dollar
BRL Brazilian Real
HKD Hong Kong Dollar
BRR INACTIVE-Brazilian Cruzeiro Real
HNL Honduras Lempira
BSD Bahamian Dollar
HRD INACTIVE-Croatian Dinar
BTN Bhutan Ngultrum
HRK Croatian Kuna
BWP Botswana Pula
HTG Haiti Gourde
BYB INACTIVE-BELARUS ROUBLE
HUF Hungarian Forint
BYR Belarussian Ruble
IDR Indonesian Rupiah
BZD Belize Dollar
IEP Irish Pound
CAD Canadian Dollar
ILS Israeli Shekel
CDF CONGO (DEM REP) FRANC
INR Indian Rupee
 CURRENCY CODE CURRENCY DESCRIPTION CURRENCY CODE CURRENCY DESCRIPTION
51
IQD Iraqi Dinar NLG Netherlands Guilder
Chapter 9: Introduction

IRR Iranian Rial NOK Norwegian Krone

ISK Icelandic Krona NPR Nepalese Rupee
ITL Italian Lira NZD New Zealand Dollar
JMD Jamaican Dollar OMR Oman Rial
JOD Jordanian Dinar PAB Panama Balboa
JPY Japanese Yen PEI INACTIVE-Peruvian Inti
KES Kenyan Shilling PEN Peruvian Nuevo Sol
KGS KYRGYZSTAN SOM PGK Papua New Guinea Kina
KHR Cambodian Riel PHP Philippine Peso
KMF Comoro Franc PKR Pakistani Rupee
CRSPSift User Guide

KPW North Korean Won PLN Polish New Zloty

KRW South Korean Won PLZ INACTIVE-Polish Zloty
KWD Kuwaiti Dinar PTE Portuguese Escudo
KYD Cayman Islands Dollar PYG Paraguay Guarani
KZT Kazakhstan Tenge QAR Qatari Rial
LAK Laos Kip ROL INACTIVE-ROMANIAN LEU
LBP Lebanese Pound RON ROMANIAN LEU (NEW)
LKR Sri Lankan Rupee RSD Serbian Dinar
LRD Liberian Dollar RUB Russian Ruble
LSL Lesotho Loti RUR INACTIVE-RUSSIAN ROUBLE (OLD)
LTL Lithuanian Litas RWF Rwanda Franc
LUF Luxembourg Franc SAR Saudi Riyal
LVL Latvian Lats SBD Soloman Islands Dollar
LYD Libyan Dinar SCR Seychelles Rupee
MAD Moroccan Dirham SDD Sudanese Dinar
MDL Moldovan Leu SDP INACTIVE-Sudanese Pound
MGF Malagasy Franc SEK Swedish Krona
MKD Macedonian Denar SGD Singapore Dollar
MMK Myanmar Kyat SHP INACTIVE-ST. HELENA POUND
MNT Mongolian Tugrik SIT Slovenian Tolar
MOP MACAO PATACA SKK Slovak Koruna
MRO Mauritania Ouguiya SLL Sierra Leone Leone
MTL Maltese Lira SOS Somali Shilling
MUR Mauritius Rupee SRG Surinam Guilder
MVR Maldives Rufiyaa STD Sao Tome & Principe Dobra
MWK Malawi Kwacha SUR INACTIVE-USSR Rouble
MXN Mexican Nuevo Peso SVC El Salvador Colon
MXP INACTIVE-Mexican Peso SYP Syrian Pound
MYR Malaysian Ringgit SZL Swaziland Lilangeni
MZM MOZAMBIQUE METICALS THB Thailand Baht
MZN MOZAMBIQUE METICAL NEW TJR INACTIVE-Tajik Ruble
NAD Namibia Dollar TND Tunisian Dinar
NGN Nigerian Naira TOP TONGA PA’ANGA
NIC INACTIVE-Nicaragua Cordoba TRL INACTIVE-Turkish Lira
NIO Nicaraguan Cordoba Oro TRY Turkish Lira (NEW)
 CURRENCY CODE CURRENCY DESCRIPTION
52
TTD Trinidad & Tobago Dollar
Chapter 9: Introduction

TWD New Taiwan Dollar

TZS Tanzania Shilling
UAH Ukraine Hryvnia
UAK INACTIVE-Ukraine Karbovanet
UDT INACTIVE-USD Per 1000 Brazilian Shares (IBES)
UGX Uganda Shilling
UNK INACTIVE-Unknown Currency
USD U.S. Dollar
UYP Uruguayan Peso
UYU Uruguayan Peso (new)
CRSPSift User Guide

UZS Uzbekistan Sum

VEB VENEZUELAN BOLIVAR
VEF VENEZUELAN BOLIVAR FUERTE
VND Vietnam Dong
VUV Vanuatu Vatu
WST Western Samoa Tala
XAF CFA (BEAC) FRANC (CENTL AFR)
XCD East Caribbean Dollar
XEU INACTIVE-EUROPEAN COMPOSITE UNIT
XOF CFA (BCEAO) FRANC (WEST AFR)
XPF French Polynesia - C.F.P. Franc
YER Yemeni Rial
YUD INACTIVE-YUGOSLAVIAN NEW DINAR
YUN SERBIA DINAR
ZAL INACTIVE-South African Financial Rand
ZAR South African Rand
ZMK Zambian Kwacha
ZRN INACTIVE-NEW ZAIRE
ZRZ INACTIVE-Zaire
ZWD ZIMBABWE DOLLAR
 Chapter 10: CRSPSIFT OPTIONS

CRSPSIFT OPTIONS - ENVIRONMENT SCANNER

Use the Environment Scanner tool search your computer for CRSP databases and automatically combine them into
valid environments.

FOLDER HINT PATHS

By default, the Environment Scanner looks for databases in a set of pre-defined locations, called Folder Hint Paths. The
Folder Hint Paths provided with CRSPSift are C:\crspdata (the default install location for CRSP databases) and C:\
crspdata\Sample 3.21 (the location of the sample datasets provided with the CRSPSift software). Each time CRSPSift
launches, the Folder Hint Paths are search for newly installed databases. When new databases are found by the Scanner, any
valid environments based on those databases are presented to you to select and add to CRSPSift.

If your databases are installed outside the default locations, you can add your own Folder Hint Paths to guide the
Environment Scanner’s search. Click the Add button next to the list of paths. Next click the Browse button and choose the
directory where you would like the Scanner to search for databases. Click OK to add the directory to the list.

To delete a Folder Hint Path, select it from the list and click the Delete button.

SCAN OPTIONS
The Environment Scanner’s Scan Options allow you to fine tune how its scan is performed. The default behavior for the
Scanner is to search defined Folder Hint Paths, scanning the top level of those folders for valid databases.

CRSPSift User Guide • PAGE 53

54 If you would like to search a directory without adding it to the list of Folder Hint Paths, click the Scan Specific
Location checkbox and then the Browse button to locate it.

SCANNING
After setting the appropriate Scan Options, click the Start Scan button to activate the Environment Scanner.
When the scan begins, the Start Scan button is replaced with the Cancel Scan button. Click this button at any
Chapter 10: Introduction

time during the process to halt the scan.

The time it takes for a scan to complete depends on the number of directories scanned.

NEW ENVIRONMENTS FOUND

When the Environment Scanner completes its search, any new, valid environments it discovered are displayed in
the New Environments Found table at the bottom of the screen. Each environment displays its attributes, such as
its default nickname, full dataset name, and the folder paths of the databases it includes.
CRSPSift User Guide

To add a discovered environments to CRSPSift, click the Create checkbox in its row. If you would like to modify
the environment’s settings before adding it, click the Edit button. See the discussion of CRSPSift Environments
for details on how environment editing options. When you are satisfied with the environment, click the Add
button. The new environment will now be available for use by CRSPSift query tools.

THE ENVIRONMENT SCAN LOG

To see a log of the databases found by the Environment Scanner, click the View Log button in the Scan Options
area. The Environment Scan Log maintains a list of the databases discovered in each run of the Environment
Scanner. The log of the most recent session is shown by default. To view a previous session, select a date and time
from the Session drop-down menu.
55 CRSPSIFT OPTIONS - SERVICES

The Services Options screen displays

technical information about various
subsystems of the CRSPSift application,
Chapter 10: Introduction

including Manager Service and Query Engine

connections. The only editable feature is the
Query Engine Connection Account.

CRSPSIFT OPTIONS - QUERY QUEUE

CRSPSift User Guide

The Query Queue lists and displays details for all batch queries, as well as any open standalone queries that have
been executed at least once during the current CRSPSift session.

SEARCHING FOR BATCH QUERIES

Using the search options, you can filter the list to display only those
queries that match certain criteria, including query owner name and
query status, as well as the dates relating to when a query is scheduled
to execute, when it started execution, and when execution ended.

MANAGING BATCH QUERIES

A drop-down menu at the bottom left of the Query Queue
screen contains actions that you can perform on a batch query.