UMDF MarketDataSpecification v2.1.7 PDF
UMDF MarketDataSpecification v2.1.7 PDF
Version: 2.1.7
Last modified: 2019/08/06
Contacts
• Services Development Department (GDS): handles all enquiries for connectivity setup and
general exchange supported services.
• Trading Certification: performs certification of all software solutions applying for MarkeData
connectivity.
+55 11 2565-5029
• Trading Support Channel: provides real time connectivity monitoring and troubleshooting.
Index
1. PREFACE ............................................................................................................................................ 8
1.1 ABBREVIATIONS ............................................................................................................................. 8
1.2 GLOSSARY..................................................................................................................................... 8
2. TRADING HOURS ............................................................................................................................... 9
2.1 TRADING SESSION HOURS.............................................................................................................. 9
2.2 EXCHANGE HOLIDAYS .................................................................................................................... 9
3. FAST INTRODUCTION ..................................................................................................................... 10
3.1 IMPLEMENTING FAST OVERVIEW .................................................................................................. 10
3.1.1 Templates .............................................................................................................................. 10
3.1.2 Message Structure ................................................................................................................ 10
3.1.3 Data Types ............................................................................................................................ 11
3.1.4 Stop Bit Encoding .................................................................................................................. 12
3.1.5 Data Redundancy Removal .................................................................................................. 13
3.1.6 Templates and Implicit Tagging ............................................................................................ 13
3.1.7 Presence Map (PMAP) .......................................................................................................... 13
3.1.8 Template IDs ......................................................................................................................... 14
3.1.9 The Dictionary Context .......................................................................................................... 14
3.1.10 Field Operators ................................................................................................................. 14
3.1.11 Sequence Numbers and Groups ...................................................................................... 16
3.1.12 The FAST Decoding Process ........................................................................................... 17
3.1.13 Transfer Decoding............................................................................................................. 17
3.1.14 Field Decoding .................................................................................................................. 17
3.1.15 Decoder State Reset for Every Message.......................................................................... 18
3.1.16 Template Implementation Considerations ........................................................................ 18
3.2 REFERENCE SOURCE CODE FOR FAST DECODING ....................................................................... 18
4. LEGACY ELECTRONIC MARKET DATA FEEDS ........................................................................... 19
4.1 BELL (FIX 4.4 OVER TCP) .......................................................................................................... 19
4.2 RLC/MMTP OVER TCP ............................................................................................................... 19
4.3 SDM OVER TCP .......................................................................................................................... 19
5. SYSTEM ARCHITECTURE ............................................................................................................... 20
5.1 MARKET DATA CHANNEL .............................................................................................................. 20
5.1.1 Incremental Stream ............................................................................................................... 21
5.1.2 Snapshot Recovery Stream .................................................................................................. 21
5.1.3 Instrument Definition Stream ................................................................................................. 21
5.1.4 TCP Replayer ........................................................................................................................ 21
5.1.5 TCP Historical Replayer ........................................................................................................ 22
5.1.6 BVMF Legacy Market Data Distribution Diagram ................................................................. 22
5.1.7 TCP Recovery and TCP Historical Replay ............................................................................ 23
5.2 FIX/FAST ENGAGEMENT RULES .................................................................................................. 24
5.2.1 FIX/FAST Templates ............................................................................................................. 24
5.2.2 Network Configuration ........................................................................................................... 25
5.2.3 Market Data Network Contingency Feed .............................................................................. 25
5.2.4 Technical Message Header ................................................................................................... 26
5.2.5 Instrument List Processing .................................................................................................... 27
5.2.6 Initial Market Data Synchronization Procedure ..................................................................... 28
5.2.7 Start of Day Heartbeats ......................................................................................................... 30
5.2.8 Stream Reset Message ......................................................................................................... 30
5.2.9 Channel Reset and Book Reset ............................................................................................ 32
6. RECOVERY ....................................................................................................................................... 35
6.1 SNAPSHOT RECOVERY OVERVIEW ................................................................................................ 35
6.2 TCP REPLAYER AND TCP HISTORICAL REPLAYER OVERVIEW........................................................ 36
6.2.1 Message Level Sequencing .................................................................................................. 39
6.2.2 Instrument Level Sequencing ................................................................................................ 40
7. MARKET DATA ENTRY TYPES ....................................................................................................... 41
2
FIX Market Data Messaging Specification Version 2.1.7
3
FIX Market Data Messaging Specification Version 2.1.7
Revision History
Date Version Description Author
Aug, 6th,2019 2.1.7 - Updated section 11 - Group phase/Instrument State Information. AYSF
Jun, 27th, 2019 2.1.6 - Updated section 10.1 – Trade, indicating new domain to tag 277. AYSF
- Updated section 10.9, indicating tag 37008 only sent for Rejection
Sep, 21st, 2017 2.1.5 AYSF
Auction Bands when PriceLimitType equals to 2.
- Added section 10.11 about volatility prices
Sep, 12nd, 2017 2.1.4 - Removed mention to UMDF 1.6 on section 12, mentioned the addition of AYSF / JLRM
the new volatility price repeating group on section 12.6
- Changed sections 13.1 and 10.1, indicating new behavior for BTB,
Jun, 2nd, 2017 2.1.3 AYSF / JLRM
explained on CL 043/2017-DO.
- Changed section 12.7 indicating tag 75-TradeDate is still being used.
Feb, 21st, 2017 2.1.2 - Changed section 10.9, indicating tag 37003-AvgDailyTradedQty is also JLRM / AYSF
sent for Derivatives, but always with 0.
- Renamed TCP Recovery / Historical Replay to TCP Replayer / Historical
Replayer, since the former ones are going to be decommissioned on
Jan, 11st, 2017 2.1.1 Nov, 2017 JLRM / AYSF
- Added section 5.1.7 explaining the changes and decommission on TCP
Recovery and Historical Replay.
- Unified Derivatives/FX and Equities specification into this single
document
- Reviewed broken links to BM&FBOVESPA new website.
- Updated chapter 4, indicating the legacy feeds that are discontinued.
- Updated diagrams on chapter 5.1.6.
- Updated template links on chapter 5.2.1.
- Separation by Derivatives and Equities on messages from chapter 7
through 10.
- Updated the description of Delete Thru on chapter 9.5.
- Removed chapter 11.5 since derivatives and equities now have similar
behavior on phases and states
Dec, 12nd, 2016 2.1.0 JLRM
- Merged chapter 14 into chapter 12, to have all Derivatives specific
behavior in a single chapter
- Removed legacy information from chapter 14 (now merged with chapter
12).
- Removed note that explained that Co-location customers did not have
access to feed B for incremental messages (the access is normal now).
- Updated sections 6.2 and 13.7 indicating TCP Recovery no longer has a
10000 messages limitation (now it can recover all messages for the
trading day).
- Changed tags 288 and 289 for book and trades for anonymous trading
on chapters 9.1 and 10.1.
- Changed sections 9.1, 10.1 and 13.1 indicating new behavior for BTB
May, 4th, 2016 2.0.15 JLRM
(ex-BTC) Security Lending System.
Mar, 4th, 2015 2.0.14 - Removed section 1.1 (What´s new on UMDF), and every mention of JLRM
UMDF GTS and UMDF MEGA. Also removed mentions of GLOBEX.
- Added note on section 9.1 warning to that the only safe way to handle
book entry references is by using tag 290-MDEntryPositionNo.
Dec, 19th, 2014 2.0.13 JLRM
- Updated sections 5.2.6 and 6.1 clarifying step 6 for Market Initial
Synchronization and Recovery regarding usage of tag 369.
- Updated section 7 changing the definition of VWAP that now includes
cross trades.
Nov, 13th, 2014 2.0.12 RNKH
- Updated section 10.9 removing statement regarding Trading Reference
Price: this tag is only applied to Economic Indicator instruments.
Jun, 10th, 2014 2.0.11 - Clarifications regarding Fixed Income on section 13.2. JLRM
- Typo in section 14.6, where there was a misspelling on the name of tag
326.
- Updated section 16.3 with the correct URL for the Channel Definitions
May, 16th, 2014 2.0.10 JLRM
file.
- Updated section 17 with the correct location for the Message Reference
specification.
- Edited chapter 1.1 removing reference of the release date for Derivatives
Apr, 15th, 2014 2.0.9 JLRM
on UMDF 2.0.
4
FIX Market Data Messaging Specification Version 2.1.7
5
FIX Market Data Messaging Specification Version 2.1.7
Jul, 11th, 2012 1.6.3 - Changed CCB to TSG (Trading Support Group) through the text. JLRM
Jun. 27th, 2012 1.6.2 - Changed contact information to the Trading Support Group JLRM
- Added a diagram describing the full TCP Recovery scenarios on section
May 18th, 2012 1.6.1 6.2. JLRM
- Small corrections on diagram in section 6.2
- Replaced chapter 15 explaining the conversion from RLC to FIX for
PUMA specific impacts
- Added 279=2 for Closing Price on Section 10.5
- Replaced SecurityExchange default value from XBSP to BVMF all over
the document
- Mention on chapter 5.3.3, indicating the MTU for PUMA is 1420 instead
of 1400 for UMDF Legacy
- Renamed TCP Replay to TCP Recovery
- On Section 6.2.2, explain that the customer must wait 10-20ms before
Apr. 5th, 2012 1.6 JLRM
requesting the message on TCP Recovery, to confirm that the message
is indeed lost
- Added section 11.5 directing to Phases and States handling on PUMA
- Added a note on chapter 10 regarding special handling of tag 1500
- On section 11.4, marked tags 625 and 326 on Snapshot (35=W) as non-
required.
- Added chapter 13.4 to describe the behavior of Option Strike Price for
derivatives products.
- Removed references to the GTS system (that has been discontinued).
Sep. 2nd, 2011 1.5.1 - On section 9.1.1, added the tag 37016-MDInserDate for equities market. JLRM
- On sections 15.13 and 15.14, added tag 37016-MDInserDate whilst
translating messages S3 and S4 from RLC.
Aug. 30th, 2011 1.5.1 - On section 3.1.3, corrected the math formula so it displays well in PDF. JLRM
- On section 3.1.3, added a clarification about the encoding of Boolean
values.
Jul. 27th, 2011 1.5.1 - On section 10.7, added Imbalance to the list of statistics that customers JLRM
must delete when an auction is over.
- Fixed diagram on section 5.3.8, changing tag 55 to 48.
- On Section 15.9, added tag 286=1 (session entry).
- Changed the description below the table on section 10.9.
Jul. 1st, 2011 1.5.1 - On section 10.8, corrected tag number for MDEntrySize from 270 to 271. JLRM
May 25th, 2011 1.5.0 - Added TradingReferencePrice handling to section 15.3 JLRM
- Added a note to the table at the end of section 5.3.4, explaining the
times it contains are in local market time.
May 5th, 2011 1.5.0 - Clarified SequenceReset (35=4) usage on all streams (5.3.7) JLRM
- Corrected red box on section 10.2, explaining Trade Volume block
(269=B) is available for derivatives now.
- Updated section 10.1, explaining the new domain ‘3’ for tag 277-
TradeCondition.
- Updated section 5.3.8 explaining that the tag 83-RptSeq won´t be sent
on 269=J (Empty book) blocks
Apr. 20th, 2011 1.5.0 - Removed conflicting template HTTP link JLRM
- In section 15.11, added strategy leg tags from message 63
- Added section 14.3 to explain about ISIN in underlying instruments.
Apr. 16th, 2011 1.5.0 - Section 9.2.2, changed MDUpdateAction. JLRM
- Section 10.5, changed domain of tag 286-OpenCloseSettlFlag.
- In section 10.9, added tag 6939-PriceBandType again.
- In section 15.9, added 286-OpenCloseSettlFlag.
- In section 15.11, added tags 1194-ExerciseStyle, 201-PutOrCall and
37012-PriceDivisor.
- In section 15.12, removed mention to ignore S0 in certain cases.
Feb. 9th, 2011 1.4.2.1 - NewSeqNo for stream reset messages should be 1 not 0. JLRM
Jan. 19th, 2011 1.4.2.0 - Typo for pages 72;73, should be tag 269, not 279 JLRM
6
FIX Market Data Messaging Specification Version 2.1.7
Jul. 27th, 2010 1.4.0.0 - Symbol removed from the spec except from SecurityList. RNKH
- Default SecurityExchange field is changed to “BVMF”.
- Domain of phases and states are changed to be compatible with new
Matching Engine.
- Imbalance and Trade Volume block included.
- Book Reset mechanism changed.
- Derivatives post-trading information included.
- Adjusted section describing the behavior for each marketplace.
May 5th, 2010 1.3.0.1 - Security Status to RLC 07 mapping added. EEW/RNKH/JML
Apr. 20th, 2010 1.3.0.0 - DayCumQty removed: TradeVolume replace it. RNKH
- Include NoMDEntryTypes in the SecurityStatus message do indicate the
entry types to be reset by client systems.
- Closing Price to RLC 5J mapping added.
Jan. 14th, 2010 1.2.2.5 - Included SettlPriceType to incremental and snapshot messages RNKH/TAT
- Included (9 and U) as new values to TradeCondition field
Jan. 13th, 2010 1.2.2.4 - RLC to FIX mapping revised RNKH/TAT
- Included Trading Statistics Reset Flag
- Included NewsSource to News message
- Included DayCumQty to incremental and snapshot messages
Jan. 8th, 2010 1.2.2.3 - Including best description for index related messages RNKH/TAT
7
FIX Market Data Messaging Specification Version 2.1.7
1. Preface
This document outlines the BVMF Unified Market Data Feed (UMDF) specification contemplating the use
of FIX 5.0/FAST protocol over UDP multicast transport and the integration of Equities, Derivatives and
FX, using the consolidated trading platform, called PUMA Trading System for both market segments.
During the transition from the legacy platforms to PUMA Trading System, the legacy feeds will remain
available, but sometime after the migration is concluded, these feeds will be discontinued. Please pay
heed to the Circular Letters, available at BVMF´s website at:
http://www.bmfbovespa.com.br/en_us/regulation/circular-letters-and-external-communications/
BVMF provides this market data feed based on the Financial Information eXchange ("FIX") Protocol. FIX
is a technical specification for electronic communication of trade-related messages. It is an open standard
managed by members of FIX Protocol Limited (http://www.fixprotocol.org/). It is assumed that the reader
of this document has basic knowledge of the FIX protocol.
1.1 Abbreviations
Abbreviation Description
BVMF Bolsa de Valores, Mercadorias & Futuros, or BM&FBOVESPA.
CBOT Chicago Board of Trade
TSG BVMF Trading Support Group.
CFI Code Classification of Financial Instruments Code.
CME Chicago Mercantile Exchange
CME Group – the holding that encompasses the CME, CBOT, NYMEX
CMEG
and other exchanges.
FIX Adapted for STreaming – a specification for data compression to
FAST
reduce bandwidth usage, especially for market data feeds.
FIX Financial Information Exchange Protocol
IP Internet Protocol
SSL Secure Socket Layer
TCP Transport Control Protocol
UDP User Datagram protocol
EQT The Equities segment, previously available as BOVESPA signal.
DER Derivatives and FX segment, previously available as BMF segment.
1.2 Glossary
Term Definition
Securities, Commodities and Futures Exchange, based in São
BM&FBOVESPA Paulo, Brazil. For more information, visit
http://www.bmfbovespa.com.br.
A broker is an individual or firm who acts as an intermediary
Broker
between a buyer and seller, usually charging a commission.
Used interchangeably with broker when referring to a firm rather
Brokerage
than an individual. Also called brokerage house or brokerage firm.
Counterparty Party to a trade.
Direct Market Access – functionality that allows end-customers, such
as hedge funds or investment banks, to directly access the
DMA
exchange electronically without the need to go over physical broker
firm infrastructure.
8
FIX Market Data Messaging Specification Version 2.1.7
Term Definition
Service that provides connectivity to third-party clients and
FIX Gateway
brokerages using the FIX protocol.
Instrument Financial capital in a readily tradable form.
A collective term for quotes, last sales, volume statistics and other
Market Data
information used by the market to evaluate trading opportunities.
The process by which two counterparties that have engaged in a
trade compare the settlement details of the offers provided by both.
Matching
Matching is done to verify all aspects of a trade and ensure that all
parties agree on the terms of the transaction.
Method of forwarding IP datagrams to a group of interested
IP Multicast
receivers.
A stock, bond or contract that has been authorized for trading on,
Security and by, a registered exchange. Each exchange has different criteria
to determine a security's eligibility for listing.
Institution that sells services to its clients. In the context of this
Vendor document, a vendor is an institution that sells access to market data
feeds and order management interfaces to an Exchange.
BVMF´s PUMA Trading System to unify the trading for all exchange
PUMA
products.
2. Trading Hours
http://www.bmfbovespa.com.br/en_us/services/trading/bm-fbovespapuma-trading-system/for-members-
and-traders/trading-hours/
http://www.bmfbovespa.com.br/en_us/services/trading/bm-fbovespapuma-trading-system/for-members-
and-traders/trading-calendar/
9
FIX Market Data Messaging Specification Version 2.1.7
3. FAST Introduction
FIX Adapted for STreaming (FAST) encoding has been developed by the FIX Market Data Optimization
Working Group. FAST is designed to optimize electronic exchange of financial data, particularly for high
volume, low latency data dissemination. This document describes implementation of FAST in receiving
and processing BVMF’s FIX/FAST-encoded electronic market data feed.
The implementation of BVMF’s market data feed is based on the FAST 1.1 specification, available at:
http://www.fixprotocol.org/fastspec
FAST is a data compression algorithm that significantly reduces bandwidth requirements and latency
between sender and receiver. FAST works especially well at improving performance during periods of
peak message rates. FAST extends the base FIX specification and assumes the use of FIX message
formats and data structures.
It compresses data by removing redundant data and doing binary encoding. It does not use general-
purpose, data compressing methods like Lempel-Ziv or arithmetic coding; instead, carefully crafted
templates are used for describing the structure of the messages. High levels of data compression with low
processing overhead and latency can be attained by using FAST.
It is not required that the decoding of a FAST message results in a FIX message; you can streamline your
market data feed processing by creating directly data structures suited to your program, if your FAST
decoder implementation supports it.
3.1.1 Templates
Every FIX message can be described by one or more FAST templates. Each template describes what
fields from the original FIX message are included, and their types and transfer encodings. The templates
are kept in a single XML file that obeys the “FAST v1.1 Template Definition Schema”, included in the
FAST 1.1 specification.
• Header;
• Body;
• Trailer (that is not encoded in FAST).
10
FIX Market Data Messaging Specification Version 2.1.7
FIX Message
Header Body Trailer
MsgSeqNUM
SendingTime
TradeDate
MsgType
(35=X)
(34)
(52)
(75) MDEntry
MDUpdateAction
MDEntryType
SecurityID
Symbol
(279)
(269)
(55)
(48)
...
FAST encoding makes no distinction between Header, Body and Trailer. The template for the message
“X” simply lists the fields, as follows:
11
FIX Market Data Messaging Specification Version 2.1.7
Example fragments of template definitions for fields (not necessarily used in official templates):
• Ascii String
<string name="SecurityID" id="48" />
• Unicode String
<string name="Text" id="58" charset="unicode" presence="optional" />
• Byte Vector
<byteVector name="Text" id="58" presence="optional">
<length name="TextLength" id="59" />
</byteVector>
• Decimal number
<decimal name="MDEntryPx" id="270" presence="optional"/>
FIX has more types, but almost all of them can be easily mapped to FAST data types (like Price →
Decimal). One exception is the UTCTimeStamp type, that’s mapped to an unsigned, 64-bit integer in a
non-standard1 way – just remove all separators of the UTCTimeStamp value (the value must have the
milliseconds part) and convert the resultant decimal string to a number. For instance, if the field
SendingTime (52) has the value 20081007-09:12:08.008 (format YYYYMMDD-HH:MM:SS.sss), encode it
to the integer “20081007091208008”:
Decimal numbers are represented as a pair of integers “mantissa” and “exponent”. For instance, the value
23.45 is 2345×10-2 and it is represented as “2345” and “-2”.
Another exception is the Boolean type that is encoded on FIX within the domain (N, Y). However, BVMF
uses the integers “0” and “1” when encoding it to FAST.
1 BVMF uses the same FAST encoding of a FIX UTCTimeStamp as the CME Group, and does not follow
the tentative FAST 1.2 specification.
12
FIX Market Data Messaging Specification Version 2.1.7
Optional fields are encoded slightly differently from mandatory fields, to take into account the special
value NULL (missing); the details can be found in the FAST specification document. We will show only
the encoding of mandatory fields.
The 8th bit of the last byte (61 hex, 0110 0001 binary) must be set to indicate that it’s the last byte,
so the last byte must be encoded as 1100 0001 binary = C1 hex. The FAST encoding will be:
42 4D 26 46 42 6F 76 65 73 70 C1
i.e.,
07 44 C0
The stop-bit encoding of the integer value 6 is binary 86, so the resulting encoding will be:
86 61 C3 A7 C3 A3 6F
• If you have a template, no metadata information need to be sent (like tags numbers and field
separators);
• Optional fields are usually absent;
• Some fields have constant or default values, and could be omitted;
• In repeating groups, some fields can have repeated or similar values.
13
FIX Market Data Messaging Specification Version 2.1.7
The BVMF encoding process always resets the dictionary for each message, and uses only the “global
dictionary”. See FAST Specification Version 1.1 for more details:
http://www.fixprotocol.org/fastspec
• Constant – The field will always contain a predetermined value. For instance, to encode a
MarketDataIncrementalRefresh message (tag 35=X), the value of tag 35 is constant and always
X, so it can be omitted. (The messages are distinguished by their template IDs.)
14
FIX Market Data Messaging Specification Version 2.1.7
• Default – The field is omitted from the message if it is equal to the default value. For instance, the
field SecurityExchange (tag 207) is usually “BVMF” and can be omitted if the value is exactly
“BVMF”.
• Copy – Omit the field if it was already used with that exact value (usually in a previous repeating
group). For instance, if the field Currency (tag 15) occurs several times in the same FIX message
with the value “BRL”, the first occurrence of that field is sent and the other occurrences are
copies, so they don’t need to be encoded.
• Delta (for numbers) – Encode the difference between the previous value and the current value. It
can save some bytes because smaller numbers are encoded with lesser bytes.
• Delta (for strings) – Encode the “string difference” between the previous value and the current
value. For instance, to encode two fields Symbol, one with the value “BMFBR123456” and the
other with the value “BMFBR789012” (both start with “BMFBR”), encode the binary value “-5” and
the string value “789012”.
• Increment –If the difference between the current value and the previous value is exactly 1 (one),
the field can be omitted.
• Tail – Encode just the “tail” difference. It’s like “delta” but the strings must have exactly the same
length. For instance, to encode the Symbol fields given above (“BMFBR123456” and
“BMFBR789012”), encode just “789012”.
15
FIX Market Data Messaging Specification Version 2.1.7
A “sequence” is a length and an ordered set of FAST groups (a FAST “sequence” is a FIX “repeating
group”). For instance, you can define a “sequence” that lists MDEntries (market data incremental refresh
blocks). You can specify directly the fields, dispensing the FAST “group”.
The terminology is somewhat confusing; just remember, “FAST Sequence” = “FIX Repeating Group”.
<sequence name="MDEntries">
<length name="NoMDEntries" id="268" />
<string name="MDEntryType" id="269">
<default value="0" />
</string>
<decimal name="MDEntryPx" id="270" presence="optional">
<copy />
</decimal>
...
</sequence>
16
FIX Market Data Messaging Specification Version 2.1.7
Encoded FIX/FAST
Message
Network
Transport Layer
Transfer Decoding
Process FIX
Message
1) The FAST Encoder translates the original FIX message into a FAST message.
2) Such message is transmitted (via UDP within a datagram, for instance), and received by the
client system. If the message needed to be split in pieces, the client must join them to get a
complete message.
3) Transfer decoding:
a. Identify template (get the template ID and find the matching template)
b. Extract binary encoded bits
c. Map bits to fields per template field
4) Field decoding: apply operators (like <copy> or <delta>) to determine values per template field.
5) Build/Process FIX Message (optional)
17
FIX Market Data Messaging Specification Version 2.1.7
• Client systems should use the defined sizes and types for each tag in the FIX Message
Specification as a guide for storing data, not just only the FAST template.
• If the structure of the underlying FIX message is changed, a new template will be generated, with
a new ID and BVMF will release a new version of the template XML file.
The field types on the FAST template may be different from the types described on the
FIX Message Specification, as a transport optimization. Always follow the FIX
message Specification when implementing the protocol.
•
Template changes should be handled by client systems without any changes to their
decoder.
The source code comes with absolutely no warranties and is not intended for production use. The
decoder is implemented in C++ and can be compiled by MSVC++ (Windows platform) and gcc/g++
(Unix/GNU platform).
ftp://ftp.bmf.com.br/FIXFAST/reference/FASTDecoder.zip
18
FIX Market Data Messaging Specification Version 2.1.7
19
FIX Market Data Messaging Specification Version 2.1.7
5. System Architecture
The market data systems at BVMF publishes a unified FIX FAST feed. These components are specific to
each market segment (Equities and Derivatives), although their output will be the same from the client
system standpoint.
There are two focal points on this market data architecture: the concept of a “market data channel” –
which defines how the feed is logically distributed according to a set of instruments and level of
information of the book; and the “FIX/FAST engagement rules” – which define the transport of the
information and how the client system should synchronize the data that is provided in the market data
channels.
For contingency purposes, BVMF provides a backup feed that is generated at its contingency site.
The backup feed contains the exact same data that is sent over the primary feed, however with different
connectivity information (different UDP multicast addresses and ports).
Besides the 3 feeds present in the channel, there is a TCP Replayer feed, global to all channels, allowing
the recovery of lost messages.
20
FIX Market Data Messaging Specification Version 2.1.7
If no data is sent through the incremental stream for more than 10 seconds, BVMF will issue a heartbeat
message for maintaining connectivity. If client systems do not receive this message within 30 seconds,
the incremental stream should be considered not functional and the book state should be considered
inconsistent.
Once the books are synchronized and the client starts using only the incremental stream, the client
should unjoin the stream as it would take up unnecessary bandwidth.
The number of snapshots sent in the snapshot recovery stream in one loop could be
less than the number of instruments assigned to the related channel. Client systems
must handle instruments with no snapshots as have empty books and statistical data
before applying incremental data.
The exchange sends more than one instrument in each SecurityList message.
The request specifies a range of messages to be retransmitted. The client system must use an
ApplicationMessageRequest message (tag 35=BW) to request the lost messages in the incremental
stream (UDP channel). For each request, BVMF should send an Application Message Request
21
FIX Market Data Messaging Specification Version 2.1.7
Acknowledgment (tag 35=BX) to report whether the request was accepted or not. After sending a positive
acknowledgment, BVMF should start resending the available requested messages wrapped in one or
more Application Raw Data Reporting messages (tag 35=URDR). To indicate the end of the
retransmission, for each ApplID (channel id) in the request, BVMF sends an ApplicationMessageReport
(tag 35=BY) message.
This method of recovery should only be used if few messages were lost. For late joiners to the market, on
in case of massive loss of messages, the snapshot recovery stream should be used.
There is a global TCP Historical Replayer feed for all channels, and the customer application can choose
to remain connected to either A or B feed through the week (disconnecting during the times the platform
is down for maintenance).
The message format is exactly the same as TCP Replayer, without the maximum message limitation, up
to the entire trading week (SequenceNumber=1). However, a maximum limit of 2000 messages per
request is still enforced.
For the complete flow of messages, refer to the chart on section 6.2.
WARNING:
BVMF does not support the usage of this feed for any other purpose than offline
charting. This feed is not supposed to be used for recovery or for real time market data
consumption.
22
FIX Market Data Messaging Specification Version 2.1.7
UMDF
FIX5.0/FAST UMDF 1.6
UMDF 1.6 Client UMDF 1.6 Converter
(to be discontinued)
PUMA
UMDF
FIX5.0/FAST UMDF 2.0 PUMA
UMDF Client Derivatives
PUMA
For Equities, the default distribution strategy is used (only UMDF 2.0 protocol, with no converter used for
legacy feeds).
On Nov, 2017 all TCP Recovery and Historical Replay session will be decommissioned, hence existing
sessions must be replaced by TCP Replayer and TCP Historical Replayer respectively. Please contact
Trading Support for assistance.
23
FIX Market Data Messaging Specification Version 2.1.7
The templates are all listed within a single XML file. The templates are subject to change by BVMF as the
system evolves and new functionality is added. When a change is done, BVMF will notify market
participants in advance for appropriate development and/or testing efforts.
On the template file header comments the customer can obtain the latest template id used for each
message available. For example:
*Incremental refresh
In this case, the latest template id for message MDIncrementalRefresh (35=X) is 145, with 138 being the
previous template id.
The customer application must be compatible at least with the current and previous
template ids for each message type specified in the template file.
Please contact the TSG (BVMF Trading Support Group) on how to get the latest template information.
In addition, template files are available at the BVMF public FTP site, at the following address:
For Production:
FAST SCP (Session Control Protocol) is not currently used by BVMF to exchange
template files.
24
FIX Market Data Messaging Specification Version 2.1.7
See the documents: Market Data Channels Definition, or contact the TSG for the list of new release,
certification and production multicast streams and TCP Replayer / Historical Replayer connection
information.
Please note that FIX/FAST multicast data is available through the RCB (Rede de Comunicação BVMF, or
BVMF Communications Network).
FIX / FAST Multicast Data is available through the RCB and RCCF2 (low latency
network).
The following diagram illustrates the primary and backup feeds distribution:
Market Market
data feed data feed
Client systems
BVMF suggests customers to sign up for both feeds, to increase stability. In case of disaster, only the
backup feed will be available.
CAUTION!
On PUMA UMDF, both Incremental feeds share the exact same messages, so it´s advised
to connect to both feeds simultaneously for better reliability and avoiding packet losses.
25
FIX Market Data Messaging Specification Version 2.1.7
Each datagram received from client system could contain one or more messages that consist in a set of a
not encoded technical header and the payload: a FAST encoded message. It is illustrated below:
UDP datagram
Technical header
FAST encoded FIX message
(unencoded)
CurrentChunk
MsgSeqNum
MsgLength
NoChunks
binary
• Allow the client system to detect sequence number gaps before decoding the message;
• Allow for breaking-up of large messages and re-composition (e.g. market data snapshots of
order depth-books may be very deep – e.g. over 100 entries for each side, bid and ask).
Before each received FIX/FAST message from UDP feed, there is the following sequence of bytes
defining a header (blue rows):
All attributes defined in the header is in “big-endian” convention, where bits and bytes are in network byte
order, where high order bits precede low order bits, and high order bytes precede low order bytes.
26
FIX Market Data Messaging Specification Version 2.1.7
MsgSeqNum – this attribute contains the same value as in the Tag 34-MsgSeqNum.
NoChunks – total number of chunks that constitutes a single FIX/FAST Message identified by
MsgSeqNum in the channel at the current trading session.
CurrentChunk – the current position of the chunk of data that constitutes a single FIX/FAST Message
identified by MsgSeqNum in the channel at the current trading session.
MsgLength – The length of the following sequence of bytes that constitutes a chunk of data.
Client systems need to reassembe all chunks of data with same MsgSeqNum in the correct order to have
a valid FIX /FAST encoded data before decoding it.
From this point on, the instrument database on the client side may be populated. Each SecurityList
message needs to be processed until the count of instruments of that channel (tag 393-
TotNoRelatedSym) is fully received. The last message in the loop will contain tag 893-LastFragment set
to ‘Y’.
Note that a SecurityList message may contain more than one instrument definition. Deleted or expired
instruments are not sent over the instrument definition stream. For deletion of instruments, the application
must process the SecurityList message sent over the incremental stream.
The following diagram illustrates correct client system processing of the instrument definition stream:
27
FIX Market Data Messaging Specification Version 2.1.7
BVMF will start issuing instrument definition messages in the instrument definition stream using the
following schedule (all times are local unless stated otherwise):
In general, for PUMA Trading System, customers may connect every day or keep connected through the
week. However, BVMF recommends that customers remain disconnected during the weekends, unless
when participating in scheduled mock tests.
28
FIX Market Data Messaging Specification Version 2.1.7
1. Contact the TSG or visit the BVMF FTP server to get the latest configuration parameters and
template files;
2. Connect to the TCP Replayer service. In case of missing packets on the incremental stream,
they can be recovered using this service;
3. Join the multicast address/UDP port of the incremental stream and start receiving the market
data incremental messages. Queue them;
4. Join the multicast address/UDP port of the security definition stream until all instruments have
been received (monitor the tag 393–TotNoRelatedSym);
5. Unjoin the security definition stream, to avoid consuming unnecessary bandwidth;
6. Join the multicast address/UDP port of the snapshot recovery stream until all snapshot
messages have been received: monitor the tag 34-MsgSeqNum whose value is cyclical and
the tag 911-TotNumReports = total number of snapshots in the current loop. Client systems
could receive and queue snapshots until total number of snapshots received and stored is
equal to the value of field TotNumReports field (tag 911) of the last snapshot message
received and the older incremental data queued is greater than the next sequence of the
lowest value of LastMsgSeqNumProcessed field (tag 369) of all snapshots stored;
7. Unjoin the snapshot recovery stream, to avoid consuming unnecessary bandwidth;
8. Start by removing from the queue the incremental stream messages applying over related
snapshots until consuming all the queued messages: discard queued 35=X and 35=f
messages from the incremental stream until tag 34-MsgSeqNum in the message has the
same value as tag 369-LastMsgSeqNumProcessed in the snapshot for each instrument in the
channel. The discarded messages contain information that was already included in the
related snapshot message; Do not remove from the queue messages of type 35=y
(SecurityList) and 35=B (News), as they are not reflected on the received snapshot.
9. Start normal processing with incremental messages.
NOTE
The number of snapshots sent in the snapshot recovery stream in one loop could be less
than the number of instruments assigned to the related channel. Client systems must
handle instruments with no snapshots as have empty books and statistical data before
applying incremental data.
29
FIX Market Data Messaging Specification Version 2.1.7
The following diagram illustrates the graphical representation of the steps listed above.
This message is issued in case of a severe failure in the exchange market data system, or regular start-
up. This message will be sent individually for each site, i.e. if the failure occurs in the primary site, only
that channel in the primary site is affected, likewise for the backup site.
This message is also sent on security definition and snapshot recovery streams when they are starting or
just after a loop is finished to indicate a new loop is commencing. The stream reset is the Sequence
Reset message (tag 35=4) with NewSeqNo field (tag 36) = 1 (set new sequence number).
30
FIX Market Data Messaging Specification Version 2.1.7
• Consider that the application sequence number has been reset, and should be started from the
value in NewSeqNo field;
• Resynchronize their order books according to the snapshot recovery stream, as if it were a start
of day synchronization.
IMPORTANT
TCP Replayer is not available for messages prior to a Sequence Reset
message in that channel.
IMPORTANT
The Sequence Reset message resetting the market data stream is sent at the
startup of the market data component, regardless of failure or regular
startup.
31
FIX Market Data Messaging Specification Version 2.1.7
35=X, 34=2
.
.
.
In case of component failure, BVMF will issue a market data incremental message with an entry type ‘J’
(Empty Book) without any instrument identification to notify client systems of book reset event. This
message will also not contain tag 83-RptSeq.
The steps to detect the Channel Reset condition and proceed to recovery process are shown below:
1) The Market Data Incremental Refresh (tag 35-MsgType = X) message, Channel Reset data block is
sent down the Incremental feed with tag 269-MDEntryType = J to indicate that there has been a
component failure and books of all instruments on the channel are corrupted;
2) The client system must empty books of all instruments related to the impacted channel;
3) The Market Data Snapshot Full Refresh (tag 35-MsgType = W) message on the Snapshot Recovery
feed will be deleted for all impacted instruments;
4) Market Data Incremental Refresh (tag 35-MsgType = X) messages will be sent at the incremental
stream to populate the book of all instruments:
a) The first incremental Market Data Incremental Refresh message have the first repeating group of
type ‘J’ containing the instrument identification indicating the starting of recovery process for the
given instrument (only sent for instruments that previously had a book);
32
FIX Market Data Messaging Specification Version 2.1.7
b) Each of recovered book entry is identified by the presence of tag 276-QuoteCondition = ‘R’. The
tag 83-RptSeq fields also reset for the subsequent messages/repeating groups for the related
instrument.
5) Once a book for a specific instrument has been recovered, BVMF will disseminate incremental real-
time market data for that instrument (book entry will not have tag 276-QuoteCondition = ‘R’), but other
instruments on the channel may still be going through the recovery process.
Below is the sequence diagram to illustrate the full order book reset dynamics:
In addition, BVMF can send a Book Reset notification for specific instrument through issuing a market
data incremental message with an entry type ‘J’ (Empty Book) with instrument identification.
The steps to detect Book Reset condition and proceed to recovery process are shown below:
1. The Market Data Incremental Refresh (tag 35-MsgType = X) message, Book Reset data block is
sent down the Incremental feed with tag 269-MDEntryType = J and the instrument identification
component to indicate the book of this instrument is corrupted;
2. The client system must empty the book of related instrument;
33
FIX Market Data Messaging Specification Version 2.1.7
3. The Market Data Snapshot Full Refresh (tag 35-MsgType = W) message on the Snapshot
Recovery feed will be deleted for the impacted instrument;
4. Market Data Incremental Refresh (tag 35-MsgType = X) messages will be sent at the incremental
stream to populate the book of the specific instrument, where each of recovered book entry is
identified by the presence of tag 276-QuoteCondition = ‘R’. The tag 83-RptSeq fields also reset
for the subsequent messages/repeating groups for the related instrument.
5. Once a book for a specific instrument has been recovered, BVMF will disseminate incremental
real-time market data for that instrument (book entry will not have tag 276-QuoteCondition = ‘R’).
Below is the sequence diagram to illustrate the order book reset for a specific instrument dynamics:
34
FIX Market Data Messaging Specification Version 2.1.7
6. Recovery
BVMF offers some options for recovering missed messages or synchronizing client systems to the latest
state: TCP Replayer and Snapshot Recovery.
Message loss is detected using the message sequence number present in the message header and tag
34-MsgSeqNum in the decoded incremental FIX message. This attribute is an incrementing number. If a
gap is detected between messages in tag 34-MsgSeqNum, this indicates a group of messages have been
missed. It should be assumed at this point that all books maintained in the client system may no longer
have the correct, latest state maintained by BVMF. Client systems must resynchronize all books to the
latest state maintained by BVMF. During this synchronization process, all books are initially assumed to
be in an incorrect state and are recovered during the synchronization process.
NOTE
UDP protocol cannot guarantee the order of packets to be maintained, thus
the customer application may receive packets out of order, and must be able
to handle that gracefully.
IMPORTANT
Before requesting a message that is presumed to be lost on the TCP
Replayer / Historical Replayer feed, the customer application must wait 10-
20ms to be sure that the message is indeed lost (not just out of order).
Some considerations:
1. Client systems should queue real-time data until all snapshot data is retrieved from a given
channel. After this, the queued data should then be applied as necessary, where all queued
incremental message with tag 34-MsgSeqNum less or equal than the value of tag 369-
LastMsgSeqNumProcessed of processed snapshot should be discarded.
2. BVMF strongly recommends that the Snapshot Recovery streams be used for recovery
purposes only. Once client systems have retrieved recovery data, client systems should
stop listening to the Snapshot Recovery streams.
Recommended procedure for recovering:
35
FIX Market Data Messaging Specification Version 2.1.7
3. Join the multicast address/UDP port of the snapshot recovery stream until all snapshot
messages have been received: monitor the tag 34-MsgSeqNum whose value is cyclical and
the tag 911-TotNumReports = total number of snapshots in the current loop. Client systems
could receive and queue snapshots until total number of snapshots received and stored is
equal to the value of field TotNumReports (tag 911) of the last snapshot message received
and the older incremental data queued is greater than the next sequence of the lowest value
of LastMsgSeqNumProcessed (tag 369) of all snapshots stored;
4. Start by removing from the queue the incremental stream messages applying over related
snapshots until consuming all the queued messages: discard queued 35=X and 35=f
messages from the incremental stream until tag 34-MsgSeqNum in the message has the
same value as tag 369-LastMsgSeqNumProcessed in the snapshot for each instrument in
the channel. The discarded messages contain information that was already included in the
related snapshot message; Do not remove from the queue messages of type 35=y
(SecurityList) and 35=B (News), as they are not reflected on the received snapshot.
5. Unjoin the snapshot recovery stream, to avoid consuming unnecessary bandwidth;
6. Start normal processing with incremental messages.
The same protocol is used for TCP Historical Replayer, with small differences on the scope of what can
be recovered, see section 5.1.5 for more details.
• The Application Message Request Acknowledgment (tag 35=BX) message is sent to confirm the
receiving of the Application Message Request (tag 35=BW) message. The ApplRespType field
(tag 1348) contains the type of acknowledgment being sent. The requested messages are resent
only when the value of this tag is “0” (Request accepted) or “1” (Request partially accepted), for
the later not all of the messages are resent, in this case the client application must iterate through
all the NoApplIDs (tag 1351) instances to check the presence and value of the ApplRespError
field, which has the reason for the error related to a specific RefApplID (tag 1355). The other
values (greater than 1) for ApplRespType indicate Negative acknowledgment and the client
application should verify and treat the error (see the message specification for more details).
• The Application Raw Data Reporting (tag 35=URDR) message is a BVMF user defined message
created to encapsulate and make feasible the transportation of the FAST encoded messages
(binary data) over a regular FIX 4.4 session using a TCP connection. The RawData field (tag 96)
contains one or more FAST encoded messages. The NoApplSeqNums field (tag 10054) is the
repeating group that contains the list of the message sequence numbers and related offset/length
to retrieve each message in the RawData field (tag 96). The ApplLastSeqNum field (tag 1350)
can be used to detect gaps (i.e., a sequence reset during the trading session). See the message
specification for more details.
• The Application Message Report (tag 35=BY) message is used to indicate that the application
resend process is complete or was interrupted because of an error. The ApplReportType field
(tag 1426) reports whether the resending was successfully completed (value=3) or there was an
error (value=4). A separate Application Message Report message is issued for each channel ID
36
FIX Market Data Messaging Specification Version 2.1.7
in the request. Thus, in all messages of this type, NoApplIDs field (tag 1351) is always equal to 1.
The field RefApplID (tag 1355) identifies the channel ID (incremental stream) being reported. This
message might be sent immediately after the Application Message Request Acknowledgment
(tag 35=BX) message (if an error occurs and messages cannot be resent), or just after the
resending of the last Raw Data Reporting message for the related channel ID. Client application
must always check the presence of the ApplRespError field to detect error occurrence and the
field’s value to know the error reason.
Some considerations:
1. TCP Historical Replayer may retrieve messages from the beginning of the current trading week
up to the request time of the current trading session. In the next week, MsgSeqNum is reset and
the messages from the previous week become unavailable.
2. TCP Replayer may retrieve messages from the beginning of the current trading session up to the
request time of the current trading session. Messages from the previous trading sessions are not
available.
3. The replayed messages from the current trading session are available until the next weekly
trading session starts. After that, MsgSeqNum is reset and the old messages become
unavailable.
4. BVMF strongly recommends that the client application should keep connected to TCP Replayer
during the whole trading session (establishing a connection for each request is not efficient and is
not recommended).
5. This type of connection conforms to FIX Session layer standard defined by FPL, but Application
Message Request (tag 35=BW), Application Message Report (tag 35=BY), Application Message
Request Acknowledgment (tag 35=BX) and Application Raw Data Reporting (tag 35=URDR).
6. Concerning the URDR (BVMF Raw Data Reporting) message, The FAST encoded messages
appended in the RawData (tag 95) field do not contain the header that is sent in the incremental
stream for fragmentation/reassembly purposes. After correctly extracting a message from the
RawData (tag 95) field using RawDataOffset (tag 10055) and RawDataLength (tag 96), the client
application can immediately submit it to the application FIX/FAST decoder routines to obtain the
final FIX.5.0SP2 MarketDataIncrementalRefresh (tag 35=X), SecurityList (tag 35=y),
SecurityStatus (tag 35=f), News (tag 35=B) and Heartbeat (tag 35=0) messages.
7. BVMF expects that the adopted FIX engine at the client application side to take care of all FIX 4.4
session layer routines (i.e., the sending of heartbeat messages during the periods of inactivity)
8. BVMF recommends to reset the sequence numbers on every logon (client application should
send the logon message with tag 141-ResetSeqNumFlag=Y).
9. Retransmission from the session level is not implemented at BVMF's side; all Resend Request
messages (35=2) are responded with a Sequence Reset (35=4 with Gap Fill). Thus, the client
application should not rely on retransmissions at the session level because this feature isn´t
available through the TCP Replayer connection.
The following sequence diagram describes an example of a successful scenario for the TCP Replayer
process:
37
FIX Market Data Messaging Specification Version 2.1.7
Client System
Heartbeat (35-MsgType=0)
Heartbeat (35-MsgType=0)
Process Request()
Application Message Request Ack (35-MsgType=BX) stating that the request is invalid
Process Request()
Application Message Request Ack (35-MsgType=BX) stating accepted request
[For each ApplID] Send [0..N] Raw Data Reporting messages (35-MsgType=URDR)
[For each ApplID] [after sending all URDR messages] send the Application Message Report (35-MsgType=BY)
WARNING
After the message is sent on the incremental feed, it usually takes 25ms on
average for it to become available on the TCP Replayer cache. In extreme
conditions, messages may take much more than that to become available,
therefore, in case of an unsuccessful recovery attempt, it is recommended
at least three extra attempts (with a 140ms pause) before resorting to a full
snapshot recovery.
NOTE
When using TCP Replayer / Historical Replayer, since both feeds are
identical, it´s preferable to use feed A always, unless this feed is not
available. Only then should the application connect to feed B.
The complete map of possible scenarios and message flow when using TCP Replayer is described in the
following diagram:
38
FIX Market Data Messaging Specification Version 2.1.7
There are two strategies that client systems can apply to determine whether this is the moment to use
TCP Replayer:
Please note that on the PUMA Trading System, as incremental feeds A and B both have messages with
synchronized MsgSeqNums, the customer is advised to check feed B before proceeding to request the
missing message from TCP Replayer. Even after the message is published on the incremental feeds, it
may take about 25ms for it to become available on TCP Replayer.
39
FIX Market Data Messaging Specification Version 2.1.7
Client systems can keep track of the instrument sequence number (tag 83-RptSeq) for every instrument
by inspecting incoming data and determining whether there is a gap in the instrument sequence number
(tag 83-RptSeq).
• If there is a gap in the instrument sequence number (tag 83-RptSeq), it indicates that data was
missed for the instrument when message loss occurred.
• If there is no gap, the data can be used immediately, and it also indicates that the book for this
instrument still has a correct current state.
It is likely that if only a small number of messages have been missed, there will be data in subsequent
messages which are not affected by the missing data. If there are 100 instruments in a channel, for
example, and the missed messages contain data for 4 of these instruments, any subsequent messages
containing data about the other instruments (not affected by message loss) are still valid.
40
FIX Market Data Messaging Specification Version 2.1.7
PVWAP =
QP
j j j
Q j j
Where:
PVWAP = Volume Weighted Average Price
Pj = price of trade j
Qj = quantity of trade j
j = each individual trade that takes place over
the defined period of time (including cross
trades).
A Imbalance X X Information related to imbalance of auctions
such as side and quantity.
B Trade volume X X The total volume traded for that security in the
trading session.
41
FIX Market Data Messaging Specification Version 2.1.7
After the start of day procedure to retrieve the list of instruments, any new updates will be sent over the
incremental stream of the market data channel. These updates will be available in the TCP Replayer
functionality as well.
Updates to the instrument definitions will also be reflected in the instrument definition stream for late
joiners, however client systems that have already constructed their instrument database as per the start
of day procedure should rely on the incremental stream updates instead.
The following sections illustrate the three possible types of instrument updates that will be sent over the
incremental channel.
42
FIX Market Data Messaging Specification Version 2.1.7
.
.
NOTE
After an instrument is created during a session, before receiving the first
35=f (SecurityStatus) message referring that instrument, its phase and state
are to be considered invalid and unknown.
.
.
43
FIX Market Data Messaging Specification Version 2.1.7
.
.
The actions to be executed by the client system receiving the incremental message are determined by tag
279-MDUpdateAction, whose value carries an instruction that can be either add, delete, change, delete
from, delete thru or overlay. The items in the order book that are affected by the action stated in tag 279
are stated in tag 290-MDEntryPositionNo, which contains a position in the order book.
For bid or offer book entries (order and price depth book), the deletion is based on the entry’s position
(tag 290-MDEntryPositionNo). For example, assume ten bids for a security. Adding a bid with 290-
MDEntryPositionNo = 4 requires the receiver to shift down other Market Data Entries, i.e. the Market Data
Entry in the 4th display position will shift to the 5th, the 5th shifts to the 6th, etc. until the 10th shifts to the
11th. BVMF will not send a modification of all entries in the 4th through 10th positions just to update the
290-MDEntryPositionNo field; the receiver of the market data must infer the change.
Similarly, deleting a Market Data Entry in the 7th position causes the 8th Market Data Entry to move into
the 7th position, the 9th to shift into the 8th position, etc. BVMF will not issue a change action to modify the
position of an entry in the book. Change updates are only sent when a value applicable to a specific tag
290-MDEntryPositionNo – such as total quantity or number of orders – changes.
BVMF publishes two types of book depth: order depth and price depth using the same MDEntryType: 0
(Bid) and 1 (Offer). To determine which type of book is currently defined in a given channel, see
“FIX/FAST Channel Definitions” documents on the website or from tag 264-MarketDepth in the Market
Data Snapshot Full Refresh (tag 35=W) message for each instrument: if it is absent, the book is order-
depth based, if present, it is price-depth based and the level is determined by the value of the tag where
the value 1 (one) indicates top of book.
44
FIX Market Data Messaging Specification Version 2.1.7
IMPORTANT
The book could be order-depth based or price-depth based depending on
channel parameters definition. Please see “FIX/FAST Channel Definitions”
documents to determine which type of book each channel supports.
Bid Offer
PosNo Size Px Px Size PosNo One entry per order: same
1 5000 10.58 11.03 7000 1 price on more than one
2 4000 10.58 11.03 2000 2 entry.
3 3000 10.57 11.05 1000 3
4 4000 10.54 4
BVMF provides the full depth of the book for order-depth book, i.e. the client will always receive updates
for all the orders that are in the order book, even if it is the last one (worst price).
In general, if a trade occurs, BVMF will send a delete or change data block to update the book. The trade
data block itself is not used to update the order book.
Below are the data blocks sent for order depth book update:
83 RptSeq X X X
48 SecurityID X X X
22 SecurityIDSource X X X
207 SecurityExchange X X X
271 MDEntrySize X X X
272 MDEntryDate X X X
273 MDEntryTime X X X
37016 MDInsertDate X X X
37017 MDInsertTime X X X
45
FIX Market Data Messaging Specification Version 2.1.7
290 MDEntryPositionNo X X X
37 OrderID X X X
For more details, please check the UMDF Message Reference document.
IMPORTANT
The only safe way to treat references to an entry in the book is by using tag
290-MDEntryPositionNo. Using other tags such as 37-OrderID is highly
disregarded and can lead to book inconsistencies.
Bid Offer
PosNo NoOrders Size Px Px Size NoOrders PosNo One entry per
1 2 9000 10.58 11.03 9000 2 1 price: more
2 1 3000 10.57 11.05 1000 1 2 than one order
3 1 4000 10.54 3 per entry.
In addition to the quantity and the price, the price-depth book also makes the number of orders that
compose a specific price available. BVMF presets the depth of the book that will be made available per
instruments, usually defaulting to 5. Client systems must determine the book-depth for an instrument from
tag 264-MarketDepth in the Market Data Snapshot Full Refresh (tag 35=W) message.
BVMF sends an add data block if there is a new price level. Client systems should then shift price levels
down, and delete any price levels past the defined depth of the book as indicated in tag 264-Market-
Depth in the Market Data Snapshot Full Refresh (tag 35 =W) message.
The change data block is sent to update characteristics of a price level without changing the price itself, or
impacting any other prices on the book (update to the order count or quantity at that price).
46
FIX Market Data Messaging Specification Version 2.1.7
New buy order is received (BUY 1000 @ 10.60), updating the top of the book (bid):
Bid
PosNo NoOrders Size Px New bottom row of
1 1 1000 10.60 the book.
2 2 9000 10.58
3 1 3000 10.57
Implicit deletion of
4 1 4000 10.54
the previous bottom
5 4 10000 10.53
row.
6 3 8000 10.50
The order with price 10.57 is deleted (CANCEL BUY 3000 @ 10.57):
MDEntryPositionNo 5
MDUpdateAction New
MDEntrySize 8000
MDEntryPx 10.50
NumberOfOrders 3
So, the book will miss the last row until the insert at the last position:
Bid
PosNo NoOrders Size Px
1 1 1000 10.60
2 2 9000 10.58
3 1 4000 10.54
4 4 10000 10.53
New bottom row will be sent by BVMF:
47
FIX Market Data Messaging Specification Version 2.1.7
Bid
PosNo NoOrders Size Px
1 1 1000 10.60 New bottom row will
2 2 9000 10.58 be sent by BM&F.
3 1 4000 10.54
4 4 10000 10.53
5 3 8000 10.50
For information on data blocks sent for price depth book update, please check the UMDF Message
Reference document.
This information is represented by a price depth book with market depth = 1 (as described at the previous
sub-topic: Price depth book), and is largely used at some client systems for comprehensive overview of
market data behavior of several instruments at same time.
This book management mode makes use of the Overlay method (see below) when updating the sole
price level for each book.
When an order is entered that causes several executions and sweeps the order book, causing several
price levels to be deleted, instead of sending deletions for several price levels, the MDUpdateAction
“Delete From” (tag 279 = 4) is used. It indicates that all positions from the position stated in tag
MDEntryPositionNo up until position 1 must be deleted. This will cause the market data entry that was in
position MDEntryPositionNo + 1 to be the first position now.
Bid Offer
PosNo Size Px Px Size PosNo
1 5000 10.58 11.03 7000 1
2 4000 10.58 11.03 2000 2
3 3000 10.57 11.05 1000 3
4 4000 10.54 4
A sell order is sent with quantity 12000 and price 10.57, which executes against the 3 existing buy orders
in the book. BVMF will send an incremental market data message with the following characteristics:
48
FIX Market Data Messaging Specification Version 2.1.7
Bid Offer
PosNo Size Px Px Size PosNo
1 4000 10.54 11.03 7000 1
2 11.03 2000 2
3 11.05 1000 3
Bid Offer
PosNo Size Px Px Size PosNo
1 5000 10.58 11.03 7000 1
2 4000 10.58 11.03 2000 2
3 3000 10.57 11.05 1000 3
4 4000 10.54 4
The market supervisor decided to cancel all bid entries, so BVMF will send an incremental market data
message with the following characteristics:
Bid Offer
PosNo Size Px Px Size PosNo
1 11.03 7000 1
2 11.03 2000 2
3 11.05 1000 3
9.6 Overlay
Another possibility for book update is the overlay method (tag 279-MDUpdateAction=5). This mode
indicates that the price level should be replaced or inserted not implying an inconsistency when that price
level is not defined. For example:
Please note that an Overlay update can be sent without containing tag 270-MDEntryPx, meaning that
price level should be removed.
49
FIX Market Data Messaging Specification Version 2.1.7
For more details on each of the blocks and the message structure, please refer to UMDF Message
Reference document.
IMPORTANT
Whenever handling trades and other real-time statistics, if the tag 1500-
MDStreamID is present, this data must be stored separately, as they may
contain information from different venues.
10.1 Trade
The trade data block is sent when a trade occurs to provide volume and trade statistics.
When a cross order is accepted in the trading system, the related market data contains a trade, tag 269-
MDEntryType = 2 (trade) with tag 277-TradeCondition containing character ‘X’.
If a repeating group with tag 269-MDEntryType = 2 (trade) contains tag 277-TradeCondition containing
character ‘R’, it informs that this is one of trade that forms the opening trade event that indicates when an
instrument is traded for the first time in the trading session in progress.
If a trade contains tag 277-TradeCondition containing character ‘L’, it indicates that the related trade is the
last trade of match or opening event.
For termo vista trades, the tag 277-TradeCondition will contain the character ‘3’, indicating this trade was
matched on that origin.
For trades happening on Strategy products, a trade on each leg is also generated. This trade is marked
with 277-TradeCondition containing value ‘1’
If a trade contains tag 277-TradeCondition containing character “RL”, this trade was generated
with a RLP (Retail Liquidity Provider) Order
Here are the FIX tags sent for a trade repeating group (X= required, C=conditional):
83 RptSeq X X X
48 SecurityID X X X
22 SecurityIDSource X X X
207 SecurityExchange X X X
50
FIX Market Data Messaging Specification Version 2.1.7
271 MDEntrySize X X X
Only sent for Termo and BTB (will be discontinued for
37014 MDEntryInterestRate C X
BTB, see chapter 13.1)
272 MDEntryDate X X X
273 MDEntryTime X X X
37016 MDInsertDate X X X
37017 MDInsertTime X X X
274 TickDirection C X X Not sent on first trades and trades of equal price
1003 TradeID X X X
NOTE 1
The last received repeating group MDEntryType (tag 269) = 2 (Trade) does not mean
it is the last traded price. Please per attention to the following fields to determine
the last traded price: MDEntryTime (tag 273) and TradeID (tag 1003).
NOTE 2
When receiving a message with the repeating group whose MDEntryType (tag 269)
= 2 (Trade) and TradeCondition (tag 277) contains U (Exchange Last), it means that
it is not a “real” trade, but only information of the last valid trade. Therefore, the
related repeating group only informs the price and quantity, and not contains other
information like buying and selling parties, trade identification, etc.
51
FIX Market Data Messaging Specification Version 2.1.7
83 RptSeq X X X
48 SecurityID X X X
22 SecurityIDSource X X X
207 SecurityExchange X X X
15 Currency X X X
272 MDEntryDate X X X
273 MDEntryTime X X X
1020 TradeVolume C X X Total traded volume for the session, TRS only sends it for E&B.
The FIX message syntax for Session High/Low/VWAP Trade Price repeating groups lie below (X=
required, C=conditional):
83 RptSeq X X X
48 SecurityID X X X
22 SecurityIDSource X X X
207 SecurityExchange X X X
52
FIX Market Data Messaging Specification Version 2.1.7
270 MDEntryPx X X X
272 MDEntryDate X X X
273 MDEntryTime X X X
The theoretical opening price is also sent on this block (indicated by the presence of the tag 286-
OpenCloseSettlFlag) and is calculated and updated based on the orders presented in the book during
every auction including the pre-opening / pre-closing auction.
83 RptSeq X X X
48 SecurityID X X X
22 SecurityIDSource X X X
207 SecurityExchange X X X
272 MDEntryDate X X X
273 MDEntryTime X X X
53
FIX Market Data Messaging Specification Version 2.1.7
83 RptSeq X X X
48 SecurityID X X X
22 SecurityIDSource X X X
207 SecurityExchange X X X
270 MDEntryPx X X X
272 MDEntryDate X X X
273 MDEntryTime X X X
286 OpenCloseSettlFlag “0”,”3”,”4” X X 3 – used for index preliminary close (equities only)
Here are the FIX tags normally sent for this type of repeating group (X= required, C=conditional):
83 RptSeq X X
48 SecurityID X X
22 SecurityIDSource X X
207 SecurityExchange X X
270 MDEntryPx X X
286 OpenCloseSettlFlag “1”, “4” X For today (1) or previous day (4)
Usually, settlement prices are sent in a regular order during the trading day. However, Market
Surveillance can issue corrections or updates to the settlement price for the previous or current session.
54
FIX Market Data Messaging Specification Version 2.1.7
Hence, it´s advised that the customer application is able to process any combination for the tags 286-
OpenCloseSettlFlag and 731-SettlPriceType.
Here are the FIX tags normally sent for this type of repeating group (X= required, C=conditional):
83 RptSeq X X X
48 SecurityID X X X
22 SecurityIDSource X X X
207 SecurityExchange X X X
272 MDEntryDate X X X
273 MDEntryTime X X X
NOTE
Customer applications are responsible for deleting the existing theoretical opening
price and imbalance information from memory after trading phase changes from Pre-
Open (tag 625-TradingSessionSubID = 21) for each instrument in the group whose
trading status is different from Pre-Open status (tag 326-SecurityTradingStatus = 21).
55
FIX Market Data Messaging Specification Version 2.1.7
Below is the basic template of both market data entry type (C) (X= required, C=conditional):
83 RptSeq X X X
48 SecurityID X X X
22 SecurityIDSource X X X
207 SecurityExchange X X X
272 MDEntryDate X X X
273 MDEntryTime X X X
– Hard limits
– Rejection band
– Auction band
– Static limits
– Quantity limit (Equities only)
Here are the FIX tags normally sent for this type of repeating group (X= required, C=conditional):
269 MDEntryType “g”, “h” X X Price band (g), Quantity band (h)
83 RptSeq X X X
48 SecurityID X X X
22 SecurityIDSource X X X
207 SecurityExchange X X X
6939 PriceBandType “1”,”2”,”3”,”4” X X Not sent for reference price and quantity limits
56
FIX Market Data Messaging Specification Version 2.1.7
1306 PriceLimitType “0”,”2” X X Not sent for reference price and quantity limits
1148 LowLimitPrice X X X
1149 HighLimitPrice X X X
272 MDEntryDate X X X
273 MDEntryTime X X X
The tunnels don´t change intraday. It is also known as “oscillation tunnel” establishing the price limits
(lower and higher) of an instrument. Any order submitted with a price below the low limit or above the high
limit will be rejected.
Below is the specific usage for each type of Band and Tunnel:
279 MDUpdateAction 0 0 0 0 0 0
269 MDEntryType g g g g g h
48 SecurityID X X X X X X
22 SecurityIDSource X X X X X X
207 SecurityExchange X X X X X X
83 RptSeq X X X X X X
272 MDEntryDate X X X X X X
273 MDEntryTime X X X X X X
1149 HighLimitPrice - X X X X -
1150 TradingReferencePrice C - - - - -
1140 MaxTradeVol - - - - - X
(tags marked with an “X” are required, those marked with “-“ are not sent, otherwise they have the specified values)
57
FIX Market Data Messaging Specification Version 2.1.7
Market Data entry type Index Value (3) is used to inform the current value of given index and described
as follows:
83 RptSeq X
48 SecurityID X
22 SecurityIDSource X
207 SecurityExchange X
37100 IndexSeq X
272 MDEntryDate X
273 MDEntryTime X
274 TickDirection C
The Composite Underlying Price (269=D) block is used to inform the price composition of BDR indexes:
279 MDUpdateAction 0
83 RptSeq X
48 SecurityID X
22 SecurityIDSource X
207 SecurityExchange X
37100 IndexSeq X
711 NoUnderlyings X
309 UnderlyingSecurityID X
305 UnderlyingSecurityIDSource X
308 UnderlyingSecurityIDExchange X
810 UnderlyingPx X
58
FIX Market Data Messaging Specification Version 2.1.7
272 MDEntryDate X
273 MDEntryTime X
83 RptSeq X
48 SecurityID X
22 SecurityIDSource X
207 SecurityExchange X
1500
MDStreamID
“E” Not sent for theoretical open
273 MDEntryTime X
59
FIX Market Data Messaging Specification Version 2.1.7
When the client system starts up, it should consider that all snapshots contain the current state of the
individual instrument. Intraday updates may be done on the instrument group level.
NOTE
Group codes may repeat amongst different exchanges, hence it is advisable that
client systems use the key group code (tag 1151 – SecurityGroup) + exchange (tag
207 – SecurityExchange).
When processing the SecurityStatus message (tag 35=f), client systems must first look for tag 1151-
SecurityGroup. This tag contains the group identification of a set of instruments. That being the case, all
individual instruments of that set will have their status changed to the value of tag 625–
TradingSessionSubID. The following message example illustrates the change of trading phase of the
group “XX” to “Pause”:
MsgType = f (SecurityStatus)
Tag Tag Name Value
1151 SecurityGroup XX
207 SecurityExchange BVMF
625 TradingSessionSubID 02
If tag 1151-SecurityGroup is not present in the message, then the message refers to an instrument,
referred by tag 48-SecurityID. The following message show a case where the instrument changes to
Pause, separating from the group Phase (tag 1174-SecurityTradingEvent=101).
MsgType = f (SecurityStatus)
Tag Tag Name Value
48 SecurityID 99999999
22 SecurityIDSource 8
207 SecurityExchange BVMF
326 SecurityTradingStatus 02
1174 SecurityTradingEvent 101
Please see the complete SecurityStatus message format at UMDF Message Reference document.
NOTE
Either tag 625 or 326 are sent at once per 35=f message. Whenever an instrument
state rejoins the group phase (1174=102), it´s safe to infer the group phase (tag 625)
from the current instrument state (tag 326).
60
FIX Market Data Messaging Specification Version 2.1.7
61
FIX Market Data Messaging Specification Version 2.1.7
Not available for 18 This instrument state is used by surveillance to prevent order
trading entry and matching by market operations command or
(Forbidden) schedule.
62
FIX Market Data Messaging Specification Version 2.1.7
For example, group “XX” may be in trading phase “Open”, but instrument ABCD that belongs to group
“XX” is in the “Pause” status – due to market surveillance command. This information is especially useful
when client systems want to determine the state of the group altogether, and outlining the individual state
of the instrument.
63
FIX Market Data Messaging Specification Version 2.1.7
On receipt of this message and flag, client systems are expected to reset the following entry types:
Description
MDEntryType
2 Last Trade
3 Index Value
4 Opening Price
4 Theoretical Price (tag 286=5)
7 Trading Session High Price
8 Trading Session Low Price
9 Trading Session VWAP Price
B Trade Volume
The same statistics will no longer be available at the snapshot recovery stream upon the receipt of
message SecurityStatus (tag 35=f) with SecurityTradingEvent field (tag 1174) = 4 (Change of Trading
Session).
64
FIX Market Data Messaging Specification Version 2.1.7
Currently, all derivatives and FX products are traded on PUMA UMDF 2.0 platform.
Hence, as such instruments are not options, the strike price (tag 202-StrikePrice) field will not be
sent, or sent as zero.
The proper way to identify these instruments is checking their security subtype (tag 762-
SecuritySubType). The following subtypes are used identify them:
Also, the Trade Volume block (269=B) also carries more detailed information regarding financial volume
(on tag 270-MDEntryPx) and number of trades (tag 271-MDEntrySize) and also informs the total traded
volume for non-electronic venues (for instance option exercise and ex-pit, 1500=O and 1500=X
respectively).
NOTE
All trade volume information is sent per instrument not per segment. All summarization
must be processed by the client.
65
FIX Market Data Messaging Specification Version 2.1.7
• New domains for tag 167-SecurityType (also replaced OPTEXC for OPTEXER, for consistency)
• Changed the way to represent information about legs, removing tags 37009-LegType and 37010-
BuyersPerspective, now using 609-LegSecurityType and 624-LegSide
• Stopped using tag 561-RoundLot in favor of the new singleton repeating group 1234-
NoLotTypeRules
• For trades (269=2), now able to replay the Exchange last trade (277=U), Leg trades (277=1) and
also Marketplace entered trades (277=2)
• Started using the new Phase/State mechanics, using tag 1174-SecurityTradingEvent (values 102
and 101, to indicate following the group or not)
• Possibility to enable new Phase/State = 101-Final Closing Call (for future use)
66
FIX Market Data Messaging Specification Version 2.1.7
67
FIX Market Data Messaging Specification Version 2.1.7
MDEntry Description
269=0,1 BTB Book Entries (Borrower, Lender)
269=2 BTB Pre-Contract (Trade)
269=J BTB Book Reset
269=B BTB Trade Volume
269=C BTB Open Interest
Special remarks:
• All market data for BTB comes with the tag 1500-MDStreamID=L to differentiate the market data
entries from other venues;
• Tag 37019-EarlyTermination indicates if the lending can be terminated earlier, also if in case of
PTO
• Tag 37025-BTBGradeDate indicates the minimum date when the borrower can call the loan
contract to an end
• Tags 288-MDEntryBuyer and 289-MDEntrySeller contains the participant codes for the
borrower and lender parties, respectively
• Tag 270-MDEntryPx carries the lender tax (as the sum of the lender tax plus the fee)
• Tag 37014-MDEntryInterestRate, that was once used to convey the borrower tax, will no longer
be used (will carry value “0” until 2017-12-04, when the tag is no longer published for BTB).
68
FIX Market Data Messaging Specification Version 2.1.7
This message (35=n) is used to carry a payload that is the unmodified RLC-Z5 message, from ProxyDiff,
that can be extracted and processed as usual by customers and vendors already capable of processing
this message.
NOTE
The Fixed Income products are currently disseminated as regular PUMA instruments,
identifiable by tag 460-Product, tag 167-SecurityType and tag 762-SecuritySubType.
Please refer to the Message Reference document (see chapter 17) for tag usage.
69
FIX Market Data Messaging Specification Version 2.1.7
For specific changes for each message, please refer to the UMDF Message Reference document.
70
FIX Market Data Messaging Specification Version 2.1.7
For specific changes for each message, please refer to the UMDF Message Reference document.
Another important remark for TCP Replayer is that there is a single IP/port centralizing the recovery for all
channels. The customer application should be able to connect a single time using a single session for all
channels.
NOTE
Customer applications must be capable of arbitrating between both incremental
feeds A and B, to be able to recover missing packages more efficiently and avoid
using the TCP Replayer feed.
71
FIX Market Data Messaging Specification Version 2.1.7
Another important remark is that the tag 272-MDEntryDate is not sent for Channel Reset messages.
– A new Unified News Channel reserved for global news broadcast that is able to send encoded
headlines and text with special characters (accented letters for instance);
– Revised news sources (tag 6940-NewsSource);
– Cross-news referencing (using tag 1472-NewsID);
– Deprecated News routing tags;
For specific changes for the News message (35=B), please refer to the UMDF Message Reference
document.
72
FIX Market Data Messaging Specification Version 2.1.7
Client system must certify against the FIX 5.0/FAST feed before being deployed in production. The
certification process consists of:
Client systems have different options of establishing connectivity to the certification environment. They
are:
• Internet VPN: in this case, the customer must be able to configure a GRE tunnel with the
exchange, to allow for multicast traffic to be sent.
For a detailed description of network connectivity options, please contact to the BVMF Customer Services
Department:
The certification, new release and production environments multicast and TCP Replayer / Historical
Replayer channel definitions are available at the following website:
http://www.bmfbovespa.com.br/en_us/services/trading/bm-fbovespapuma-trading-system/for-developers-
and-vendors/umdf-unified-market-data-feed/
Changes to the multicast addresses/ports and TCP Replayer / Historical Replayer information will be
notified by the exchange if applicable. Please keep in mind that message rates in the certification
environment may be substantially lower than in production.
73
FIX Market Data Messaging Specification Version 2.1.7
http://www.bmfbovespa.com.br/en_us/services/trading/bm-fbovespapuma-trading-system/for-developers-
and-vendors/umdf-unified-market-data-feed/
74