CFG IP Integration Spec - CModel Addendum
CFG IP Integration Spec - CModel Addendum
CModel Addendum
Version: 23Q1
Revision: 0.0
Intel Confidential
CFG IP Integration Specification: CModel
Addendum
About this document
This document describes the CFG C++ Model and usage model where user can use as reference
when integrating the model into their target system.
As part of Intel Inclusive language Initiative (I3), CFG will rename master to main and slave to
secondary. During transition, these terms are interchangeable.
Audience
Prerequisites
Related documents
Customer support
For technical support about this product and general information, contact CFG Support.
Intel Confidential 2
Revision History
Revision Date Updates
0.0 Mar 08, 2023 Initial Release
Intel Confidential 3
Contents
About this document ................................................................................................................. 2
Audience .................................................................................................................................... 2
Prerequisites .............................................................................................................................. 2
Related documents..................................................................................................................... 2
Intel Confidential 4
1.8 Known limitations ..................................................................................................................... 23
Intel Confidential 5
1 C++ NOC MODEL INTEGRATION
The C++ NoC model consists of a dynamically linked library containing all functions of the model,
and a header file containing the APIs that are used to run the model. The C++ model is named
libnocstudio.so for Linux and nocstudio.lib for Windows. The C++ header files are named
nocstudio.h.
NOTE: In multi-die or multi-NOC designs, current implementation requires all NOCs to have
the same mesh size in order for these global variables to function correctly.
struct NocFlit_base {
brif_t src; // Source bridge interface id
int qos; // QoS
FlitPos pos; // Position of flit in packet
void* payload; // Pointer to payload
};
A flit object of type NocFlit or MCNocFlit must be constructed for injection into the NoC,
depending on whether the Flit is Unicast or Multicast. The NocFlit is unicast and has one
destination, while the MCNocFlit supports multicast transmissions through the NoC, having a
vector of destinations. Flit injection process must adhere to the protocol requirements of the
Intel Confidential 6
interface. The FlitPos field determines the position of the flit in the packet; i.e. Start of Packet
(Sop), End of Packet(Eop), Middle of packet (Neither Sop nor Eop) or single flit packet (SopEop).
The source and destination of the NocFlit/MCNocFlit are represented as brif_t (typedef int),
encoding the bridge id and the interface id. The brif_t (bridge_interface ID) of a source or
destination is derived as (bridge_id << 2) | interface_id.
The payload field is a generic (void) pointer that is disregarded by the internals of the model, and
returned unchanged upon flit ejection.
For an injection to be successful, the flit provided must match a valid traffic hop specified in the
imported script file with which the model was configured. Flit injections must adhere to
following protocol:
1. SOP/EOP logic
a. A series of flits must be started with an SOP flit
b. An EOP flit must come at some point after an SOP to complete the series
c. The first injection from any interface must always be an SOP flit
2. Flits for different traffic flows cannot be interleaved when injecting into the same
interface
3. Only one flit may be injected per interface per cycle
During each cycle, a flit may be injected at every input interface of all bridges of the NoC by
calling the appropriate inject_flit() API depending on whether the injected flit is unicast or
multicast. The injection API returns true if flit injection is successful, and false if injection fails.
Injection may fail because of protocol errors, invalid message, or invalid fields in the flow. It may
also fail if there is flow control asserted at the injection interface. In this case, the same flit may
be injected again in the next cycle.
Intel Confidential 7
Table 1 Injection error cause and string values
Invalid src_br Cannot get a valid bridge from source bridge ID <bridge id>.
Invalid dest_br Cannot get a valid bridge from destination bridge ID <bridge id>.
Invalid src_ifce Cannot get a valid transmitting interface from Bridge ID <bridge id>
and Interface ID <interface id>.
Invalid dest_ifce Cannot get a valid receiving interface from Bridge ID <bridge id> and
Interface ID <interface id>.
Invalid src_br (regbus) Source bridge is Regbus Main. Currently not supported.
Invalid message Cannot create a valid packet. No traffic flows match flit attributes.
Invalid message No packet is being sent; need SOP to start a new packet.
Intel Confidential 8
Invalid message Flits are being interleaved. QoS and destination of flits from same
packet must match.
Flow control Can inject only one flit per interface every cycle.
bool set_credit_return_callback(Sim* s,
const brif_t src,
function<void(const brif_t src)> cb,
string* error_string = NULL);
set_credit_return_callback() sets the callback function to be called upon the transmitting bridge
returning a credit to the host via the given interface. The NocFlit should be destroyed inside this
callback function.
Table 2 Credit return callback setting error cause and string values
Invalid src ifce id Source interface ID <bridge id> must be in range [0-3].
Intel Confidential 9
Invalid src_br Tx bridge for Bridge ID <bridge id> not found.
Invalid src_ifce Tx interface for Bridge ID <bridge id> and Interface ID <interface id>
not found.
The following event-based API is used for flit ejection. Note: The original polling-based
eject_flit() and can_eject_flit() APIs have been removed.
set_eject_flit_callback() sets the callback function to be called upon the receiving bridge
receiving a flit at the given interface. When it is called, the ownership of the NocFlit returns to
the user code. The callback function is responsible for calling send_credit_rxif() to indicate to
the receiving bridge that it will be able to send one more packet to the host without overflowing
the host. There is no MCNocFlit version of eject callbacks because only NocFlit is ejected.
1. Flits injected from a given source to a given destination arrive (are ejected) in the order
that they were injected
2. As long as the payload pointers of all injected flits of a packet are consistent with each
other (i.e. they all point to the same payload), the payload pointers of all ejected flits of
the packet will be consistent with each other
Table 3 Flit ejection callback setting and credit issuing error cause and string values
Intel Confidential 10
Invalid dest_br Rx bridge for Bridge ID <bridge id> not found.
Invalid dest_ifce Rx interface for Bridge ID <bridge id> and Interface ID <interface id>
not found.
The following API is used to advance the internal clock of the NoC model.
advance_time() advances the simulation time by one clock cycle, consuming all inputs of all blocks
and producing outputs during that clock. The simulation is clocked at the highest clock frequency
of all the clocks domains in the NoC.
An example of how to include various model files and use the NoC model APIs is provided in
APITest.cpp in the NocStudio release package.
The command script must be scrubbed, as the NoC model does not allow standard NocStudio
simulator commands such as run, cont, etc. Therefore, all such commands should be removed
from the command script. If not removed, these commands will be ignored while configuring
the NoC model. The command that generates NoC IP, gen_rtl, is ignored, as the model library is
disabled from generating RTL and verification files. Currently, all commands that write to disk
Intel Confidential 11
are disabled, including sim_stats, gen_image, create_trace_files, etc., and are ignored during
model configuration.
Once the script file is ready, the following API is used to configure the model.
The function takes the input stream of the commands from the script file as an argument and
returns a NULL pointer in case of error(s) in script processing, mapping of the NoC, or Sim
creation, and writes a description of the error in error_string if the pointer is not NULL.
Mesh cannot be Error: Cannot extract a NocStudio grid from Console; new_mesh not
created called?
Traffic is not mapped Error: Cannot run sim unless traffic is mapped.
The following API is used to set the performance statistics logging level.
set_perf_logging() adjusts the level of logging performed by the simulator for purposes of
performance statistics generation. Lower values consume fewer resources per simulation cycle,
increasing simulation speed.
Intel Confidential 12
Table 5 Performance statistics logging levels
The following API is used to set the simulation safety check level.
void set_sim_safety_check_level(Sim* s,
api_mode_sim_safety_check_level_t level);
Intel Confidential 13
• No traffic flows match flit attributes
1.5 UTILITIES
The NoC model also includes utilities for modifying the NoC and querying information.
The following API is used to run arbitrary commands on the NoC, as if they were typed into the
NocStudio console.
bool run_commands(Sim* s,
const vector<string>& cmds,
ostream& cmd_out = cout,
string* error_string = NULL);
run_commands() takes a vector of strings and runs them sequentially on the NoC of the given
simulator. It also accepts an optional cmd_out parameter containing a reference to an output
stream, to enable users to redirect the output of the commands to a file or a parser, for instance
to inspect error messages to determine why a command failed. If no output stream is given,
output is written to standard output cout.
• Not modify NoC construction, such as merely changing the FIFO depth of a link
• End with a map command after modifying the construction of the NoC by, for instance,
adding a host, changing the skip_enable property of a router, or adding traffic
Consequently, it returns true only if all commands succeeded and the NoC remains in a
‘Mapped’ state after executing them.
Intel Confidential 14
One or more commands failed Command(s) failed; see standard output / 'cmd_out'
stream for details.
Given set of commands modified NoC NoC is no longer in 'Mapped' state after running
construction without re-running map commands.
The following API is used to query the occupancy, throughput or latency of a channel.
struct EventStats {
double minimum; // Minimum value of the stat gathered
double maximum; // Maximum value of the stat gathered
double average; // Average value of the stat gathered
double count; // Number of samples of the event stat
void merge(const EventStats& e); // To merge two stats
};
EventStats query_end_to_end_latency(Sim* s,
brif_t src, brif_t dest,
string* error_string = NULL);
EventStats query_stats(Sim* s,
const string& channel_name,
api_mode_stats_t type,
string* error_string = NULL);
query_end_to_end_latency() is used to get the end to end latency of the messages from a source
to a destination. If either or both of the src and dest are -1, then the function returns the event
stats associated with all the src/dest.
query_stats()is used to get the occupancy, throughput or latency of a specified channel. The
type of statistics to collect is controllable via the third argument to the function. The allowed
types are Occupancy, Throughput and Latency.
Intel Confidential 15
Invalid channel name Invalid channel name <channel name> specified.
Channel name contains Specified channel name must match exactly 1 channel. Wildcards are
a wildcard not supported.
bool reset_stats(Sim* s,
string& channel_name,
api_mode_stats_t type,
string* error_string = NULL);
reset_stats() allows users to reset the logging structures associated with the Occupancy,
Throughput or Latency of channel(s). The channels can be VCs of routers or Interfaces of
bridges, and can include wildcards. The function returns true if the resetting of stats was
successful.
The following API is used to enable or disable the writing of detailed performance simulator
messages to an output stream.
bool set_perf_sim_log_file(Sim* s,
bool en,
ostream* os,
string* error_string = NULL);
set_perf_sim_log_file() allows users to write the verbose output displayed in NocStudio console
output upon issuing a step command to a file or an output stream instead. It accepts a Boolean
value of whether to enable or disable logging, and a mandatory pointer to an output stream to
which the messages will be written (if enabling).
The function returns false if enabling logging, but the given output stream is null or not in a
“good” state.
Intel Confidential 16
1.6 SUMMARY OF NOC MODEL APIS
The NoC model uses the following commands, summarized below:
1. create_sim
a. input: input stream of the script file
b. output: pointer to model simulator
c. about: creates the sim pointer from a script
2. advance_time
a. input: simulator pointer
b. about: runs the simulator for one clock cycle
3. inject_flit
a. input: simulator pointer, NocFlit/MCNocFlit reference, payload
b. output: bool value of successful injection
c. about: injects a flit
4. set_credit_return_callback
a. input: simulator pointer, source bridge_interface ID, pointer to user-implemented
callback function
b. output: bool value of successful setting of callback function
c. about: sets the callback function that handles a credit issued by a transmitting
bridge to a host
5. set_eject_flit_callback
a. input: simulator pointer, destination bridge_interface ID, pointer to user-
implemented callback function
b. output: bool value of successful setting of callback function
c. about: sets the callback function that handles an ejected flit and issues a credit
6. send_credit_rxif
a. input: simulator pointer, destination bridge_interface ID
b. output: bool value of successful issuing of credit
c. about: to be called by user-implemented ‘eject flit’ callback function to issue a
credit to the receiving bridge
7. set_perf_logging
a. input: desired level
b. about: sets the performance statistics logging level
8. set_sim_safety_check_level
a. input: desired level
b. about: sets the extent of safety checks performed by simulation routines
9. run_commands
a. input: simulator pointer, vector of commands, output stream
b. output: bool value of whether all commands succeeded and NoC remains in
‘Mapped’ state
c. about: runs a list of commands on the NoC
10. query_stats
a. input: simulator pointer, channel name as string, logging type
Intel Confidential 17
b. output: EventStats containing the min, max, avg and number of events of the
requested statistic
c. about: returns the throughput, occupancy, or latency of a channel
11. query_end_to_end_latency
a. input: simulator pointer, src bridge_interface ID, dest bridge_interface ID
b. output: EventStats containing the min, max, avg and number of events of the
requested statistic
c. about: returns the end to end latency of the messages from source(s) to
destinations(s)
12. reset_stats
a. input: simulator pointer, channel name as string, logging type
b. output: true if the reset of stats was successful
c. about: resets the stat logger associated with the channel(s)
13. set_perf_sim_log_file
a. input: simulator pointer, bool value of whether to enable or disable logging,
output stream
b. output: bool value of whether given output stream is good
c. about: enables or disables writing of simulator step output to an output stream
The following code snippet describes AXI Flits. Note that it inherits from NocFlit_base, so in
addition to these fields, there are also source, qos, pos and payload fields. The source field is
used to determine both which bridge in the model sent this flit as well as the flit type, i.e.
whether the flit is carried on AR, AWW, R, B, AC, etc. More details on interface_id ↔ message
type mappings in the AXI/ACE interfaces section <LINK HERE>. The qos field carries the
AR/AWQOS value used to differentiate different traffic classes. All qos values from 0 to 15 are
allowed. Pos is used the model’s implementation AMBA protocols and must be correctly
specified for each flit; use SopEop for single-flit requests like AR and B and a sequence of Sop,
Middle, Middle, …, Eop for multi-beat AWW and R. The payload pointer can be populated in
AMBA simulations, but its behavior will be different because of the presence of in-NoC agents
Intel Confidential 18
like LLC or CCC. A request’s payload pointer will only be received at the secondary if no
changes to the request were made by the NoC, otherwise a null pointer will be found in this
field. When the NoC generates a response, this response will have the same payload pointer
value as the request, and it is recommended for secondaries to maintain this property so that
requests and responses can be correlated.
Instead of a destination, AxiFlits have an address, which is an unsigned 64-bit value indicating
the byte offset of memory to read or write. The three following fields are “security” bits that are
also used in destination lookup.
The aid field is used for ordering related requests; the model guarantees that responses to
requests with the same aid and request interface (ar/aw) will be presented to the main in the same
order as the requests; this may require serializing the requests or reordering the responses.
The data_bytes field indicates the number of bytes of data being requested and must be specified
for both reads and writes. If the data payload is null, data will be ignored for this operation; no
data will be modified or returned. Otherwise the data payload must point to at least data_bytes
of memory to be read from or written to. The secondary (or agent) processing this transaction
Intel Confidential 19
will read or write data to this location to complete the request. This memory should be considered
reserved for this transaction until the transaction is complete.
The decode_error field must be left false on requests and will be set to true when a request’s
address does not indicate a destination in the main bridge’s routing table.
The *_access fields are for coherent and cachable requests, and indicate whether the request
needs/is allowed to go to an in-NoC CCC or LLC (or a customer-provided cache added with
add_cache).
//! when sending RACK/WACK into fabric, use these command types to
//! distinguish read from write (as same ifce is used for both)
const ace_commands_t RACK_CMD = ace_commands_t::ReadClean;
const ace_commands_t WACK_CMD = ace_commands_t::WriteClean;
//! AceFlits are used for snoop requests and ace requests and responses; they
can only be sent to and from Ace Main bridges
struct AceFlit : AxiFlit {
bool pass_dirty = false; //!< sent as part of snoop response and as part of
read response to initiating ace main
bool is_shared = false; //!< sent as part of snoop response and as part of
read response to initiating ace main
bool has_data = false; //!< sent as part of snoop response
ace_commands_t ace_command; //!< sent as part of ace requests
from main to interconnect; WACK
and RACK use ReadClean and
Intel Confidential 20
WriteClean commands to indicate
read/write
ace_snoop_t snoop_command; //!< sent as part of snoop requests
The two big additions are the ace and snoop command fields, but there are also a few boolean
fields used for snooping.
Requests coming from an ACE main must have the ace_command set to one of the enumeration
values in ace_commands_t. These correspond to the various commands defined in the ACE spec.
The src brif_t must have the correct ACE interface for the command being sent; it is not allowed
to send, for example, ReadClean on the AWW interface. Other ACE fields of request messages
are unused.
When a snoop is sent to an ACE main, a flit will arrive on the AXI_SN interface of that main with
the snoop_command field set to a value from ace_snoop_t. These values correspond to the
various snoop messages in the ACE spec. Other ACE fields of snoop messages are unused.
The response to a snoop message must have the pass_dirty, is_shared and has_data flags set to
indicate these details of the snoop response. If there is data, multiple flits can be sent with
appropriate pos values to carry that data back to the CCC.
The response to a read request will have valid pass_dirty and is_shared values to indicate the
state of this cache line in the requesting main’s cache. Other ACE fields of read response (and all
ACE fields of write response) messages are unused.
After receiving a coherent response, a RACK/WACK message must be sent back to the CCC to
complete the exchange. As both of these messages use a single interface, the ace_command field
is given either the value RACK_CMD or WACK_CMD to indicate the type of the message.
The AXI and ACE protocols use a VALID/READY protocol for flow control. This scheme uses a
VALID signal sent by the transmitter to indicate that the data pins are currently valid. The
received sends a READY signal to indicate if it is ready to receive data. At the end of each cycle,
if both VALID and READY were asserted, the sender and receiver know that data transfer
happened successfully and the next data transfer can happen in the next cycle. Because NSIP
Intel Confidential 21
uses credit-based flow control, the flow control of the Modeling API follows this nomenclature,
but is adapted in behavior for AXI/ACE interfaces to match VALID/READY semantics.
First, the number of credits for an AMBA interface is always 1; this means that until a credit is
returned, no more data can be sent. Second, the data transfer is considered complete only when
credit is returned. In the case that VALID and READY are both asserted in the same cycle, this
means that the credit will be returned in the same cycle as the data was sent, allowing more
data to be sent in the next cycle. In the case that READY is deasserted during a cycle that
VALID is asserted, the valid/ready protocol requires the same data to be presented on the
interface until READY is asserted. This is modeled by data being injected, but it will not make
progress through the NoC model until READY is sent back to the injector in the form of a credit.
After data is injected, further data cannot be injected until READY is asserted (i.e. a credit is
returned on that interface).
In this translation, it is not possible to assert READY before VALID, but this can be modeled
equivalently by asserting READY (sending a credit) immediately after VALID (inject_flit).
To indicate a source or destination id, the concept of brif is extended to give identifiers to AXI
and ACE interfaces as follows:
0 AR AR
1 AWW AWW SN
2 R SN RESP R
3 B ACK B
To aid in programming against this interface, constants are provided to give names to these
values.
Intel Confidential 22
// Axi/Ace bridge interfaces are mapped to integers according to the
following:
const int AXI_AR = 0; // tx on main, rx on secondary
const int AXI_AWW = 1; // tx on main, rx on secondary
const int AXI_R = 2; // rx on main, tx on secondary
const int AXI_B = 3; // rx on main, tx on secondary
const int AXI_SN = 1; // rx on main
const int AXI_SN_RESP = 2; // tx on main
const int AXI_ACK = 3; // tx on main
The AXI AW and W channels are represented in the model as a single interface, AWW. This
interface transports both address and data, modeled by having the first injected flit be the AW
data and all following flits be data (i.e. W). Injecting single-flit write requests is not supported.
Note: Under some rare circumstances, the NOC may present a single-flit write request and this
is considered a flaw and will be fixed in a future release.
The AXI protocol permits interleaved R responses from secondaries, where data/response for a
different transaction can be sent before all data for the first transaction was complete
(RLAST). In current release, the modeling API expects responses to be presented to the model
in a contiguous fashion.
Note: When the secondary host model fails to comply, the modeling API behavior is
undetermined.
In Statistical mode, the model is simply counting packet flits and bubbles to see how many
packets can flow through the NOC. The exact source and destination is not perfectly modeled
as long as the injected bandwidth matches design’s specification. As an example, in a coherency
design with one CPU, one CCC and one MEM. The transactions initiated by the CCC (to the
MEM) are independently modeled and they have no direct relationship to the CPU-to-CCC
transactions. In the NOC performance modeling world, it’s not perfect but sufficient. Please
note that this is the default mode for NocStudio PerfSim.
In address-aware mode, using the same example earlier, the model will not initiate a transaction
from CCC to MEM until the CPU issues coherent request to CCC and there is a directory miss.
Please note that this mode is the default mode for Platform Architect flow.
Intel Confidential 23
Feature Supported Limited support Not supported
Statistical mode AXI, ACE CSB2 vAXI3
Address aware mode AXI CSB2 NA
PA-Ultra AXI5, ACE, CCC, AHB1, APB1 IOCB, CSB4
LLC and ICCC
Note:
Intel Confidential 24
Intel Corporation
Intel Confidential 25