Sustaining high-speed data streams on USB

The new technology helps engineers achieve high-performance applications on USB, which previously was possible only on an internal bus such as PCI. This white paper describes the challenges associated with high-speed data acquisition on USB and reasons why NI signal streaming technology is essential to any USB data acquisition (DAQ) device. Since the introduction of the PCI bus in 1993, the adoption and use of PC-based data acquisition technologies have grown to an all-time high, with more than 6 million new measurement channels sold in 2006 from National Instruments alone. With more than 2 billion ports around the world (usb.org), the Universal Serial Bus (USB) has radically changed the data acquisition landscape, as more and more engineers now prefer to build a data acquisition system based on the external, portable, and easy-to-use USB bus. As experienced by early adopters of USB-based test, measurement, and control products, a significant performance gap exists between the USB devices and their PCI counterparts. This performance gap between PCI and USB DAQ products is not caused by the inability to place high-performance components on USB DAQ devices but by the sheer nature of the Universal Serial Bus. USB exhibits lower bandwidth, slower latency and host-initiated data transfers. Because of these challenges, early USB-based applications were limited to data storage, basic data acquisition, and a replacement for GPIB and serial in instrument control. To optimize the use of the Universal Serial Bus and deliver high-performance data acquisition, National Instruments created NI signal streaming technology. NI signal streaming combines three innovative hardware- and software-level design elements to enable sustained high-speed and bidirectional data streams over USB. The new technology helps engineers achieve high-performance applications on USB, which previously was possible only on an internal bus such as PCI.

Challenges of USB for data acquisition before NI signal streaming

At 53 MB/s theoretical, the bandwidth of Hi-Speed USB suffices for most data acquisition applications. However, existing USB DAQ hardware and software designs are unable to take advantage of the provided bandwidth. Data acquisition often involves multiple I/O tasks such as analog input, analog output, digital input, and digital output as well as multiple counter/timers. To operate all of these tasks simultaneously, a data acquisition device must process multiple streams at the same time and be able to quickly move data to and from PC memory. PCI and PCI Express are ideal for data acquisition designs because these buses provide high bandwidth, low latency, bus mastering, and DMA channels to stream data directly to PC memory. USB and other external buses such as Ethernet are inferior to plug-in buses because they lack equivalent features. Unlike PCI designs, USB DAQ devices are limited by a host-driven serial protocol with no DMA channels. The operating system must initiate any request to transfer data to or from the device. This adds significant overhead to data transfers because devices are no longer able to take control of the bus and transfer data without CPU interaction. Additionally, data moving to or from the device to PC memory is controlled by the computer processor, which adds significant communication overhead. In the past, data acquisition designers had two options to address these limitations compensate for them or ignore them. Designers compensated for the limitations by installing large amounts of expensive memory on the devices. The additional memory provided a place for the data to reside on the device before the CPU could process the appropriate instructions to retrieve the data. While this may have helped to alleviate buffer overflows for a limited amount of time, the device latency increased because data ended up stored in large stacks of memory in an unusable location. Ignoring the limitations often meant designing data acquisition devices capable of only one type of high-speed I/O task (in other words, clocked analog inputs combined with static analog outputs and digital I/O on a device). These solutions were not ideal because they made devices increasingly more expensive while still prohibiting performance similar to that of PCI-based designs. NI signal streaming technology takes a different approach to handling the concerns by enabling sustained high-speed and bidirectional data streams over USB. Before explaining NI signal streaming, it is important to review USB communication and legacy USB DAQ designs.

USB communication overview

USB, a serial protocol that transfers data such as addresses, requests, and confirmations in packets, shares available bandwidth across all devices present. USB systems incorporate three components the host (typically a PC or laptop), a USB device, and the USB cable. The USB host initiates all data transfers to a USB device, thus requiring the transmission of three or more packets for a simple transaction host request, data transfer, and host confirmation. Information is sent to and from a USB device through its endpoints. An endpoint is equivalent to a gate through which all data must pass to get on or off the USB device. Endpoints can transfer data only in one direction, requiring devices to have multiple endpoints to achieve both input and output functions. Data is transferred to or from the device endpoints to the host controller using four basic types of transfers, one for USB bus control and three others for data transfer: Interrupt data transfer provides low-volume data with timely and reliable data transfers; often used for a peripheral device such as a keyboard or mouse. Isochronous data transfer offers prenegotiated bandwidth with possible data loss; often used when on-time data delivery is

more important than data accuracy, such as streaming audio and video. Bulk data transfer delivers large data transfers with no loss of data; often used for applications where lots of data must be transferred with no loss of data such as external hard drives. With built-in error checking and large data transfer capabilities, bulk data transfer is the logical choice for USB data acquisition. But in practice, bulk transfers can achieve fast performance when only a single stream of data is sent through the USB port. Any additional streams can considerably reduce transfer rates, especially if the transfers are bidirectional.

Traditional approach
Three technical issues contribute to the inability of USB to meet throughput and latency expectations for high-performance data acquisition. The first issue is the inefficient use of a processor for data management. The current standard for data management between the data acquisition front end and USB endpoints is a slow, commercial off-the-shelf (COTS) processor. Its purpose is to divert the appropriate information to each individual USB endpoint. Compared to a custom design, it is an inexpensive solution with a fairly easy implementation. Figure 1. Traditional USB DAQ architectures use a low-performance processor to connect the DAQ front end to the USB endpoints. These processors are instruction-based, single-threaded pieces of silicon that create a switch-like behavior. Only able to run a single instruction at any given time, they are the bottleneck of all typical USB DAQ devices. Any higher-priority tasks, such as handling control requests from the host, may delay the processor in handling data transfers, causing low throughputs and possible buffer underflow/overflow errors. Due to its architecture, a processor cannot simultaneously manage several data streams and handle incoming control requests, making high-performance data acquisition impossible. In other words, USB cannot achieve the PCI DMA-like performance of streaming multiple I/O types simultaneously. Another issue is latency. Latency is defined as the time delay between initiating a request for data and the beginning of actual data transfer. USB has higher latency than PCI because of its OS-based access and serial nature. A typical single-point voltage acquisition could require dozens of register-level instructions, which, when multiplied by 1 ms of latency, is unacceptable. Finally, there is a trade-off between data throughput and latency. To have a fast data throughput rate, engineers need to move larger data sets at every transfer. Doing so reduces the responsiveness of the system and increases system latency, thus leaving the device unresponsive during each large transfer. What is truly needed is a data transfer method that uses both the streaming capabilities of isochronous data transfers and the 100 percent error-free transfer of bulk. To address these needs, National Instruments created an innovative patent-pending USB data streaming technology NI signal streaming.

NI signal streaming approach to USB DAQ

NI implemented three main design improvements with NI signal streaming the ability to quickly transfer large data sets through USB, optimized message-based instructions, and increased responsiveness through added intelligence on the USB device.

Fast data transfers

There are two requirements for transferring large amounts of data quickly through USB. First, because USB is a host-initiated communication protocol, a software driver needs to relentlessly send data requests in quick succession. Then, on the device side, data needs to be made available to the USB endpoints as fast as possible to fulfill the rapid incoming host requests. On the host side, NI-DAQmx driver software actively ensures a continuous supply of data requests. The requests are queued and ready to be sent from the USB host even as it is receiving and processing incoming data from the device for the user. On the device side, NI signal streaming avoids processor-based switch-like behavior by using the NI Signal Streaming Controller, a custom controller that concurrently runs multiple tasks and maps data streams directly to USB endpoints. NI signal streaming works with the NI-STC 2 system timing controller to deliver four onboard, high-speed DMA channels directly into four USB endpoints. This frees resources for the processor to work on other tasks, such as message translation. Onboard DMA transfers ensure that data is always ready to be read or written at the USB endpoints. Figure 2. NI signal streaming, which adds the NI Signal Streaming Controller to provide onboard DMA transfers from the DAQ front end to the USB endpoints, reduces overhead introduced by the onboard processor.

Message-based instructions
As discussed earlier, USB has a higher latency than PCI. This delay is amplified when a single-point voltage measurement requires the transmission of several dozen register-level commands. NI signal streaming provides message-based communication to avoid lengthy transmissions the host needs to send only a single, high-level message such as acquire.commit through USB. The device processor then converts the message into the dozens of register-level commands that are necessary to properly set up the system timing controller. Figure 3. Using message-based instructions avoids dozens of register-level commands.

Device-side communication intelligence

Finally, NI signal streaming resolves the trade-off between data throughput and latency. For fast throughput, large data sets are continuously transferred, leaving an immediate request for a smaller data set impossible to fulfill. For traditional USB DAQ devices, the USB host waits for a large data transfer to complete before it releases any data to the user.

For devices with NI signal streaming, added intelligence on the device allows it to prematurely stop large data set transfers and send an early completion notification to the USB host without compromising data integrity. This effectively allows the device to respond faster to requests and send smaller sets of data when needed.

High-performance DAQ on USB

Faster data transfers and message-based instructions contribute to higher data throughputs while the additional device-side intelligence combined with message-based instructions contribute to lower latencies. These enable a single-point acquisition performance increase of up to 1,600 percent for analog input and up to 250 percent for analog output. Figure 4. Single-Point Analog Input and Analog Output Performance Chart Using new USB DAQ designs that integrate NI signal streaming technology with Hi-Speed USB, National Instruments can leverage USB as a high-performance bus for its industry-leading NI M Series and NI CompactDAQ devices. M Series USB devices provide integrated signal connectivity with performance rivaling its PCI counterparts. They are available in both bus- and wall-powered formats. NI CompactDAQ is a scalable data acquisition system with integrated signal conditioning and hot-swappable C Series modules. Both platforms also include other industry-leading NI technologies developed for high-performance and high-precision data acquisition. All USB DAQ devices with NI signal streaming feature four DMA-like channels, capable of streaming up to four high-speed operations simultaneously across USB. Figure 5. NI Industry-Leading USB Data Acquisition Platforms

Summary
USB has evolved from a low-speed peripheral bus for accessories, such as mice and keyboards, to a high-speed bus capable of satisfying data-intensive applications. NI signal streaming combined with the error correction, noise rejection, power management, and power distribution inherent to the USB protocol, yields a robust, secure, and reliable USB bus for use in measurement and control products. With the introduction of NI signal streaming technology, USB devices now rival the performance users typically expect from plug-in buses. By addressing performance concerns with fast data transfers, reduced communication overhead, and increased device responsiveness, NI signal streaming is a necessary technology for any USB DAQ device. Whether streaming multiple measurement types in the megahertz range or logging sensor data at several hertz, NI signal streaming provides the performance required for USB DAQ applications.