Data Acquisition

Buffer

Fig. 18 Acquisition ringbuffer

Scan and Scan Size

One scan is the portion of data that consists of exactly one sample for each sampled channel on a board.

So if there are 2 analog channels and 1 counter channel active, the scan would logically hold three values. (AI0, AI1, CNT0).

The scan-size therefore directly derives from this information. It describes the memory-consumption of one scan in Bytes. In above example, when using the AI-channels in 24Bit mode (consuming 32Bit per Sample) the resulting scan size would be:

ScanSize := sizeof(AI0) + sizeof(AI1) + sizeof(CNT0)

Scansize := 32Bit + 32Bit + 32Bit

Scansize := 4 Byte + 4 Byte + 4 Byte

Scansize := 12 Byte

So one scan would have the size of 12 Byte.

The scan size cannot be directly controlled by the application as it directly depends on the number and type of activated channels.

Usually the application does not have to know very detailed about one scan and its layout inherently, as there are ways to get this information from the API in an abstracted way at runtime.

Block and Block Size

One block is a collection of n scans.

It is only meant as a logical unit and does not directly influence the driver in any way. Usually it is set up in accordance with the polling-interval of the application.

The block-size can be set to any arbitrary value > 0. A standard use case would set it to SampleRate * pollingIntervall. For Example:

BlockSize := SampleRate * pollingIntervall

BlockSize := 2000 SPS * 0.1 sec

BlockSize := 200

This has to be set by the application.

Block Count

This defines how many blocks the buffer shall be able to hold. This allows the application to control how big the backlog of data shall be and thus how much time the application may spend with tasks not related to the acquisition – so that peaks in computation times won’t lead to lost acquisition data.

It can be set to any value > 0, and is only limited by the total available memory.

For example:

BlockCount := 50

This has to be setup by the application.

Total Buffer Size

The total buffer size is calculated based on the above described information.

Buffersize := ScanSize * BlockSize * BlockCount

In our example:

BufferSize := 12 Bytes * 200 * 50

BufferSize := 12 Bytes * 10000

Buffersize := 12000 Bytes

Synchronous Data Channels

Each sampling period produces one sample for each channel and consumes “Scan Size” amount of data in the buffer. There are currently three kinds of synchronous data in the buffer: analog channel samples, counter channel samples and digital channel samples.

The driver itself maintains a separate read- and write-pointer into this buffer. So the hardware can add new samples independent of the applications data-processing.

The driver will notify the application with an error-code if a buffer-overrun occurs. That is, if the application processes data too slow, so that the new samples have already overwritten unprocessed old ones.

The application then can freely decide how to handle this error case.

Buffer Setup and Buffer Ownership

The buffer itself is completely maintained inside the API – so the applications do not have to bother with allocation and de-allocation issues, which usually come with having a buffer.

However – to allow the application a fine granulated control over the buffer, it is able and obligated to indicate to the API the desired size of the buffer in terms of logical units, by using the integer-based functions. The application decides, how many scans one block shall hold, and how many blocks shall be allocated. The actual size in bytes is then calculated by the API and the buffer is allocated.

Buffer Readout from Application Point of View

The ring-buffer is exposed to the application by providing the related pointer information.

The API will provide:

• Start-pointer of the ring buffer

• End-Pointer of the ring buffer

• Pointer to the first unprocessed scan

Together with the information how many unprocessed samples are available the application iterates directly over the ring buffer.

This approach allows a minimal internal overhead on data-access.

Scan Descriptor

As mentioned before a scan is the portion of data containing exactly one sample per used channel. Without knowledge about its internal layout, this would just be a binary stream with arbitrary length.

But the application does not need to know implicitly about the layout of the data. This would be undesirable, as the layout may change with coming driver versions or coming hardware. For example, when a new type of synchronous data will be added, inherent hardcoded knowledge within the application would immediately break the data-readout mechanism of the application.

So after setting up the acquisition environment, the API can be queried about the layout.

The detailed layout-information will be returned as an XML-string.

Listing 9 BoardProperties - ScanDescriptor Example

<ScanDescriptor>
    <BoardId0>
        <ScanDescription version="2" scan_size="96" byte_order="little_endian" unit="bit">
            <Channel type="Analog" index="3" name="AI3">
                <Sample offset="32" size="24" />
            </Channel>
        </ScanDescriptor>
    </BoardId0>
</ScanDescriptor>

Scan Descriptor Structure

The following API string command returns the scan information for a specific Board:

DeWeGetParamStruct_str( "BoardId0", "ScanDescriptor_V2", Buf, sizeof(Buf));

The returned XML document correlates with the following hierarchy:

1. <ScanDescriptor> : XML Element. Max. Occurrences: 1.

2. <BoardID0> : XML Element. Max. Occurrences: 1.

3. <ScanDescription> : XML Element. Max. Occurrences: 1.

4. <Channel> : XML Element. Max. Occurrences: Unbounded.

5. <Sample> : XML Element. Max. Occurrences: Unbounded.

Please be aware that the scan descriptor annotates only the enabled channels for a specific Board. In case no channel is enabled, the API returns an empty scan descriptor with “scan_size” set to the value 0.

The API considers disabled channels and therefore the returned “scan_size” and “offsets” are being returned accordingly.

The following list depicts all possible XML Elements and their XML attributes and values of the returned scan descriptor XML document:

Table 3 TEDS XML description

Element	Attribute	Description
ScanDescriptor		ScanDescriptor root element
BoardIdXX		Selected board elememt “BoardID0”
ScanDescription		Describes the scan for the requested board
	version	Scan descriptor’s document version (Should be 2)
	scan_size	The size of the scan expressed in unit
	byte_order	The byte order of the scan (“little_endian”)
	unit	The unit of “scan_size” attribute (“bit”)
Channel		Channel element
	type	Value: string “Analog”, “Counter”, “Discrete”
	index	The channel index on the specific board
	name	API name of the channel “AI0”
Sample		Detailed sample description
	offset	The offset within the whole scan
	size	the size of the sample
	subChannel	Optional attribute for counter sub channels

Warning

When requesting a scan descriptor with command “ScanDescriptor” (Version 1), some boards may not be able to return a valid scan descriptor for analog 24bit channels. Therefore, always use “ScanDescriptor_V2”.

Warning

“ScanDescriptor” version 1 is deprecated and will be removed. It will return the V2 document in future.

Scan Descriptor Example Source Code

The next example extends the Quickstart app with a valid Scan Descriptor support class.

Listing 10 Scan Descriptor example

#include "dewepxi_load.h"
#include "dewepxi_apicore.h"
#include "dewepxi_apiutil.h"
#include "pugixml.hpp"
#include <functional>
#include <iomanip>
#include <iostream>
#include <string>
#include <vector>


/**
 * Functor to be implemented by applications interested in sample data.
 */
using AddSampleFunctor = std::function<void(const char*, uint32_t, 
    const void*, uint32_t, uint32_t, uint32_t, uint32_t)>;


/**
 * ScanDescripterDecode
 * Uses ScanDescriptor_V2 xml allowing generic
 * sample decoding and processing.
 */
class ScanDescriptorDecoder
{
public:
    ScanDescriptorDecoder(const std::string& sd_xml, AddSampleFunctor f)
        : m_datasink(f)
    {    
        parseScanDescriptor(sd_xml);
    }

    /**
     * parseScanDescriptor - parses V2 xml and build internal processing vector
     */
    void parseScanDescriptor(const std::string& sd_xml)
    {
        pugi::xml_document sd_doc;
        if (pugi::status_ok == sd_doc.load_string(sd_xml.c_str()).status)
        {
            auto scan_description_node = 
                sd_doc.select_node("ScanDescriptor/*/ScanDescription").node();
            if (scan_description_node)
            {
                if (2 != scan_description_node.attribute("version").as_int())
                {
                    throw std::runtime_error("Unsupported version");
                }

                m_scan_size_bytes 
                    = scan_description_node.attribute("scan_size").as_int() / 8;
                // Can be safely ignored:
                // bit
                // byte_order
                // buffer_direction
                // buffer

                auto channel_nodes = scan_description_node.select_nodes("Channel");
                for (auto channelx : channel_nodes)
                {
                    auto channel = channelx.node();

                    auto sample = channel.child("Sample");

                    m_scan_desc_vec.push_back(
                        SDData{ std::string(channel.attribute("name").as_string()) 
                                , std::string(channel.attribute("type").as_string())
                                , channel.attribute("index").as_uint()
                                , sample.attribute("size").as_uint()
                                , sample.attribute("offset").as_uint()
                        });
                }
            }
            else
            {
                throw std::runtime_error("ScanDescriptor unexpected element");
            }
        }
        else
        {
            throw std::runtime_error("ScanDescriptor parse error");
        }
    }

    /**
     * Process sample blocks 
     * @param avail_samples a continuous block of samples
     * @return incremented read_pos
     */
    sint64 processSamples(sint64 read_pos, int avail_samples)
    {
        auto read_pos_ptr = reinterpret_cast<const char*>(read_pos);

        for (const auto& sd : m_scan_desc_vec)
        {
            uint32_t offset_bytes = sd.sample_offset / 8;
            auto read_pos_ptr_chn = read_pos_ptr + offset_bytes;
            uint32_t channel_bit_mask = (1 << sd.sample_size) - 1;
            uint32_t channel_bit_msb = sd.sample_size;

            m_datasink(sd.name.c_str(), sd.index, read_pos_ptr_chn, avail_samples,
                channel_bit_mask, channel_bit_msb, m_scan_size_bytes);
        }

        return read_pos + (avail_samples * m_scan_size_bytes);
    }

private:
    AddSampleFunctor m_datasink;
    uint32_t m_scan_size_bytes;

    struct SDData
    {
        std::string name;
        std::string channel_type;
        uint32_t index;
        uint32_t sample_size;
        uint32_t sample_offset;
    };
    std::vector<SDData> m_scan_desc_vec;
};


/**
 * Data Sink - App interface
 * @param channel is the name of the channel
 * @param first_sample pointer to the first sample
 * @param nr_samples number of samples available
 * @param bitmask bit mask to apply to get the valid sample value
 * @param stride is the offset to the next sample
 */
void addSample(const char* channel_name,
            uint32_t channel_index,
            const void* first_sample,
            uint32_t nr_samples,
            uint32_t channel_bitmask,
            uint32_t channel_bit_msb,
            uint32_t stride)
{
    const char* data_ptr = static_cast<const char*>(first_sample);

    for (uint32_t i = 0; i < nr_samples; ++i)
    {
        uint32_t value = (*reinterpret_cast<const uint32_t*>(data_ptr)) & channel_bitmask;
        std::cout << channel_name << ": " << std::hex << value  << std::endl;
        data_ptr += stride;
    }
}

/**
 * Print samples in channel per column
 * Uses Functor interface like "addSample"
 */
class FormattedOutput
{
public:
    FormattedOutput(int num_channels, int block_size)
        : m_num_channels(num_channels)
        , m_block_size(block_size)
    {
        m_output_buffer.resize(m_num_channels);
    }

    void operator()(const char* channel_name,
        uint32_t channel_index,
        const void* first_sample,
        uint32_t nr_samples,
        uint32_t channel_bitmask,
        uint32_t channel_bit_msb,
        uint32_t stride)
    {
        const char* data_ptr = static_cast<const char*>(first_sample);

        m_output_buffer[channel_index].chn_name = channel_name;

        // copy samples in separate buffer per channel
        for (uint32_t i = 0; i < nr_samples; ++i)
        {
            // The bitmask has to be applied to cut out the correct value.
            // The MSB has to be checked to sign extend negative values
            int32_t value = (*reinterpret_cast<const int32_t*>(data_ptr)) & channel_bitmask;
            if ((value & (1 << (channel_bit_msb - 1))) != 0)
            {
                value |= ~channel_bitmask;
            }

            m_output_buffer[channel_index].chn_sample_buffer.push_back(value);
            data_ptr += stride;
        }

        // print channels in separate columns
        if (channel_index + 1 >= m_num_channels)
        {
            // Channel names
            for (uint32_t chn = 0; chn < m_num_channels; ++chn)
            {
                std::cout << std::setw(10) << m_output_buffer[chn].chn_name << ", ";
            }
            std::cout << std::endl;


            for (uint32_t i = 0; i < nr_samples; ++i)
            {
                for (uint32_t chn = 0; chn < m_num_channels; ++chn)
                {
                    auto value = m_output_buffer[chn].chn_sample_buffer[i];
                    std::cout << std::setw(10) << std::hex << value << ", ";
                }

                std::cout << std::endl;
            }

            m_output_buffer.clear();
            m_output_buffer.resize(m_num_channels);
        }
    }

    struct ChannelSamples
    {
        std::string chn_name;
        std::vector<int32_t> chn_sample_buffer;
    };

    std::vector<ChannelSamples> m_output_buffer;
    uint32_t m_num_channels;
    int m_block_size;
};



int main(int argc, char* argv[])
{
    int boards = 0;
    int avail_samples = 0;
    const int32_t buffer_block_size = 10;
    int64_t buf_end_pos = 0;        // Last position in the ring buffer
    int buff_size = 0;              // Total size of the ring buffer
    char scan_descriptor[8192] = { 0 };

    FormattedOutput output(4, buffer_block_size);

    // Basic SDK Initialization
    DeWePxiLoad();

    // boards is negative for simulation
    DeWeDriverInit(&boards);

    // Open boards
    // 0: chassis controller
    // 1: TRION3-1850-MULTI
    DeWeSetParam_i32(0, CMD_OPEN_BOARD, 0);
    DeWeSetParam_i32(0, CMD_RESET_BOARD, 0);
    DeWeSetParam_i32(1, CMD_OPEN_BOARD, 0);
    DeWeSetParam_i32(1, CMD_RESET_BOARD, 0);

    // Enable all analog channels
    DeWeSetParamStruct_str("BoardID1/AIAll", "Used", "True");

    // Configure acquisition properties
    DeWeSetParam_i32(1, CMD_BUFFER_0_BLOCK_SIZE, buffer_block_size);
    DeWeSetParam_i32(1, CMD_BUFFER_0_BLOCK_COUNT, 200);
    DeWeSetParamStruct_str("BoardID1/AcqProp", "SampleRate", "100");

    // Apply settings
    DeWeSetParam_i32(1, CMD_UPDATE_PARAM_ALL, 0);

    // Get buffer configuration
    DeWeGetParam_i64(1, CMD_BUFFER_0_END_POINTER, &buf_end_pos);
    DeWeGetParam_i32(1, CMD_BUFFER_0_TOTAL_MEM_SIZE, &buff_size);

    // Get scan descriptor
    DeWeGetParamStruct_str("BoardId1", "ScanDescriptor_V2", scan_descriptor, sizeof(scan_descriptor));
    
#if 1
    // Connect to formatted output
    ScanDescriptorDecoder sd_decoder(scan_descriptor, output);
    
#else
    // or addSample function
    //ScanDescriptorDecoder sd_decoder(scan_descriptor, &addSample);
#endif

    // Start acquisition
    DeWeSetParam_i32(1, CMD_START_ACQUISITION, 0);

    // Measurement loop and sample processing
    int64_t read_pos = 0;
    int64_t* read_pos_ptr = 0;
    int sample_value = 0;

    // Break with CTRL+C only
    while (1)
    {
        // Get the number of samples available
        DeWeGetParam_i32(1, CMD_BUFFER_0_AVAIL_NO_SAMPLE, &avail_samples);
        if (avail_samples <= 0)
        {
            Sleep(100);
            continue;
        }

        // Get the current read pointer
        DeWeGetParam_i64(1, CMD_BUFFER_0_ACT_SAMPLE_POS, &read_pos);

        // Process samples in CMD_0_BUFFER_BLOCK_SIZE
        // to handle ring buffer wrap arounds only here:
        auto avail_samples_to_process = avail_samples;
        while (avail_samples_to_process > 0)
        {
            read_pos = sd_decoder.processSamples(read_pos, buffer_block_size);
            avail_samples_to_process -= buffer_block_size;

            // Handle the ring buffer wrap around
            if (read_pos >= buf_end_pos)
            {
                read_pos -= buff_size;
            }
        }

        DeWeSetParam_i32(1, CMD_BUFFER_0_FREE_NO_SAMPLE, avail_samples);
    }


    // Stop acquisition
    DeWeSetParam_i32(1, CMD_STOP_ACQUISITION, 0);

    // Free boards and unload SDK
    DeWeSetParam_i32(0, CMD_CLOSE_BOARD, 0);
    DeWeSetParam_i32(1, CMD_CLOSE_BOARD, 0);
    DeWeDriverDeInit();
    DeWePxiUnload();

    return 0;
}

ADC Delay

AD converters have a conversion time. Analog samples may appear in later scans than time-wise corresponding digital channels. The ADCdelay is used to allow the application to align samples of analog and digital channel types.

Table 4 ADC delay effect on samples (for ADCDelay = 4)

Sample Nr	AI0	AI1	CNT0	DI0
0	invalid	invalid	0	0
1	invalid	invalid	1	1
2	invalid	invalid	2	2
3	invalid	invalid	3	3
4	0	0	4	4
5	1	1	5	5
6	2	2	6	6
7	3	3	7	7

After acquisition start the samples from index 0 to 3 (== ADCDelay) are marked as invalid. There will be values, but because of AD conversion time they will be more or less randomized.

The value of ADCDelay is board dependend and can be requested with CMD_BOARD_ADC_DELAY.

Note

Please have look at the example “ADCDelay” showing a way for applying the ADC delay to realign AI samples to the other channels.

Sample Rate

Synchronous Acquisition

Asynchronous Acquisition

Data Output