How to configure sampling on Data Stream

When you configure a stream and choose to associate all of your domains, you can configure the Sampling option to filter the percentage of data you’ll be streaming. It’s a viable option to reduce the costs of data collection and analysis.

Access Azion Console > Data Stream.
To configure the stream, follow the steps in How to configure Data Stream main settings.
On Domains > Options, choose All Curent and Future Domains.
On Sampling (%), add the value of the percentage of data you want to stream, from 0 to 100. Example: 60.
- Data Stream will collect data randomly according to the percentage you choose.
Finish configuring the Destination section.
Click the Save button.

Access Real-Time Manager > Data Stream.
To configure the stream, follow the steps in How to configure Data Stream main settings.
On Options, choose All Domains. This will associate all of the account’s current and future domains to this stream.
On Sampling (%), add the value of the percentage of data you want to stream, from 0 to 100. Example: 60.
- Data Stream will collect data randomly according to the percentage you choose.
Finish configuring the Destination section.
Click the Save button.

For this section, you’ll be creating a stream with an Apache Kafka connector via API. See other connector options in the Data Stream Endpoints table.

Run the following GET request in your terminal, replacing [TOKEN VALUE] with your personal token:

curl --location 'https://api.azionapi.net/data_streaming/streamings' \
--header 'Accept: application/json; version=3' \
--header 'Authorization: Token [TOKEN VALUE]' \
--header 'Content-Type: application/json' \
--data '{
    "name": "Kafka Connector",
    "template_id": 2,
    "domain_ids": [1656613172],
    "data_source": "http",
    "endpoint": {
        "endpoint_type": "kafka",
        "kafka_topic": "mykafka.dts.topic",
        "bootstrap_servers": "infra.my.net:9094,infra.my.net:9094"
    },
    "all_domains": true,
    "sampling_percentage": 60
}'

You’ll receive a response similar to this:

{
  "results": {
    "id": 1594,
    "name": "Kafka Connector",
    "template_id": 2,
    "data_source": "http",
    "active": true,
    "endpoint": {
      "endpoint_type": "kafka",
      "use_tls": false,
      "kafka_topic": "mykafka.dts.topic",
      "bootstrap_servers": "infra.my.net:9094,infra.my.net:9094"
    },
    "all_domains": true,
    "sampling_percentage": 60
  },
  "schema_version": 3
}

Wait a few minutes for the changes to propagate and your stream will be updated.