How to configure sampling on Data Stream

When you configure a stream and choose to associate all of your domains, you can configure the Sampling option to filter the percentage of data you’ll be streaming. It’s a viable option to reduce the costs of data collection and analysis.

Access Azion Console > Data Stream.
To configure the stream, follow the steps in How to configure Data Stream main settings.
On Domains > Options, choose All Curent and Future Domains.
On Sampling (%), add the value of the percentage of data you want to stream, from 0 to 100. Example: 60.
- Data Stream will collect data randomly according to the percentage you choose.
Finish configuring the Destination section.
Click the Save button.

For this section, you’ll be creating a stream with an Apache Kafka connector via API. See other connector options in the Data Stream Endpoints table.

Run the following POST request in your terminal, replacing [TOKEN VALUE] with your personal token:

curl --request POST \
  --url https://api.azion.com/v4/data_stream/streams \
  --header 'Accept: application/json' \
  --header 'Authorization: [TOKEN VALUE]' \
  --header 'Content-Type: application/json' \
--data '{
    "name": "Kafka Connector",
    "template_id": 2,
    "workloads": [1656613172],
    "data_source": "http",
    "endpoint": {
        "endpoint_type": "kafka",
        "kafka_topic": "mykafka.dts.topic",
        "bootstrap_servers": "infra.my.net:9094,infra.my.net:9094"
    },
    "all_domains": true,
    "sampling_percentage": 60
}'

You’ll receive a response similar to this:

{
  "results": {
    "id": 1594,
    "name": "Kafka Connector",
    "template_id": 2,
    "data_source": "http",
    "active": true,
    "endpoint": {
      "endpoint_type": "kafka",
      "use_tls": false,
      "kafka_topic": "mykafka.dts.topic",
      "bootstrap_servers": "infra.my.net:9094,infra.my.net:9094"
    },
    "all_domains": true,
    "sampling_percentage": 60
  },
  "schema_version": 3
}

Wait a few minutes for the changes to propagate and your stream will be updated.