How to use Amazon S3 to receive data from Data Stream

While configuring streams, you need to set up a specific endpoint to stream your Azion data.

Continue reading for a step by step on how to connect an Amazon S3 endpoint to receive data from Data Stream.

Amazon S3 requirements

To get started, you must complete a few configurations in Amazon S3:

Create an Amazon AWS account.
Register an Identity and Access Management (IAM) credential.
Create an Amazon S3 bucket.
Optionally, you can also upload an object.
Optionally, you can set up and manage Server Side Encryption (SSE) with Amazon S3-Managed Keys (SSE-S3).

You’ll also need to save and create the following information:

Region
Access Key
Secret Key

Region

This is the region where your Amazon S3 is running.

On the top bar of your dashboard, right next to the help button with the ’?’ icon, you find a dropdown menu, the Regions selector, where you can see the name of the region and its code.
Find the code and keep it to use it later. It looks similar to this: us-east-1

You can find more details on the AWS Regions and Zones documentation.

Access Key and Secret Key

Access the Identify and Access Management (IAM) panel.
Click the Create new access key button.
A pop-up will appear on the screen with your key information.
Click on Display Access Key to display your key information as indicated below:
- Access Key ID, with a format similar to AKIA5LVCRTER524RAIA3
- Secret Access Key, with a format similar to N9UTjunb299M5ryfKdG2cKh2+Pe2mBiGZXaBpVff
Save the two keys information in a safe place to use them later on Azion Data Stream.

You can also download the EMP file with the information for the keys.

For more details on setting up Amazon S3, see the AWS documentation page.

Configuring the endpoint in Data Stream

Next, follow these steps to configure the new endpoint you created in Amazon S3 in your Azion Data Stream.

You can find detailed steps for the entire configuration on the How to use Data Stream guide.

In the Destination configurations:

On the Connector dropdown menu, select Simple Storage Service (S3).
On URL, add the URL of your S3 host.
On Bucket Name, add the name of your S3 bucket on AWS.
On Region, add the region configured on AWS.
On Access Key, add the access key you created on AWS.
On Secret Key, add the secret key you created on AWS.
On Object Key Prefix (optional), add the prefix name of your uploaded object. Example: WAF_
On Content Type, add the format of the object that’ll be created in your S3 bucket. It can be plain/text or application/gzip.
Make sure the Active switch is on.
Click the Save button.

On the Endpoint Type dropdown menu, select Simple Storage Service (S3).
On Host URL, add the URL of your S3 host.
On Bucket Name, add the name of your S3 bucket on AWS.
On Region, add the region configured on AWS.
On Access Key, add the access key you created on AWS.
On Secret Key, add the secret key you created on AWS.
On Object Key Prefix (optional), add the prefix name of your uploaded object. Example: WAF_
On Content Type (optional), add the format of the object that’ll be created in your S3 bucket. It can be plain/text or application/gzip.
Make sure the Active switch is on.
Click the Save button.

After saving the configurations, your data will be streamed to the newly configured endpoint.

You can keep track of the calls made by Data Stream to Amazon S3 on Real-Time Events. To do so, select Data Source > Data Stream and choose the filters options as you wish.

Watch a video tutorial on how to use Amazon S3 with Data Stream on Azion’s YouTube channel:

Trademarks

Amazon Simple Storage Service is a registered trademark of Amazon Web Services in the United States and other countries.