Skip to main content

How to Configure an AWS S3 Object Storage

Login in to your AWS account and navigate to the S3 console.

1. Begin by Creating an S3 Bucket​

  1. Login in to your AWS account and navigate to the AWS S3 console dashboard

  2. Click the "Create Bucket" button to start the process.

Thumbnail

2. General Configuration Setup:​

  1. AWS Region: Select the region where you want the bucket to be created. In this example, the region is US East (N. Virginia) us-east-1.

  2. Bucket Type: Choose General purpose for a standard bucket with redundancy across multiple Availability Zones. Alternatively, select Directory for low-latency use cases where data is stored in a single Availability Zone.

  3. Bucket Name: Enter a unique name for your bucket. For example: blink-runner-object-storage.

    tip

    Follow AWS bucket naming rules for a valid name.

    Thumbnail

  4. Under Object Ownership:

  • Select ACLs disabled (recommended). This ensures that all objects in the bucket are owned by your AWS account, and access is specified via bucket policies only. Alternatively, you can enable ACLs enabled if you need objects to have different ownership rules.

  • Ensure that Bucket owner enforced is shown as the Object Ownership setting.

  1. Set Block Public Access Settings

    1. Enable the Block all public access option. This is highly recommended for security unless you specifically need public access for your use case.
    2. Ensure all the following options are checked:
      • Block public access to buckets and objects granted through new access control lists (ACLs).
      • Block public access to buckets and objects granted through any access control lists (ACLs).
      • Block public access to buckets and objects granted through new public bucket or access point policies.
      • Block public and cross-account access to buckets and objects through any public bucket or access point policies.

3. Create a New IAM User​

Set up a user to manage access to the S3 bucket:

  1. Navigate to the IAM Console and to the IAM (Identity and Access Management) service.

  2. Specify user details by providing a user name and then click the "Next" button.

Thumbnail

4. Set Permissions​

  1. choose the "Attach policies directly" option.
Thumbnail
  1. Next, Click the "Create Policy" option.
Thumbnail

  1. Paste the provided JSON policy in the policy editor and proceed.

    {
      "Version": "2012-10-17",
      "Statement": [
        {
          "Sid": "AllowListObjects",
          "Effect": "Allow",
          "Action": ["s3:ListBucket"],
          "Resource": ["arn:aws:s3:::<bucket>"]
        },
        {
          "Sid": "AllowObjectsCRUD",
          "Effect": "Allow",
          "Action": ["s3:DeleteObject", "s3:GetObject", "s3:PutObject"],
          "Resource": ["arn:aws:s3:::<bucket>/*"]
        }
      ]
    }
    Thumbnail

5. Review and Finalize the Policy​

  1. Review the Policy Details: Once you've reviewed and confirmed the details, click the Create policy button at the bottom right of the screen.

    Thumbnail

6. Review and Create User​

  1. Carefully review all the details, and complete the process by clicking "Create User".
Thumbnail

7. Enable Security Features for the IAM User​

  1. Navigate to the Security Credentials Tab:

  2. Create an Access Key: In the Access keys section, click the Create access key button and follow the steps:

    1. Purpose: Select the appropriate use case for the access key (e.g., programmatic access).
    2. Download Credentials: After creating the access key, securely download and store the key pair (Access Key ID and Secret Access Key). This is your only chance to view and save the secret access key.
    tip

    Treat the access key as sensitive information and avoid hardcoding it into your applications. Use AWS Secrets Manager or environment variables instead.

    Thumbnail

8. Add Optional Description Tags​

After creating and securely storing the AWS credentials, use them to deploy the Blink Runner. Follow the appropriate instructions for Kubernetes or Docker deployment:

Deploying the Runner in Kubernetes​

  1. Run the Helm command provided by Blink, but make sure to include the additional configurations below to enable object storage for the runner:

     --set storage.s3.accessKeyId=<access_key_id> \
     --set storage.s3.secretAccessKey=<secret_access_key> \
     --set config.storage.execution.bucket=blink-runner-bucket \
     --set config.storage.execution.region=us-east-1
  • Replace <access_key_id> and <secret_access_key> with the credentials created in Step 7
  • Replace blink-runner-bucket with the bucket name created in Step 2.
  • Ensure the region matches your bucket’s region (e.g., us-east-1).

Deploying the Runner in Docker​

Run the Helm command provided by Blink in the UI, but make sure to include the additional configurations below to enable object storage for the runner:

  -e STORAGE_S3_ACCESS_KEY_ID=YOUR_AWS_ACCESS_KEY \
  -e STORAGE_S3_SECRET_ACCESS_KEY=YOUR_AWS_SECRET_KEY \
  -e STORAGE_EXECUTION_BUCKET='{"region":"us-east-1","bucket":"your-bucket-name"}' \
  • Replace <YOUR_AWS_ACCESS_KEY> and <YOUR_AWS_SECRET_KEY> with the credentials created in Step 7.
  • Replace <auth_key> with the authentication key for the runner.
  • Ensure the region matches your bucket’s region (e.g., us-east-1).
  • Replace blink-runner-bucket with the bucket name created in Step 2.