Skip to main content
Version: v4.12

Connect to S3

Collect the connection details and certificates

To connect to S3 buckets through the Cyral sidecar, follow these steps:

  1. Get the sidecar endpoint address and port number used for S3 connections:

    • As a Cyral admin, open the Cyral control plane UI.
      • Click Sidecars and click on the name of your sidecar.
      • Copy the sidecar host from the top of the page, right below the sidecar name.
      • The port can be collected from the Bindings tab.
    • As an end-user:
      • Go to the Cyral data access portal and select the desired S3 repository.

      • Follow the steps to configure the AWS CLI with the Cyral CLI.

      • Open the AWS configuration file, find the profile created (or updated) by the Cyral CLI and copy the proxy endpoint, as illustrated below:

        [profile <your-profile-name>]
        ca_bundle = /home/<user>/.aws/cyral_ca_bundle.pem
        s3 =
        proxy = http://<sidecar-host>:<port>
        s3api =
        proxy = http://<sidecar-host>:<port>
  2. Download the certificate authority (CA) bundle to verify TLS/SSL connections.

    The data traffic is encrypted with the sidecar's own certificate. To properly operate, client applications should use the sidecar's CA bundle to verify TLS/SSL connections. This step is optional, but recommended. Client applications can also be configured to not verify the Cyral-provided TLS/SSL certificates.

    • In case you already made the AWS CLI configuration using the Cyral CLI, the CA bundle file should be already present in your local machine. Its location is specified in the AWS configuration file for the profile you created with the Cyral CLI tool, referenced by the ca_bundle key. Please check the example configuration payload above for a reference to the CA bundle file.

    • Alternatively, you can ask your admin to provide you the CA bundle to use with your encrypted connections.

Connect to S3

  1. Make sure you have followed the steps described above.

  2. Connect to S3 using your preferred client. User authentication relies on the credentials you collected from the Cyral Control Plane UI. If present, Cyral policies are enforced. All traffic is sent through the Cyral sidecar, which logs all data activity to the log location configured in your Cyral installation.

Connect using the AWS SDK for Python:

In the proxies configuration block (in the code below), specify the host and port of the sidecar in the format http://<sidecar_endpoint>:<sidecar_port> for both http and https keys. The scheme in the URL must be http. The final connection between client applications and the S3 servers will still be TLS encrypted.

For the example code below, it is assumed that the sidecar host and port are example-sidecar-endpoint.com:453.

Set verify to the path of the Cyral CA bundle.

import boto3
from botocore.config import Config

# reference AWS documentation:
# https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html

def sidecar_example():
sidecar_certificate_bundle = '/path/to/cyral_ca_bundle.pem'
sidecar_endpoint = 'example-sidecar-endpoint.com'
sidecar_port = 453

region = 'us-east-2'

s3_client = boto3.client('s3',
aws_access_key_id="YOUR_AWS_ACCESS_KEY_ID",
aws_secret_access_key="YOUR_AWS_SECRET_ACCESS_KEY",
region_name=region,
verify=sidecar_certificate_bundle,
config=Config(
proxies={
"http": f"http://{sidecar_endpoint}:{sidecar_port}",
"https": f"http://{sidecar_endpoint}:{sidecar_port}",
}),
)

print('Running List Tables command through the sidecar')
result = s3_client.list_tables()
print(result)

if __name__ == "__main__":
sidecar_example()

Alternatively, the following environment variables are applicable to the Python AWS SDK HTTP_PROXY, HTTPS_PROXY, and AWS_CA_BUNDLE. If a shell session is configured with these variables, all data traffic will go through the sidecar, potentially including traffic not related to S3 itself, so the code snipped presented above is preferable in favor of environment variables.

It is recommended to keep certificate validation always enabled. However, it can be temporarily disabled, to perform connectivity validation, by setting verify=False.

Note on other AWS SDKs

AWS offers SDKs for a wide range of programming languages, while here we only show configuration examples for a subset of them. If examples for your programming language are missing from this page, we encourage you to either look for examples in AWS official documentation at https://aws.amazon.com/developer/tools/, or contact Cyral support for help.