The AWS S3 connector is designed to fetch data using the Amazon Simple Storage Service API. For authentication to the REST interfaces, for each AWS account a key/secret pair has to be created in the AWS Portal. When sending API requests to AWS, it is possible to sign the requests, so AWS can identify who sent them. Sign requests with the AWS access key, which consists of an access key ID and a secret access key. For enterprise-wide access, an IAM (Identity and Access Management) role-based method is recommended.
User-based Authentication
•Access Key ID
The Access Key for connecting to the Security Token Service (STS)
•Secret Access Key
The Secret Access Key for connecting to the STS.
For using the connectors, the following AWS policy is required. It defines read-only access to the services EC2 and RDS (background information on configuring IAM policies/roles/users is provided below in this document):
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "ec2:DescribeRegions", "ec2:DescribeInstances", "ec2:DescribeReservedInstances", "ec2:DescribeImages", "rds:DescribeDBInstances", "s3:HeadBucket", "s3:ListBucket", "s3:GetObject" ], "Resource": "*" } ] } |
The resources scope can of course be restricted to the desired scope, e.g. only one specific s3 bucket should be accessible.
Assign this policy to an IAM user and create an access key for the user:
Role-based Authentication
For large organizations, role-based access is easier to maintain than individual key-secret pairs and policies per AWS account. The organization should configure an AWS trust relationship.
For example, IAM roles can be used to grant permissions to applications running on instances. It is possible to specify permissions for IAM roles by creating a policy in the JSON format. These policies are similar to the ones created for the IAM users. When making a change to a role, the change is propagated to all instances.
Therefore for a connection the following parameters are required:
•Session Token: The Session token for connecting to the STS.
•Assume Role Arn Chain: Use the STS to assume the given roles one after another.
•External ID: Provide an external ID for cross-account access with the STS.
•List Accounts: List all accounts in the OU and fetch the data for all of them.
•Bucket name: The name of the bucket form which to download the files.
•AWS Regions: Comma-seperated list of AWS regions from where to fetch the data. Omit or set to empty string in order to fetch from all active regions.
•Object Keys: A list of keys of the S3 Object(s) which should be downloaded. Cannot be combined with the Object Key Pattern parameter.
•Filter for License Models: Only extract databases which have the specified license model. Possible valutes: \"bring-your-own-license\", \"license-included\",\"general-public-license\".
How to get the credentials is described in the following URL: https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys.
The official documentation about IAM roles can be found here: https://docs.aws.amazon.com/IAM/latest/UserGuide/tutorial_cross-account-with-roles.html.