Accessing Firehose with Data Access Server
Create a Firehose delivery stream and set up access control for it.
Firehose Delivery Stream Queries
-
From a terminal prompt, create a new delivery datastream 'SalesDataDeliveryStream'.
-
(–delivery-stream-type : KinesisStreamAsSource)
aws firehose create-delivery-stream --delivery-stream-name SalesDataDeliveryStream --delivery-stream-type KinesisStreamAsSource --kinesis-stream-source-configuration "KinesisStreamARN=arn:aws:kinesis:us-east-1:857494200836:stream SalesDataStream,RoleARN=arn:aws:iam::857494200836:role/privacera_user_role" --extended-s3-destination-configuration "BucketARN=arn:aws:s3:::sales-data-stream-bucket,RoleARN=arn:aws:iam::857494200836:role/privacera_user_role" --region us-east-1
-
Options:
-delivery-stream-name: delivery datastream name
-delivery-stream-type: input source type (Kinesis DataStream or DirectPut)
-kinesis-stream-source-configuration (Source Kinesis DataStream ARN)
-extended-s3-destination-configuration (Destination S3 bucket ARN)
-region
It will show the following result An error occurred (403)
-
(–delivery-stream-type : DirectPut)3 nju88ik;
aws firehose create-delivery-stream --delivery-stream-name SalesDataDeliveryStream --delivery-stream-type DirectPut --extended-s3-destination-configuration "BucketARN=arn:aws:s3:::sales-data-stream-bucket,RoleARN=arn:aws:iam::857494200836:role/privacera_user_role” --region us-east-1
It will show the following result An error occurred (403).
This indicates that the current user doesn’t have permission to perform this operation.
-
-
Check the audit log for the related event (Access Manager > Audit).
-
Create three Ranger policies for this scenario.
-
Firehose Policy to have CreateDeliveryStream on SalesDataDeliveryStream.
-
S3 Policy to provide on input-data bucket location and for OutputLocation to query.
-
Kinesis Policy required for source input (Required if –delivery-stream-type : KinesisStreamAsSource).
-
Firehose Policy
-
On the Privacera Portal home page, expand Access Management and click Resource Policies from the left menu.
-
On the Resource Policies page, go to privacera_kinesis and then select Firehose to create policy for Firehose.
-
Enter the following details as:
-
Policy Name: SalesDataDeliveryStreamPolicy
-
kinesis_firehose: SalesDataDeliveryStream (Firehose datastream which you want to allow the user to create)
-
Under Allow Conditions, click the '+' icon and select the below:
-
User: User’s username to which you want to allow access.
-
Add Permission as: CreateDeliveryStream
-
-
S3 Policy
-
On the Resource Policies page, go to privacera_kinesis and then click Add New Policy to create policy for S3.
-
Enter the following details as:
-
Policy Name: SalesDataDeliveryStreamPolicy
-
bucket: sales-data-stream-bucket (destination S3 bucket)
-
object: *
-
Under Allow Conditions, click '+' icon and select the below:
-
User: username to which you want to allow access.
-
Permission: metadata read, metadata write, write
-
-
Kinesis Policy
-
On the Resource Policies page, go to privacera_kinesis and click Add New Policy and then select Firehose to create policy for Kinesis.
-
Enter the following details as:
-
Policy Name: SalesDataStreamPolicy
-
kinesis_datastream: SalesDataStream
-
Under Allow Conditions, click '+' icon and select the below:
-
User: User’s username to which you want to allow access.
-
Permission: GetRecords, GetShardIterartor, DescribeStream
-
-
-
Now, run the query from Step 1.
aws firehose create-delivery-stream --delivery-stream-name SalesDataDeliveryStream --delivery-stream-type KinesisStreamAsSource --kinesis-stream-source-configuration "KinesisStreamARN=arn:aws:kinesis:us-east-1:857494200836:stream/SalesDataStream,RoleARN=arn:aws:iam::857494200836:role/privacera_user_role" --extended-s3-destination-configuration "BucketARN=arn:aws:s3:::sales-data-stream-bucket,RoleARN=arn:aws:iam::857494200836:role/user_role" --region us-east-1