Skip to content

Customizing EBS Root Volume Size and Auto Termination Policy

If you want to customize the EBS root volume size and auto-termination policy for the EMR cluster, you can do so by updating the configuration in the emr-template.json file.

  • SSH into the instance where Privacera Manager is installed.
  • Run the following command to navigate to the /config directory.
    Bash
    cd ~/privacera/privacera-manager/config
    
  • Run the following command to open the .yml file to be edited.
    Bash
    vi custom-vars/vars.emr.yml
    
  • To increase the EBS root volume size, modify the following property:
    Bash
    1
    2
    3
    # Uncomment the below property to set EBS root volume to the desired size in GB.
    # By default, it is set to 20.
    EMR_EBS_ROOT_VOLUME_SIZE: "<PLEASE_CHANGE>"
    
  • To set the auto-termination policy, modify the following property:
    Bash
    # Uncomment the below property to set EMR Auto termination time in seconds.
    EMR_AUTO_TERMINATION_TIMEOUT: "<PLEASE_CHANGE>"
    
  • Once the properties are configured, run Privaera Manager to generate the updated emr-template.json file. Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    cd ~/privacera/privacera-manager
    ./privacera-manager.sh setup
    
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    cd ~/privacera/privacera-manager
    ./pm_with_helm.sh upgrade
    
    Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

    Bash
    cd ~/privacera/privacera-manager
    ./privacera-manager.sh post-install
    
  • After the post-install, create a new cluster using the newly generated emr-template.json file from the output directory.

To increase the EBS root volume size and set the auto-termination policy, update the resources configuration in emr template as shown below. Then, create a new emr cluster with this template:

privacera-emr-resources
  "Resources":{
    "EMRCLUSTER":{
      "Type":"AWS::EMR::Cluster",
      "Properties":{
        "EbsRootVolumeSize":<SIZE_IN_GB>,
        "AutoTerminationPolicy": {
          "IdleTimeout": <TIME_IN_SECONDS>
        }
      }
    }
  }
}

Comments