Skip to content

Configure Multiple Master Nodes (MMN) in EMR

If you are using multiple master nodes in Amazon EMR, you need to update the EMR creation script to include the Privacera configurations for multiple master nodes.

  1. SSH to the instance where Privacera is installed.

  2. Run the following command to navigate to the /config directory.

    Bash
    1
    cd ~/privacera/privacera-manager/config

  3. Run the following command to open the .yml file to be edited.

    Bash
    1
    vi custom-vars/vars.emr.yml

  4. Add the below property and set the value to the desired number of master nodes. For example, to set the number of master nodes to 3, add the following property:

    Bash
    1
    EMR_MASTER_NODE_COUNT: 3

  5. Once the properties are configured, run the following commands to update your Privacera Manager platform instance:

    Step 1 - Setup which generates the helm charts. This step usually takes few minutes.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh setup
    Step 2 - Apply the Privacera Manager helm charts.
    Bash
    1
2
    cd ~/privacera/privacera-manager
./pm_with_helm.sh upgrade
    Step 3 - Post-installation step which generates Plugin tar ball, updates Route 53 DNS and so on.

    Bash
    1
2
    cd ~/privacera/privacera-manager
./privacera-manager.sh post-install

  6. After the post-install, create a new cluster with newly generated emr-template.json file from output directory.

Update hive-site configuration in emr template as below and create new emr cluster with this template.

privacera-emr-config
JSON
1
2
3
4
5
"CoreInstanceGroup": {
  "Name": "Core Instance Group",
  "InstanceCount": 3,
  "InstanceType": "<instance-type>"
}

Comments