Greenplum 6.7.1 on AWS

How to install a three-node Greenplum Database cluster with segment mirroring

Source: https://twitter.com/Greenplum

Overview

II. System Requirements

III. Part 1 — Create AWS Instances (~15 minutes)

IV. Part 2 — Setup on Each Node (~15 minutes)

V. Part 3 — Initialize Greenplum Database (~15 minutes)

Introduction

The basic structure of a Greenplum cluster involves one master node and one or more segment nodes each running an independent Postgres instance. The master node serves as the entry point for client requests and segments nodes each store a portion of accessible data. Useful information can be found on the Pivotal Docs page for Greenplum here.

Source: https://gpdb.docs.pivotal.io/6-7/admin_guide/intro/arch_overview.html

System Requirements

NOTE: This tutorial assumes basic familiarity with creating and administering AWS instances, using SSH, using SCP, and basic Linux administration using BASH.

Part 1 — Create AWS Instances

AWS Instance Details:

  • Minimum recommended instance type: r5ad.large
  • Number of instances: 3
  • Subnet: 172.31.0.0/16 (Use your default subnet; anywhere this subnet is used in the tutorial, substitute your own)
  • Inbound Security Rules: SSH from My IP; SSH from subnet; All TCP from subnet; All ICMP IPv4 from subnet
Sample inbound rules (personal IP address edited out)

NOTE: Listed Security Rules are not secure. Specific ports to consider include the various ports for Greenplum to communicate among nodes and the Postgres default port. Specific info about ports can be found in this Pivotal Docs page.

Part 2—Setup on Each Node

  • Download Greenplum 6.7.1 deb package from GitHub.
Source: https://github.com/greenplum-db/gpdb/releases
  • Secure copy the package to each instance.
  • SSH to each instance and change the hostname.
# Master Node
sudo hostnamectl set-hostname master;
sudo reboot;
# Segment Node 1
sudo hostnamectl set-hostname seg1;
sudo reboot;
# Segment Node 2
sudo hostnamectl set-hostname seg2;
sudo reboot;
  • Edit /etc/hosts file on with the new hostnames corresponding to their private IPs.
Sample lines for /etc/hosts file
  • Edit /etc/ssh/sshd_config to allow PasswordAuthentication.
Set “PasswordAuthentication” to “yes”
  • Restart the sshd service.
sudo service sshd restart
  • Create storage areas for Greenplum to use.
# Master Node
sudo mkdir -p /gpdata/master;
sudo mkdir -p /gpdata/mirror
# Segment Nodes
sudo mkdir -p /gpdata/primary;
sudo mkdir -p /gpdata/mirror
  • Install the deb package.
sudo dpkg -i greenplum-db-6.7.1-ubuntu18.04-amd64.deb
  • Create the gpadmin group and user.
sudo groupadd gpadmin;
sudo useradd gpadmin -r -m -g gpadmin;
sudo chsh -s /bin/bash gpadmin;
sudo passwd gpadmin
  • Assign ownership of storage areas to “gpadmin” user.
  • Assign ownership of Greenplum files to “gpadmin” user.
sudo chown -R gpadmin:gpadmin /gpdata;
sudo chown -R gpadmin:gpadmin /usr/local/greenplum-db-6.7.1
  • Log in as gpadmin and create a key pair.
su gpadminssh-keygen -t rsa -b 4096 
# Accept defaults; Do not input a password
  • As gpadmin, import Greenplum environment variables.
source /usr/local/greenplum-db-6.7.1/greenplum_path.sh
  • (Optional) Add lines to gpadmin’s .bashrc for ease of use and source the edited file.
echo "source /usr/local/greenplum-db-6.7.1/greenplum_path.sh; export MASTER_DATA_DIRECTORY=/gpdata/master/gpseg-1; cd ~" >> ~/.bashrc; source ~/.bashrc
Sample optional lines for .bashrc for gpadmin

Part 3—Initialize Greenplum Database

  • Log in as gpadmin to the master node and run ssh-copy-id for all three nodes. You will need to input the gpadmin password for each node.
ssh-copy-id masterssh-copy-id seg1ssh-copy-id seg2
Sample run of ssh-copy-id
  • Create a file with the hostname of each node in a file called “hostlist”.
Sample hostlist file
  • Create a file with only the segment hostnames in a file called “hostlist_segonly”.
  • Run the “gpssh-exkeys” command that comes with the Greenplum installation.
gpssh-exkeys -f hostlist
  • Copy the example intialization file to the current directory (you may want to switch to /home/gpadmin).
cp /usr/local/greenplum-db-6.7.1/docs/cli_help/gpconfigs/gpinitsystem_config ./
  • Change the following lines in the copied initializtion file:
declare -a DATA_DIRECTORY=(/data1/primary /data1/primary /data1/primary /data2/primary /data2/primary /data2/primary)
declare -a DATA_DIRECTORY=(/gpdata/primary /gpdata/primary /gpdata/primary /gpdata/primary)
# The above line forces the creation of four segments per segment nodes because the location "/gpdata/primary" appears four times.
# Any locations or combination of locations can be added here.
# Appropriate segmentation is addressed in the Pivotal Docs
here.
MASTER_HOSTNAME=mdw
MASTER_HOSTNAME=master
MASTER_DIRECTORY=/data/master
MASTER_DIRECTORY=/gpdata/master
#DATABASE_NAME=name_of_database
DATABASE_NAME=testdatabase_1
#MACHINE_LIST_FILE=/home/gpadmin/gpconfigs/hostfile_gpinitsystem
MACHINE_LIST_FILE=/home/gpadmin/hostlist_segonly
Sample gpinitsystem_config file
  • Run the Greenplum initialization.
gpinitsystem -c gpinitsystem_config
  • Enter “Y” when prompted to “Continue with Greenplum creation”.
Greenplum initialization success message
  • Enable segment mirroring using the “gpaddmirrors” utility. You will need to supply the location “/gpdata/mirror” multiple times.
gpaddmirrors -p 10000
  • Supply the location “/gpdata/mirror” when prompted. This will happen based on the number of segments you specified per host.
  • Enter “Y” when prompted to “Continue with add mirrors procedure”.
  • Run the “gpstate” command to find out if all segments were created successfully. Because we have 2 segment nodes with 4 segments each, each segment having a corresponding mirror segment, we should see 16 segments.
gpstate -b
Sample gpstate run
  • Access the test database created as part of the initialization.
/usr/local/greenplum-db-6.7.1/bin/psql -h master -d testdatabase_1 -U gpadmin
Accessing test database

A man with a passion for information technology.