Step-by-step transformations on arbitrary text input

Outline

  • Introduction (including software versions used)
  • Creating AWS Instance
  • Installation (Spark, Kafka)
  • Spark job 1: Output raw data to console
  • Spark job 2: Run custom functions on input and output as new column
  • Spark job 3: Parse JSON and output specific fields
  • Spark job 4

For when you really just don’t want to pay for vSphere

Outline

  • Introduction
  • Requirements
  • Part 0: Installing VirtualBox and Extension Pack on the Command Line
  • Part 1: Creating and Deleting a VirtualBox VM using “VBoxManage”
  • Part 2: Enabling RDP access for a VirtualBox VM
  • Part 3: Additional Considerations

Introduction

When I was…

CDC-like data pipeline using MySQL binary logs

Outline

  • Introduction
  • Creating Security Groups and EC2 Instances (~15 min)
  • Installing MySQL and Configuring to Allow Binary Log Reading (~15 min)
  • Installing/Configuring Kafka and Debezium Connector (~15 min)

Introduction

In my previous set of tutorials, I explained how to use the Debezium connector to stream…

Creating a CDC data pipeline: Part 3

Outline

  • Introduction
  • Creating Security Groups and EC2 Instances (~5 min)
  • Installing Graphite Carbon, Graphite Web, and StatsD (~15 minutes)
  • Installing Grafana (~5 min)
  • Configuring StatsD (~5 min)
  • Starting All Pipeline Services (~10 min)
  • Configuring Grafana and Creating a Dashboard (~10 min)
  • Completed Python File

How to install a three-node Greenplum Database cluster with segment mirroring

Source: https://twitter.com/Greenplum

Overview

I. Introduction

II. System Requirements

III. Part 1 — Create AWS Instances (~15 minutes)

IV. Part 2 — Setup on Each Node (~15 minutes)

V. Part 3 — Initialize Greenplum Database (~15 minutes)

Introduction

Greenplum Database uses an MPP (massively parallel processing) database architecture that is able to take advantage of…

Sandeep Kattepogu

A man with a passion for information technology.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store