Skip to main content

AWS Terraform

The deployment consists of a few docker containers, database for storing metadata, and persistent file storage for storing files

This document contains instructions to deploy Datagrok using Terraform on AWS ECS cluster with AWS RDS and AWS S3.

We considered a lot of typical security nuances during the Terraform code development. As a result, you will create a Datagrok infrastructure in AWS that applies to all standard security policies.

More information about Datagrok design and components:

Basic usage​

Prerequisites​

  1. Check that you have required permissions on AWS account to perform Terraform deployment to ECS. The cloudformation permissions are optional.

  2. Check that you have required S3 permissions and DynamoDB permissions

  3. Create access token in Docker Hub

  4. Come up with two endpoints: DATAGROK_DNS, CVM_DNS. Datagrok requires two endpoints: DATAGROK_DNS and CVM_DNS. Users will use DATAGROK_DNS to access Datagrok Web UI, and requests CVM_DNS will be sent automatically by Datagrok Client.

Deploy Datagrok components​

  1. Download Terraform Template: link .

  2. Replace the placeholders in the template with the actual values

    • SET_YOUR_S3_BUCKET_FOR_TERRAFORM_STATE - the unique S3 bucket name to store Terraform state
    • SET_YOUR_REGION - target region to create Datagrok Infrastructure

  3. Setup AWS credentials to run Terraform code

  4. Apply Terraform code

    terraform init --upgrade
    terraform apply -target module.datagrok_core
    terraform apply -target module.datagrok_cvm

  5. After the Datagrok container starts, the Datagrok server will deploy the database. You can check the status by checking the running task log in CloudWatch

Configure Datagrok settings​

  1. Go into the web browser to DATAGROK_DNS, login to Datagrok using username admin and unique password, which was generated during terraform run. You can find it in the GROK_PARAMETERS environment variable in the datagrok task definition.

  2. Edit settings in the Datagrok (Tools | Settings...). Remember to click Apply to save new settings.

    • Scripting:
      • CVM URL Client: https://<CVM_DNS>
    • Dev:
      • CVM Url: https://<CVM_DNS>

Advanced usage​

The Terraform code is highly configurable. Feel free to adapt the code and variables to meet your needs and requirements. Terraform modules documentation: