AWS CloudFormation

The deployment consists of a few docker containers, database for storing metadata, and persistent file storage for storing files

This document contains instructions to deploy Datagrok using CloudFormation on AWS ECS cluster with AWS RDS and AWS S3.

We considered a lot of typical security nuances during the CloudFormation template development. As a result, you will create a Datagrok infrastructure in AWS that applies to all standard security policies.

More information about Datagrok design and components:

• Architecture
• Infrastructure

Prerequisites

1. Check that you have required permissions on AWS account to perform CloudFormation deployment to ECS.

Deploy Datagrok components

We prepared specific template for every need of our customers, answer simple questions below to use the right one for you.

Would you like to use an existing VPC in your AWS account?

Yes

Datagrok stand will be put in an existing VPC you choose upon creation.

Do you use Route53 as DNS provider?

Yes

Requirements

1. Create Route53 public hosted zone

How to deploy

1. Use the link to open CloudFormation template and fill all required parameters.

   1. Specify stack name. To meet AWS naming requirements, name must be shorter than 10 symbols and correspond S3 Bucket naming rules. We use 'datagrok' by default, but you may prefer to also specify env in the stack name.

2. Wait until AWS completes the deployment. The stack status will be 'CREATE_COMPLETE.' The script created datagrok stand in existing VPC using existing Route53 hosted zone. Your Datagrok instance is now ready to use.

   If you see one of the following statuses then something went wrong: CREATE_FAILED, ROLLBACK_IN_PROGRESS, ROLLBACK_COMPLETE, ROLLBACK_FAILED. Check the stack events for more information about error.

3. Enter the platform datagrok.<subdomain> using the admin user. To get the password:

   1. Go to stack Outputs. Find DatagrokAdminPassword and click on the link to open AWS Secret Manager.
   2. Click 'Retrieve secret value' and copy password. It is a generated password for the first admin login.
   3. To increase security, change the password for the admin user on the first login. Datagrok will ignore the admin password from secrets on subsequent restarts.

4. Complete the initial setup in platform and you are ready to use Datagrok.

No

Our CloudFormation scripts support external DNS providers, however, it will require a few manual steps to configure the endpoint.

Requirements

1. Datagrok requires an endpoint: DATAGROK_DNS. Users will use it to access Datagrok Web UI.

2. Create RSA SSL certificate for DATAGROK_DNS.

   • If you use AWS ACM service for SSL certificates
     1. Generate ACM certificate in AWS which will be valid for endpoint DATAGROK_DNS.
     2. Copy AWS ARN for the created certificate. It should look like this: arn:aws:acm:<region>:<account_id>:certificate/<certificate_id>.
   • If you do not use AWS ACM service for SSL certificates, you can create a certificate for DATAGROK_DNS endpoint any way you are already using. Wildcard certificate also suffices.
     1. Upload certificate to AWS ACM
     2. Copy AWS ARN for the created certificate(s). It should look like this: arn:aws:acm:<region>:<account_id>:certificate/<certificate_id>.

How to deploy

1. Use the link to open CloudFormation template and fill all required parameters.

   1. Specify stack name. To meet AWS naming requirements, name must be shorter than 10 symbols and correspond S3 Bucket naming rules. We use 'datagrok' by default, but you may prefer to also specify env in the stack name.
   2. DatagrokArnSSLCertificate: Specify AWS ACM ARN for DATAGROK_DNS from the 2nd prerequisites step.

2. Wait until AWS completes the deployment. The stack status will be 'CREATE_COMPLETE.' The script created datagrok stand with all required infrastructure from scratch using external DNS service and existing AWS ACM certificate. Your Datagrok instance is now ready to use.

   If you see one of the following statuses then something went wrong: CREATE_FAILED, ROLLBACK_IN_PROGRESS, ROLLBACK_COMPLETE, ROLLBACK_FAILED. Check the stack events for more information about error.

3. As you chose the fulfillment option with external DNS, you need to create CNAME DNS records for Datagrok Load Balancer. To get the Load Balancer endpoints for DNS record:

   1. Go to stack Outputs. Copy value for DatagrokLoadBalancerDNSName .
   2. Use copied DNS namesto create CNAME DNS record, for example
      • Host: DATAGROK_DNS, Target: DatagrokLoadBalancerDNSName

4. Enter the platform DATAGROK_DNS using admin user. To get the password:

   1. Go to stack Outputs. Find DatagrokAdminPassword and click on the link to open AWS Secret Manager.
   2. Click 'Retrieve secret value' and copy password. It is a generated password for the first admin login.
   3. To increase security, change the password for the admin user on the first login. Datagrok will ignore the admin password from secrets on subsequent restarts.

5. Complete the initial setup in platform and you are ready to use Datagrok.

No

Datagrok stand will create VPC and all required network resources itself.

Do you use Route53 as DNS provider?

Yes

Requirements

1. Create Route53 public hosted zone

How to deploy

1. Use the link to open CloudFormation template and fill all required parameters.

   1. Specify stack name. To meet AWS naming requirements, name must be shorter than 10 symbols and correspond S3 Bucket naming rules. We use 'datagrok' by default, but you may prefer to also specify env in the stack name.

2. Wait until AWS completes the deployment. The stack status will be 'CREATE_COMPLETE.' The script created datagrok stand with all required infrastructure from scratch using existing Route53 hosted zone. Your Datagrok instance is now ready to use.

   If you see one of the following statuses then something went wrong: CREATE_FAILED, ROLLBACK_IN_PROGRESS, ROLLBACK_COMPLETE, ROLLBACK_FAILED. Check the stack events for more information about error.

3. Enter the platform datagrok.<subdomain> using the admin user. To get the password:

   1. Go to stack Outputs. Find DatagrokAdminPassword and click on the link to open AWS Secret Manager.
   2. Click 'Retrieve secret value' and copy password. It is a generated password for the first admin login.
   3. To increase security, change the password for the admin user on the first login. Datagrok will ignore the admin password from secrets on subsequent restarts.

4. Complete the initial setup in platform and you are ready to use Datagrok.

No

Our CloudFormation scripts support external DNS providers, however, it will require a few manual steps to configure the endpoint.

Requirements

1. Come up with an endpoint: DATAGROK_DNS. Users will use DATAGROK_DNS to access Datagrok Web UI.

2. Create RSA SSL certificate for DATAGROK_DNS.

   • If you use AWS ACM service for SSL certificates
     1. Generate ACM certificate in AWS which will be valid for both endpoints: DATAGROK_DNS.
     2. Copy AWS ARN for the created certificate. It should look like this: arn:aws:acm:<region>:<account_id>:certificate/<certificate_id>.
   • If you do not use AWS ACM service for SSL certificates, you can create a certificate for DATAGROK_DNS endpoint any way you are already using. Wildcard certificate also suffices.
     1. Upload certificate to AWS ACM
     2. Copy AWS ARN for the created certificate(s). It should look like this: arn:aws:acm:<region>:<account_id>:certificate/<certificate_id>.

How to deploy

1. Use the link to open CloudFormation template and fill all required parameters.

   1. Specify stack name. To meet AWS naming requirements, name must be shorter than 10 symbols and correspond S3 Bucket naming rules. We use 'datagrok' by default, but you may prefer to also specify env in the stack name.
   2. DatagrokArnSSLCertificate: Specify AWS ACM ARN for DATAGROK_DNS from the 2nd prerequisites step.

2. Wait until AWS completes the deployment. The stack status will be 'CREATE_COMPLETE.' The script created datagrok stand with all required infrastructure from scratch using external DNS service and existing AWS ACM certificate. Your Datagrok instance is now ready to use.

   If you see one of the following statuses then something went wrong: CREATE_FAILED, ROLLBACK_IN_PROGRESS, ROLLBACK_COMPLETE, ROLLBACK_FAILED. Check the stack events for more information about error.

3. As you chose the fulfillment option with external DNS, you need to create CNAME DNS records for Datagrok Load Balancer. To get the Load Balancer endpoints for DNS record:

   1. Go to stack Outputs. Copy value for DatagrokLoadBalancerDNSName.
   2. Use copied DNS names to create CNAME DNS records, for example
      • Host: DATAGROK_DNS, Target: DatagrokLoadBalancerDNSName

4. Enter the platform DATAGROK_DNS using the admin user. To get the password:

   1. Go to stack Outputs. Find DatagrokAdminPassword and click on the link to open AWS Secret Manager.
   2. Click 'Retrieve secret value' and copy password. It is a generated password for the first admin login.
   3. To increase security, change the password for the admin user on the first login. Datagrok will ignore the admin password from secrets on subsequent restarts.

5. Complete the initial setup in platform and you are ready to use Datagrok.

Update Datagrok components

You can update your Datagrok deployment without re-creating infrastructure. Before updating, we recommend backing up the database and persistent storage. Refer to your internal backup procedures.

How to update

Use the same deployment script and an updated version of the deployment profile:

1. Click Update > Replace current template, and provide the new template URL corresponding to your deployment configuration.
2. Specify new versions of image tags (see the latest version).
3. Click Next, skip optional settings, and proceed to Review.
4. If the stack enters a failed state (e.g., UPDATE_ROLLBACK_IN_PROGRESS), check events for details.

• CloudFormation will not replace database or file storage during update.
• Your platform URL, admin credentials, and uploaded files will remain unchanged.
• If you previously customized your deployment with environment variables, they will persist unless explicitly modified in parameters.