AWS CloudFormation (ECS)

Avoid for new installations

The ECS path is stable and supported, but is targeted for deprecation in a future release. For new stands, prefer the AWS CloudFormation (EKS) template — it provisions the same RDS + S3 logical resources and installs the Helm chart on EKS. The two templates share resource IDs, so an existing ECS stack can be migrated to EKS in place via Update stack → Replace current template without re-creating the database or bucket.

The deployment consists of a few docker containers, database for storing metadata, and persistent file storage for storing files

This document contains instructions to deploy Datagrok using CloudFormation on AWS ECS cluster with AWS RDS and AWS S3.

We considered a lot of typical security nuances during the CloudFormation template development. As a result, you will create a Datagrok infrastructure in AWS that applies to all standard security policies.

More information about Datagrok design and components:

• Architecture
• Infrastructure

Prerequisites

1. Check that you have required permissions on AWS account to perform CloudFormation deployment to ECS.

Deploy Datagrok components

We prepared specific template for every need of our customers, answer simple questions below to use the right one for you.

Would you like to use an existing VPC in your AWS account?

Yes

Datagrok stand will be put in an existing VPC you choose upon creation.

Do you use Route53 as DNS provider? (existing VPC)

Yes

Requirements

1. Create Route53 public hosted zone

How to deploy

1. Use the link to open CloudFormation template and fill all required parameters.

   1. Specify stack name. To meet AWS naming requirements, name must be shorter than 10 symbols and correspond S3 Bucket naming rules. We use 'datagrok' by default, but you may prefer to also specify env in the stack name.

2. Wait until AWS completes the deployment. The stack status will be 'CREATE_COMPLETE.' The script created datagrok stand in existing VPC using existing Route53 hosted zone. Your Datagrok instance is now ready to use.

   If you see one of the following statuses then something went wrong: CREATE_FAILED, ROLLBACK_IN_PROGRESS, ROLLBACK_COMPLETE, ROLLBACK_FAILED. Check the stack events for more information about error.

3. Enter the platform datagrok.<subdomain> using the admin user. To get the password:

   1. Go to stack Outputs. Find DatagrokAdminPassword and click on the link to open AWS Secret Manager.
   2. Click 'Retrieve secret value' and copy password. It is a generated password for the first admin login.
   3. To increase security, change the password for the admin user on the first login. Datagrok will ignore the admin password from secrets on subsequent restarts.

4. Complete the initial setup in platform and you are ready to use Datagrok.

No

Our CloudFormation scripts support external DNS providers, however, it will require a few manual steps to configure the endpoint.

Requirements

1. Datagrok requires an endpoint: DATAGROK_DNS. Users will use it to access Datagrok Web UI.

2. Create RSA SSL certificate for DATAGROK_DNS.

   • If you use AWS ACM service for SSL certificates
     1. Generate ACM certificate in AWS which will be valid for endpoint DATAGROK_DNS.
     2. Copy AWS ARN for the created certificate. It should look like this: arn:aws:acm:<region>:<account_id>:certificate/<certificate_id>.
   • If you do not use AWS ACM service for SSL certificates, you can create a certificate for DATAGROK_DNS endpoint any way you are already using. Wildcard certificate also suffices.
     1. Upload certificate to AWS ACM
     2. Copy AWS ARN for the created certificate(s). It should look like this: arn:aws:acm:<region>:<account_id>:certificate/<certificate_id>.

How to deploy

1. Use the link to open CloudFormation template and fill all required parameters.

   1. Specify stack name. To meet AWS naming requirements, name must be shorter than 10 symbols and correspond S3 Bucket naming rules. We use 'datagrok' by default, but you may prefer to also specify env in the stack name.
   2. DatagrokArnSSLCertificate: Specify AWS ACM ARN for DATAGROK_DNS from the 2nd prerequisites step.

2. Wait until AWS completes the deployment. The stack status will be 'CREATE_COMPLETE.' The script created datagrok stand with all required infrastructure from scratch using external DNS service and existing AWS ACM certificate. Your Datagrok instance is now ready to use.

   If you see one of the following statuses then something went wrong: CREATE_FAILED, ROLLBACK_IN_PROGRESS, ROLLBACK_COMPLETE, ROLLBACK_FAILED. Check the stack events for more information about error.

3. As you chose the fulfillment option with external DNS, you need to create CNAME DNS records for Datagrok Load Balancer. To get the Load Balancer endpoints for DNS record:

   1. Go to stack Outputs. Copy value for DatagrokLoadBalancerDNSName .
   2. Use copied DNS names to create CNAME DNS record, for example
      • Host: DATAGROK_DNS, Target: DatagrokLoadBalancerDNSName

4. Enter the platform DATAGROK_DNS using admin user. To get the password:

   1. Go to stack Outputs. Find DatagrokAdminPassword and click on the link to open AWS Secret Manager.
   2. Click 'Retrieve secret value' and copy password. It is a generated password for the first admin login.
   3. To increase security, change the password for the admin user on the first login. Datagrok will ignore the admin password from secrets on subsequent restarts.

5. Complete the initial setup in platform and you are ready to use Datagrok.

No

Datagrok stand will create VPC and all required network resources itself.

Do you use Route53 as DNS provider? (new VPC)

Yes

Requirements

1. Create Route53 public hosted zone

How to deploy

1. Use the link to open CloudFormation template and fill all required parameters.

   1. Specify stack name. To meet AWS naming requirements, name must be shorter than 10 symbols and correspond S3 Bucket naming rules. We use 'datagrok' by default, but you may prefer to also specify env in the stack name.

2. Wait until AWS completes the deployment. The stack status will be 'CREATE_COMPLETE.' The script created datagrok stand with all required infrastructure from scratch using existing Route53 hosted zone. Your Datagrok instance is now ready to use.

   If you see one of the following statuses then something went wrong: CREATE_FAILED, ROLLBACK_IN_PROGRESS, ROLLBACK_COMPLETE, ROLLBACK_FAILED. Check the stack events for more information about error.

3. Enter the platform datagrok.<subdomain> using the admin user. To get the password:

   1. Go to stack Outputs. Find DatagrokAdminPassword and click on the link to open AWS Secret Manager.
   2. Click 'Retrieve secret value' and copy password. It is a generated password for the first admin login.
   3. To increase security, change the password for the admin user on the first login. Datagrok will ignore the admin password from secrets on subsequent restarts.

4. Complete the initial setup in platform and you are ready to use Datagrok.

No

Our CloudFormation scripts support external DNS providers, however, it will require a few manual steps to configure the endpoint.

Requirements

1. Come up with an endpoint: DATAGROK_DNS. Users will use DATAGROK_DNS to access Datagrok Web UI.

2. Create RSA SSL certificate for DATAGROK_DNS.

   • If you use AWS ACM service for SSL certificates
     1. Generate ACM certificate in AWS which will be valid for both endpoints: DATAGROK_DNS.
     2. Copy AWS ARN for the created certificate. It should look like this: arn:aws:acm:<region>:<account_id>:certificate/<certificate_id>.
   • If you do not use AWS ACM service for SSL certificates, you can create a certificate for DATAGROK_DNS endpoint any way you are already using. Wildcard certificate also suffices.
     1. Upload certificate to AWS ACM
     2. Copy AWS ARN for the created certificate(s). It should look like this: arn:aws:acm:<region>:<account_id>:certificate/<certificate_id>.

How to deploy

1. Use the link to open CloudFormation template and fill all required parameters.

   1. Specify stack name. To meet AWS naming requirements, name must be shorter than 10 symbols and correspond S3 Bucket naming rules. We use 'datagrok' by default, but you may prefer to also specify env in the stack name.
   2. DatagrokArnSSLCertificate: Specify AWS ACM ARN for DATAGROK_DNS from the 2nd prerequisites step.

2. Wait until AWS completes the deployment. The stack status will be 'CREATE_COMPLETE.' The script created datagrok stand with all required infrastructure from scratch using external DNS service and existing AWS ACM certificate. Your Datagrok instance is now ready to use.

   If you see one of the following statuses then something went wrong: CREATE_FAILED, ROLLBACK_IN_PROGRESS, ROLLBACK_COMPLETE, ROLLBACK_FAILED. Check the stack events for more information about error.

3. As you chose the fulfillment option with external DNS, you need to create CNAME DNS records for Datagrok Load Balancer. To get the Load Balancer endpoints for DNS record:

   1. Go to stack Outputs. Copy value for DatagrokLoadBalancerDNSName.
   2. Use copied DNS names to create CNAME DNS records, for example
      • Host: DATAGROK_DNS, Target: DatagrokLoadBalancerDNSName

4. Enter the platform DATAGROK_DNS using the admin user. To get the password:

   1. Go to stack Outputs. Find DatagrokAdminPassword and click on the link to open AWS Secret Manager.
   2. Click 'Retrieve secret value' and copy password. It is a generated password for the first admin login.
   3. To increase security, change the password for the admin user on the first login. Datagrok will ignore the admin password from secrets on subsequent restarts.

5. Complete the initial setup in platform and you are ready to use Datagrok.

Service versions

Every service has a dedicated CloudFormation parameter (DatagrokVersion, GrokPipeVersion, GrokSpawnerVersion, GrokConnectVersion, JKGVersion, GrokRegistryProxyVersion, RabbitmqVersion) for picking the container image tag. The defaults match the EKS template — both templates ship the same images. See Images and versions for the canonical set and pinning conventions across deployment paths.

Update Datagrok components

You can update your Datagrok deployment without re-creating infrastructure. Before updating, back up the database and persistent storage.

How to update

1. Click Update > Replace current template, and provide the new template URL corresponding to your deployment configuration.
2. Specify new image tag versions (see Release history).
3. Click Next, skip optional settings, and proceed to Review.
4. If the stack enters a failed state (e.g., UPDATE_ROLLBACK_IN_PROGRESS), check events for details.

note

• CloudFormation does not replace the RDS database or S3 bucket on update.
• Your platform URL, admin credentials, and uploaded files remain unchanged.
• Customisations applied via stack parameters persist; environment-variable overrides applied outside the stack do not.