AWS CloudFormation
The deployment consists of a few docker containers, database for storing metadata, and persistent file storage for storing files
This document contains instructions to deploy Datagrok using CloudFormation on AWS ECS cluster with AWS RDS and AWS S3.
We considered a lot of typical security nuances during the CloudFormation template development. As a result, you will create a Datagrok infrastructure in AWS that applies to all standard security policies.
More information about Datagrok design and components:
Prerequisites
- Check that you have required permissions on AWS account to perform CloudFormation deployment to ECS.
Deploy Datagrok components
We prepared specific template for every need of our customers, answer simple questions below to use the right one for you.
Would you like to use an existing VPC in your AWS account?
- Yes
- No
Datagrok stand will be put in an existing VPC you choose upon creation.
Do you use Route53 as DNS provider?
- Yes
- No
Requirements
- Create Route53 public hosted zone
How to deploy
-
Use the link to open CloudFormation template and fill all required parameters.
- Specify stack name. To meet AWS naming requirements, name must be shorter than 10 symbols and correspond S3 Bucket naming rules. We use 'datagrok' by default, but you may prefer to also specify env in the stack name.
-
Wait until AWS completes the deployment. The stack status will be 'CREATE_COMPLETE.' The script created datagrok stand in existing VPC using existing Route53 hosted zone. Your Datagrok instance is now ready to use.
If you see one of the following statuses then something went wrong: CREATE_FAILED, ROLLBACK_IN_PROGRESS, ROLLBACK_COMPLETE, ROLLBACK_FAILED. Check the stack events for more information about error.
-
Enter the platform
datagrok.<subdomain>
using theadmin
user. To get the password:- Go to stack Outputs. Find DatagrokAdminPassword and click on the link to open AWS Secret Manager.
- Click 'Retrieve secret value' and copy password. It is a generated password for the first admin login.
- To increase security, change the password for the admin user on the first login. Datagrok will ignore the admin password from secrets on subsequent restarts.
-
Complete the initial setup in platform and you are ready to use Datagrok.
Our CloudFormation scripts support external DNS providers, however, it will require a few manual steps to configure the endpoint.
Requirements
-
Come up with two endpoints:
DATAGROK_DNS
,CVM_DNS
. Datagrok requires two endpoints:DATAGROK_DNS
andCVM_DNS
. Users will useDATAGROK_DNS
to access Datagrok Web UI, and requestsCVM_DNS
will be sent automatically by Datagrok Client. -
Create RSA SSL certificate for
DATAGROK_DNS
andCVM_DNS
.- If you use AWS ACM service for SSL certificates
- Generate ACM certificate in AWS
which will be valid for both endpoints:
DATAGROK_DNS
,CVM_DNS
. - Copy AWS ARN for the created certificate. It should look like
this:
arn:aws:acm:<region>:<account_id>:certificate/<certificate_id>
.
- Generate ACM certificate in AWS
which will be valid for both endpoints:
- If you do not use AWS ACM service for SSL certificates, you can create a certificate for
DATAGROK_DNS
,CVM_DNS
endpoints any way you are already using. Wildcard certificate also suffices.- Upload certificate to AWS ACM
- Copy AWS ARN for the created certificate(s). It should look like
this:
arn:aws:acm:<region>:<account_id>:certificate/<certificate_id>
.
- If you use AWS ACM service for SSL certificates
How to deploy
-
Use the link to open CloudFormation template and fill all required parameters.
- Specify stack name. To meet AWS naming requirements, name must be shorter than 10 symbols and correspond S3 Bucket naming rules. We use 'datagrok' by default, but you may prefer to also specify env in the stack name.
DatagrokArnSSLCertificate
: Specify AWS ACM ARN forDATAGROK_DNS
andCVM_DNS
from the 2nd prerequisites step.
-
Wait until AWS completes the deployment. The stack status will be 'CREATE_COMPLETE.' The script created datagrok stand with all required infrastructure from scratch using external DNS service and existing AWS ACM certificate. Your Datagrok instance is now ready to use.
If you see one of the following statuses then something went wrong: CREATE_FAILED, ROLLBACK_IN_PROGRESS, ROLLBACK_COMPLETE, ROLLBACK_FAILED. Check the stack events for more information about error.
-
As you chose the fulfillment option with external DNS, you need to create CNAME DNS records for CVM and Datagrok Load Balancers. To get the Load Balancer endpoints for DNS record:
- Go to stack Outputs. Copy values for DatagrokLoadBalancerDNSName and CvmLoadBalancerDNSName.
- Use copied DNS names to create CNAME DNS records, for example
- Host:
DATAGROK_DNS
, Target: DatagrokLoadBalancerDNSName - Host:
CVM_DNS
, Target: CvmLoadBalancerDNSName
- Host:
-
Enter the platform
DATAGROK_DNS
usingadmin
user. To get the password:- Go to stack Outputs. Find DatagrokAdminPassword and click on the link to open AWS Secret Manager.
- Click 'Retrieve secret value' and copy password. It is a generated password for the first admin login.
- To increase security, change the password for the admin user on the first login. Datagrok will ignore the admin password from secrets on subsequent restarts.
-
Complete the initial setup in platform and you are ready to use Datagrok.
Datagrok stand will create VPC and all required network resources itself.
Do you use Route53 as DNS provider?
- Yes
- No
Requirements
- Create Route53 public hosted zone
How to deploy
-
Use the link to open CloudFormation template and fill all required parameters.
- Specify stack name. To meet AWS naming requirements, name must be shorter than 10 symbols and correspond S3 Bucket naming rules. We use 'datagrok' by default, but you may prefer to also specify env in the stack name.
-
Wait until AWS completes the deployment. The stack status will be 'CREATE_COMPLETE.' The script created datagrok stand with all required infrastructure from scratch using existing Route53 hosted zone. Your Datagrok instance is now ready to use.
If you see one of the following statuses then something went wrong: CREATE_FAILED, ROLLBACK_IN_PROGRESS, ROLLBACK_COMPLETE, ROLLBACK_FAILED. Check the stack events for more information about error.
-
Enter the platform
datagrok.<subdomain>
using theadmin
user. To get the password:- Go to stack Outputs. Find DatagrokAdminPassword and click on the link to open AWS Secret Manager.
- Click 'Retrieve secret value' and copy password. It is a generated password for the first admin login.
- To increase security, change the password for the admin user on the first login. Datagrok will ignore the admin password from secrets on subsequent restarts.
-
Complete the initial setup in platform and you are ready to use Datagrok.
Our CloudFormation scripts support external DNS providers, however, it will require a few manual steps to configure the endpoint.
Requirements
-
Come up with two endpoints:
DATAGROK_DNS
,CVM_DNS
. Datagrok requires two endpoints:DATAGROK_DNS
andCVM_DNS
. Users will useDATAGROK_DNS
to access Datagrok Web UI, and requestsCVM_DNS
will be sent automatically by Datagrok Client. -
Create RSA SSL certificate for
DATAGROK_DNS
andCVM_DNS
.- If you use AWS ACM service for SSL certificates
- Generate ACM certificate in AWS
which will be valid for both endpoints:
DATAGROK_DNS
,CVM_DNS
. - Copy AWS ARN for the created certificate. It should look like
this:
arn:aws:acm:<region>:<account_id>:certificate/<certificate_id>
.
- Generate ACM certificate in AWS
which will be valid for both endpoints:
- If you do not use AWS ACM service for SSL certificates, you can create a certificate for
DATAGROK_DNS
,CVM_DNS
endpoints any way you are already using. Wildcard certificate also suffices.- Upload certificate to AWS ACM
- Copy AWS ARN for the created certificate(s). It should look like
this:
arn:aws:acm:<region>:<account_id>:certificate/<certificate_id>
.
- If you use AWS ACM service for SSL certificates
How to deploy
-
Use the link to open CloudFormation template and fill all required parameters.
- Specify stack name. To meet AWS naming requirements, name must be shorter than 10 symbols and correspond S3 Bucket naming rules. We use 'datagrok' by default, but you may prefer to also specify env in the stack name.
DatagrokArnSSLCertificate
: Specify AWS ACM ARN forDATAGROK_DNS
andCVM_DNS
from the 2nd prerequisites step.
-
Wait until AWS completes the deployment. The stack status will be 'CREATE_COMPLETE.' The script created datagrok stand with all required infrastructure from scratch using external DNS service and existing AWS ACM certificate. Your Datagrok instance is now ready to use.
If you see one of the following statuses then something went wrong: CREATE_FAILED, ROLLBACK_IN_PROGRESS, ROLLBACK_COMPLETE, ROLLBACK_FAILED. Check the stack events for more information about error.
-
As you chose the fulfillment option with external DNS, you need to create CNAME DNS records for CVM and Datagrok Load Balancers. To get the Load Balancer endpoints for DNS record:
- Go to stack Outputs. Copy values for DatagrokLoadBalancerDNSName and CvmLoadBalancerDNSName.
- Use copied DNS names to create CNAME DNS records, for example
- Host:
DATAGROK_DNS
, Target: DatagrokLoadBalancerDNSName - Host:
CVM_DNS
, Target: CvmLoadBalancerDNSName
- Host:
-
Enter the platform
DATAGROK_DNS
using theadmin
user. To get the password:- Go to stack Outputs. Find DatagrokAdminPassword and click on the link to open AWS Secret Manager.
- Click 'Retrieve secret value' and copy password. It is a generated password for the first admin login.
- To increase security, change the password for the admin user on the first login. Datagrok will ignore the admin password from secrets on subsequent restarts.
-
Complete the initial setup in platform and you are ready to use Datagrok.