Skip to main content

Git policy

In Datagrok, we think the project's development should be efficient and straightforward. With that in mind, we created a Git policy that enables time-effective development without losing end product quality. Always use it during development in the public repository.

Git Flow​

We separated the Git Flow into two types: flow for experienced contributors and flow for general contributors.

Productivity tips for git​

  1. Configure the commit authorship. Set your name and email address correctly.

    git config user.name "<Name> <Surname>"
    git config user.email "<email@address>"

  2. Lint and test your changes before committing.

  3. Keep your local branch up to date by frequently fetching changes from the remote server. It will prevent bugs, rework, and the tiresome resolution of conflicts. Start your work by fetching changes.

    <Tabs>
    <TabItem value="master" label="Master branch" default>

    Do not create unnecessary merge loops. Always use the --rebase flag to pull changes from the master branch.

    Set the git configuration to enforce rebase pull strategy for master branch. It is a one-time procedure.

    git config --global pull.rebase false
    git config --global branch.master.rebase true
    git config --global branch.main.rebase true
    git config --global rebase.autostash true

    Then you should be able to update master branch using simple git pull command.

    For more information on how to work with git, check the detailed instructions in codeowners flow.

    </TabItem>
    <TabItem value="branch" label="Development branches">

    Fetch changes from master branch to keep your code up-to-date.

    git pull
    git merge origin/master
    git push
    </TabItem>
    </Tabs>

  4. Keep your branch name meaningful. Use the agreed standard: <first letter of your name><your surname>/<task ID>/<meaningful short description>. Task ID should be a full ID from any task tracking system. For example, jdoe/GROK-123/description for Jira issues or jdoe/#123/description for GitHub issues.

  5. If git checkouts fail on Windows with an error "Filename too long", run the command below:

    git config --system core.longpaths true

Codeowners flow​

The codeowners are a small group of people with good knowledge of the code base, excellent development skills, and attention to detail. They have a right to push to the master branch skipping the creation of development branches, pull requests, code review, and all other steps required for general development. This way, codeowners can contribute directly to the project and get the profits of the accelerated development.

However, with great power comes great responsibility. The codeowners are responsible for the code review on pull requests to the master branch. They are the owners of the code quality of the project.

Detailed git instructions

  1. Clone the repository if you do not have the local repository. It is a one-time procedure.

    # git clone <repository_url>
    git clone https://github.com/datagrok-ai/public.git

  2. Change the branch to the master branch.

    # git checkout <main_branch_name>
    git checkout master

  3. Pull the latest changes from the remote server. Start your work by pulling the changes from the remote server. It will prevent bugs, rework, and the tiresome resolution of conflicts.

    git pull --rebase

  4. Make changes in the files.

  5. Test the changes. You can use the datagrok toolkit to test packages

  6. Add files to commit. We recommend not mixing refactoring with a new feature on one commit. The changes should be atomic with the exact purpose.

    git add -p

  7. Create a commit with changes. Write descriptive and meaningful messages according to the commit message rules. Changelogs include all commit messages.

    # git commit -m '<commit message>'
    git commit -m '[closes] <ticket ID>: <scope>: <subject>
    [optional description]
    '

  8. Push the commit right after the creation. Pushing one commit at a time helps to avoid unexpected GitHub Actions behavior.

    git push

General flow​

The general contributors first push code to the branches and get a code review from the code owner. Then, the code owner merges the changes to the master branch. Pushing straight to the master branch is forbidden for general contributors. They have to follow the full git flow, which is quite similar to the trunk-based development flow.

Git Flow General

Detailed git instructions

  1. Clone the repository if you do not have the local repository. It is a one-time procedure.

    # git clone <repository_url>
    git clone https://github.com/datagrok-ai/public.git

  2. Change the branch to the master branch.

    # git checkout <main_branch_name>
    git checkout master

  3. Pull the latest changes from the remote server. Start your work by pulling the changes from the remote server. It will prevent bugs, rework, and the tiresome resolution of conflicts.

    git pull

  4. Create a git branch. It would be best to name it according to the task; the name must be meaningful. Use the agreed standard: <first letter of your name><your surname>/<task ID>/<meaningful short description>. Task ID should be a full ID from any task tracking system. For example, jdoe/GROK-123/description for Jira issues or jdoe/#123/description for GitHub issues.

    # git checkout -b <first letter of your name><you surname>/<task ID>/<meaningful short description>
    git checkout -b jdoe/GROK-123/description

    # git push –set-upstream origin <first letter of your name><you surname>/<task ID>/<meaningful short description>
    git push --set-upstream origin jdoe/GROK-123/description

  5. Make changes in the files.

  6. Test the changes. You can use the datagrok toolkit to test packages

  7. After all the required changes are made, merge changes from the master branch to your branch. Fix all the conflicts if it is required. Push the merged changes right after the merge.

    git pull
    # git merge origin/<main_branch_name>
    git merge origin/master
    # Push changes immediately after merge. This is very important to avoid conflicts in the future.
    git push

  8. Add files to commit. We recommend not mixing refactoring with a new feature on one commit. The changes should be atomic with the exact purpose.

    git add -p

  9. Create a commit with changes. Write descriptive and meaningful messages according to the commit message rules. Changelogs include all commit messages. If all the commits have descriptive messages, there will be no need to rewrite the message during a merge.

    # git commit -m '<commit message>'
    git commit -m '[closes] <ticket ID>: <scope>: <subject>
    [optional description]
    '

  10. Push the commit right after the creation. Pushing one commit at a time helps to avoid unexpected GitHub Actions behavior.

    git push

  11. Repeat steps 5-9 until you get the expected result

  12. Create a pull request from your branch to the master branch and ask for a review from Code Owner.

  13. After the approval, merge the pull request using the 'Squash and merge' option. All changes from the development branch will be merged into the master branch as one squashed commit. Check and rewrite the commit message if needed. The commit message must meet the commit message rules. Changelogs include all commit messages.

  14. Repeat 2-12 steps every time you start working on a new task

Release flow​

The new branch release/X.Y.Z is created during the release of the latest version of the platform. The release branch is required to create a stable and reliable release of public Datagrok JS API and Datagrok Docker image. After the tests and approvals, the CI/CD tool bumps the version of JS API and Datagrok Docker image. Then it generates the release notes for the new version. The version of JS API coincide with the Datagrok Docker image version. Commit with new version, and release notes should be cherry-picked from the release branch to the master branch. All the procedures are performed automatically by CI/CD tool.

note

For now, there is no need in the release branch for any software other than JS API and Datagrok Docker Image.

gitflow-release.svg

Detailed git instructions

  1. Clone the repository if you do not have the local repository. It is a one-time procedure.

    # git clone <repository_url>
    git clone https://github.com/datagrok-ai/public.git

  2. Change the branch to the master branch.

    # git checkout <main_branch_name>
    git checkout master

  3. Pull the latest changes from the remote server.

    git pull

  4. Create a release branch. Name it according to the release: release/<release number>. For example, release/1.2.3

    # git checkout -b release/<release number>
    git checkout -b release/1.2.3
    # git push –-set-upstream origin release/<release number>
    git push --set-upstream origin release/1.2.3

  5. Run the release job in CI/CD tool.

  6. If any test has failed and the changes are required, make changes in the master branch and then cherry-pick them to the release branch.

    git pull
    # git cherry-pick <commit with required changes>
    git cherry-pick 123456789abcdefghijklmnopqrstuvwxyz

  7. Restart the release job in CI/CD tool.

  8. Repeat steps 5-6 until you get the expected result. The release job should complete successfully and push the new version and release notes to the release branch.

  9. Cherry-pick the commit with the new version from the release branch to the master branch

    # Pull changes in the release branch
    git pull
    git checkout master

    # Pull changes in the master branch
    git pull

    # git cherry-pick <commit with version change and release notes from release branch>
    git cherry-pick zyxwvutsrqponmlkjihgfedcba987654321

  10. Repeat 2-9 steps every time you start working on a new release

Commit message policy​

In the repositories, we follow the convention, which is very close to Conventional Commits specification and based on Angular Commit Message Guidelines. It provides an easy set of rules for creating an explicit commit history which is easier to follow. Also, we built automation on top of commit messages which generates release notes based on tickets from task trackers and commit messages. The most important part of a commit message is that it should be clear, meaningful, and descriptive. Use your common sense for controversial aspects which are uncovered in the documentation. The Conventional Commits scheme is fully documented on the official website; below is our cheat sheet to help you understand the basics of committing to our products.

Commit message rules​

Each commit message consists of a header and a body structured as follows:

[closes] <ticket ID>: <scope>: <subject>

[optional description]

Every line in the commit should be 70 chars or less.

Samples
GROK-123: JS API: Update rxjs dependency version

Fix the CVE-0000-99999 vulnerability of old version rxjs in
JS API package.json
https://community.datagrok.ai/t/topic-for-dependency/000
closes #123: Chem | Descriptors: New super incredible feature release

BREAKING CHANGE: deleted the method superCool in chem.ts
GROK-456: GitHub Actions: Change in the release notes generation flow
#456: Library Bio: Create documentation for package usage (WIP)

Commit header​

The header is an essential part in the commit message. It should be 70 chars or less.

[closes] <ticket ID>: <scope>: <subject>

Example:

GROK-321: Grok Connect: Fix vulnerability in Postgres provider (WIP)
closes #321: Grok Connect | Postgres: Fix vulnerability in provider
Ticket ID (references in commitlint)​

The ticket ID is highly recommended in the commit message. If there is a ticket, mention it in the commit message. The ticket ID is a complete id from Jira or GitHub. The change type for release notes will be determined by ticket type, such as Bug or Feature.

note

Commits without ticket ID won't be included to release notes

Example:

  • GROK-1234 - Jira issue in GROK project
  • #1093 - GitHub issue
Scope​

The scope is REQUIRED in the commit message. Use the software name that was changed as scope. For example, the scope for package-specific changes is the package name.

Example:

  • JS API
  • Bio
  • Chem
  • Library Bio
  • Tools
  • Grok Connect
  • GitHub Actions
  • Help
  • etc.
Subject​

The subject is REQUIRED in the commit message. The subject is a succinct description of the change. In most cases, the ticket name will suit the subject.

Commit body​

The body is OPTIONAL in the commit message. Every line in the body of the commit should be 70 chars or less. The body is a detailed description of the change; It is needed if the subject is not enough to express the change. For example, add a link to the community forum topic to the body. If you need to break the behavior of the existing functionality in the JS API, library, or package API, add BREAKING CHANGE: <description what change breaks> to the body.

<BLANK LINE>
<optional description>

Example:


Fix the Uncontrolled Resource Consumption vulnerability in log4j
dependency for Grok Connect (CVE-2021-44228)

Commit linting​

Every commit MUST be checked by commitlint on pre-commit hook.

Installation

npm install --location=global @commitlint/config-conventional @commitlint/cli
pip install pre-commit
pre-commit install --install-hooks --hook-type pre-commit --hook-type commit-msg

Every commit is checked in GitHub Actions and reported broken if it does not apply to the commit message rules