Scribd
Upload
68 views

Git Workflow - Overview

The document outlines the standard Git workflow for making changes or adding features to a Git project. It involves creating a local branch from the local master branch, pushing changes to a Data Bricks workspace for testing, pulling changes back to the local branch, pushing the local branch to the remote DevOps repository, creating a pull request for review, and merging the changes once approved. Low-level steps provide details on commands for cloning a repo, checking out a branch, adding and committing changes, and pushing to remote. Common issues with running PowerShell scripts are also noted.

Uploaded by

Rajul Rahmad
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
68 views

Git Workflow - Overview

The document outlines the standard Git workflow for making changes or adding features to a Git project. It involves creating a local branch from the local master branch, pushing changes to a Data Bricks workspace for testing, pulling changes back to the local branch, pushing the local branch to the remote DevOps repository, creating a pull request for review, and merging the changes once approved. Low-level steps provide details on commands for cloning a repo, checking out a branch, adding and committing changes, and pushing to remote. Common issues with running PowerShell scripts are also noted.

Uploaded by

Rajul Rahmad
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

10/6/2020 Git Workflow - Overview

Git Workflow
Last updated by | Kirby Prowting | May 28, 2020 at 10:48 AM GMT+1

Standard git workflow when making a change/adding features to a git project

High Level Workflow

Prerequisites
Have Git Bash installed on your local machine, download Git Bash 
Have Visual Studio Code installed on your local machine, download VSC 

1. Create a local folder for your Git repos - perform a Git Bash pull from master in DevOps and create a
master locally here
2. Create a local branch from your local master branch
3. Run PowerShell script to push your branch to Data Bricks. This allows you to develop and test in Data
Bricks
4. Once happy with code changes/fixes, run PowerShell script to pull down from Data Bricks back into
your local branch
5. Perform a Git Bash Push on your local branch to DevOps, if this does not already exist in DevOps a
branch will be created for you
6. Create a pull request for changes to be reviewed and accepted by another team member

Low level steps


1. Open git bash - once git is installed on your local machine you can simply create a folder where you
want to store git repos locally. Once you're in this folder, right click and hit git bash
https://dev.azure.com/marsanalytics/PETCARE DATA PLATFORM/_wiki/wikis/PETCARE-DATA-PLATFORM.wiki/1015/Git-Workflow 1/2
10/6/2020 Git Workflow - Overview

2. Run git clone <link to repo> - Skip this step if you have already cloned the repo
3. Run cd <repo name> - Change directory into the git repo within git bash. If you've already cloned the
repo, open git bash in your repo folder rather instead of the step above
4. Run git branch -d <branch name> - If any branches on your local machine still exist, delete them. This
minimizes your risk of falling into a merge conflict later on
5. Run git pull - This pulls the most up to date version of the master repo down to your local machine.
It's best practice to always do a git pull before starting development - we want to make sure we always
start with the most up-to-date master to avoid conflicts later on
6. Run git checkout -b <branch name> - This creates a new branch and moves you in to it
7. Generate a bearer token in databricks  & create a text file mybearertoken.txt containing the token
you just generated, and save it in the parent directory of the repo you just cloned on your local
machine. This is used so that the powershell scripts used to push our files up to the databricks
workspace can access it (note: you only need to do this once)
8. Run code . to open VS Code and ensure the powershell plugin is installed. On the left where it says
'choose folder', open up the repo for the project you're working on
9. Open a powershell integrated terminal - if it only says 'powershell', open up a .ps1 file and it should
change to integrated
10. Run the powershell command which pushes the notebook to the workspace. For Kyte, this is:
& ".\adb.cicd.tools\pipeline.ps1" -PushNoteBooksToWorkspace -environment localdev
Once we have pushed files to our databricks workspace we can start making changes in databricks
notebooks. Also ensure the Mars VPN is turned on at this step as the powershell script determines your
email address dynamically so it know's where to put your repo in databricks
11. Make any changes in databricks
12. Once you have made your changes, pull the notebooks back down from databricks to your local area.
When this happens, git will be able to track any changes that have been made. For Kyte, the command
is:
& ".\adb.cicd.tools\pipeline.ps1" -pullNoteBooksFromWorkspace -environment localdev

13. Open git bash and run git status - this is good practice as this command shows you which files (or
notebooks) git has detected a change in. It's a sort of QC that what you expect to have changed is the
thing that's changed
14. Now we are back to normal git workflow. Run: git add -A - this adds all changed files to the staging
area
15. Run git commit -m "<custom commit message>" - This is a description on the change you have made
which will show up in Azure Dev Ops

16. Run git push - this pushes your local changes up to your working branch in Azure Dev Ops, ready to
merge back into the master branch. If this is the first time you are pushing from this particular working
branch, you need to run the below which links your working branch to a branch in Azure Dev Ops:
git push --set-upstream origin <branch name>

17. Open your repo in Azure DevOps and create a pull request - this is essentially a request to merge the
changes you've just made into the master branch for that repo.
18. Finally, someone approves and completes the pull request and your changes are merged in
19. Go to step 1

Common Issues
https://stackoverflow.com/questions/56199111/visual-studio-code-cmd-error-cannot-be-loaded-
because-running-scripts-is-disabl 

https://dev.azure.com/marsanalytics/PETCARE DATA PLATFORM/_wiki/wikis/PETCARE-DATA-PLATFORM.wiki/1015/Git-Workflow 2/2

You might also like