Migration To Git

Recommended Reading

• Official Gerrit introduction: https://gerrit-review.googlesource.com/Documentation/intro-quick.html.
• Gerrit tutorial: http://www.vogella.com/tutorials/Gerrit/article.html

Instructions for Simantics Developers

Gerrit

1. Request a Gerrit account. If your already have a SVN account, it should work also in Gerrit.
2. Generate SSH Public/Private key. If you already have one available, skip this step
   • Windows
     • Install PuTTY (msi) or download only PuTTYgen (exe) (http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html)
     • Run PuTTYgen and generate a key
       • For Type of key to generate, select SSH-2 RSA
       • Press Generate button
       • Move your mouse over the window until the progress bar is finished
       • Write a Key passphrase to protect your private key in case your local machine is compromised.
       • Save the generated public and private keys in C:\Users\{username}\.ssh as follows:
         • Start cmd.exe and create the target directory using mkdir .ssh
         • Save private key using Conversions -> Export OpenSSH Key. The name id_rsa is preferred because it will work out of the box with all tools.
         • At the top of the window, there is a text field labeled Public key for pasting into OpenSSH authorized_keys. Copy the public key from there and save it into a text file at C:\Users\{username}\.ssh\id_rsa.pub
   • Linux
     • Generate public and private keys using ssh-keygen -t rsa
3. Go to http://www.simantics.org:8088/r/ and log in with your Gerrit username/password
4. Select your name in the top-right corner and press Settings
5. Go to section SSH Public Keys
6. Open the file containing the generated public key in a text editor and copy-paste its contents to the box in Gerrit and and press Add
7. Add and verify your contact information (name + e-mail). This will allow administrators to add you to the necessary groups to get commit access:
   • Go to section Contact Information and enter your Full Name there
   • Add a Preferred Email address using the button Register New Email
   • Wait for the authorization mail to arrive. It will look something like the following and contain a longish link that tends to end with == characters. Note that outlook tends to leave the trailing == characters out of the link it generates automatically. That's why it is recommended to copy the full URL from the mail into your browser to verify your e-mail address.

Subject:  [Gerrit Code Review] Email Verification

Welcome to Gerrit Code Review at www.simantics.org.

To add a verified email address to your user account, please
click on the following link while signed in as .....:

http://www.simantics.org:8088/r/#/../...............................................................................==

If you have received this mail in error, you do not need to take any
action to cancel the account.  The address will not be activated, and
you will not receive any further emails.

If clicking the link above does not work, copy and paste the URL in a
new browser window instead.

This is a send-only email address.  Replies to this message will not
be read or answered.

8. Finally, ask User:Tuukka Lehtonen to add you to the necessary groups and wait for this to happen.

Eclipse

The following instructions are written for Eclipse Neon. There might be some differences if you are using Mars.

1. Go to preferences (General / Network Connections / SSH2) and ensure that SSH2 home points to the directory where you saved the private key.
2. Open Git perspective
3. Press button Clone a Git Repository and add the clone to this view (third button in the toolbar of Git view)
4. Select Clone URI (Don't select Gerrit, it doesn't work very well at the moment)
5. Go back to Gerrit. Press Projects in the top-left menu and List in the submenu.
6. Find a row named simantics/platform and press (gitweb) link at the last column.
7. Copy the URI starting with ssh: (ssh://{username}@www.simantics.org:29418/simantics/platform.git) and paste it to URI field in Clone Git Repository dialog.
8. This should automatically fill the other fields.
9. Press Next. If your private key has a passphrase, Eclipse now asks it.
10. Select all branches to be cloned (default) and press Next.
11. Give a local directory where the repository is cloned to and press Finish.
12. After the cloning is finished, select the platform repository in the Git Repositories view and select Import Projects... from context menu.
13. Press Next, select all projects but org.simantics.root and press Finish.
14. Expand the tree view of platform, expand Branches under it and Local under it. Open context menu on master branch and select Configure Branch.... Select Rebase checkbox and press Ok.

The repository you cloned contains the target platform definition that needs to be set in order for the imported Simantics codebase to compile. You can find the required target platform definition file from org.simantics.sdk.build.targetdefinition/org.simantics.sdk.build.targetdefinition.target. Open it in the target definition editor and set it as the active target platform by pressing the link Set as Target Platform at the top right corner of the editor.

Congratulations, you've now set up your workspace for developing the Simantics codebase in Eclipse!

Updating your local repository and working space

1. Open context menu on platform in the Git perspective and select Pull.

Committing your changes back to shared repository

1. Select platform in the Git perspective and open Git Staging view.
2. Unstaged Changes shows all files you have modified.
3. Drag&drop those changes you want to commit to the box Staged Changes.
4. Write a commit message
5. Ensure that Add Change-Id button is toggled down and the commit message contains a row starting Change-Id:.
   • This is required by Gerrit to be able to bind commits to specific reviews. Gerrit will replace the default zero-valued Id with a generated one when the change is pushed for review.
6. Press Commit and Push...
7. This opens a dialog that asks the Gerrit Branch and a Topic. Topic can be added optionally to tie multiple changes together.
   • Example of a topic: http://www.simantics.org:8088/r/#/q/topic:svn
8. Go to Gerrit. In the top-left menu select My and select Changes.
9. Your change can be seen now in the section Outgoing reviews.
10. Click the change. This opens a review page.
11. Ask someone to review your change. This can be done also in Gerrit by pressing Add... button in the right of the Reviewers field.
12. If you need to make modifications to your changes based on reviews, you'll need to Amend the earlier changes to send a new patch set to Gerrit. In this case, ensure that the Amend (Edit Previous Commit) button is toggled down and you should be able to modify the earlier commit message and push the amended changes.
13. When somebody has reviewed and accepted the change with +2, the review page will contain a Submit button that can be used to merge the change to the master branch.

Commit message format

In Git, the first row of the commit message is used for shortlog and generating patch file names so keep it short, concise and as descriptive as possible. The commit box in the UI shows a vertical line marking the maximum length of a commit message line and the view will inform you if you're exceeding limits. Note that the commit message box will automatically break long lines of text into separate lines to help you write a cleanly formatted commit message.

To keep commits attached to Redmine issues, use refs #nnn, where nnn is the issue number. This must be specified before Change-Id:. The Simantics platform Gerrit is configured to show refs #nnn lines as hyperlinks to platform Redmine issues, i.e. to URL https://www.simantics.org/redmine/issues/nnn.

Gerrit reviews require a row starting Change-Id: as the last row of the commit message (excluding Signed-off-by rows). This allows Gerrit to bind commits to specific reviews. Gerrit will replace the default zero-valued Id with a generated one when the change is pushed for review.

All in all, a typical commit message should look like this:

Short description of my change
 
A longer description of my changes
that can span multiple lines.
 
refs #nnn

Change-Id: I0000000000000000000000000000000000000000

Simantics Member Read-only Git Access

1. Register to member wiki at https://www.simantics.org/members/index.php
2. Log into Gerrit using the credentials found at https://www.simantics.org/members/index.php/Code_Repository_Access
3. Clone http://member@www.simantics.org:8088/r/p/simantics/platform.git using the HTTP password found at http://www.simantics.org:8088/r/#/settings/http-password

The instructions for using Eclipse with this git repository apply also for the generic member account. Just remember to replace the cloned SSH repository address with the HTTP address shown above.

Testing Reviewed Changes

To test reviewed code changes locally, first you must have an Eclipse IDE setup according to the instructions in #Instructions for Simantics Developers or #Simantics Member Read-only Git Access.

Now, let's say for example we wish to grab a patch set from a review, such as review #72 (http://www.simantics.org:8088/r/72).

1. Open Git perspective
2. Select root node of the cloned simantics-platform repository from the Git Repositories view
3. From the context menu of the repository, select Fetch from Gerrit...
4. In the Change field, type the change number 72 and press CTRL+SPACE to start content assist. Content assist will provide you with the selections 72 - 1, and 72 - 2, i.e. patch sets 1 and 2 of change 72. After selecting a patch set, the field will be filled with the value refs/changes/72/72/1 or refs/changes/72/72/2 depending on the patch set you selected.
5. The default settings are otherwise recommended, i.e. Create a Local Branch and Checkout new branch to switch to the new branch immediately.

There you have it, a local branch that has exactly the same repository state as in the code to be reviewed. Start testing!

Q & A

Will the complete SVN history be transferred to Git?

Sadly, no it will not, at least for everything that is considered part of the current SVN Simantics SDK (see revisions list on page https://www.simantics.org/jenkins/job/target-simantics-head-sdk/lastStableBuild/).

The reason for not attempting to salvage the full history into git is in the Simantics SVN repository's structure. In SVN a "project structure" is a single combination of the folders trunk, branches and tags. In git branches and tags do not exist like this as separate directories in the repository itself. The git-svn tool is perfectly capable of transforming a single project structure into git repository with all its history, branches and tags. At some point in the past we for decided to split the SVN repository contents into multiple project structures in the hope of modularizing the platform SDK. That hope never quite realized and what's worse, that structure has made platform release engineering much more laborous and complex than it would be with a single project structure. Now in this move to Git we decided to simplify things and just put the platform SDK projects back in a single monolithic structure of bundles, features, tests and release engineering projects. All in all, we just didn't have enough git expertise to be able to transfer all the history scattered in these multiple project structures into a single git repository and moving the current SVN project structures each into its own git repository was not an enticing option either.

So how are we doing it right now? We have a separate migration git repository that contains a python script that simply exports all listed projects from SVN into the simantics-platform git repository structure. Our simantics-platform git repository contains a branch called 'svn'. The SVN exporting python script was originally used to initially populate the git repository, back when we were first testing the migration. Another python script is then used to bring in changes from SVN by exporting on top of the existing simantics-platform git repository. These changes are then committed into the svn branch and merged into master from there. We are not using git svn fetch because we're just not proficient enough with git-svn to be able to use it separately for each and every project that has moved location in git already. The git master branch has already been developed onwards, but merging changes from the svn branch to it is still very easy.

Note that the SVN repository will remain in place to keep all old versions of the platform that are still needed for maintaining existing products. We will not be moving these old branches/versions into git.

Why are there .keep files in Git repository folders?

• https://git.wiki.kernel.org/index.php/GitFaq#Can_I_add_empty_directories.3F
• http://stackoverflow.com/questions/115983/how-can-i-add-an-empty-directory-to-a-git-repository

Our recommendation is to add an empty file named .keep to the empty folder to ensure it is kept in git.

SVN to Git Transition TODO

• Set a date for when SVN trunks will be changed to read-only
• Double check all instructions:
  • Basic Eclipse IDE installation with Simantics graph compiler
  • Git instructions and platform development workflow
  • Gerrit instructions and practices
    • Only project owners shall have the right for direct push to git repositories. Others must always go through gerrit review. This is enforced via Gerrit access rights.
• Test and migrate all products trunk versions on top of the simantics-platform git master branch.
• E-mail developers with instructions or links to instructions about the change.

The next step is to start thinking about moving your project/product over to Gerrit!