Development Practices

If it isn't in the wiki, it doesn't exist.

Contributing to the Simantics codebase

The development of the Simantics platform is coordinated by the Simantics Technical Board. The Technical Board consists of three to six members that are selected among the THTH Simantics Division member organisations by the Simantics General Meeting. The duties of the Simantics Technical Board are defined in the THTH Simantics Division Rules and Regulations.

The Simantics Technical Board shall:

1. prepare a proposal for the Management Board for the exact content of the new releases of Simantics
2. prepare a proposal for the Management Board for the content of the training sessions
3. update simantics.org website both public and member areas

All contributions made to Simantics modules go through the Simantics Technical Board. The Simantics Technical Board meetings are arranged every week. These meetings are documented in the Simantics ticket management system (Redmine).

Using version control

First things first, read through Subversion best practices.

Branching

In Simantics, only new major releases may contain breaking changes. Therefore in Simantics we strive to use a specialization of the Branch-When-Needed system, i.e. Branch-Every-Major-Change. This allows us to keep continuous builds (hudson) working with less disruption, and enables easier pre-release testing, migration and peer review.

The rules are:

• Users commit their day-to-day work on /trunk.
• Rule #1: /trunk must compile and pass regression tests at all times. Committers who violate this rule are publically humiliated.
• Rule #2: a single commit (changeset) must not be so large so as to discourage peer-review.
• Rule #3: if rules #1 and #2 come into conflict (i.e. it's impossible to make a series of small commits without disrupting the trunk), then the user should create a branch and commit a series of smaller changesets there. This allows peer-review without disrupting the stability of /trunk.
• Rule #4: if API is changed, old version in the trunk must be tagged, and version number of the trunk must be increased. If API changes break other trunk code, development must continue in its own branch. Non-incremental change in an ontology is counted as major change (prevents using existing databases); new branch must be created.

Pros

/trunk is guaranteed to be stable at all times. The hassle of branching/merging is somewhat rare.

The word stable here implies mainly API stability but also functional stability. Functional stability must be enforced through both automatic and manual testing. If there are no test cases, functional stability becomes really hard to enforce. Therefore testing is key.

Violating these rules will result removal of your breaking commits from the SVN.

Integrating Branches

We obviously do not want development to continue eternally in branches so that changes are never merged between branches or integrated back to trunk. Branches are usually private or shared between a group of developers that work on a particular task. The process for integration to trunk is as follows:

• The Simantics Technical Board (STB) and its weekly meetings will be the place for the developers to advertise and present their changes to the board.
• The following things need to be documented for the changes in some way:
  • The what and why for the changes
  • Migration instructions from previous version (trunk):
    • A direct patch to fix problems
    • or plain-text instructions
  • When integration is done, mail must be sent to Simantics developers list about the change.
  • ...

TODO more description

Versioning

In Eclipse/OSGi conventions, components (bundles, plug-ins) are versioned <major>.<minor>.<service>[.<qualifier>].

Our approach to component versioning is as follows:

• All components are in one of two states: incubation or production
• Incubation components: <major> = 0, API breakage is allowed between <minor> versions
• Production components: <major> ≥ 1, API breakage is allowed between <major> versions

Creating a new plugin

Before a release, the plugin is in incubation phase and there are no quality or other requirements. The repository https://www.simulationsite.net/svn/simantics-incubator may be used to share or co-develop the plugin with other developers, but the plugin must not be included in the features defined in https://www.simulationsite.net/svn/simantics repository.

If the new plugin is a result of refactoring some already existing plugin into two pieces or otherwise is closely related to an other plugin under development, the new plugin can be committed to the same branch of https://www.simulationsite.net/svn/simantics where the other plugin is developed in. Note that whenever this kind of piecing happens, it has the potential to affect the consistency of other components or products. In such cases the refactor should be carried out in branches, not in trunk.

Naming a plugin

If the plugin is intended to be part of the official Simantics release, its name should begin with org.simantics followed by a word that describes the plugin. If the functionality of the same domain is divided into multiple plugins, the name may have more parts. The following suffixes are used:

• .feature for feature projects.
• .product A plugin defining a product.
• <plugin>.ui Provides UI-functionality for <plugin>. Separating base functionality and UI (that depends on SWT) is usually recommendable so that headless applications do not have unnecessary dependencies.
  • Example: org.simantics.simulation, org.simantics.simulation.ui

NOTE! If the the UI is light (e.g. wizards, dialogs, renderers, few external dependencies) the UI code can reside in <plugin>, a separate <plugin>.ui is not required. In such a case, the UI dependencies (to plugins such as org.eclipse.swt) are optional.

Naming in feature projects

Features have two distinct names: project name and feature ID. The project name is defined in the .project file. It shall contain the .feature suffix since the project name affects how Eclipse checks the project out into workspaces from version control systems. The feature ID is defined in feature.xml and it shall not contain the .feature suffix.

This naming convention exists for many reasons:

• P2 automatically appends .feature.group to to identification of installable units (IU) generated from features. Artifacts with .feature.feature.group ID tail .
• Works well with Buckminster and its resource maps.

Publising a standalone plugin

1. Ensure that the plugin satisfies the following quality requirements:
   • The plugin contains no compilation errors (against which dependencies???)
   • All public classes and interfaces are documented (javadoc)
   • Each public package contains a package-info.java containing package-related comments and annotations (see [1] chapter 7.4)
   • examples/
   • unittests/
   • Version information
   • (optional) Build Script (targets: clean, build, clean-build)
     • [name]_[version]-src.zip
     • [name]_[version].jar
     • [name]_[version].zip
       • doc/
         • manual.pdf
         • changelog.txt
       • examples/
       • unittests/
       • javadoc/ (optional)
       • [name]_[version]-src.zip
       • [name]_[version].jar
     • [name]_[version]-project.zip (Contains everything in workspace)
     • See Example: Non-OSGi
2. Create a homepage for the plugin in wiki (for example org.simantics.databoard) that contains:
   • Development documentation that is not written in javadoc (basic concepts, components etc.) [We should discuss more about what and how to document] and links to user documentation if the plugin contributes new user interfaces.
   • Change log
   • Plugin roadmap
   • Who are responsible for the plugin
   • (optional) Download to artifacts created by Build script
3. Commit the plugin to https://www.simulationsite.net/svn/simantics/<moduleName>/trunk
4. Create a tag of the plugin to https://www.simulationsite.net/svn/simantics/<moduleName>/tags/<version>
5. Inform other developers about the new plugin through the mailing list [simantics-developers@simantics.org]. See also Mailing Lists.

Including a new plug-in to existing features

Inclusion of a plug-in in a feature must always be carefully considered. Consider the following before including anything:

• Cohesion: Does the plug-in logically belong in the considered feature.xml?
• Dependencies: What does the new plug-in depend on? Which dependencies are optional? Do the dependencies of the feature's other plug-ins intersect naturally with those of the new plug-in?
• Permission: do not add anything without permission from the feature maintainer or send a patch and hope it will be merged.

It is highly recommended to have plug-ins included in a single feature only. Instead of including plug-ins in multiple features, prefer creating highly logical and cohesive features that can acyclically include each other. In general new completely new functionality should be put in its own feature. If you are unable to decide where a plug-in should be included, ask yourself is the plug-in properly cohesive, are its dependencies minimal?

Note that it is not technically mandatory that each plug-in only resides in one feature - it is simply good practice and will most likely help in training people to use the platform and also make maintainance easier. Having multiple features including the same plug-in will not break a PDE build, but will make the product structure more difficult to understand.

Modifying a feature will most likely break:

1. Project sets
   • FIX: add a line for the new plug-in in all project sets that include the modified feature
2. Hudson PDE builds that depend on the feature
   • TODO: link to instructions for general hudson and pde build configuration

Releasing a new version of a component

Here, a component stands for a plug-in, a set of plug-ins or a feature.

1. Increment version number(s), incl. MANIFEST.MF, plugin.xml, feature.xml, build.xml, homepage.
   • Bug fixes increment service number, API changes minor number and large releases major number.
2. Run test cases or test suite
3. Check java docs are compiled without errors. Read Javadocs
4. Update homepage
5. (optional) run build.xml and upload generated artifacts to homepage
6. Create SVN tag: trunk/ → tags/[version]
7. Inform other developers Simantics developers list.

Making refactorizations that affect other plugins

1. Plan refactorization beforehand with the maintainers of all affected plugins
2. Create a new branch for all affected plugins
3. Refactor, different possibilities
   • One person refactors all plugins
   • Every maintainer refactors his/her own plugins
     • Create rename scripts for other maintainers to help them
4. Check that everything works
5. If changes were large inform other developers in the mailing list
6. Commit plugins to trunk

Developing and maintaining a released plugin

Generally, people need to be able to commit their changes continuously to the SVN location dedicated for a component. Often committing is necessary for transferring their changes to another machine or just to keep their code safe. We can't allow developers committing API changes or broken temporary code into SVN locations that other projects are using - those need to be stable to some degree. Hence, the active development and API breaking must occur in SVN locations that are documented to be unstable and modifiable only by component maintainer(s). These locations shall be released to the general public according to the roadmap of the component. For every release, a tag shall be created. For every release external dependencies need to be documented: component name, version to use (tag). Releases are versioned. Version numbers are composed of three (3) segments: 3 integers and a string respectively named major.minor.service. Read [2] and [3].

1. the major segment indicates breakage in the API
2. the minor segment indicates "externally visible" changes
3. the service segment indicates bug fixes and the change of development stream (the semantics attached to development stream is new to this proposal, see below)

By default, the following SVN layout should be used for components:

component/

trunk/

• The main development branch, unstable by nature. This version should only be used by active developers of this component.

branches/

• Branches should be created:
  • when preparing for a release, for stabilizing the component while letting active development run along in trunk.
  • for working on bug-fix releases.
  • when working on a larger changes, trying out things or working with other developers. These kinds of branches should either be integrated back to where they were branched from or removed as obsolete.

tags/

• Released versions of the component. Other developers and project-sets should be using these for development if the component isn't already deployed into the target platform or it is necessary to have the code in your workspace for some other reason.

Developers may initially feel that creating tickets for every measly issue is too time consuming but it is necessary to make any kind of change tracking possible for releases. Obviously during the initial development of a component the code changes a lot and all the time. This is exactly why we have an incubator for starting projects that does not impose these strict rules but is more like a playground where the developer can freely "hack around" disregarding all these policies. A good developer will get started on design, testing and documentation also right from the start. When the developer feels his component is ready for consumption of the general public and the component passes peer review, it shall be ceremonially transferred from incubation into the official project. Each official component should have documentation, unit tests and possibly examples or really good reasons for not having any.

Modifying the target platform

The target platform is a composition of features and plug-ins that contains all the bundles that you as a developer can build your code against when starting from a blank table. It is a mixture of the standard Eclipse RCP platform, other general Eclipse components and also some Simantics components.

It makes sense to deploy into the target platform only if your plug-ins:

• are general, i.e. reusable by many other components
• have little dependencies
• are stable, both by API and functionally

Deploying plug-ins into the target platform will most likely break:

1. Hudson PDE builds that depend on the feature
   • Remove the deployed plug-ins from the plugins/ directory of the build structure to take the deployed plug-ins from the target platform
   • The target platform(s) used by the builds must be updated
   • TODO: link to instructions for general hudson and PDE build configuration
2. Project sets
   • FIX: remove lines matching the plug-ins that were moved into the target platform

Deploying plug-ins for the target platform

TODO: these instructions are out-of-date for Simantics 1.4 and the proper use of P2 repositories

Standard Procedure

1. Test your deployed components.
2. Read the instructions below and deploy your plug-ins and features as P2 repositories
3. Make your built P2 repository available on the internet and notify the target platform maintainer to grab your changes and update the official target platform.

There are two main ways of deploying for the target platform:

• deploying plug-ins and fragments directly
• deploying plug-ins and fragments through features

These steps can be performed using the Eclipse export wizard through the Plug-in Development/Deployable features and Plug-in Development/Deployable plug-ins and fragments wizards

When considering whether your component should have a feature or not, consider these points:

• Eclipse products/applications are always made up of either a set of features or a set of plug-ins. In order for someone to use your component in an product, either your features or your plug-ins need to be included somewhere in the product. If a product is using features and you don't provide one, the product either has to create a new feature and include it or include your plug-ins in his own features.
• Features can be used as a way to give a collective version all plug-ins of a component. The feature will identify the versions of its included plug-ins but these need not be identical to the feature version. Not all plug-ins have to change versions while a feature version changes each time a new release of the component is made.
  • IMPORTANT: The feature deployment build will produce a P2 repository containing content.jar, artifacts.jar and two directories, features/ and plugins/. The features-directory will contain the feature as JAR files which is correct when used as a P2 repository. But if you want to install your deployed features into the current target platforms, the feature jars must be unpacked with the JAR file names as the directory names' Don't unpack directly into the features directory, the feature JARs do not contain a common parent folder. For example, if you've built your feature with an ID my.feature and version 1.0.0, the build will produce a file my.feature_1.0.0.jar. The contents of this jar must be unpacked into features/my.feature_1.0.0/. If you copied the JAR files into the features directory and unpack them there, do not leave the JARs in the features directory. This will also break the features.

Building Products

The best way to currently build products is to use the Eclipse tool Buckminster for the job.

External links

• Buckminster Homepage
• Buckminster Definitive Guide