Tutorial: Database Development

Basic concepts

Session

Database session / connection that allows reading and writing the database using transactions.

Transaction

A database operation; either read, or write operation. Simantics supports three types of transactions: Read, Write, and WriteOnly.

Resource

A single object in the database.

Statement

Triple consisting of subject, predicate, and object

Literal

primitive value (int, double, String,..) or an array of primitive values.

Reading the database

Literals

In this example we set a Label to show a Resource’s name. A resource's name is defined using the L0.HasName property relation.

To support Layer0 URI ↔ resource mapping mechanism (see Layer0.pdf section 4), resource names have certain restrictions.

• A single resource cannot have more than one statement objects with L0.ConsistsOf-predicate that have the same name. For example the following graph snippet in Graph File Format does not work with the URI → resource discovery mechanism:

a : L0.Entity
    L0.ConsistsOf
        a : L0.Entity
        a : L0.Entity
        b : L0.Entity

Synchronous read with a return value

One way to read information stored in the database is to use synchronous reads with a return value. Return value can be any Java class, and here it is String.

try {
	String name = session.syncRequest(new Read<String>() {
		@Override
		public String perform(ReadGraph graph) throws DatabaseException {
			Layer0 l0 = Layer0.getInstance(graph);
			return graph.getRelatedValue(resource, l0.HasName);
				
		}});
	label.setText(name);
} catch (DatabaseException e) {
	// handle exception here
}

Database connection may fail during the transaction and send an exception. Therefore you have to handle the exception in some way. Here we have wrapped the transaction inside try-catch block. In real code the most common thing to do is to let the exception propagate back to UI code.

All literals can be read the same way, you just have to provide correct relation. There are also other methods to read literals that behave differently:

• getRelatedValue(): returns literal or throws and exception if value is not found
• getPossibleRelatedValue(): returns literal value or null, if the value does not exist.

Asynchronous read

Asynchronous read allows creating requests that are processed in the background. If you don’t need the result immediately, it is recommended to use asynchronous reads for performance reasons or to prevent blocking UI threads.

session.asyncRequest(new ReadRequest() {
	@Override
	public void run(ReadGraph graph) throws DatabaseException {
		Layer0 l0 = Layer0.getInstance(graph);
		final String name =  graph.getRelatedValue(resource, l0.HasName);
		Display.getDefault().asyncExec(new Runnable() {
			@Override
			public void run() {
				if (!label.isDisposed())
					label.setText(name);
			}
		});
	}
});

Here we use separate Runnable to set the label’s text (SWT components must be accessed from main / SWT thread). Note that we have to check if the label is disposed before setting the text.

Asynchronous read with a Procedure

Using a procedure allows you to handle possible exceptions caused by failed asynchronous transaction.

session.asyncRequest(new Read<String>() {
	@Override
	public String perform(ReadGraph graph) throws DatabaseException {
		Layer0 l0 = Layer0.getInstance(graph);
		return graph.getRelatedValue(resource, l0.HasName);
	}
}, new Procedure<String>() {
	@Override
	public void exception(Throwable t) {
		// handle exception here
	}

	@Override
	public void execute(final String name) {
		Display.getDefault().asyncExec(new Runnable() {
			@Override
			public void run() {
				if (!label.isDisposed())
					label.setText(name);
			}
		});
	}
});

Asynchronous read with a SyncListener

Using SyncListener with a read transaction leaves a listener to the database that notifies changes in the read data. There exist also Listener and AsyncListener interfaces. SyncListener provides a ReadGraph for the listener, AsyncListener provides AsyncReadGraph for the user. Using Listener provides no graph access at all, just a notification of a changed read request result.

session.asyncRequest(new Read<String>() {
	@Override
	public String perform(ReadGraph graph) throws DatabaseException {
		Layer0 l0 = Layer0.getInstance(graph);
		return graph.getRelatedValue(resource, l0.HasName);
	}
}, new SyncListener<String>() {
	@Override
	public void exception(ReadGraph graph, Throwable throwable) throws DatabaseException {
		// handle exception here
	}
			
	@Override
	public void execute(ReadGraph graph, final String name) throws DatabaseException {
		Display.getDefault().asyncExec(new Runnable() {
			@Override
			public void run() {
				if (!label.isDisposed())
					label.setText(name);
			}
		});
	}
	
	@Override
	public boolean isDisposed() {
		return label.isDisposed();
	}

});

Here the listener updates the label when the name changes. The method isDisposed() controls lifecycle of the listener, and here when the label is disposed, the listener is released and it stops updating the label.

Triggering a read transaction (and its SyncListener) explicitely or without changes in data

Idea is to embed an ExternalRead object (trigger) into a read request chain. Thus, the chain will be re-evaluated if the trigger is launched. Note, that the trigger has to be embedded with a listener. This guarantees that the trigger is kept alive.

final TriggeringExternalRead trigger = new TriggeringExternalRead();

session.asyncRequest(new Read<String>() {
	@Override
	public String perform(ReadGraph graph) throws DatabaseException {

                // embed a trigger with listener into this read request chain
                graph.syncRequest(trigger, new Listener<Integer>(){

				@Override
				public void execute(Integer result) {
					// TODO Auto-generated method stub
					
				}

				@Override
				public void exception(Throwable t) {
					// TODO Auto-generated method stub
					
				}

				@Override
				public boolean isDisposed() {
					// TODO Auto-generated method stub
					return false;
				}});
		); 
		// end of embeding

		Layer0 l0 = Layer0.getInstance(graph);
		return graph.getRelatedValue(resource, l0.HasName);
	}
});

To launch a trigger in other code section just call its fire() -method.

	trigger.fire();

Resources

Reading resources is similar to reading literals. Here is an example that reads all resources that are connected to a Resource with a ConsistsOf-relation:

try {
	Collection<Resource> children = session.syncRequest(new Read<Collection<Resource>>() {
		@Override
		public Collection<Resource> perform(ReadGraph graph) throws DatabaseException {
			Layer0 l0 = Layer0.getInstance(graph);
			return graph.getObjects(resource, l0.ConsistsOf);
		}
	});
} catch (DatabaseException e) {
	// handle exception here
}

Same asynchronous, Procedure, and Listener mechanisms work with reading resources. For example, you can use listener to be notified if resources are added or removed:

session.asyncRequest(new Read<Collection<Resource>>() {
	@Override
	public Collection<Resource> perform(ReadGraph graph) throws DatabaseException {
		Layer0 l0 = Layer0.getInstance(graph);
		return graph.getObjects(resource, l0.ConsistsOf);
	}
},new AsyncListener<Collection<Resource>>() {
	@Override
	public void exception(AsyncReadGraph graph, Throwable throwable){
		// handle exception here
	}
			
	@Override
	public void execute(AsyncReadGraph graph, Collection<Resource> result) {
		// this is run every time a resource is added or removed
	}
			
	@Override
	public boolean isDisposed() {
		// this must return true when the listener is not needed
		// anymore to release the listener
		return false;
	}
});

Writing the database

Explicit clustering

Here we show how you can influence the way resources are grouped by the underlying implemention. You can significantly improve the dynamic behaviour of the database by grouping resources that are likely to be referenced in the same context/operation.

try {
        session.syncRequest(new WriteRequest() {
            @Override
            public void perform(WriteGraph graph) throws DatabaseException {
                Layer0 l0 = Layer0.getInstance(graph);
                try {
                    graph.newClusterSet(l0.InstanceOf);
                } catch (ClusterSetExistException e) {
                    // Cluster set exits, no problem.
                }
                Resource newResource = graph.newResource(l0.InstanceOf);
                graph.claim(newResource, l0.InstanceOf, null, l0.Entity); 
            }
        });

Here is the same effect with setting a default for newResource method.

        session.syncRequest(new WriteRequest() {
            @Override
            public void perform(WriteGraph graph) throws DatabaseException {
                Layer0 l0 = Layer0.getInstance(graph);
                Resource oldClusterSet = graph.setClusterSet4NewResource(l0.InstanceOf);
                try {
                    Resource newResource = graph.newResource();
                    graph.claim(newResource, l0.InstanceOf, null, l0.Entity);
                } finally {
                    graph.setClusterSet4NewResource(oldClusterSet);
                }
            }
        });

Setting Literals

Here we set a name of a resource to “New name”. Both synchronous and asynchronous versions are similar, the only difference is that with synchronous way you have to handle the exception, while using an asynchronous write without a Procedure does not let you know if the transaction failed. Any exceptions will just be logged by the database client library.

try {
	session.syncRequest(new WriteRequest() {
		@Override
		public void perform(WriteGraph graph) throws DatabaseException {
			Layer0 l0 = Layer0.getInstance(graph);
			graph.claimLiteral(resource, l0.HasName, "New name");
		}
	});
} catch (DatabaseException e) {
	// TODO Auto-generated catch block
	e.printStackTrace();
}

session.asyncRequest(new WriteRequest() {	
	@Override
	public void perform(WriteGraph graph) throws DatabaseException {
		Layer0 l0 = Layer0.getInstance(graph);
		graph.claimLiteral(resource, l0.HasName, "New name");
	}
});

Using a Procedure with an asynchronous write transaction is similar to read transactions, so we don’t have separate example of that. Note that listeners can only be used with read transactions, not write transactions.

Creating new Resources

Same synchronous and asynchronous mechanisms apply to writing new Resources. The difference is that with asynchronous you can use a Callback to check if write failed, or separate WriteResult and a Procedure to return and check custom results of a write transaction.

Here is an example that creates a new library, gives it a name “New Library” and connects it to given resource.

session.asyncRequest(new WriteRequest() {
	@Override
	public void perform(WriteGraph graph) throws DatabaseException {
		Layer0 l0 = Layer0.getInstance(graph);
		Resource newResource = graph.newResource();
		graph.claim(newResource, l0.InstanceOf, l0.Library);
		graph.claimLiteral(newResource, l0.HasName, "New Library");
		graph.claim(resource, l0.ConsistsOf, newResource);
				
	}
});

The key points here are:

• New resource are created with WriteGraph.newResource()
• You must set resource’s type using InstanceOf-relation.
• If you do not connect created resources anywhere, the resources become orphan resources. For orphan resources there is no statement path to/from the database root resource. They cannot be traversed and therefore found from the database.

Deleting resources

Here is an example that removes everything the given resource consists of directly. In the end it removes all the statements from the given resource, thereby essentially removing the resource itself. A resource is considered to not exist if it has no statements, since there's nothing to define its nature.

session.asyncRequest(new WriteRequest() {
	@Override
	public void perform(WriteGraph graph) throws DatabaseException {
		Layer0 l0 = Layer0.getInstance(graph);
                for (Resource consistsOf : graph.getObjects(resource, l0.ConsistsOf)) {
                    // Remove all statements (consistsOf, ?p, ?o) and their possible inverse statements
                    graph.deny(consistsOf);
                }
                // Remove resource itself by removing any statements to/from it.
                graph.deny(resource);
	}
});

Another example, that deletes the name of the given resource.

session.asyncRequest(new WriteRequest() {
	@Override
	public void perform(WriteGraph graph) throws DatabaseException {
		Layer0 l0 = Layer0.getInstance(graph);
                graph.denyLiteral(resource, l0.HasName);
	}
});

The key points here are:

• deny -methods are used to delete statements from the database
• denyLiteral -methods are used to delete literal values from the database
• A resource that has no statements does not exist.

Example case: Product and Purchase management

In this example, we will implement a simple product and purchase management system. The system will allow user to define new products, and create new purchase orders. To keep the example simple, the products have just a name and unit cost, and purchase orders will have name of the customer and a list of ordered products.

We have already implemented skeleton for the system. It contains basic user interface for creating products and purchases, but it lacks several features that we are going to add during this tutorial. The base code can be found from SVN:

• https://www.simulationsite.net/svn/simantics/tutorials/trunk/org.simantics.example
• https://www.simulationsite.net/svn/simantics/tutorials/trunk/org.simantics.example.feature

Ontology that we are using is defined in Example.pgraph file. It is in org.simantics.example/graph folder. To run the example application, use simantics-example.product. After starting the example product, you should get a user interface like the one in Figure 1.

Eclipse will complain that AsyncPurchaseEditorFinal.java does not compile, but it does not prevent
starting the application. 

Figure 1: Tutorial application after one created product (Product 1) and one purchase.

When you use the application you may notice following:

• Name of the customer or the date of the order is not shown anywhere.
• When purchase editor is open, new products won’t show up in it. Only closing and opening the editor shows the products.
• Adding new products to purchases can be done only in Example Browser
• While you can order several products, you cannot order multiple items of the same product.

Org.simantics.example plug-in is split into four packages:

• editors: Purchase Editor implementation.
• handlers: UI actions.
• project: Project binding. See … for more details.
• resource: Taxonomy generated from the used ontology.

The most important parts of this tutorial are editors. AsyncPurchaseEditor and classes in handlers-package.

Adding the missing information to UI

First we have to add new Text widgets to the Purchase Editor. At the beginning of AsyncPurchaseEditor.java you see:

// UI controls
private final Text totalCost;
private final TableViewer productViewer;
private final TableViewerColumn nameColumn;
private final TableViewerColumn costColumn;

Add new line after totalCost:

private final Text customer;
private final Text date;

Then we have to instantiate text fields, create labels that inform user what the text fields mean, and setup the layout properly. In the beginning of the constructor modify:

Label l = new Label(this, SWT.NONE);
l.setText("Total Price:");
GridDataFactory.fillDefaults().applyTo(l);

totalCost = new Text(this, SWT.NONE);
totalCost.setEditable(false);
GridDataFactory.fillDefaults().grab(true, false).applyTo(totalCost);

To:

Label l = new Label(this, SWT.NONE);
l.setText("Customer:");
GridDataFactory.fillDefaults().applyTo(l);
        
customer = new Text(this, SWT.NONE);
customer.setEditable(false);
GridDataFactory.fillDefaults().grab(true, false).applyTo(customer);

l = new Label(this, SWT.NONE);
l.setText("Date:");
GridDataFactory.fillDefaults().applyTo(l);

date = new Text(this, SWT.NONE);
date.setEditable(false);
GridDataFactory.fillDefaults().grab(true, false).applyTo(date);

l = new Label(this, SWT.NONE);
l.setText("Total Price:");
GridDataFactory.fillDefaults().applyTo(l);

totalCost = new Text(this, SWT.NONE);
totalCost.setEditable(false);
GridDataFactory.fillDefaults().grab(true, false).applyTo(totalCost);

And finally, we have to update text fields to show the data. Modify updateUI(Model m) method to contain:

customer.setText(m.boughtBy);
date.setText(m.boughtAt);

Completed code is in org.simantics.example.phases.AsyncPurchaseEditorMiddle.hava

Fixing the editor updates

Currently the Purchase Editor loads products in the purchase when the editor is opened, but if new products are added, editor’s user interface won’t update. To fix this, modify AsyncPurhaseEditor’s setInput(Resource) method. Currently the method is:

public void setInput(Resource r) {
    this.input = r;
    if (input == null)
        return;

    session.asyncRequest(new Read<Model>() {
       @Override
       public Model perform(ReadGraph graph) throws DatabaseException {
           return loadModel(graph, input);
       }
    }, new Procedure<Model>() {
          @Override
          public void exception(Throwable throwable) {
                ExceptionUtils.logAndShowError(throwable);
          }
          @Override
          public void execute(final Model result) {
            // Set the loaded model as the model viewed by the UI.
            getDisplay().asyncExec(new Runnable() {
               @Override
               public void run() {
                  if (isDisposed())
                     return;
                  updateUI(result);
               }
            });
         }
   });
}

Hint: You have to replace the Procedure with an Listener.

Answer:

public void setInput(Resource r) {
   this.input = r;
   if (input == null)
      return;

   session.asyncRequest(new Read<Model>() {
      @Override
      public Model perform(ReadGraph graph) throws DatabaseException {
         return loadModel(graph, input);
      }
   }, new Listener<Model>() {
        @Override
        public boolean isDisposed() {
        	return AsyncPurchaseEditor.this.isDisposed();
        }

        @Override
        public void exception(AsyncReadGraph graph, Throwable throwable) {
        	ExceptionUtils.logAndShowError(throwable);
        }
            
        @Override
        public void execute(final Model result) {
            // Set the loaded model as the model viewed by the UI.
            getDisplay().asyncExec(new Runnable() {
               @Override
               public void run() {
                  if (AsyncPurchaseEditor.this.isDisposed())
                     return;
                  updateUI(result);
               }
            });
        }
    });
}

After this modification the editor should update its contents when the data is changed. You may try to add new product to purchase order when the editor is open to check that it really works.

Completed code is in org.simantics.example.phases.AsyncPurchaseEditorMiddle2.hava

Adding Context Menu to Purchase Editor

We are going to add context menu to Purchase editor. The context menu will be used for adding new products to a purchase.

There are several ways to implement the context menu in Eclipse. For sake of simplicity, we are not going to implement context menu that allows contributions from plug-in definitions (plugin.xml).

The code that Example Browser uses in its menu is in handlers.AddProductsContributionItem. Sadly, we cannot reuse that implementation, since it uses Eclipse’s selection mechanism to interpret current purchase, while we want to use the purchase that editor has opened.

Put this class to AsyncPurchaseEditor.java:

public class AddProductsContributionItem extends CompoundContributionItem {

  public AddProductsContributionItem() {
    super();
  }

  public AddProductsContributionItem(String id) {
    super(id);
  }

  @Override
  protected IContributionItem[] getContributionItems() {

    ProductManagerImpl manager = SimanticsUI.getProject().getHint(ProductManager.PRODUCT_MANAGER);

    Map<Resource, String> products = manager.getProducts();

    final Resource purchase = input;
            
    IContributionItem[] result = new IContributionItem[products.size()];
    int i=0;
    System.out.println("Products count = " + products.size());
    for(Map.Entry<Resource, String> entry : products.entrySet()) {
      final Resource product = entry.getKey();
      result[i++] = new ActionContributionItem(new Action("Add " + entry.getValue()) {
        @Override
        public void run() {
          SimanticsUI.getSession().asyncRequest(new WriteRequest() {
            @Override
            public void perform(WriteGraph graph) throws DatabaseException {
              Layer0 l0 = Layer0.getInstance(graph);
              if(!graph.hasStatement(purchase, l0.ConsistsOf, product))
                graph.claim(purchase, l0.ConsistsOf, product);
              }
           });
         }
      });
    }
    return result;
  }
}

It is the same code as handlers.AddProductsContributionItem, but the selection mechanism has been replaced with the editor input.

To add context menu to the editor, add this code to the end of AsyncPurchaseEditor’s constructor:

final MenuManager manager = new MenuManager();
manager.add(new AddProductsContributionItem());
Menu menu = manager.createContextMenu(productViewer.getControl());

this.addDisposeListener(new DisposeListener() {
			
    @Override
    public void widgetDisposed(DisposeEvent e) {
	manager.dispose();
    }
});
        
productViewer.getControl().setMenu(menu);

Completed code is in org.simantics.example.phases.AsyncPurchaseEditorMiddle3.hava

Supporting multiple items of the same product

Current ontology is designed so that one purchase order contains products, but product count cannot be stored anywhere [Figure 2].

Figure 2: Purchases contain only links to products, and there is no way to store product count.

To add support for product count we have to change the ontology. We have to introduce new type “ProductPurchase” that will have link to ordered Product and property for product count [Figure 3].

Figure 3: Purchases contain ProductCounts that link to Products.

Open the ontology (example.pgraph) and add these to ExampleOntology:

EXAMPLE.ProductCount <R L0.HasProperty : L0.FunctionalRelation
    L0.HasDomain EXAMPLE.ProductPurchase
    L0.HasRange L0.Double

EXAMPLE.ProductPurchase <T L0.Entity
    @L0.singleProperty EXAMPLE.ProductCount

After this clean the applications workspace, since now it has a database with old ontologies.

To get the application working, we must update AddProductContributionsItem.java and AsyncPurchaseEditor.java. We start with the AddProduct…java. Currently its wrote transaction is:

if(!graph.hasStatement(purchase, l0.ConsistsOf, product))
   graph.claim(purchase, l0.ConsistsOf, product);

This just creates a link between the Purchase and selected Product. We need to create an instance of ProductPurchase link it to the Purchase and to selected Product:

 for (Resource productPurchase : graph.getObjects(purchase, l0.ConsistsOf)) {
    if (graph.hasStatement(productPurchase, l0.ConsistsOf, product)) {
       double amount = graph.getRelatedValue(productPurchase, e.ProductCount);
       graph.claimLiteral(productPurchase, e.ProductCount, amount + 1.0);
       return;
    }
}
	
Resource productPurchase = graph.newResource();
graph.claim(productPurchase, l0.InstanceOf, e.ProductPurchase);
graph.claimLiteral(productPurchase, e.ProductCount, 1.0);
graph.claim(purchase, l0.ConsistsOf, productPurchase);
 graph.claim(productPurchase, l0.ConsistsOf, product);

This code checks first if the product is already in the purchase, and increases count by one if it is.

Then we edit AsyncPurchaseEditor.java. Start by adding “amount” to Item:

class Item {
   String name;
   double costs;
   double amount;
}

Next, change loadModel() method to read the data in current form and include product count:

if (g.isInstanceOf(modelResource, ex.Purchase)) {
   Resource purchase = modelResource;
   m.boughtAt = g.getRelatedValue(purchase, ex.BoughtAt);
   m.boughtBy = g.getRelatedValue(purchase, ex.BoughtBy);

   List<Item> items = new ArrayList<Item>();
   for (Resource productPurchase : g.getObjects(purchase, g.getBuiltins().ConsistsOf)) {
     	Resource product = g.getSingleObject(productPurchase, b.ConsistsOf);
      Item i = new Item();
      i.name = g.getRelatedValue(product, b.HasName);
      i.costs = g.getRelatedValue(product, ex.Costs);
      i.amount = g.getRelatedValue(productPurchase, ex.ProductCount);
      items.add(i);
   }
   m.items = items.toArray(new Item[items.size()]);
}

Then update AddProductContributionItem inside AsyncPurchaseEditor.java to match the previous edit.

To show the product count we have to add new column to the table. First, create a member for the column:

private final TableViewerColumn amountColumn;

Then initialize the column in the AsyncPurchaseEditor’s constructor:

amountColumn = new TableViewerColumn(productViewer, SWT.LEFT);
amountColumn.getColumn().setWidth(100);
amountColumn.getColumn().setText("Amount");
amountColumn.setLabelProvider(new CellLabelProvider() {
    @Override
    public void update(ViewerCell cell) {
        Item i = (Item) cell.getElement();
        cell.setText(String.valueOf(i.amount));
    }
});

And last, change updateUI() method to calculate total costs correctly and update the table’s layout:

private void updateUI(Model m) {
   // Update the UI based on the contents of the specified model.
   double total = 0;
   for (Item i : m.items) {
      total += i.costs * i.amount;
   }
   totalCost.setText(String.valueOf(total));

   customer.setText(m.boughtBy);
   date.setText(m.boughtAt);
        
   productViewer.setInput(m);

   nameColumn.getColumn().pack();
   costColumn.getColumn().pack();
   amountColumn.getColumn().pack();
}

After these modifications, your application should look like in Figure 4.

Figure 4: Amount column has been added into the table.

Completed code is in org.simantics.example.phases.AsyncPurchaseEditorFinal.java and completed ontology in example.pgraph.end