Org.simantics.scenegraph

From Developer Documents
Jump to navigation Jump to search

org.simantics.scenegraph (SVN) is a scene graph library for 2D rendering with a focus on supporting remote rendering using a client-server architecture.

Overview

org.simantics.scenegraph provides a generic serializable scene graph datastructure ( wikipedia:Scene graph ) for displaying 2D and 3D graphics.

The idea of scene graph is that it is an independent datastructure that can be rendered as such. It can only access local variables, thus it provides a strict interface between the visualization and program logic. Mainly for this reason scene graph is an ideal datastructure for rendering graphics remotely.

Architecture

org.simantics.scenegraph is designed to be used for rendering graphics locally on desktop application, but also remotely from server to client. For this reason, it has some features that a normal scene graph implementation doesn't have.

In org.simantics.scenegraph, the scene graph datastructure is a tree consisting of ParentNodes and Nodes.

  • Node is a leaf in the tree, thus it cannot have children, but it should be used to render visible objects.
  • ParentNode is used to group a set of nodes (children) and to give some common parameters for the children.

To enable scene graph datastructure synchronization from a server to a client, a mechanism for detecting changes in the datastructure is required. For this purpose, all changes to Node variables must be made through a public method (usually setter). These methods that change local variables are marked with annotations, thus the solution is very similar to what Hibernate uses with entities.

Node methods can have four different annotations, three for using scene graph remotely, and one for synchronizing node variables automatically from the graph:

  • SyncField(..) When a method annotated with this is called, the fields listed in the annotation are updated to the client side (if running in remote mode)
  • ServerSide Methods with ServerSide annotation are intended to be executed on server-side, thus it can access resources outside the scenegraph datastructure (using interfaces)
  • ClientSide Methods with this annotation are executed on client-side, thus it should only access resources inside the scenegraph datastructure
  • PropertySetter This annotation is used to map graph properties to node automatically. This annotation can only be used in setter methods, that take one argument. Notice that the argument type must match with the object that is read from the graph (i.e., for double values you must use Double type instead of double, float, or Float).

Without these annotations, the Node will not work properly when using scenegraph remotely.

These annotations (except PropertySetter) are handled with cglib MethodInterceptor which is a proxyclass for the actual class implementation, and contains an interceptor for performing actions before and after the actual method is called. For this reason, the Node classes should not be instantiated directly, but through ParentNode.addNode method. Cglib support is implemented in org.simantics.scenegraph.remote plugin, and thus org.simantics.scenegraph plugin does not depend on cglib.

ParentNode and Node classes implement only functionalities for creating and modifying the scene graph datastructure but do not contain any methods for rendering.

2D

Java 2D API compatible implementation of the scene graph is in org.simantics.scenegraph.g2d package:

  • G2DNode is a baseclass for all leaf nodes that perform 2D rendering
  • G2DParentNode is a baseclass for the 2D compatible parent nodes
  • IG2DNode is a common interface for G2DNode and G2DParentNode which should be used when traversing through the data structure.
  • G2DSceneGraph G2DParentNode implementation containing event listeners and some extra features. This should always be the root component of a G2D scene graph datastructure.

Example G2DNode: 

public class BackgroundNode extends G2DNode {
    private static final long serialVersionUID = -1L;	
    protected Rectangle2D bounds = null;
	
    @SyncField("bounds")
    public void setBounds(Rectangle2D bounds) {
        this.bounds = bounds;
    }
	
    @Override
    public void render(Graphics2D g) {
        g.fill(bounds);
    }
}

Because org.simantics.scenegraph is an independent scene graph datastructure, it can be rendered as such. Rendering can be performed by calling G2DSceneGraph.render(Graphics2D g2d) method. No changes are allowed to the scene graph while the structure is rendered (and rendering methods should not make any changes to the structure either, excluding transient node fields that are not synchronized between server and client).

3D

JOGL compatible implementation of the scenegraph can be found under package org.simantics.scenegraph.g3d.jogl in a separate plug-in org.simantics.scenegraph.g3d.

  • JOGLNode
  • JOGLParentNode
  • JOGLPaintable
  • JOGLSceneGraph

See org.simantics.scenegraph.g3d.eclipse.JOGLLocalView for an example how to use org.simantics.scenegraph with OpenGL.

Serialization

For serializing the scene graph datastructure there is class called org.simantics.scenegraph.remote.Serializer. The Serializer simply listens for any changes in the scene graph, and translates those changes into events which are then serialized and sent to ObjectOutputStream. To enable Serializer to listen for changes in the scene graph, the scene graph root node must be initialized with cglib. For this purpose there is a static method in Serializer: 

 G2DSceneGraph root = Serializer.newRemoteSceneGraphNode(G2DSceneGraph.class);
 Serializer serializer = new Serializer(root, input, output);
 serializer.start();

After this, the root node can be used normally. Any child added to it will be created by cglib automatically, hence Serializer works automatically with these whole scene graph data structure.

On the client side, Serializer can be directly attached to object streams, and it will create G2DSceneGraph object for you: 

 Serializer serializer = new Serializer(input, output);
 serializer.start();
 G2DSceneGraph root = (G2DSceneGraph)serializer.getRoot();

Because the system works asynchronously, you should actually attach UpdateListener to serializer in real case.

Notice that you have to attach the serializer to the scene graph before any other operation. Serializer is an instance of PropertyChangeListener which will be set to listen the changes in the scene graph datastructure. If there are changes made before the serializer is attached, these changes will not be serialized.

Serializer uses ObjectInput and ObjectOutput to serialize and unserialize the scene graph, thus it can be used to write the scenegraph to a file and from a file, or to synchronize the scenegraph in a client-server system.

Using scenegraph

  • Create scenegraph
G2DSceneGraph sg = new G2DSceneGraph();
sg.setRootPane(instanceOfJComponent); // In case you want to use swing components, you must set rootPane.
// Register event handlers
addKeyListener(sg);
addMouseWheelListener(sg);
addMouseListener(sg);
addMouseMotionListener(sg);
addFocusListener(sg);
  • Create node
ShapeNode shape = sg.addNode(ShapeNode.class);
shape.setShape(new Rectangle2D.Double(10, 10, 100, 100));
shape.setColor(Color.RED);
  • Render
sg.render(g2d);
  • Move shape
shape.setTransform(AffineTransform.getTranslateInstance(5, 5));
  • Updating node from stateless object
ShapeNode shape = sg.getOrCreateNode("bounds", ShapeNode.class);
shape.setShape(new Rectangle2D.Double(10, 10, 100, 100));
shape.setColor(Color.RED);

Scenegraph in Simantics

  • Each canvas participant that wants to draw something should implement methods annotated with @SGInit and @SGCleanup
public class Participant extends AbstractCanvasParticipant {
   protected ShapeNode node = null;

   @SGInit
   public void initSG(G2DParentNode parent) {
       node = parent.addNode(ShapeNode.class);
       node.setZIndex(PAINT_PRIORITY);
       node.setShape(new Rectangle2D.Double(1, 1, 20, 20));
   }

   @SGCleanup
   public void cleanupSG() {
       node.remove();
       node = null;
   }
}
  • Optionally, you can specify a designation for @SGInit that specifies how you want your nodes to be transformed, i.e. in control space or in canvas space. The default value is canvas space. Use the scene graph viewer to understand how this affects your scene graph.
@SGInit(designation = SGDesignation.CONTROL)
  • Each diagram element can implement interface SceneGraph to give their own contribution to the scene graph.
public class ElementSGNode implements SceneGraph {
   public static final Key SG_NODE = new SceneGraphNodeKey(Node.class, "SG_NODE");

   @Override
   public void init(final IElement e, G2DParentNode parent) {
       ShapeNode node = (ShapeNode) e.getHint(SG_NODE);
       if (node == null) {
           node = parent.addNode(ShapeNode.class);
           e.setHint(ElementHints.KEY_SG_NODE, node);
       }
       node.setShape(new Rectangle2D.Double(1, 1, 20, 20));
   }

   @Override
   public void cleanup(IElement e) {
       Node node = e.removeHint(SG_NODE);
       if (node != null)
           node.remove();
   }
}

org.simantics.scenegraph features

org.simantics.scenegraph package contains lots of readymade nodes and features. For a list of g2d nodes, see org.simantics.scenegraph.g2d.nodes.

Swing support

It is possible to integrate Swing components to 2D scenegraph by using org.simantics.scenegraph.g2d.nodes.swing.ComponentNode class. See for example MonitorNode or Trend2DNode for an example.

Eclipse view

org.simantics.scenegraph.eclipse package contains local and remote view of 2D and 3D scene graphs. You can try out these demos by executing the project as Eclipse Application and choosing the views "SceneGraph Remote View", "SceneGraph Local View", "JOGL SceneGraph Local View", "JOGL SceneGraph Remote View".

2D-3D integration

It is possible to create a G2DParentNode that works as a wrapper between 2D and 3D scenegraphs. This is not implemented yet.

Node Bounds

In order to perform any kind of rendering optimization in the future some kind of spatial subdivision methods need to be employed. For example Trolltech QT GraphicsView [1] implements BSP-based partitioning to accelerate interaction and rendering in its 2D systems. Bounds information is also useful for picking facilitation.

Currently scene graph nodes do have bounds information, and it is used in rendering to filter off nodes that are not visible in the window. However, this information is not currently used to render view partially. Partial rendering is something that needs to be implemented in the future.

Implementation

  • Each node has a method getBoundsInLocal() which returns a Rectangle2D instance describing the 2D boundaries of the node in the element's local coordinate system.
  • Node base implementation classes have a method getBounds() that returns the node boundaries with local transforms applied.
  • Nodes that have children (i.e., a parent node that does not draw anything) will return union of its childrens bounds.
  • G2DSceneGraph root node will cache bounds information.
  • Bounds will need caching in order to be efficiently implemented in parent nodes. One option is to implement a mechanism for propagating transformation changes up in the node tree to keep parent nodes aware of transformation changes in their children. Possily implement a listener in parent nodes that can be registered into any transformable node.

Node ⇔ ID mapping

(ticket:871)

In DOM implementations the client has the possibility to search for document nodes by their ID (getElementById(String)). Similarly scene graph nodes can be given identifications based on which they can be looked up from the scene graph through ILookupService which is implemented by scene graph root node G2DSceneGraph.

Normally the scene graph is a tree of nodes. Giving nodes scene graph -locally unique ID's through the lookup mechanism makes it possible to create links between nodes. The links effectively make the scene graph more like a DAG (Directed Acyclic Graph) or even a DG (Dependency Graph). Just as in DOM, the id attribute is not mandatory. Node lookup will only be possible if an ID is provided.

The lookup facility is available at the root node of the scene graph. Considering performance, having to always traverse to the root node to make lookups is a very likely bottleneck. This problem has been alleviated by using transient root node caches in ParentNode. This will naturally make the scene graph a bit more space consuming.

Using LookupService

Giving a lookup ID to a node
Lookup ID's are arbitrary strings.
    // Create a new node
    G2DSceneGraph root = ...;
    Node child1 = root.addNode("node child 1", G2DParentNode.class);
    Node child2 = root.addNode("node child 2", G2DParentNode.class);

    // Give it a lookup ID
    child2.setLookupId("child 2 lookup id");
    // OR
    NodeUtil.map(child2, "child 2 lookup id");


Performing a lookup
    // ID -> INode
    INode lookedUpNode = NodeUtil.lookupNode(child1, "child 2 lookup id");
    //OR
    INode lookedUpNode = root.lookupNode("child 2 lookup id");
    assert lookedUpNode == child2;

    // INode -> ID
    String lookedUpId = NodeUtil.lookupId(child2);
    assert lookedUpId.equals("child 2 lookup id");

    // See NodeUtil for more utility methods for lookups and associations

Main interfaces

ParentNode

org.simantics.scenegraph.ParentNode

Add child node

Add node simply creates a new node object with the given class and sets it as a child of the parent node. 

 public final <TC> TC addNode(Class<TC> a);
 public <TC> TC addNode(String id, Class<TC> a);

Get or create node

If you are not sure if the node already exists, you should give a unique id for the node, and check if the node exists in the scene graph (actually you can only check if its a child of the parent node). For this purpose, there is a method called getOrCreateNode that simply checks if the the node exists and returns it if it does. Otherwise the method will create a new node using addNode. 

 public final <TC> TC getOrCreateNode(String id, Class<TC> a);

Remove whole subtree

To remove node and all its children from the scenegraph, a method called remove can be called. This removes node from the parent, and disposes itself and its children. After this method is called, the node and its children are disposed, and thus should not be used anymore. 

  public void remove();

Delete node

Delete method can be used to remove a single node from the scene graph tree, so that its children will still be part of the scene graph. This method basically moves nodes children under its parent, and disposes itself. After this method is called, only the node is disposed. 

 public void delete();

Remove all children

To remove nodes children, you can use removeNodes-method. This method removes and disposes all children of the node (and their children). 

 public final void removeNodes();

INode

org.simantics.scenegraph.INode

Append parent node

To add a node in the middle of the scene graph tree structure, you can use appendParent-method. Basically this method creates a new parent node, adds it as a child of the original parent. Then the node is removed from its parent and added under its new parent. 

 public <TC> TC appendParent(String id, Class<TC> nc);

Questions

How to implement Sysdyn -scenario

In this scenario, diagram consists of nodes and arcs between them. Nodes are text boxes that scale according to the text inside them. Arcs go from the center of one node to another so that it clipped to the edges of the nodes. An arrow is drawn in the head of the arc (at the point where the arc is clipped). The following editing operations are supported

  1. Changing the name of a node
  2. Moving a node. The arcs should be updated during the operation.
  3. Changing the curvature of a arc. This is done by dragging some point on the arc to other point.

See [2]. The operations should be reponsive so an implementation that does not need server for updates during the operations is preferred.

Specific questions:

  • Can nodes refer to each other? For example, can arc be implemented as
public class ArcNode extends G2DNode {
    protected IConnectable tail = null;
    protected IConnectable head = null;
    protected double angle = 0.0;

    @Override
    public void render(Graphics2D g) {
        calculate an arc from tail to head with given angle
        clip the arc to the clip bounds of the tail and the head
        (or check if connectables have been changed and update the arc if necessary)
    }
}

where 

public interface IConnectable {
    double getX();
    double getY();
    Rectangle2D getClipBounds();
}

and 

public class TextNode extends G2DNode implements IConnectable { ... }

...answer

In theory, nodes could refer to each other. However, after serialization the references wouldn't work, thus the references must be id based: 

  public class ArcNode extends G2DNode {
    protected String tailId = null;
    protected String headId = null;
    protected double angle = 0.0;

    @SyncField("tailId")
    public void setTailId(String id) {
        this.tailId = id;
    }
    @SyncField("headId")
    public void setHeadId(String id) {
        this.headId = id;
    }
    @SyncField("angle")
    public void setAngle(double angle) {
        this.angle = angle;
    }
    @Override
    public void render(Graphics2D g) {
        calculate an arc from tail to head with given angle
        clip the arc to the clip bounds of the tail and the head
        (or check if connectables have been changed and update the arc if necessary)
    }
}
  • How derived fields in the nodes are updated when server updates the client-side node?
public class TextNode extends G2DNode implements IConnectable {
    // The properties of the node
    protected double centerX = 0.0;
    protected double centerY = 0.0;
    protected String label = "";

    // Auxiliary values computed in updateDerivedFields
    protected Rectangle2D clipBounds = null;	   
    protected float textX;
    protected float textY;

    private void updateDerivedFields() {
        ...
    }
}

...answer:

The methods in IConnectable should be actually implemented by every G2DNode (Hence, we might combine these interfaces). Notice that you cannot use serialization annotations with private methods. 

public class TextNode extends G2DNode implements IConnectable {
    // The properties of the node
    protected double centerX = 0.0;
    protected double centerY = 0.0;
    protected String label = "";

    // Auxiliary values computed in updateDerivedFields
    protected Rectangle2D clipBounds = null;	   
    protected float textX;
    protected float textY;

    @SyncField({"clipBounds, textX, textY"})
    protected void updateDerivedFields() {
        // Update fields, but create clone of clipBounds to enable comparison between old and new value
    }
}
  • How the operation 3 is implemented in client-side? For example, when an arc node notifies that it is being dragged, it should prevent other nodes from getting any mouse events. It should also have some place to store the state of the current operation so that this kind of transient structures do not have to be stored to the nodes. In the applet, this is implemented as:
   class Mover implements ActionHandler {
       public void handleDrag(double x, double y) {
           angle = Arcs.angleOfArc(tail.getX(), tail.getY(), x, y, head.getX(), head.getY());
       }
       public void handleRelease(double x, double y) {}        
   }

   @Override
   public ActionHandler handlePress(double x, double y) {
       double dx = x-cx;
       double dy = y-cy;
       double dist = dx*dx + dy*dy;
       if(dist < (r+SELECTION_TOLERANCE)*(r+SELECTION_TOLERANCE) &&
           dist > (r-SELECTION_TOLERANCE)*(r-SELECTION_TOLERANCE)) {
           double angle = Arcs.normalizeAngle(Math.atan2(-dy, dx));
           if(Arcs.areClockwiseOrdered(angle0, angle, angle1))
               return new Mover();
       }
       return null;
   }

...answer 

public class ArcNode extends G2DNode {
   protected double angle;
   protected boolean drag = false;

   public void handleEvent(AWTEvent event) {
       if(!(event instanceof MouseEvent)) return;
       MouseEvent me = (MouseEvent)event;
       if(me.getID() == MouseEvent.MOUSE_PRESSED) {
           double dx = me.getPoint().getX()-cx;
           double dy = me.getPoint().getY()-cy;
           double dist = dx*dx + dy*dy;
           if(dist < (r+SELECTION_TOLERANCE)*(r+SELECTION_TOLERANCE) &&
               dist > (r-SELECTION_TOLERANCE)*(r-SELECTION_TOLERANCE)) {
               double angle = Arcs.normalizeAngle(Math.atan2(-dy, dx));
               if(Arcs.areClockwiseOrdered(angle0, angle, angle1))
                   drag = true;
           }
           me.consume();
       } else if(me.getID() == MouseEvent.MOUSE_DRAGGED && drag == true) {
           G2DNode tail = findNode(tailId); // NOTE: findMode not implemented yet
           G2DNode head = findNode(headId);
           angle = Arcs.angleOfArc(tail.getX(), tail.getY(), me.getPoint().getX(), me.getPoint().getY(), head.getX(), head.getY());
           me.consume();
       } else if(me.getID() == MouseEvent.MOUSE_RELEASED) {
           drag = false;
       }
   }
}

Development Ideas

Links

Current Development

See also

Contact

juha-pekka.laine@semantum.fi

Retrieved from "https://dev.simantics.org/index.php?title=Org.simantics.scenegraph&oldid=1168"