Difference between revisions of "Org.simantics.scenegraph"

From Developer Documents
Jump to navigation Jump to search
 
(75 intermediate revisions by 3 users not shown)
Line 17: Line 17:
 
To enable scene graph datastructure synchronization from a server to a client, a mechanism for detecting changes in the datastructure is required. For this purpose, all changes to Node variables must be made through a public method (usually setter). These methods that change local variables are marked with annotations, thus the solution is very similar to what [http://www.hibernate.org/ Hibernate] uses with entities.
 
To enable scene graph datastructure synchronization from a server to a client, a mechanism for detecting changes in the datastructure is required. For this purpose, all changes to Node variables must be made through a public method (usually setter). These methods that change local variables are marked with annotations, thus the solution is very similar to what [http://www.hibernate.org/ Hibernate] uses with entities.
  
Node methods can have three different annotations:
+
Node methods can have four different annotations, three for using scene graph remotely, and one for synchronizing node variables automatically from the graph:
 
* '''SyncField(..)''' When a method annotated with this is called, the fields listed in the annotation are updated to the client side (if running in remote mode)
 
* '''SyncField(..)''' When a method annotated with this is called, the fields listed in the annotation are updated to the client side (if running in remote mode)
 
* '''ServerSide''' Methods with ServerSide annotation are intended to be executed on server-side, thus it can '''access resources outside the scenegraph datastructure''' (using interfaces)
 
* '''ServerSide''' Methods with ServerSide annotation are intended to be executed on server-side, thus it can '''access resources outside the scenegraph datastructure''' (using interfaces)
 
* '''ClientSide''' Methods with this annotation are executed on client-side, thus it should only '''access resources inside the scenegraph datastructure'''
 
* '''ClientSide''' Methods with this annotation are executed on client-side, thus it should only '''access resources inside the scenegraph datastructure'''
 +
* '''PropertySetter''' This annotation is used to map graph properties to node automatically. This annotation can only be used in setter methods, that take one argument. Notice that the argument type must match with the object that is read from the graph (i.e., for double values you must use Double type instead of double, float, or Float).
  
 
Without these annotations, the Node will not work properly when using scenegraph remotely.
 
Without these annotations, the Node will not work properly when using scenegraph remotely.
  
These annotations are handled with cglib MethodInterceptor which is a proxyclass for the actual class implementation, and contains an interceptor for performing actions before and after the actual method is called. For this reason, the Node classes should not be instantiated directly, but through ParentNode.addNode method.
+
These annotations (except PropertySetter) are handled with cglib MethodInterceptor which is a proxyclass for the actual class implementation, and contains an interceptor for performing actions before and after the actual method is called. For this reason, the Node classes should not be instantiated directly, but through <code>ParentNode.addNode</code> method. Cglib support is implemented in org.simantics.scenegraph.remote plugin, and thus org.simantics.scenegraph plugin does not depend on cglib.
 +
 
 +
{{tip|Scene graph sub-nodes must not be instantiated directly, but through <code>ParentNode.addNode</code> or <code>ParentNode.getOrCreate</code> methods. Only the root node may be instantiated directly.}}
  
 
ParentNode and Node classes implement only functionalities for creating and modifying the scene graph datastructure but do not contain any methods for rendering.  
 
ParentNode and Node classes implement only functionalities for creating and modifying the scene graph datastructure but do not contain any methods for rendering.  
Line 33: Line 36:
 
* '''G2DNode''' is a baseclass for all leaf nodes that perform 2D rendering  
 
* '''G2DNode''' is a baseclass for all leaf nodes that perform 2D rendering  
 
* '''G2DParentNode''' is a baseclass for the 2D compatible parent nodes
 
* '''G2DParentNode''' is a baseclass for the 2D compatible parent nodes
* '''G2DPaintable''' is a common interface for G2DNode and G2DParentNode which should be used when traversing through the data structure. (FIXME: get rid of this and inherit G2DParentNode from G2DNode)
+
* '''IG2DNode''' is a common interface for G2DNode and G2DParentNode which should be used when traversing through the data structure.
 
* '''G2DSceneGraph''' G2DParentNode implementation containing event listeners and some extra features. This should always be the root component of a G2D scene graph datastructure.
 
* '''G2DSceneGraph''' G2DParentNode implementation containing event listeners and some extra features. This should always be the root component of a G2D scene graph datastructure.
  
Line 54: Line 57:
 
</pre>
 
</pre>
  
Because org.simantics.scenegraph is an independent scene graph datastructure, it can be rendered as such. Rendering can be performed by calling ''G2DSceneGraph.render(Graphics2D g2d)'' method. No changes are allowed to the scene graph while the structure is rendered (and rendering methods should not make any changes to the structure either, excluding transient node fields that are not synchronized between server and client).
+
Because ''org.simantics.scenegraph'' is an independent scene graph datastructure, it can be rendered as such. Rendering can be performed by calling ''G2DSceneGraph.render(Graphics2D g2d)'' method. No changes are allowed to the scene graph while the structure is rendered (and rendering methods should not make any changes to the structure either, excluding transient node fields that are not synchronized between server and client).
  
 
=== 3D ===
 
=== 3D ===
  
JOGL compatible implementation of the scenegraph can be found under package ''org.simantics.scenegraph.g3d.jogl''
+
JOGL compatible implementation of the scenegraph can be found under package ''org.simantics.scenegraph.g3d.jogl'' in a separate plug-in [[svn:2d/trunk/org.simantics.scenegraph.g3d|org.simantics.scenegraph.g3d]].
 
* '''JOGLNode'''  
 
* '''JOGLNode'''  
 
* '''JOGLParentNode'''
 
* '''JOGLParentNode'''
Line 68: Line 71:
 
=== Serialization ===
 
=== Serialization ===
  
For serializing the scene graph datastructure there is class called [[svn:2d/trunk/org.simantics.scenegraph/src/org/simantics/scenegraph/serializer/Serializer.java|''org.simantics.scenegraph.serializer.Serializer'']]. This Serializer can be attached to an existing ParentNode if you want to serialize your local scene graph, or you can call it without ParentNode when the Serializer will act as a client.
+
For serializing the scene graph datastructure there is class called [[svn:2d/trunk/org.simantics.scenegraph.remote/src/org/simantics/scenegraph/remote/Serializer.java|''org.simantics.scenegraph.remote.Serializer'']]. The Serializer simply listens for any changes in the scene graph, and translates those changes into events which are then serialized and sent to ObjectOutputStream. To enable Serializer to listen for changes in the scene graph, the scene graph root node must be initialized with cglib. For this purpose there is a static method in Serializer:
 +
 
 +
  G2DSceneGraph root = Serializer.newRemoteSceneGraphNode(G2DSceneGraph.class);
 +
  Serializer serializer = new Serializer(root, input, output);
 +
  serializer.start();
 +
 
 +
After this, the root node can be used normally. Any child added to it will be created by cglib automatically, hence Serializer works automatically with the whole scene graph data structure.
 +
 
 +
On the client side, Serializer can be directly attached to object streams, and it will create G2DSceneGraph object for you:
 +
 
 +
  Serializer serializer = new Serializer(input, output);
 +
  serializer.start();
 +
  G2DSceneGraph root = (G2DSceneGraph)serializer.getRoot();
 +
 
 +
Because the system works asynchronously, you should actually attach UpdateListener to serializer in real case.  
 +
 
 
Notice that you have to attach the serializer to the scene graph before any other operation. Serializer is an instance of ''PropertyChangeListener'' which will be set to listen the changes in the scene graph datastructure. If there are changes made before the serializer is attached, these changes will not be serialized.
 
Notice that you have to attach the serializer to the scene graph before any other operation. Serializer is an instance of ''PropertyChangeListener'' which will be set to listen the changes in the scene graph datastructure. If there are changes made before the serializer is attached, these changes will not be serialized.
  
Line 128: Line 146:
 
             e.setHint(ElementHints.KEY_SG_NODE, node);
 
             e.setHint(ElementHints.KEY_SG_NODE, node);
 
         }
 
         }
         e.setShape(new Rectangle2D.Double(1, 1, 20, 20));
+
         node.setShape(new Rectangle2D.Double(1, 1, 20, 20));
 
     }
 
     }
 
   
 
   
Line 141: Line 159:
 
== org.simantics.scenegraph features ==
 
== org.simantics.scenegraph features ==
  
[[svn:2d/trunk/org.simantics.scenegraph/src/org/simantics/scenegraph|org.simantics.scenegraph]] package contains lots of readymade nodes and features. For a list of g2d nodes, see [[svn:2d/trunk/org.simantics.scenegraph/src/org/simantics/scenegraph/g2d/nodes|org.simantics.scenegraph.g2d.nodes]].
+
[[svn:2d/trunk/org.simantics.scenegraph/src/org/simantics/scenegraph|org.simantics.scenegraph]] package contains lots of readymade nodes and features. For a list of g2d nodes, see [[Scene graph node list]].
  
 
===Swing support===
 
===Swing support===
  
It is possible to integrate Swing components to 2D scenegraph by using [[svn:2d/trunk/org.simantics.scenegraph/src/org/simantics/scenegraph/g2d/nodes/ComponentNode.java|org.simantics.scenegraph.g2d.nodes.ComponentNode]] class. See for example MonitorNode or Trend2DNode for an example.
+
It is possible to integrate Swing components to 2D scenegraph by using [[svn:2d/trunk/org.simantics.scenegraph/src/org/simantics/scenegraph/g2d/nodes/swing/ComponentNode.java|org.simantics.scenegraph.g2d.nodes.swing.ComponentNode]] class. See for example MonitorNode or Trend2DNode for an example.
  
 
===Eclipse view===
 
===Eclipse view===
Line 157: Line 175:
 
=== Node Bounds ===
 
=== Node Bounds ===
  
In order to perform any kind of rendering optimization in the future some kind of spatial subdivision methods need to be employed. For example Trolltech QT GraphicsView [http://doc.trolltech.com/4.5/graphicsview.html] implements BSP-based partitioning to accelerate interaction and rendering in its 2D systems. Bounds information is also useful for picking facilitation.
+
Scene graph nodes may either have specific spatial boundaries or have infinite boundaries. Nodes with infinite boundaries are omnipresent, i.e. should always be rendered. Node boundaries are necessary for doing things like spatial subdivision or picking.
 
 
Currently scene graph nodes do not have any kind of bounds information, but the ideas to implement bounds checking will be written under this section.
 
  
 
==== Implementation ====
 
==== Implementation ====
* Each node has a method ''getBoundsInLocal()'' which returns a Rectangle2D instance describing the 2D boundaries of the node in the element's local coordinate system.
+
* Each node has a method ''getBoundsInLocal()'' which returns a Rectangle2D instance describing the 2D boundaries of the node in the element's local coordinate system. A null return value is interpreted as having infinite boundaries (omnipresence).
 
* Node base implementation classes have a method ''getBounds()'' that returns the node boundaries with local transforms applied.
 
* Node base implementation classes have a method ''getBounds()'' that returns the node boundaries with local transforms applied.
 
* Nodes that have children (i.e., a parent node that does not draw anything) will return union of its childrens bounds.
 
* Nodes that have children (i.e., a parent node that does not draw anything) will return union of its childrens bounds.
* G2DSceneGraph root node will cache bounds information.
 
 
* Bounds will need caching in order to be efficiently implemented in parent nodes. One option is to implement a mechanism for propagating transformation changes up in the node tree to keep parent nodes aware of transformation changes in their children. Possily implement a listener in parent nodes that can be registered into any transformable node.
 
* Bounds will need caching in order to be efficiently implemented in parent nodes. One option is to implement a mechanism for propagating transformation changes up in the node tree to keep parent nodes aware of transformation changes in their children. Possily implement a listener in parent nodes that can be registered into any transformable node.
  
== Questions ==
+
=== Spatial Subdivision ===
 +
 
 +
In order to perform feasible visibility optimization during rendering some kind of spatial subdivision methods need to be employed. For example Trolltech QT GraphicsView [http://doc.trolltech.com/4.5/graphicsview.html] implements BSP-based partitioning to accelerate interaction and rendering in its 2D systems. Node boundaries are an essential requirement to performing this subdivision.
 +
 
 +
==== Implementation ====
 +
* [[svn:2d/trunk/org.simantics.scenegraph/src/org/simantics/scenegraph/g2d/nodes/spatial|org.simantics.g2d.nodes.spatial]] contains [[svn:2d/trunk/org.simantics.scenegraph/src/org/simantics/scenegraph/g2d/nodes/spatial/RTreeNode.java|RTreeNode]] that implements rectangular boundary -based subdivision of all of its direct child nodes.
 +
* Rendering an ''RTreeNode'' includes performing a lookup into the R-Tree structure using the current clipping viewport. As a result, we will get the set of direct children that are visible within that viewport and therefore optimize the rendering process by quickly pruning what needs to be processed.
 +
 
 +
=== Node &hArr; ID mapping ===
 +
 
 +
;([https://www.simantics.org/redmine/issues/871 ticket:871])
 +
 
 +
In DOM implementations the client has the possibility to search for document nodes by their ID (getElementById(String)). Similarly scene graph nodes can be given identifications based on which they can be looked up from the scene graph through [[svn:2d/trunk/org.simantics.scenegraph/src/org/simantics/scenegraph/ILookupService.java|ILookupService]] which is implemented by scene graph root node [[svn:2d/trunk/org.simantics.scenegraph/src/org/simantics/scenegraph/g2d/G2DSceneGraph.java|G2DSceneGraph]].
 +
 
 +
Normally the scene graph is a tree of nodes. Giving nodes scene graph -locally unique ID's through the lookup mechanism makes it possible to create links between nodes. The links effectively make the scene graph more like a DAG ([[wikipedia:Directed_acyclic_graph|Directed Acyclic Graph]]) or even a DG ([[wikipedia:Dependency graph|Dependency Graph]]). Just as in DOM, the ''id'' attribute is not mandatory. Node lookup will only be possible if an ID is provided.
 +
 
 +
The lookup facility is available at the root node of the scene graph. Considering performance, having to always traverse to the root node to make lookups is a very likely bottleneck. This problem has been alleviated by using transient root node caches in [[svn:2d/trunk/org.simantics.scenegraph/src/org/simantics/scenegraph/ParentNode.java|ParentNode]]. This will naturally make the scene graph a bit more space consuming.
 +
 
 +
==== Using LookupService ====
 +
 
 +
; Giving a lookup ID to a node:
 +
<pre>
 +
G2DSceneGraph root = new G2DSceneGraph();
 +
 
 +
// Create new nodes
 +
Node child1 = root.addNode("node child 1", G2DParentNode.class);
 +
Node child2 = root.addNode("node child 2", G2DParentNode.class);
 +
 
 +
// Give a lookup ID
 +
child2.setLookupId("child 2 lookup id");
 +
// OR
 +
NodeUtil.map(child2, "child 2 lookup id");
 +
</pre>
 +
 
 +
; Performing a lookup:
 +
<pre>
 +
// ID -> INode
 +
INode lookedUpNode = NodeUtil.lookupNode(child1, "child 2 lookup id");
 +
assert lookedUpNode == child2;
 +
//OR
 +
lookedUpNode = root.lookupNode("child 2 lookup id");
 +
assert lookedUpNode == child2;
 +
 
 +
// INode -> ID
 +
String lookedUpId = NodeUtil.lookupId(child2);
 +
assert lookedUpId.equals("child 2 lookup id");
 +
 
 +
// See NodeUtil for more utility methods for lookups and associations
 +
</pre>
 +
 
 +
== Main interfaces ==
 +
 
 +
=== ParentNode ===
 +
 
 +
[[svn:2d/trunk/org.simantics.scenegraph/src/org/simantics/scenegraph/ParentNode.java|org.simantics.scenegraph.ParentNode]]
 +
 
 +
==== Add child node ====
 +
 
 +
Add node simply creates a new node object with the given class and sets it as a child of the parent node.
 +
 
 +
  public final <TC> TC addNode(Class<TC> a);
 +
  public <TC> TC addNode(String id, Class<TC> a);
 +
 
 +
==== Get or create node ====
 +
 
 +
If you are not sure if the node already exists, you should give a unique id for the node, and check if the node exists in the scene graph (actually you can only check if its a child of the parent node). For this purpose, there is a method called getOrCreateNode that simply checks if the the node exists and returns it if it does. Otherwise the method will create a new node using addNode.
 +
 
 +
  public final <TC> TC getOrCreateNode(String id, Class<TC> a);
 +
 
 +
==== Remove whole subtree ====
 +
 
 +
To remove node and all its children from the scenegraph, a method called remove can be called. This removes node from the parent, and disposes itself and its children. After this method is called, the node and its children are disposed, and thus should not be used anymore.
 +
 
 +
  public void remove();
 +
 
 +
==== Delete node====
 +
 
 +
Delete method can be used to remove a single node from the scene graph tree, so that its children will still be part of the scene graph. This method basically moves nodes children under its parent, and disposes itself. After this method is called, only the node is disposed.
 +
 
 +
  public void delete();
 +
 
 +
==== Remove all children ====
 +
 
 +
To remove nodes children, you can use removeNodes-method. This method removes and disposes all children of the node (and their children).
 +
 
 +
  public final void removeNodes();
  
=== How to implement Sysdyn -scenario ===
+
=== INode ===
  
In this scenario, diagram consists of nodes and arcs between them. Nodes are text boxes that scale according to the text inside them. Arcs go from the center of one node to another so that it clipped to the edges of the nodes. An arrow is drawn in the head of the arc (at the point where the arc is clipped). The following editing operations are supported
+
[[svn:2d/trunk/org.simantics.scenegraph/src/org/simantics/scenegraph/INode.java|org.simantics.scenegraph.INode]]
# Changing the name of a node
 
# Moving a node. The arcs should be updated during the operation.
 
# Changing the curvature of a arc. This is done by dragging some point on the arc to other point.
 
See [http://www.simulationsite.net/~niemisto/sysdyn/]. The operations should be reponsive so an implementation that does not need server for updates during the operations is preferred.
 
  
Specific questions:
+
==== Append parent node ====
* Can nodes refer to each other? For example, can arc be implemented as
+
 
public class ArcNode extends G2DNode {
+
To add a node in the middle of the scene graph tree structure, you can use appendParent-method. Basically this method creates a new parent node, adds it as a child of the original parent. Then the node is removed from its parent and added under its new parent.
    protected IConnectable tail = null;
+
 
    protected IConnectable head = null;
+
  public <TC> TC appendParent(String id, Class<TC> nc);
    protected double angle = 0.0;
 
 
    @Override
 
    public void render(Graphics2D g) {
 
        calculate an arc from tail to head with given angle
 
        clip the arc to the clip bounds of the tail and the head
 
        (or check if connectables have been changed and update the arc if necessary)
 
    }
 
}
 
where
 
public interface IConnectable {
 
    double getX();
 
    double getY();
 
    Rectangle2D getClipBounds();
 
}
 
and
 
public class TextNode extends G2DNode implements IConnectable { ... }
 
...answer
 
  In theory, nodes could refer to each other. However, after serialization the references wouldn't work, thus the references must be id based:
 
 
  public class ArcNode extends G2DNode {
 
    protected String tailId = null;
 
    protected String headId = null;
 
    protected double angle = 0.0;
 
 
    @SyncField("tailId")
 
    public void setTailId(String id) {
 
        this.tailId = id;
 
    }
 
    @SyncField("headId")
 
    public void setHeadId(String id) {
 
        this.headId = id;
 
    }
 
    @SyncField("angle")
 
    public void setAngle(double angle) {
 
        this.angle = angle;
 
    }
 
    @Override
 
    public void render(Graphics2D g) {
 
        calculate an arc from tail to head with given angle
 
        clip the arc to the clip bounds of the tail and the head
 
        (or check if connectables have been changed and update the arc if necessary)
 
    }
 
}
 
  
* How derived fields in the nodes are updated when server updates the client-side node?
+
== Scenarios ==
public class TextNode extends G2DNode implements IConnectable {
 
    // The properties of the node
 
    protected double centerX = 0.0;
 
    protected double centerY = 0.0;
 
    protected String label = "";
 
 
    // Auxiliary values computed in updateDerivedFields
 
    protected Rectangle2D clipBounds = null;  
 
    protected float textX;
 
    protected float textY;
 
 
    private void updateDerivedFields() {
 
        ...
 
    }
 
}
 
...answer:
 
The methods in IConnectable should be actually implemented by every G2DNode (Hence, we might combine these interfaces). Notice that you cannot use serialization annotations with private methods.
 
public class TextNode extends G2DNode implements IConnectable {
 
    // The properties of the node
 
    protected double centerX = 0.0;
 
    protected double centerY = 0.0;
 
    protected String label = "";
 
 
    // Auxiliary values computed in updateDerivedFields
 
    protected Rectangle2D clipBounds = null;  
 
    protected float textX;
 
    protected float textY;
 
 
    @SyncField({"clipBounds, textX, textY"})
 
    protected void updateDerivedFields() {
 
        // Update fields, but create clone of clipBounds to enable comparison between old and new value
 
    }
 
}
 
  
* How the operation 3 is implemented in client-side? For example, when an arc node notifies that it is being dragged, it should prevent other nodes from getting any mouse events. It should also have some place to store the state of the current operation so that this kind of transient structures do not have to be stored to the nodes. In the applet, this is implemented as:
+
* [[Scene Graph: Sysdyn-scenario]]
    class Mover implements ActionHandler {
 
        public void handleDrag(double x, double y) {
 
            angle = Arcs.angleOfArc(tail.getX(), tail.getY(), x, y, head.getX(), head.getY());
 
        }
 
        public void handleRelease(double x, double y) {}       
 
    }
 
 
    @Override
 
    public ActionHandler handlePress(double x, double y) {
 
        double dx = x-cx;
 
        double dy = y-cy;
 
        double dist = dx*dx + dy*dy;
 
        if(dist < (r+SELECTION_TOLERANCE)*(r+SELECTION_TOLERANCE) &&
 
            dist > (r-SELECTION_TOLERANCE)*(r-SELECTION_TOLERANCE)) {
 
            double angle = Arcs.normalizeAngle(Math.atan2(-dy, dx));
 
            if(Arcs.areClockwiseOrdered(angle0, angle, angle1))
 
                return new Mover();
 
        }
 
        return null;
 
    }
 
...answer
 
public class ArcNode extends G2DNode {
 
    protected double angle;
 
    protected boolean drag = false;
 
 
    public void handleEvent(AWTEvent event) {
 
        if(!(event instanceof MouseEvent)) return;
 
        MouseEvent me = (MouseEvent)event;
 
        if(me.getID() == MouseEvent.MOUSE_PRESSED) {
 
            double dx = me.getPoint().getX()-cx;
 
            double dy = me.getPoint().getY()-cy;
 
            double dist = dx*dx + dy*dy;
 
            if(dist < (r+SELECTION_TOLERANCE)*(r+SELECTION_TOLERANCE) &&
 
                dist > (r-SELECTION_TOLERANCE)*(r-SELECTION_TOLERANCE)) {
 
                double angle = Arcs.normalizeAngle(Math.atan2(-dy, dx));
 
                if(Arcs.areClockwiseOrdered(angle0, angle, angle1))
 
                    drag = true;
 
            }
 
            me.consume();
 
        } else if(me.getID() == MouseEvent.MOUSE_DRAGGED && drag == true) {
 
            G2DNode tail = findNode(tailId); // NOTE: findMode not implemented yet
 
            G2DNode head = findNode(headId);
 
            angle = Arcs.angleOfArc(tail.getX(), tail.getY(), me.getPoint().getX(), me.getPoint().getY(), head.getX(), head.getY());
 
            me.consume();
 
        } else if(me.getID() == MouseEvent.MOUSE_RELEASED) {
 
            drag = false;
 
        }
 
    }
 
}
 
  
== Development Ideas ==
+
=== Implementing a 2D node ===
=== Shared Context ===
+
* TODO: implementing your own G2DNode
A scene graph in itself forms a kind of context. It would sometimes be useful to be able to store context local data accessible to any node within that graph, e.g. for having different kinds of caches in the graph.
+
** render, assumptions
 +
** getBoundsInLocal
  
One way to implement this would be to add a data map to the scene graph root node (SceneGraphNode) which can be made available to all nodes by traversing to root.
+
=== Hooking your node to UI events ===
  
=== Template Nodes ===
+
By default, 2D nodes do not receive UI events. To receive events, do the following:
It is very common in rendering of graphical scenes that a single type of graphical entity is rendered multiple times with different parametrisations, i.e. instantiated.
+
# Override <em>int getEventMask()</em>, use constants from EventTypes.
 +
# Override <em>init()</em>, add <em>addEventHandler(this)</em>
 +
# Override <em>cleanup()</em>, add<em>removeEventhandler(this)</em>
 +
# Override any of the following methods in <em>G2DNode</em> or <em>G2DParentNode</em>
 +
#* handleEvent
 +
#* handleKeyEvent, keyReleased, keyPressed
 +
#* handleCommand
 +
#* handleFocusEvent
 +
#* mouseButtonPressed, mouseButtonReleased, mouseClicked, mouseDoubleClicked
 +
#* mouseMoved, mouseDragged
 +
#* mouseEntered, mouseExited
 +
#* mouseWheelMoved
 +
#* handleTimeEvent
  
For the scene graph this would mean that it needs to be able to store the subgraph of an instantiated element and allow it to be referenced from the element instance graph locations. Also a mechanism for separating static data and instance data would be needed.
+
All event handler methods return boolean values where a <code>true</code> return value indicates that the handler consumed the event and it should not be propagated further.
  
==== IDEA: getNodeById(String) ====
+
=== Picking ===
Just as in DOM implementations the client has the possibility to search for document nodes by their ID, linking between scene graph nodes could be achieved through the use of scene graph -locally unique ID's. This kind of lookup facility could again be made available in the scene graph root node.
+
* TODO: picking
  
=== Basic scene graph modification operations ===
+
=== Coordinate system transformations ===
 +
* TODO: global <-> local coordinate system transformations
  
* Add child node
+
=== Mouse capture/grabbing ===
* Get or create node (Either get child by id, or add new child and return it)
+
* TODO: mouse capture
* Delete subtree (Remove node and its children)
 
* Delete node (Delete node from tree tree and move its children to its parent)
 
* Remove all children
 
* Append parent node (Append new node between specific child and parent)
 
  
 
== Links ==
 
== Links ==
 
* [[org.simantics.g2d]]
 
* [[org.simantics.g2d]]
 
* [[org.simantics.diagram]]
 
* [[org.simantics.diagram]]
 +
* [[org.simantics.scenegraph.ui]]
 
* [http://cglib.sourceforge.net cglib]
 
* [http://cglib.sourceforge.net cglib]
 
* [https://svgsalamander.dev.java.net SVG Salamander]
 
* [https://svgsalamander.dev.java.net SVG Salamander]
  
 
== Current Development ==
 
== Current Development ==
* [https://www.simulationsite.net/trac/simantics/query?group=milestone&status=assigned&status=new&status=reopened&col=id&col=summary&col=status&col=type&col=priority&col=milestone&component=org.simantics.scenegraph&order=priority Open Tickets]
+
* [https://www.simantics.org/redmine/projects/simantics/issues?query_id=7 Open Tickets]
* [https://www.simulationsite.net/trac/simantics/query?group=milestone&status=closed&col=id&col=summary&col=status&col=type&col=priority&col=milestone&component=org.simantics.scenegraph&order=priority Closed Tickets]
 
  
== See also ==
+
==Contact==
 
 
* [[TP:Animated Symbols]]
 
  
==Contact==
+
* [[User:Tuukka Lehtonen]]
  
juha-pekka.laine@semantum.fi
+
[[Category: Diagram Development]]

Latest revision as of 22:52, 13 October 2011

org.simantics.scenegraph (SVN) is a scene graph library for 2D rendering with a focus on supporting remote rendering using a client-server architecture.

Overview

org.simantics.scenegraph provides a generic serializable scene graph datastructure ( wikipedia:Scene graph ) for displaying 2D and 3D graphics.

The idea of scene graph is that it is an independent datastructure that can be rendered as such. It can only access local variables, thus it provides a strict interface between the visualization and program logic. Mainly for this reason scene graph is an ideal datastructure for rendering graphics remotely.

Architecture

org.simantics.scenegraph is designed to be used for rendering graphics locally on desktop application, but also remotely from server to client. For this reason, it has some features that a normal scene graph implementation doesn't have.

In org.simantics.scenegraph, the scene graph datastructure is a tree consisting of ParentNodes and Nodes.

  • Node is a leaf in the tree, thus it cannot have children, but it should be used to render visible objects.
  • ParentNode is used to group a set of nodes (children) and to give some common parameters for the children.

To enable scene graph datastructure synchronization from a server to a client, a mechanism for detecting changes in the datastructure is required. For this purpose, all changes to Node variables must be made through a public method (usually setter). These methods that change local variables are marked with annotations, thus the solution is very similar to what Hibernate uses with entities.

Node methods can have four different annotations, three for using scene graph remotely, and one for synchronizing node variables automatically from the graph:

  • SyncField(..) When a method annotated with this is called, the fields listed in the annotation are updated to the client side (if running in remote mode)
  • ServerSide Methods with ServerSide annotation are intended to be executed on server-side, thus it can access resources outside the scenegraph datastructure (using interfaces)
  • ClientSide Methods with this annotation are executed on client-side, thus it should only access resources inside the scenegraph datastructure
  • PropertySetter This annotation is used to map graph properties to node automatically. This annotation can only be used in setter methods, that take one argument. Notice that the argument type must match with the object that is read from the graph (i.e., for double values you must use Double type instead of double, float, or Float).

Without these annotations, the Node will not work properly when using scenegraph remotely.

These annotations (except PropertySetter) are handled with cglib MethodInterceptor which is a proxyclass for the actual class implementation, and contains an interceptor for performing actions before and after the actual method is called. For this reason, the Node classes should not be instantiated directly, but through ParentNode.addNode method. Cglib support is implemented in org.simantics.scenegraph.remote plugin, and thus org.simantics.scenegraph plugin does not depend on cglib.

Tip.png Scene graph sub-nodes must not be instantiated directly, but through ParentNode.addNode or ParentNode.getOrCreate methods. Only the root node may be instantiated directly.

ParentNode and Node classes implement only functionalities for creating and modifying the scene graph datastructure but do not contain any methods for rendering.

2D

Java 2D API compatible implementation of the scene graph is in org.simantics.scenegraph.g2d package:

  • G2DNode is a baseclass for all leaf nodes that perform 2D rendering
  • G2DParentNode is a baseclass for the 2D compatible parent nodes
  • IG2DNode is a common interface for G2DNode and G2DParentNode which should be used when traversing through the data structure.
  • G2DSceneGraph G2DParentNode implementation containing event listeners and some extra features. This should always be the root component of a G2D scene graph datastructure.

Example G2DNode:

public class BackgroundNode extends G2DNode {
    private static final long serialVersionUID = -1L;	
    protected Rectangle2D bounds = null;
	
    @SyncField("bounds")
    public void setBounds(Rectangle2D bounds) {
        this.bounds = bounds;
    }
	
    @Override
    public void render(Graphics2D g) {
        g.fill(bounds);
    }
}

Because org.simantics.scenegraph is an independent scene graph datastructure, it can be rendered as such. Rendering can be performed by calling G2DSceneGraph.render(Graphics2D g2d) method. No changes are allowed to the scene graph while the structure is rendered (and rendering methods should not make any changes to the structure either, excluding transient node fields that are not synchronized between server and client).

3D

JOGL compatible implementation of the scenegraph can be found under package org.simantics.scenegraph.g3d.jogl in a separate plug-in org.simantics.scenegraph.g3d.

  • JOGLNode
  • JOGLParentNode
  • JOGLPaintable
  • JOGLSceneGraph

See org.simantics.scenegraph.g3d.eclipse.JOGLLocalView for an example how to use org.simantics.scenegraph with OpenGL.

Serialization

For serializing the scene graph datastructure there is class called org.simantics.scenegraph.remote.Serializer. The Serializer simply listens for any changes in the scene graph, and translates those changes into events which are then serialized and sent to ObjectOutputStream. To enable Serializer to listen for changes in the scene graph, the scene graph root node must be initialized with cglib. For this purpose there is a static method in Serializer:

 G2DSceneGraph root = Serializer.newRemoteSceneGraphNode(G2DSceneGraph.class);
 Serializer serializer = new Serializer(root, input, output);
 serializer.start();

After this, the root node can be used normally. Any child added to it will be created by cglib automatically, hence Serializer works automatically with the whole scene graph data structure.

On the client side, Serializer can be directly attached to object streams, and it will create G2DSceneGraph object for you:

 Serializer serializer = new Serializer(input, output);
 serializer.start();
 G2DSceneGraph root = (G2DSceneGraph)serializer.getRoot();

Because the system works asynchronously, you should actually attach UpdateListener to serializer in real case.

Notice that you have to attach the serializer to the scene graph before any other operation. Serializer is an instance of PropertyChangeListener which will be set to listen the changes in the scene graph datastructure. If there are changes made before the serializer is attached, these changes will not be serialized.

Serializer uses ObjectInput and ObjectOutput to serialize and unserialize the scene graph, thus it can be used to write the scenegraph to a file and from a file, or to synchronize the scenegraph in a client-server system.

Using scenegraph

  • Create scenegraph
G2DSceneGraph sg = new G2DSceneGraph();
sg.setRootPane(instanceOfJComponent); // In case you want to use swing components, you must set rootPane.
// Register event handlers
addKeyListener(sg);
addMouseWheelListener(sg);
addMouseListener(sg);
addMouseMotionListener(sg);
addFocusListener(sg);
  • Create node
ShapeNode shape = sg.addNode(ShapeNode.class);
shape.setShape(new Rectangle2D.Double(10, 10, 100, 100));
shape.setColor(Color.RED);
  • Render
sg.render(g2d);
  • Move shape
shape.setTransform(AffineTransform.getTranslateInstance(5, 5));
  • Updating node from stateless object
ShapeNode shape = sg.getOrCreateNode("bounds", ShapeNode.class);
shape.setShape(new Rectangle2D.Double(10, 10, 100, 100));
shape.setColor(Color.RED);

Scenegraph in Simantics

  • Each canvas participant that wants to draw something should implement methods annotated with @SGInit and @SGCleanup
public class Participant extends AbstractCanvasParticipant {
   protected ShapeNode node = null;

   @SGInit
   public void initSG(G2DParentNode parent) {
       node = parent.addNode(ShapeNode.class);
       node.setZIndex(PAINT_PRIORITY);
       node.setShape(new Rectangle2D.Double(1, 1, 20, 20));
   }

   @SGCleanup
   public void cleanupSG() {
       node.remove();
       node = null;
   }
}
  • Optionally, you can specify a designation for @SGInit that specifies how you want your nodes to be transformed, i.e. in control space or in canvas space. The default value is canvas space. Use the scene graph viewer to understand how this affects your scene graph.
@SGInit(designation = SGDesignation.CONTROL)
  • Each diagram element can implement interface SceneGraph to give their own contribution to the scene graph.
public class ElementSGNode implements SceneGraph {
   public static final Key SG_NODE = new SceneGraphNodeKey(Node.class, "SG_NODE");

   @Override
   public void init(final IElement e, G2DParentNode parent) {
       ShapeNode node = (ShapeNode) e.getHint(SG_NODE);
       if (node == null) {
           node = parent.addNode(ShapeNode.class);
           e.setHint(ElementHints.KEY_SG_NODE, node);
       }
       node.setShape(new Rectangle2D.Double(1, 1, 20, 20));
   }

   @Override
   public void cleanup(IElement e) {
       Node node = e.removeHint(SG_NODE);
       if (node != null)
           node.remove();
   }
}

org.simantics.scenegraph features

org.simantics.scenegraph package contains lots of readymade nodes and features. For a list of g2d nodes, see Scene graph node list.

Swing support

It is possible to integrate Swing components to 2D scenegraph by using org.simantics.scenegraph.g2d.nodes.swing.ComponentNode class. See for example MonitorNode or Trend2DNode for an example.

Eclipse view

org.simantics.scenegraph.eclipse package contains local and remote view of 2D and 3D scene graphs. You can try out these demos by executing the project as Eclipse Application and choosing the views "SceneGraph Remote View", "SceneGraph Local View", "JOGL SceneGraph Local View", "JOGL SceneGraph Remote View".

2D-3D integration

It is possible to create a G2DParentNode that works as a wrapper between 2D and 3D scenegraphs. This is not implemented yet.

Node Bounds

Scene graph nodes may either have specific spatial boundaries or have infinite boundaries. Nodes with infinite boundaries are omnipresent, i.e. should always be rendered. Node boundaries are necessary for doing things like spatial subdivision or picking.

Implementation

  • Each node has a method getBoundsInLocal() which returns a Rectangle2D instance describing the 2D boundaries of the node in the element's local coordinate system. A null return value is interpreted as having infinite boundaries (omnipresence).
  • Node base implementation classes have a method getBounds() that returns the node boundaries with local transforms applied.
  • Nodes that have children (i.e., a parent node that does not draw anything) will return union of its childrens bounds.
  • Bounds will need caching in order to be efficiently implemented in parent nodes. One option is to implement a mechanism for propagating transformation changes up in the node tree to keep parent nodes aware of transformation changes in their children. Possily implement a listener in parent nodes that can be registered into any transformable node.

Spatial Subdivision

In order to perform feasible visibility optimization during rendering some kind of spatial subdivision methods need to be employed. For example Trolltech QT GraphicsView [1] implements BSP-based partitioning to accelerate interaction and rendering in its 2D systems. Node boundaries are an essential requirement to performing this subdivision.

Implementation

  • org.simantics.g2d.nodes.spatial contains RTreeNode that implements rectangular boundary -based subdivision of all of its direct child nodes.
  • Rendering an RTreeNode includes performing a lookup into the R-Tree structure using the current clipping viewport. As a result, we will get the set of direct children that are visible within that viewport and therefore optimize the rendering process by quickly pruning what needs to be processed.

Node ⇔ ID mapping

(ticket:871)

In DOM implementations the client has the possibility to search for document nodes by their ID (getElementById(String)). Similarly scene graph nodes can be given identifications based on which they can be looked up from the scene graph through ILookupService which is implemented by scene graph root node G2DSceneGraph.

Normally the scene graph is a tree of nodes. Giving nodes scene graph -locally unique ID's through the lookup mechanism makes it possible to create links between nodes. The links effectively make the scene graph more like a DAG (Directed Acyclic Graph) or even a DG (Dependency Graph). Just as in DOM, the id attribute is not mandatory. Node lookup will only be possible if an ID is provided.

The lookup facility is available at the root node of the scene graph. Considering performance, having to always traverse to the root node to make lookups is a very likely bottleneck. This problem has been alleviated by using transient root node caches in ParentNode. This will naturally make the scene graph a bit more space consuming.

Using LookupService

Giving a lookup ID to a node
G2DSceneGraph root = new G2DSceneGraph();

// Create new nodes
Node child1 = root.addNode("node child 1", G2DParentNode.class);
Node child2 = root.addNode("node child 2", G2DParentNode.class);

// Give a lookup ID
child2.setLookupId("child 2 lookup id");
// OR
NodeUtil.map(child2, "child 2 lookup id");
Performing a lookup
// ID -> INode
INode lookedUpNode = NodeUtil.lookupNode(child1, "child 2 lookup id");
assert lookedUpNode == child2;
//OR
lookedUpNode = root.lookupNode("child 2 lookup id");
assert lookedUpNode == child2;

// INode -> ID
String lookedUpId = NodeUtil.lookupId(child2);
assert lookedUpId.equals("child 2 lookup id");

// See NodeUtil for more utility methods for lookups and associations

Main interfaces

ParentNode

org.simantics.scenegraph.ParentNode

Add child node

Add node simply creates a new node object with the given class and sets it as a child of the parent node.

 public final <TC> TC addNode(Class<TC> a);
 public <TC> TC addNode(String id, Class<TC> a);

Get or create node

If you are not sure if the node already exists, you should give a unique id for the node, and check if the node exists in the scene graph (actually you can only check if its a child of the parent node). For this purpose, there is a method called getOrCreateNode that simply checks if the the node exists and returns it if it does. Otherwise the method will create a new node using addNode.

 public final <TC> TC getOrCreateNode(String id, Class<TC> a);

Remove whole subtree

To remove node and all its children from the scenegraph, a method called remove can be called. This removes node from the parent, and disposes itself and its children. After this method is called, the node and its children are disposed, and thus should not be used anymore.

  public void remove();

Delete node

Delete method can be used to remove a single node from the scene graph tree, so that its children will still be part of the scene graph. This method basically moves nodes children under its parent, and disposes itself. After this method is called, only the node is disposed.

 public void delete();

Remove all children

To remove nodes children, you can use removeNodes-method. This method removes and disposes all children of the node (and their children).

 public final void removeNodes();

INode

org.simantics.scenegraph.INode

Append parent node

To add a node in the middle of the scene graph tree structure, you can use appendParent-method. Basically this method creates a new parent node, adds it as a child of the original parent. Then the node is removed from its parent and added under its new parent.

 public <TC> TC appendParent(String id, Class<TC> nc);

Scenarios

Implementing a 2D node

  • TODO: implementing your own G2DNode
    • render, assumptions
    • getBoundsInLocal

Hooking your node to UI events

By default, 2D nodes do not receive UI events. To receive events, do the following:

  1. Override int getEventMask(), use constants from EventTypes.
  2. Override init(), add addEventHandler(this)
  3. Override cleanup(), addremoveEventhandler(this)
  4. Override any of the following methods in G2DNode or G2DParentNode
    • handleEvent
    • handleKeyEvent, keyReleased, keyPressed
    • handleCommand
    • handleFocusEvent
    • mouseButtonPressed, mouseButtonReleased, mouseClicked, mouseDoubleClicked
    • mouseMoved, mouseDragged
    • mouseEntered, mouseExited
    • mouseWheelMoved
    • handleTimeEvent

All event handler methods return boolean values where a true return value indicates that the handler consumed the event and it should not be propagated further.

Picking

  • TODO: picking

Coordinate system transformations

  • TODO: global <-> local coordinate system transformations

Mouse capture/grabbing

  • TODO: mouse capture

Links

Current Development

Contact