Difference between revisions of "Data View"

From Developer Documents
Jump to navigation Jump to search
m (Created page with '= Background and motivation = The synthesis of the Simantics architecture has its foundation on a series of VTT research projects on software architectures and data models [http...')
 
m
Line 1: Line 1:
= Background and motivation =
+
= The big picture =
  
The synthesis of the Simantics architecture has its foundation on a series of VTT research projects on software architectures and data models [http://lib.tkk.fi/Diss/2002/isbn9513860124/] and 20 years of experience from implementation of an industrial process simulator [http://apros.vtt.fi]. The resulting synthesis was put into motion in 2005 when a decision was made to start a development of an ontology-based approach for modelling and simulation. The synthesis is a balance between concepts of theoretical computer science and mathematical logic and concrete user needs in industrial simulation and modelling. The position of VTT between the academia and the industry has given the development team a broad view of the state of the art in modelling and simulation.
+
[[Image:SimanticsDataViewBigPicture.png|800px]]
  
= On objectives and the architecture =
+
= Semantic data model =
  
The architecture of a software system can very naturally be judged by the way it fulfills the most important objectives of the system. The objective of Simantics can in one phrase can be described as
+
== Conceptual model ==
  
* Simantics shall increase the productivity of modelling and simulation
+
=== Semi-structured semantic data ===
  
Productivity should be measured on many levels from Kernel Developer to Model User. It should cover the development of Simantics itself and integration of new tools into Simantics as well as end user tasks. Key means in increasing productivity are
+
A semi-structured data model is a model which represents the data and the rules about the structure of the data in the same database. The Simantics semantic data model is a semi-structured graph of statements about resources.  
  
* Availability. In many cases data and tools needed to perform an engineering task exist but are not practically available to the user. The architecture should make this data and tools available.
+
[[Image:SimanticsDataViewStatements.png|800px]]
* Accessibility. The user should be able to accomplish as much as possible with as little effort as possible. Engineers rely on tools to do their work. The architecture must endorse the emergence of powerful and easy to use tools.
 
* Automation. Suitable automation can in many cases create order of magnitude increases in productivity. The architecture should lower the effort for creating automation.
 
* Accumulation. People reach higher by accumulating knowledge and building on past results. The architecture should endorse practices which accumulate work into reusable results.
 
* Quality. In current state of the art error prevention and error recovery incur large costs in engineering. Better confidence in achieved quality also leads to more efficient solutions. The architecture should provide support for raising the quality of engineering work.
 
* Teamwork. Modelling and simulation is increasingly networked and global. Efficient teams are needed for increasing productivity. The architecture should take teamwork as a given fact and support it in all its structures.
 
  
Being a platform for modelling and simulation, the requirements for real-time operations are also a key factor in determining the architecture.
+
The following statements about the Simantics semantic data model apply
  
= Architectural reasoning =
+
*Each resource can contain at most one attached primitive value
 +
*The native access model is localized e.g. list predicates for subject or objects for (subject, predicate) pair
 +
*Some edges in the graph have a corresponding reverse edge but this is optional
 +
*The set of statements is partitioned and the partitions can be independently processed
 +
*The semantics of the model are built on layered ontologies
  
== Availability ==
+
[[Image:SimanticsDataViewLayeredSemantics.png|800px]]
  
* Make integration of existing data and tools into Simantics as easy as possible. Make sure that existing models can be represented using generic semantic representations. Bring existing data models into Simantics as is. Provide toolkits for implementing communications with the tools.
+
=== Primitive data ===
* Create powerful generic tools for integrating data and tools inside Simantics.
 
* Keep solutions to different problems as clean and as separate as possible. Integrate these solutions.
 
* Make most useful all-purpose tools freely available parts of the platform.
 
  
== Accessibility ==
+
Some data does not benefit from semantic representation. This is the case for example when
  
* Keep down complexity. Use few powerful abstractions to accomplish many tasks.
+
*Data is numeric or textual
* Concentrate effort on a few powerful and generic tools to make then highly productive.
+
*Data has a clear primitive structure e.g. array
* Use layered approaches to expose powerful tools step by step.
+
*There is really a lot data
 +
*The data should be treated with a different granularity of versioning than semantic data.
  
== Automation ==
+
Simantics includes its own [[Databoard_Specification|primitive data modelling system]]. The system
  
* Make the system extensible both in representation and in program logic.
+
*Specifies primitive data type schemas
* Make parts of the system highly configurable and open.
+
**Selected semantic information can also be embedded into the primitive data model
* Design interfaces to allow external modification.
+
*Specifies powerful primitive data structure access and manipulation interfaces
 +
**Arbitrary data sources can be exposed through these interfaces
  
== Accumulation ==
+
The semantic graph can be used to store one piece of primitive data per resource. This data is manipulated atomically and versioned alongside the semantic graph.
  
* Develop standard models and toolkits which support abstraction, decomposition and reuse
+
==== Configuration and Valuations ====
* Focus on long term shared data management inside organisations
 
  
== Quality ==
+
A general modelling contract in Simantics involves the following concepts
  
* Increase the semantic content of data
+
*Configuration. A semantic graph which describes the structure of a model.
* Increase automation both in model creation and in quality assurance
+
*Variable. An identifier constructed from the configuration.
 +
*Valuation. An assignment of values for a set of variables.
 +
*Value. A primitive data valued function of time. Can be sampled or accessed as a time series.
  
== Teamwork ==
+
[[Image:SimanticsArchitectureRealtimeData.png|800px]]
  
* Process all data using a shared repository model with basic multiple user functionality such as transactions, version control and access control.
+
The contract is that all primitive data for a model should be accessed from some valuation.
  
== Real-time ==
+
Simantics exposes the valuations and their structure using the semantic data model. The persistency model (database, workspace or memory) of semantic valuations is application specific.
  
* Make a low-compromise real-time data access subsystem for transferring runtime data.
+
[[Image:SimanticsArchitectureValuations.png|1000px]]
* Prepare for different profiles of data volatility.
 
  
= Architectural structures =
+
The semantic valuation is needed as a uniform interface for accessing the data model produced by a semantic model.
  
[[File:ArchitecturalModel1.png]]
+
*The configuration can be expressive and terse
 +
**A simple configuration can produce a complex set of variables e.g. even an infinite set. The valuation can be lazily produced.
 +
**The complex variable formation logic is hidden behind the valuation
 +
**The model user only operates on the results
 +
*There are usually multiple values for a given variable
 +
**E.g. default, known good configurations and several experiments
 +
**The valuation directly supports this by specifying a valuation identifier which is inserted in the middle of the variable identifier
 +
*The values can come from arbitrary sources
 +
**E.g. semantic configuration, running experiment, historical archive, sensor
 +
**The model user accesses data by value using supplied interfaces. There is no need to know where the data came from.
 +
*Computational variables can be freely implemented
 +
**E.g. expressions based on variable identities can be evaluated by value or they can even be sent to the simulator as expressions.
 +
**The implementation for new values does not know how dependent values are produced
  
== Data driven architecture ==
+
===== Variable typing =====
  
Simantics is a set of services built on top of a semantic data model. The main focus points of the architecture are
+
Some general semantics for variables are established even at the Layer0 level. These semantics can be freely extended by applications. Variables can belong to some of the following categories
  
* Emphasis on agility and expressivity. Arbitrary different models can co-exist in the same database.
+
*Fixed. In many cases e.g. model component names are unique and persistent across all valuations.
* Emphasis on large scale industrial applications. Large data sets of semantic and primitive data need to be processes.
+
*Configuration. Some variables do not change value during simulation, but there can be several known configuration values e.g. initial states for experiments.
* Emphasis on real-time applications. Fast modification and tracking of changes.
+
*Simulator. Some variables have their values produced by a black box simulator process but other e.g. initial values are also stored semantically within Simantics
* Emphasis on teamwork.
+
*Simulator only. Some simulator variables are only available from the black box simulator. Simantics stores initial values for these variables only as blobs, which are transferred to the simulator as black boxes.
* Emphasis on data model integration with client Java code. Client is supported in the implementation of the data semantics.
+
*Derived. Some variables have values computed from other variable values.
 +
*Dynamic. Some variables are semantically declared but their occurrence in valuations is optional.
 +
*Fully dynamic. Some variables are not semantically declared can occur in valuations only. These variables can be analysed based on their primitive data content only.
  
Simantics aims to enable a data-driven approach in all phases of the model-assisted decision support process.
+
===== Valuation stack =====
  
The data model is detailed in [[Data View]]
+
A valuation does not have to supply values for all variables in a model configuration. A model user accesses values using a valuation stack which is guaranteed to produce values for all variables.
  
== Plugin architecture ==
+
[[Image:SimanticsDataViewValuationStack.png|1000px]]
  
Simantics products are built up from a set of plugins. The main implications of a plugin architecture are
+
==== Large primitives ====
  
* Plugins are software components with well-defined interfaces and dependencies
+
Some models contain large pieces (> megabytes) of primitive data which needs to be specially processed. Key questions are
* Plugin-based software products can be configured by adding and removing plugins
 
  
== Integration architecture ==
+
*Partial reads and writes (e.g. of a terabyte file)
 +
*Data transfer between server and client
 +
*File access support (e.g. native state files of simulators stored in the semantic database and accessed by simulators as files)
  
A central goal of Simantics is to be able to bring existing tools and models into the Simantics framework for integration with other tools and models already in Simantics. Simantics does this by
+
[[Image:SimanticsArchitectureLargePrimitiveData.png|800px]]
  
* Implementing the native data model of the integration target in Simantics. The emphasis is on easy synchronisation and is supported by automatic tools and frameworks (solver toolkit).
+
== Client models ==
* Integrating the native data model with other models using semantic mappings.
 
  
== Layered architecture ==
+
=== Sessions and databases ===
  
Simantics aims to increase modelling productivity by adopting best practices in modelling such as hierarchical decomposition and component-based reuse. Simantics does this by
+
The semantic database is accessed by using a session which provides shared access to a working copy of the database for multiple users. Key points are
  
* Providing a few well-thought templates (e.g. structural, DEVS co-simulation) with support for modelling cases in typical engineering systems.  
+
*Concurrent use models. The session provides both transaction-locked concurrent use and publish/synchronize based operation.
* Providing support to translate these templates into native tool formats increasing the expressivity of the tools native capability.
+
*Authentication and access control. These are not yet supported in Simantics 1.0
* Providing implementation frameworks for common tasks such as solver distribution.
 
  
A focal use case of Simantics is user interface development. Simantics aims to make this quick and easy by
+
[[Image:SimanticsArchitectureSessions.png|800px]]
  
* Including a common set of typical user interface tools. These include 2D diagramming, Model browsing, Property sheets, Charts, Spreadsheet and Documentation editing.
+
=== Database client access interface ===
* Using a semantic data driven configuration of the tools
 
* Making the tools extensible via plugins
 
* Providing frameworks for easy interaction between the tools including clipboard, DnD, workspace selection and requests.
 
  
== Team architecture ==
+
The database session interface allows reading and writing the semantic database using requests. Key points are
  
The emphasis on teamwork in Simantics is implemented using the following means
+
*Transaction model. The client provides a traditional read-write locking scheme. Reads are processed concurrently while writes require exclusive locking. Fairness requirements for the locking scheme are not specified.
 +
*Caching of requests. Often-used read requests can be cached to achieve necessary read performance.
 +
*Change listening. The client interface supports change listening with automatic dependency analysis. This mechanism allows the database to be used in [[wikipedia:Reactive_programming|reactive programming]] style.
 +
*Threading model.  The client supports both synchronous and asynchronous request formulation. Asynchronous model automatically distributes the computation on available processor cores.
 +
*The connection between change sets and requests. Write requests form change sets, which can be further annotated with metadata.
  
* A shared semantic database with history and access management. The approach follows the example of highly successful and popular systems such as CVS and SVN
+
[[Image:SimanticsArchitectureRequests.png|800px]]
* Integrated documentation tools with fast manual content creation and support for transferring user interface data (e.g. diagrams, tables, requests) into documents
 
* Emphasis on diagramming and diagram self-documentation for making models accessible
 
  
= Structure of the technical specification =
+
=== Runtime client extensions ===
  
More details of the technical specification can be found in the following sections
+
==== Adapters ====
  
[[Data View]]
+
The adaption of resources into Java interfaces is a key method of enforcing the semantics of the data model. The adapter model includes
  
[[Component View]]
+
*A way to declare adapter interfaces in plugins
 +
*A way to contribute adapter implementations in plugins
 +
*A way to retrieve an adapter for a resource
  
[[Deployment View]]
+
The mechanism is described in [[Resource Adaptation]].
  
[[Security View]]
+
==== Virtual graph ====
 +
 
 +
The Simantics database client can extend the persistent semantic graph with more transient resources and statements. This mechanism can be used to e.g. expose valuations and other workspace-persistent data.
 +
 
 +
[[Image:SimanticsArchitectureTransientSemanticData.png|800px]]
 +
 
 +
The following statements apply
 +
 
 +
*The life cycle of transient contributions is managed by a workspace-persistent entity
 +
**Transient resources can be persistent within the life cycle of the workspace
 +
**This entity ensures that co-existing subgraphs share the available identifier space
 +
*Transient contributions are dynamically attached and detached to a database Session.
 +
**The subgraph is applied on top of the persistent graph and below the query layer
 +
 
 +
==== Generic Relation Indexing ====
 +
 
 +
Layer0 Generic Relations can be used to construct per model indices of useful data, which then can be quickly searched using the Lucene search engine.
 +
 
 +
[[Image:SimanticsDataViewIndexing.png|800px]]
 +
 
 +
= Semantics =
 +
 
 +
== Ontology language (Layer0) ==
 +
 
 +
[[Image:SimanticsArchitectureLayer0Concepts.png|800px]]
 +
 
 +
Layer0 describes the baseline semantics for Simantics applications. Layer0 involves
 +
 
 +
* Generic concepts used in all Simantics modelling
 +
* Common activities encountered in all Simantics modelling
 +
* Adapter interfaces, which implement the intended semantics of the concepts and the activities
 +
 
 +
[[Image:SimanticsArchitectureLayer0.png|800px]]
 +
 
 +
* Concepts
 +
** Type. A resource can be an instance of several types. The types assert structural constraints, contribute relations and specify adapters.
 +
** Relation. All predicates in the semantic graph are part of a relation hierarchy. Structural constraints involve relations and basic queries support relation hierarchies.
 +
** Requirement. A validity requirement, which can used to check semantic instances against their specification
 +
** Data type. Specifies the structure and units of a piece of primitive data.
 +
** User. Users are needed in change management, documentation and access control.
 +
** Adapter. Adapter specifies a requirement for the client to supply an implementation of a certain interface.
 +
** URI. A URI can be used to identify and locate resources within a semantic database.
 +
** Subgraph. Specifies a set of statements for a resource. Needed in e.g. copy and delete activities.
 +
** Library. Libraries are used to structure large amounts of data. Libraries are browsed.
 +
** Project. Private data is organised in projects. A project specifies its required execution environment.
 +
** Model. A model spans a variable space.
 +
** Variable. A model determines a set of variables which have primitive data values.
 +
** Valuation. A valuation contains values for variables.
 +
** Label. A label is a textual representation of a resource.
 +
** Viewpoint. A viewpoint specifies means to browse the semantic graph.
 +
** Predicate. A logical predicate as e.g. in Prolog.
 +
** Function. A function with n inputs and a single output.
 +
** Index. A workspace-persistent realization of a predicate
 +
** Activation. Used to activate and deactivate a concept.
 +
** Operation. A runnable action, which makes a modification to the semantic graph.
 +
** Extent. For determining a local extent of an instance. For determining subgraphs.
 +
 
 +
* Activities
 +
** Adaption. Retrieves an implementation of an interface for a resource.
 +
** Analysis. Computes some result based on the semantic graph.
 +
** Validation. Checks whether some constraints are satisfied.
 +
** Access determination. Determines whether access to data is granted.
 +
** Structure determination. Determines allowed structures in the graph.
 +
** Shared identification. Establishes identity across semantic databases.
 +
** Instantiation. Creates new instances based on types.
 +
** Copying. Creates a copy of a subgraph in the same database or in some other database.
 +
** Deletion. Deletes a subgraph.
 +
** Activation. Transfers a concept between active and inactive states. Activation can involve displaying and initiation of certain processes.
 +
** Mapping. Enforces rules between models.
 +
** Editing. Transforms a model based on input.
 +
** Variable discovery. Determination of the variable space based on the configuration structure of the model.
 +
** Instance discovery. Finding e.g. shared instances of a type from the database.
 +
** Browsing. Traversing the semantic graph using tree representations.
 +
 
 +
Layer 0 is further specified in [[Layer0 ontology]]
 +
 
 +
== Project management ==
 +
 
 +
The top level organization or a Simantics database is depicted in the image below.
 +
 
 +
[[Image:ConceptualModel3.png|800px]]
 +
 
 +
== Projects ==
 +
 
 +
Each Simantics workspace is started into a specific project. [[org.simantics.project|Simantics projects]] use ''features'' for associating life-cycle behavior to projects, much like ''[http://help.eclipse.org/galileo/index.jsp?topic=/org.eclipse.platform.doc.isv/reference/extension-points/org_eclipse_core_resources_natures.html project natures]'' in Eclipse.
 +
 
 +
[[Image:SimanticsDataViewProjects.png|1000px]]
 +
 
 +
== Subgraph transfer mechanism ==
 +
 
 +
The subgraph transfer mechanism is needed for identifying a set of statements to transfer from and to a database.
 +
 
 +
Use-cases, where the mechanism is needed:
 +
* Copy-paste
 +
* Export-import
 +
* Ontology evolution
 +
 
 +
There have been at least two different prototype implementations of subgraph transfer
 +
(org.simantics.layer0.utils.extent and org.simantics.layer0.utils.extent2).
 +
Both have their own shortcomings.
 +
 
 +
There are two major technical challenges:
 +
* How to define which statements belong to the subgraph
 +
** First approach is based on Includes- and Propagates-relations.
 +
** Second approach is based on classification of relations and defining parent resources for all resources.
 +
** See also [[Subgraphs]]
 +
** We have so far tried to find a generic solution for this problem. There are however often application specific concerns. A robust solution should probably involve some generic solution that can be customized with adapters.
 +
*** Even if adapters solve the problem of local propagation of extents, we still need a conceptual model that defines the whole extent.
 +
* How to serialize resource references
 +
** Several serialization formats have been defined (but the format is not really the problem):
 +
*** [https://www.simulationsite.net/svn/simantics/documentation/trunk/doc/word/TransferableGraph.doc Transferable Graph]
 +
*** org.simantics.layer0.utils.extent2.SubgraphWriter (not documented anywhere)
 +
*** [[Graph exchange format]]
 +
** The most important issue is serializability of resources.
 +
 
 +
== Validation ==
 +
 
 +
To maximally utilize the increased semantic content of the semantic database a validation mechanism is defined for
 +
 
 +
* Checking the semantic configuration against semantic rules
 +
* Checking the validity of the semantic implications of the semantic configuration (e.g. checking the flattening of a structural model)
 +
* Checking real time data against semantic rules
 +
 
 +
 
 +
== Browsing ==
 +
 
 +
[[Image:SimanticsArchitectureBrowsing.png|1000px]]
 +
 
 +
The following statements describe the browsing solution in Simantics
 +
 
 +
*The browser builds up its representation using Contributions e.g. ViewpointContributions and Labelers.
 +
*Viewpoints and Labelers are determined using Evaluators bound to Java class of input objects.
 +
*The ontology for browsing specifies a viewpoint made up from contributions to named contexts (e.g. browser identity or model) with adapter-modelled acceptance criteria
 +
*New contributions are described in ontologies and implemented as Java adapters.
 +
*Each browser has its own unique identifier string
 +
*Contributions are contributed and bound to contexts using Eclipse extension points
 +
 
 +
=== Concepts and their contracts ===
 +
 
 +
; Evaluator
 +
: A Java object which determines e.g. Viewpoints and Labelers for given input.
 +
; Viewpoint
 +
: Determines a set of child objects for given input object.
 +
; Labeler
 +
: Determines a label for an object.
 +
; Context
 +
: A set of keys and associated values which are used to steer the browsing process. A mandatory INPUT object is used in Evaluators.
 +
 
 +
= Framework ontologies =
 +
 
 +
[[Image:Ontologies.jpg]]
 +
 
 +
== Mapping ==
 +
 
 +
Mapping framework in Simantics consists of several different levels. From the lowest to the highest:
 +
; [[Layer0:Trigger|Triggers]]
 +
: Provides a uniform interface to mappings and other graph modifying software components. This specification defines how a mapping is attached to a resource and how it is used. Actual implementation can be written directly in Java or using higher level mechanisms.
 +
; [[org.simantics.scl|SCL]] mappings
 +
: A datalog based language that is meant for writing queries and transformations.
 +
; Data driven mappings
 +
: A mapping defined using lower level definitions that can be customized with specific relations. For example [[svn:modeling/trunk/modeling_ontologies/modeling.graph|mapping.graph]] defines this kind of mapping. Graphical user intefaces can be built for defining this kind of mappings.
 +
 
 +
== Structural modelling ==
 +
 
 +
The structural ontology models hierarchically decomposed and connected components and their flattening.
 +
 
 +
The data model is described in [[Structural ontology]].
 +
 
 +
== Diagramming ==
 +
 
 +
Diagrams are used to configure models in various domains. The diagram model is a self-sufficient graphical model which is mapped into domain specific simulation models directly or via structural ontology.
 +
 
 +
[[Image:SimanticsArchitectureDiagramming.png|center|frame|Conceptual diagramming model. ([[Image:SimanticsArchitectureDiagramming.graphml|image source]])]]
  
 
[[Category:Development Documents]]
 
[[Category:Development Documents]]
 
[[Category:Technical Specifications]]
 
[[Category:Technical Specifications]]
 +
 +
=Address Space=
 +
Address Space is the structure of the data when a model is exposed in an communication interface. It consists of nodes and variables.
 +
 +
There are two ''types'' of address spaces:
 +
*'''Configuration''' Configuration model consists of initialization values, such as parameters.
 +
*'''Runtime''' Runtime model consists of variables and constants.
 +
 +
 +
There are two ''structures'' of address spaces:
 +
*'''Structural''' address space is a complicated model of instances and classes. The structure is a graph.
 +
*'''Flattened''' address space is a flattened simplification of the previous model. It is a tree.
 +
 +
 +
There is no <u>structural</u> address space of runtime model.
 +
 +
The <u>structure</u> of configuration flat model is equal or a sub-set of the a runtime flat model.

Revision as of 09:23, 30 September 2010

The big picture

SimanticsDataViewBigPicture.png

Semantic data model

Conceptual model

Semi-structured semantic data

A semi-structured data model is a model which represents the data and the rules about the structure of the data in the same database. The Simantics semantic data model is a semi-structured graph of statements about resources.

SimanticsDataViewStatements.png

The following statements about the Simantics semantic data model apply

  • Each resource can contain at most one attached primitive value
  • The native access model is localized e.g. list predicates for subject or objects for (subject, predicate) pair
  • Some edges in the graph have a corresponding reverse edge but this is optional
  • The set of statements is partitioned and the partitions can be independently processed
  • The semantics of the model are built on layered ontologies

SimanticsDataViewLayeredSemantics.png

Primitive data

Some data does not benefit from semantic representation. This is the case for example when

  • Data is numeric or textual
  • Data has a clear primitive structure e.g. array
  • There is really a lot data
  • The data should be treated with a different granularity of versioning than semantic data.

Simantics includes its own primitive data modelling system. The system

  • Specifies primitive data type schemas
    • Selected semantic information can also be embedded into the primitive data model
  • Specifies powerful primitive data structure access and manipulation interfaces
    • Arbitrary data sources can be exposed through these interfaces

The semantic graph can be used to store one piece of primitive data per resource. This data is manipulated atomically and versioned alongside the semantic graph.

Configuration and Valuations

A general modelling contract in Simantics involves the following concepts

  • Configuration. A semantic graph which describes the structure of a model.
  • Variable. An identifier constructed from the configuration.
  • Valuation. An assignment of values for a set of variables.
  • Value. A primitive data valued function of time. Can be sampled or accessed as a time series.

SimanticsArchitectureRealtimeData.png

The contract is that all primitive data for a model should be accessed from some valuation.

Simantics exposes the valuations and their structure using the semantic data model. The persistency model (database, workspace or memory) of semantic valuations is application specific.

SimanticsArchitectureValuations.png

The semantic valuation is needed as a uniform interface for accessing the data model produced by a semantic model.

  • The configuration can be expressive and terse
    • A simple configuration can produce a complex set of variables e.g. even an infinite set. The valuation can be lazily produced.
    • The complex variable formation logic is hidden behind the valuation
    • The model user only operates on the results
  • There are usually multiple values for a given variable
    • E.g. default, known good configurations and several experiments
    • The valuation directly supports this by specifying a valuation identifier which is inserted in the middle of the variable identifier
  • The values can come from arbitrary sources
    • E.g. semantic configuration, running experiment, historical archive, sensor
    • The model user accesses data by value using supplied interfaces. There is no need to know where the data came from.
  • Computational variables can be freely implemented
    • E.g. expressions based on variable identities can be evaluated by value or they can even be sent to the simulator as expressions.
    • The implementation for new values does not know how dependent values are produced
Variable typing

Some general semantics for variables are established even at the Layer0 level. These semantics can be freely extended by applications. Variables can belong to some of the following categories

  • Fixed. In many cases e.g. model component names are unique and persistent across all valuations.
  • Configuration. Some variables do not change value during simulation, but there can be several known configuration values e.g. initial states for experiments.
  • Simulator. Some variables have their values produced by a black box simulator process but other e.g. initial values are also stored semantically within Simantics
  • Simulator only. Some simulator variables are only available from the black box simulator. Simantics stores initial values for these variables only as blobs, which are transferred to the simulator as black boxes.
  • Derived. Some variables have values computed from other variable values.
  • Dynamic. Some variables are semantically declared but their occurrence in valuations is optional.
  • Fully dynamic. Some variables are not semantically declared can occur in valuations only. These variables can be analysed based on their primitive data content only.
Valuation stack

A valuation does not have to supply values for all variables in a model configuration. A model user accesses values using a valuation stack which is guaranteed to produce values for all variables.

SimanticsDataViewValuationStack.png

Large primitives

Some models contain large pieces (> megabytes) of primitive data which needs to be specially processed. Key questions are

  • Partial reads and writes (e.g. of a terabyte file)
  • Data transfer between server and client
  • File access support (e.g. native state files of simulators stored in the semantic database and accessed by simulators as files)

SimanticsArchitectureLargePrimitiveData.png

Client models

Sessions and databases

The semantic database is accessed by using a session which provides shared access to a working copy of the database for multiple users. Key points are

  • Concurrent use models. The session provides both transaction-locked concurrent use and publish/synchronize based operation.
  • Authentication and access control. These are not yet supported in Simantics 1.0

SimanticsArchitectureSessions.png

Database client access interface

The database session interface allows reading and writing the semantic database using requests. Key points are

  • Transaction model. The client provides a traditional read-write locking scheme. Reads are processed concurrently while writes require exclusive locking. Fairness requirements for the locking scheme are not specified.
  • Caching of requests. Often-used read requests can be cached to achieve necessary read performance.
  • Change listening. The client interface supports change listening with automatic dependency analysis. This mechanism allows the database to be used in reactive programming style.
  • Threading model. The client supports both synchronous and asynchronous request formulation. Asynchronous model automatically distributes the computation on available processor cores.
  • The connection between change sets and requests. Write requests form change sets, which can be further annotated with metadata.

SimanticsArchitectureRequests.png

Runtime client extensions

Adapters

The adaption of resources into Java interfaces is a key method of enforcing the semantics of the data model. The adapter model includes

  • A way to declare adapter interfaces in plugins
  • A way to contribute adapter implementations in plugins
  • A way to retrieve an adapter for a resource

The mechanism is described in Resource Adaptation.

Virtual graph

The Simantics database client can extend the persistent semantic graph with more transient resources and statements. This mechanism can be used to e.g. expose valuations and other workspace-persistent data.

SimanticsArchitectureTransientSemanticData.png

The following statements apply

  • The life cycle of transient contributions is managed by a workspace-persistent entity
    • Transient resources can be persistent within the life cycle of the workspace
    • This entity ensures that co-existing subgraphs share the available identifier space
  • Transient contributions are dynamically attached and detached to a database Session.
    • The subgraph is applied on top of the persistent graph and below the query layer

Generic Relation Indexing

Layer0 Generic Relations can be used to construct per model indices of useful data, which then can be quickly searched using the Lucene search engine.

SimanticsDataViewIndexing.png

Semantics

Ontology language (Layer0)

SimanticsArchitectureLayer0Concepts.png

Layer0 describes the baseline semantics for Simantics applications. Layer0 involves

  • Generic concepts used in all Simantics modelling
  • Common activities encountered in all Simantics modelling
  • Adapter interfaces, which implement the intended semantics of the concepts and the activities

SimanticsArchitectureLayer0.png

  • Concepts
    • Type. A resource can be an instance of several types. The types assert structural constraints, contribute relations and specify adapters.
    • Relation. All predicates in the semantic graph are part of a relation hierarchy. Structural constraints involve relations and basic queries support relation hierarchies.
    • Requirement. A validity requirement, which can used to check semantic instances against their specification
    • Data type. Specifies the structure and units of a piece of primitive data.
    • User. Users are needed in change management, documentation and access control.
    • Adapter. Adapter specifies a requirement for the client to supply an implementation of a certain interface.
    • URI. A URI can be used to identify and locate resources within a semantic database.
    • Subgraph. Specifies a set of statements for a resource. Needed in e.g. copy and delete activities.
    • Library. Libraries are used to structure large amounts of data. Libraries are browsed.
    • Project. Private data is organised in projects. A project specifies its required execution environment.
    • Model. A model spans a variable space.
    • Variable. A model determines a set of variables which have primitive data values.
    • Valuation. A valuation contains values for variables.
    • Label. A label is a textual representation of a resource.
    • Viewpoint. A viewpoint specifies means to browse the semantic graph.
    • Predicate. A logical predicate as e.g. in Prolog.
    • Function. A function with n inputs and a single output.
    • Index. A workspace-persistent realization of a predicate
    • Activation. Used to activate and deactivate a concept.
    • Operation. A runnable action, which makes a modification to the semantic graph.
    • Extent. For determining a local extent of an instance. For determining subgraphs.
  • Activities
    • Adaption. Retrieves an implementation of an interface for a resource.
    • Analysis. Computes some result based on the semantic graph.
    • Validation. Checks whether some constraints are satisfied.
    • Access determination. Determines whether access to data is granted.
    • Structure determination. Determines allowed structures in the graph.
    • Shared identification. Establishes identity across semantic databases.
    • Instantiation. Creates new instances based on types.
    • Copying. Creates a copy of a subgraph in the same database or in some other database.
    • Deletion. Deletes a subgraph.
    • Activation. Transfers a concept between active and inactive states. Activation can involve displaying and initiation of certain processes.
    • Mapping. Enforces rules between models.
    • Editing. Transforms a model based on input.
    • Variable discovery. Determination of the variable space based on the configuration structure of the model.
    • Instance discovery. Finding e.g. shared instances of a type from the database.
    • Browsing. Traversing the semantic graph using tree representations.

Layer 0 is further specified in Layer0 ontology

Project management

The top level organization or a Simantics database is depicted in the image below.

ConceptualModel3.png

Projects

Each Simantics workspace is started into a specific project. Simantics projects use features for associating life-cycle behavior to projects, much like project natures in Eclipse.

SimanticsDataViewProjects.png

Subgraph transfer mechanism

The subgraph transfer mechanism is needed for identifying a set of statements to transfer from and to a database.

Use-cases, where the mechanism is needed:

  • Copy-paste
  • Export-import
  • Ontology evolution

There have been at least two different prototype implementations of subgraph transfer (org.simantics.layer0.utils.extent and org.simantics.layer0.utils.extent2). Both have their own shortcomings.

There are two major technical challenges:

  • How to define which statements belong to the subgraph
    • First approach is based on Includes- and Propagates-relations.
    • Second approach is based on classification of relations and defining parent resources for all resources.
    • See also Subgraphs
    • We have so far tried to find a generic solution for this problem. There are however often application specific concerns. A robust solution should probably involve some generic solution that can be customized with adapters.
      • Even if adapters solve the problem of local propagation of extents, we still need a conceptual model that defines the whole extent.
  • How to serialize resource references
    • Several serialization formats have been defined (but the format is not really the problem):
    • The most important issue is serializability of resources.

Validation

To maximally utilize the increased semantic content of the semantic database a validation mechanism is defined for

  • Checking the semantic configuration against semantic rules
  • Checking the validity of the semantic implications of the semantic configuration (e.g. checking the flattening of a structural model)
  • Checking real time data against semantic rules


Browsing

SimanticsArchitectureBrowsing.png

The following statements describe the browsing solution in Simantics

  • The browser builds up its representation using Contributions e.g. ViewpointContributions and Labelers.
  • Viewpoints and Labelers are determined using Evaluators bound to Java class of input objects.
  • The ontology for browsing specifies a viewpoint made up from contributions to named contexts (e.g. browser identity or model) with adapter-modelled acceptance criteria
  • New contributions are described in ontologies and implemented as Java adapters.
  • Each browser has its own unique identifier string
  • Contributions are contributed and bound to contexts using Eclipse extension points

Concepts and their contracts

Evaluator
A Java object which determines e.g. Viewpoints and Labelers for given input.
Viewpoint
Determines a set of child objects for given input object.
Labeler
Determines a label for an object.
Context
A set of keys and associated values which are used to steer the browsing process. A mandatory INPUT object is used in Evaluators.

Framework ontologies

Ontologies.jpg

Mapping

Mapping framework in Simantics consists of several different levels. From the lowest to the highest:

Triggers
Provides a uniform interface to mappings and other graph modifying software components. This specification defines how a mapping is attached to a resource and how it is used. Actual implementation can be written directly in Java or using higher level mechanisms.
SCL mappings
A datalog based language that is meant for writing queries and transformations.
Data driven mappings
A mapping defined using lower level definitions that can be customized with specific relations. For example mapping.graph defines this kind of mapping. Graphical user intefaces can be built for defining this kind of mappings.

Structural modelling

The structural ontology models hierarchically decomposed and connected components and their flattening.

The data model is described in Structural ontology.

Diagramming

Diagrams are used to configure models in various domains. The diagram model is a self-sufficient graphical model which is mapped into domain specific simulation models directly or via structural ontology.

Conceptual diagramming model. (File:SimanticsArchitectureDiagramming.graphml)

Address Space

Address Space is the structure of the data when a model is exposed in an communication interface. It consists of nodes and variables.

There are two types of address spaces:

  • Configuration Configuration model consists of initialization values, such as parameters.
  • Runtime Runtime model consists of variables and constants.


There are two structures of address spaces:

  • Structural address space is a complicated model of instances and classes. The structure is a graph.
  • Flattened address space is a flattened simplification of the previous model. It is a tree.


There is no structural address space of runtime model.

The structure of configuration flat model is equal or a sub-set of the a runtime flat model.