This project has retired. For details please refer to its Attic page.
Apache Stanbol - Execution Metadata

Execution Metadata

The execution metadata holds detailed information about an ongoing/completed enhancement process. Basically they describe how the ExecutionPlan provided by the Chain was executed by the EnhancementJobManager. Both the ExecutionMetadata and the ExecutionPlan are provided with the ContentItem as an own content part of the type MGraph with the URI "http://stanbol.apache.org/ontology/enhancer/executionmetadata#ChainExecution". For users of the Stanbol Enhancer the Execution Metadata are of interest to:

Execution Metadata Ontology

The RDFS schema used for the execution plan is defined as follows:

Execution Metadata

Example

The following example uses the same properties as used within the ExecutionPlan section. To make it easier to see the relations between the execution metadata and the execution plan, the triples of the execution plan are included at the end of this example.

This example describes the following situation:

The RDF graph with the Execution Metadata:

urn:exec
    rdf:type em:ChainExecution
    em:executionPlan urn:execPlan
    em:enhances urn:contentItem1
    em:defaultChain "true"
    em:started 2012-01-11T12.13.14.156
    em:completed 2012-01-11T12.13.15.157
    em:status em:StatusFailed
    em:statusMessage "Unable to execute EnhancementEngine 'new' \
        (Message: No NER model for language 'de' is available)."
    em:executionPart urn:exec1, urn:exec2, urn:exec3, urn:exec4, urn:exec5

urn:exec1
    rdf:type em:EngineExecution
    em:executionPart urn:exec
    em:executionNode urn:node1
    em:status em:StatusCompleted
    em:started 2012-01-11T12.13.14.160
    em:completed 2012-01-11T12.13.14.250

urn:exec2
    rdf:type em:EngineExecution
    em:executionPart urn:exec
    em:executionNode urn:node2
    em:status StatusFailed
    em:statusMessage "No NER model for language 'de' is available"
    em:started 2012-01-11T12.13.14.253
    em:completed 2012-01-11T12.13.14.289

urn:exec3
    rdf:type em:EngineExecution
    em:executionPart urn:exec
    em:executionNode urn:node5
    em:status StatusCompleted
    em:started 2012-01-11T12.13.14.253
    em:completed 2012-01-11T12.13.15.150

The Execution Plan: (copy from the example provided in the ExecutionPlan section)

urn:execPlan
    rdf:type ep:ExecutionPlan
    ep:hasExecutionNode urn:node1, urn:node2, urn:node3, urn:node4, urn:node5
    ep:chain "demoChain"

urn:node1
    rdf:type stanbol:ExecutionNode
    ep:inExecutionPlan urn:execPlan
    stanbol:engine langId

urn:node2
    rdf:type ep:ExecutionNode
    ep:inExecutionPlan urn:execPlan
    ep:dependsOn urn:node1
    ep:engine ner

urn:node3
    rdf:type ep:ExecutionNode
    ep:inExecutionPlan urn:execPlan
    ep:dependsOn urn:node1
    ep:engine dbpediaLinking

urn:node4
    rdf:type ep:ExecutionNode
    ep:inExecutionPlan urn:execPlan
    ep:dependsOn urn:node1
    ep:engine geonamesLinking

urn:node5
    rdf:type ep:ExecutionNode
    ep:inExecutionPlan urn:execPlan
    ep:engine zemanta
    ep:optional "true"^^xsd:boolean

Creation/Management of Execution Metadata

This section is primarily intended for implementors of EnhancementJobManager. However it might also provide insights for users that want/need to monitor the state of enhancement processes as it describes what information are added when to the Execution Metadata.

When the EnhancementJobManager starts the Enhancement of a ContentItem it needs to check if the ContentItem already contains ExecutionMetadata in the ContentPart with the URI "http://stanbol.apache.org/ontology/enhancer/executionmetadata#ChainExecution". If this is the case it needs to initialize itself based on the pre-existing information. If no ExecutionMetadata are present, a new EnhancementProcess needs to be created based on the parsed Chain. Differences between this two cases are explained in the following two sub sections.

Initialization

If no ExecutionMetadata are present within a parsed ContentItem, a new EnhancementProcess needs to be set up. This includes the following steps:

  1. Get the ExecutionPlan for the parsed enhancement Chain. If no chain is parsed the default chain need to be acquired by using the ChainManager.
  2. Create the content part for the ExecutionMetadata with the ContentItem and add the information of the ExecutionPlan to it.
  3. Create the initial ExecutionMetadata. This includes the 'em:ChainExecution' instance for the 'ep:ExecutionPlan' as well as 'em:EngineExecution' instances for all 'ep:ExecutionNode's defined by the execution plan. All such 'em:Execution' instances MUST BE created with the 'em:ExecutionStatus' 'em:StatusSheduled'.

The ExecutionMetadataHelper utility of the "org.apache.stanbol.enhancer.servicesapi" module contains utility methods for initializing execution metadata.

Continuation

If the parsed ContentItem does already contain ExecutionMetadata in the content part with the URI "http://stanbol.apache.org/ontology/enhancer/executionmetadata#ChainExecution" the EnhancementJobManager MUST follow the following steps to continue an EnhancementProcess.

  1. Check if the contained ExecutionMetadata are valid
    • If a 'em:ChainExecution' node is present that 'em:enhances' the parsed ContentItem
    • If the ExecutionPlan is included and if the value of the 'ep:chain' property for the 'ep:ExecutionPlan' resource corresponds to the name of the Chain parsed in the request.
  2. Check the status of all 'em:Execution' instances
    • reset the status of 'em:Execution's that are in-progress to scheduled.
    • TODO: here we could also retry the execution of failed 'em:Execution's

Note that with an continuation the ExecutionPlan MUST NOT be updated. It MUST BE also NOT checked if a Chain with the name as stored in the ExecutionMetadata is still present. Note also that configuration changes of EnhancementEngine will affect the continuation of the enhancement process.

The ExecutionMetadataHelper utility of the "org.apache.stanbol.enhancer.servicesapi" module contains utility methods for reading and validating pre-existing execution metadata.

Execution State Management

The following metadata need to be updated by the EnhancementJobManager when:

The ExecutionMetadataHelper utility of the "org.apache.stanbol.enhancer.servicesapi" module contains utility methods to preform state transitions on 'em:Execution' instances.

Using ExecutionMetadata

This section provides some examples on how to access and retrieve information from the ExecutionMetadata.

Accessing ExecutionMetadata

The ExecutionMetadata and the ExecutionPlan are stored in a content part with with URI "http://stanbol.apache.org/ontology/enhancer/executionmetadata#ChainExecution" with the ContentItem. The following code segment can be used to retrieve the RDF graph with the ExecutionMetadata:

ContentItem ci; //the ContentItem
//the URI is available as constant of the ExecutionMetadata class
UriRef contentPartURI = ExecutionMetadata.CHAIN_EXECUTION;

MGraph executionMetadata = ci.getPart(contentPartURI,MGraph.class);

The ExecutionMetadata are stored as read-/writeable RDF graph. To parse a read-only version to other components one can use the "getGraph()" method defined by MGraph.

Getting details about the em:ChainExecution

The following code segments show how to access information about the execution of the enhancement process for a ContentItem. All directly accessed methods in the examples below are static imports from one of the following two utility classes part of the "org.apache.stanbol.enhancer.servicesapi" module.

This code example first gets the ChainExecution, ExecutionPlan and Chain name for the enhanced content item. In a second step metadata of all executed EnhancementEngines are retrieved.

ContentItem ci; //the ContentItem
MGraph em; //the ExecutionMetadata

//get the ChainExecution, ExecutionPlan and the name of the Chain
NonLiteral ce = getChainExecution(em,ci.getUri());
if(ce != null){
    NonLiteral ep = getExecutionPlan(em,ce);
    String chainName = getString(em,ep,ExecutionPlan.CHAIN);
} else {
    log.warn("ExecutionMetadata of not contain information for "
        + "ContentItem {}!",ci.getUri());
}

//get the EngineExecutions and the name of the Engines
Set<NonLiteral> executions = getExecutions(em,ce);
for(NonLiteral ex : executions){
    NonLiteral en = getExecutionNode(em,ex);
    if(en != null){
        String engineName = getEngine(em,en);
        boolean optional = isOptional(em,en);
    } else { //maybe a sub-chain execution
        //currently not supported, but might
        //added in future versions
    }
    UriRef status = getStatus(em,ex);
    Date started = getStarted(em,ex);
    Date completed = getCompleted(em,ex);
}