class Mapper[Input <: Identifiable[Input], Output <: Identifiable[Output]] extends UnaryComputation[Input, Output]
The Mapper Computation, which is a subclass of UnaryComputation. In short, this takes a input, the computation turns that into a list of outputs. In the example, this would be either Clone() or CommitExtraction().
- Input
The type of the data that needs to be computed. Within the example, this is bigglue.examples.GitID for GitClone(), and bigglue.examples.GitRepo for GitCommitExtraction(). This needs to be an Identifiable type.
- Output
The type of the data that ends up being computed. Within the example, this is bigglue.examples.GitRepo for GitClone(), and bigglue.examples.GitCommitInfo for GitCommitExtraction(). This also needs to be an Identifiable type.
- Alphabetic
- By Inheritance
- Mapper
- UnaryComputation
- UnaryChecker
- OutputStoreChecker
- InputStoreChecker
- ConfigChecker
- Computation
- ConfigBuildsPlatform
- ConfigBuilds
- AnyRef
- Any
- by any2stringadd
- by StringFormat
- by Ensuring
- by ArrowAssoc
- Hide All
- Show All
- Public
- All
Instance Constructors
-
new
Mapper(op: (Input) ⇒ List[Output])
- op
The computation to be run by the computation. For both, this is this section that's in between the parentheses.
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
- def +(other: String): String
-
def
-->[End <: Identifiable[End]](p: Pipe[Output, End]): PartialMapperPipe[Input, Output, End]
This is part of the series of function calls that make the pipeline.
This is part of the series of function calls that make the pipeline. In the example, this is called when stating (Clone()-->clonedMap) or (CommitExtraction()-->commitInfoMap). Simply put, this creates a part of the pipeline that connects the mapper to the output data store and the rest of the pipeline after that.
- End
The type of data store that is at the end of the pipeline. In the example, this would be bigglue.examples.GitCommitGroups
- p
The part of the pipe that follows the mapper computation. In the example, for Clone() it's clonedMap:--CommitExtraction()-->commitInfoMap:-+FindAuthor()+->authorMap, and for BB it's commitInfoMap:-+FindAuthor()+->authorMap.
- returns
A PartialMapperPipe which links the mapper together with the pipe that starts with the output data store.
- def ->[B](y: B): (Mapper[Input, Output], B)
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
check(conf: PipeConfig, inputMap: DataStore[Input], outputMap: DataStore[Output]): Unit
Check routine for unary data processors
Check routine for unary data processors
- conf
the config file to check
- inputMap
the input store to check
- outputMap
the output store to check
- Definition Classes
- UnaryChecker
-
def
checkConfig(conf: PipeConfig): Unit
Check that paths in configuration file relevant to host class are well-formed.
Check that paths in configuration file relevant to host class are well-formed. Default does nothing.
- conf
the config file to check
- Definition Classes
- ConfigChecker
-
def
checkInput(inputMap: DataStore[Input]): Unit
Check that input store is compatible with the host class.
Check that input store is compatible with the host class. Default does nothing.
- inputMap
the input store to check
- Definition Classes
- InputStoreChecker
-
def
checkOutput(outputMap: DataStore[Output]): Unit
Check that output store is compatible with the host class.
Check that output store is compatible with the host class. Default does nothing.
- outputMap
the output store to check
- Definition Classes
- OutputStoreChecker
-
def
clone(): AnyRef
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
val
configOpt: Option[PipeConfig]
- Definition Classes
- ConfigBuilds
-
var
configOption: ConfOpt
- Definition Classes
- Computation
-
def
constructBuilder(config: PipeConfig = null): PlatformBuilder
- Definition Classes
- ConfigBuildsPlatform → ConfigBuilds
- def ensuring(cond: (Mapper[Input, Output]) ⇒ Boolean, msg: ⇒ Any): Mapper[Input, Output]
- def ensuring(cond: (Mapper[Input, Output]) ⇒ Boolean): Mapper[Input, Output]
- def ensuring(cond: Boolean, msg: ⇒ Any): Mapper[Input, Output]
- def ensuring(cond: Boolean): Mapper[Input, Output]
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
- def formatted(fmtstr: String): String
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
-
def
getConfig(config: PipeConfig): PipeConfig
- Definition Classes
- ConfigBuilds
- def getOp: (Input) ⇒ List[Output]
-
def
getUnaryPlatform(): UnaryPlatform[Input, Output]
- Definition Classes
- UnaryComputation
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
-
def
init(conf: PipeConfig, inputMap: DataStore[Input], outputMap: DataStore[Output]): Unit
This initializes the mapper computation.
This initializes the mapper computation. It creates a UnaryPlatform with PlatformBuilder.mapperPlatform, then sets itself as the platform's computation, and then initializes the platform with UnaryPlatform.init.
- conf
The configuration file to build from. Note: If there is a GitIDs-->GitRepos section or GitRepos-->GitCommitInfo section in the bigglue section of the configuration file, it will overwrite part of the configuration file with the values within the step's section. You can see this in effect with GitIDs-->GitRepos, which changes the connector from an bigglue.connectors.instances.IncrTrackerJobQueue to a bigglue.connectors.instances.JobQueue.
- inputMap
The DataStore that data is being sent in from. Within the example, for gitID:--Clone()-->clonedMap, this would be gitID. Likewise, for clonedMap:--CommitExtraction()-->commitInfoMap, this would be clonedMap. Both of these Data Stores are bigglue.store.instances.solr.SolrDataMap within the example.
- outputMap
The DataStore that data is being sent to after computation. Within the example, for gitID:--Clone()-->clonedMap, this would be clonedMap. Likewise, for clonedMap:--CommitExtraction()-->commitInfoMap, this would be commitInfoMap. Both of these Data Stores are bigglue.store.instances.solr.SolrDataMap within the example.
-
def
init(conf: PipeConfig, inputMap: DataStore[Input], outputMap: DataStore[Output], platform: UnaryPlatform[Input, Output]): Unit
This connects the newly initialized platform to the Input Data Store by making it so the Input Data Store sends data down to the platform.
This connects the newly initialized platform to the Input Data Store by making it so the Input Data Store sends data down to the platform.
- conf
The configuration file.
- inputMap
The Input Data Store
- outputMap
The Output Data Store
- platform
- Definition Classes
- UnaryComputation
-
def
init(config: PipeConfig, platform: Platform): Unit
- Definition Classes
- Computation
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
def
name(newName: String): UnaryComputation[Input, Output]
- Definition Classes
- UnaryComputation
-
var
name: String
- Definition Classes
- UnaryComputation
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
-
def
persist(): Unit
If the computation has been initialized, it calls Platform.persist on the platform.
If the computation has been initialized, it calls Platform.persist on the platform. This gives the platform the responsibility of what data to send/resend down the pipeline.
- Definition Classes
- Computation
-
var
platformOpt: Option[Platform]
- Definition Classes
- Computation
-
def
run(): Unit
- Definition Classes
- Mapper → Computation
-
def
setConfig(pipeConfig: PipeConfig): Unit
- Definition Classes
- ConfigBuilds
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
terminate(): Unit
- Definition Classes
- Computation
-
def
toStep(conf: PipeConfig, step: String): PipeConfig
- Definition Classes
- Computation
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
def
tryCompute(input: Input): Option[List[Output]]
Given an input, this attempts to compute the list of outputs, stamp it with the version of the computation, give it provenance information, and then put it in the output map to be sent further down the pipeline.
Given an input, this attempts to compute the list of outputs, stamp it with the version of the computation, give it provenance information, and then put it in the output map to be sent further down the pipeline. Calls bigglue.curators.VersionCurator.stampVersion, bigglue.curators.ProvenanceCurator.reportProvenance, and DataStore.put.
- input
The input that needs to be computed.
- returns
If the computation succeeded, the list of outputs that were produced by the input. If the computation didn't succeed, it report the error and returns None.
-
var
unaryPlatformOpt: Option[UnaryPlatform[Input, Output]]
- Definition Classes
- UnaryComputation
-
val
versionOpt: Option[String]
- Definition Classes
- Computation
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
- def withConfig(newConfigOption: ConfOpt): Mapper[Input, Output]
- def →[B](y: B): (Mapper[Input, Output], B)