class BigActorReducerPlatform[Input <: Identifiable[Input], Output <: Identifiable[Output]] extends BigActorUnaryPlatform[Input, Output]
This is the default platform for the reducer.
- Input
The type of the data that needs to be computed. bigglue.examples.GitCommitInfo in this case, which is the type of commitInfoMap. This needs to be an Identifiable type.
- Output
The type of the data that the input ends up being reduced to. Within the example, this will be bigglue.examples.GitCommitGroups.
- Alphabetic
- By Inheritance
- BigActorReducerPlatform
- BigActorUnaryPlatform
- BigActor
- UnaryPlatform
- UnaryChecker
- OutputStoreChecker
- InputStoreChecker
- ConfigChecker
- Platform
- AnyRef
- Any
- by any2stringadd
- by StringFormat
- by Ensuring
- by ArrowAssoc
- Hide All
- Show All
- Public
- All
Instance Constructors
-
new
BigActorReducerPlatform(name: String = ...)
- name
The name of the platform. This is usually "platform-actor" followed by a random ID.
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
def
+(other: String): String
- Implicit
- This member is added by an implicit conversion from BigActorReducerPlatform[Input, Output] to any2stringadd[BigActorReducerPlatform[Input, Output]] performed by method any2stringadd in scala.Predef.
- Definition Classes
- any2stringadd
-
def
->[B](y: B): (BigActorReducerPlatform[Input, Output], B)
- Implicit
- This member is added by an implicit conversion from BigActorReducerPlatform[Input, Output] to ArrowAssoc[BigActorReducerPlatform[Input, Output]] performed by method ArrowAssoc in scala.Predef.
- Definition Classes
- ArrowAssoc
- Annotations
- @inline()
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
implicit
def
actorSystem: ActorSystem
- Definition Classes
- BigActor
-
implicit
val
actorSystemOpt: Option[ActorSystem]
- Definition Classes
- BigActor
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
check(conf: PipeConfig, inputMap: DataStore[Input], outputMap: DataStore[Output]): Unit
Check routine for unary data processors
Check routine for unary data processors
- conf
the config file to check
- inputMap
the input store to check
- outputMap
the output store to check
- Definition Classes
- UnaryChecker
-
def
checkConfig(conf: PipeConfig): Unit
Check that paths in configuration file relevant to host class are well-formed.
Check that paths in configuration file relevant to host class are well-formed. Default does nothing.
- conf
the config file to check
- Definition Classes
- ConfigChecker
-
def
checkInput(inputMap: DataStore[Input]): Unit
Check that input store is compatible with the host class.
Check that input store is compatible with the host class. Default does nothing.
- inputMap
the input store to check
- Definition Classes
- InputStoreChecker
-
def
checkOutput(outputMap: DataStore[Output]): Unit
Check that output store is compatible with the host class.
Check that output store is compatible with the host class. Default does nothing.
- outputMap
the output store to check
- Definition Classes
- OutputStoreChecker
-
def
clone(): AnyRef
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
var
computationOpt: Option[Computation]
What computation this platform ends up running.
What computation this platform ends up running. (Mapper, Reducer, PairwiseComposer) This ends up being set in init() by both UnaryPlatform and BinaryPlatform.
- Definition Classes
- Platform
-
def
compute(job: Input): Unit
- Definition Classes
- BigActor
-
def
compute_(job: Input): Unit
Given an input, it updates the outputs by computing a new output from a previous output,
Given an input, it updates the outputs by computing a new output from a previous output,
- job
The input that's updating the outputs,
- Definition Classes
- BigActorReducerPlatform → BigActor
-
def
computer: ActorRef
This gets the BigActorWorker.
This gets the BigActorWorker.
- Definition Classes
- BigActor
-
val
computerOpt: Option[ActorRef]
- Definition Classes
- BigActor
-
def
ensuring(cond: (BigActorReducerPlatform[Input, Output]) ⇒ Boolean, msg: ⇒ Any): BigActorReducerPlatform[Input, Output]
- Implicit
- This member is added by an implicit conversion from BigActorReducerPlatform[Input, Output] to Ensuring[BigActorReducerPlatform[Input, Output]] performed by method Ensuring in scala.Predef.
- Definition Classes
- Ensuring
-
def
ensuring(cond: (BigActorReducerPlatform[Input, Output]) ⇒ Boolean): BigActorReducerPlatform[Input, Output]
- Implicit
- This member is added by an implicit conversion from BigActorReducerPlatform[Input, Output] to Ensuring[BigActorReducerPlatform[Input, Output]] performed by method Ensuring in scala.Predef.
- Definition Classes
- Ensuring
-
def
ensuring(cond: Boolean, msg: ⇒ Any): BigActorReducerPlatform[Input, Output]
- Implicit
- This member is added by an implicit conversion from BigActorReducerPlatform[Input, Output] to Ensuring[BigActorReducerPlatform[Input, Output]] performed by method Ensuring in scala.Predef.
- Definition Classes
- Ensuring
-
def
ensuring(cond: Boolean): BigActorReducerPlatform[Input, Output]
- Implicit
- This member is added by an implicit conversion from BigActorReducerPlatform[Input, Output] to Ensuring[BigActorReducerPlatform[Input, Output]] performed by method Ensuring in scala.Predef.
- Definition Classes
- Ensuring
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
var
errorCuratorOpt: Option[ErrorCurator[Input]]
- Definition Classes
- UnaryPlatform
-
def
finalize(): Unit
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
def
formatted(fmtstr: String): String
- Implicit
- This member is added by an implicit conversion from BigActorReducerPlatform[Input, Output] to StringFormat[BigActorReducerPlatform[Input, Output]] performed by method StringFormat in scala.Predef.
- Definition Classes
- StringFormat
- Annotations
- @inline()
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
-
def
getErrorCurator(): ErrorCurator[Input]
- Definition Classes
- UnaryPlatform
-
def
getInputMap(): DataStore[Input]
- Definition Classes
- UnaryPlatform
-
def
getInputs(): Seq[Input]
- Definition Classes
- UnaryPlatform
-
def
getOutputMap(): DataStore[Output]
- Definition Classes
- UnaryPlatform
-
def
getProvenanceCurator(): ProvenanceCurator[Input, Output]
- Definition Classes
- UnaryPlatform
-
def
getUpstreamConnector(): Connector[Input]
- Definition Classes
- UnaryPlatform
-
def
getVersionCurator(): VersionCurator[Output]
- Definition Classes
- UnaryPlatform
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
-
val
infoConfig: Config
- Definition Classes
- BigActor
-
def
init(conf: PipeConfig, inputMap: DataStore[Input], outputMap: DataStore[Output], builder: PlatformBuilder): Unit
This sets up the platform by connecting the Input Map and Output Map to the platform, as well as any other initialization that needs to be done Along with that, it also sets up the actor system, and gets the list of all of the worker actors.
This sets up the platform by connecting the Input Map and Output Map to the platform, as well as any other initialization that needs to be done Along with that, it also sets up the actor system, and gets the list of all of the worker actors.
- conf
The configuration file needed to initialize.
- inputMap
The Map that data gets sent in from.
- outputMap
The Map that data gets sent out to.
- builder
The builder that created the platform. This was called with bigglue.computations.Mapper.init or bigglue.computations.Reducer.init
- Definition Classes
- BigActorReducerPlatform → BigActorUnaryPlatform → UnaryPlatform
-
def
initConnector(conf: PipeConfig, builder: PlatformBuilder): Unit
This just initializes the connectors and connects them to the platform.
This just initializes the connectors and connects them to the platform.
- conf
The configuration file needed to initialize.
- builder
The builder that created the platform. This was called with bigglue.computations.Mapper.init or bigglue.computations.Reducer.init
- Definition Classes
- BigActorUnaryPlatform → UnaryPlatform
-
var
inputMapOpt: Option[DataStore[Input]]
- Definition Classes
- UnaryPlatform
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- var mergals: Map[Identity[Output], Seq[Input]]
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
-
var
outputMapOpt: Option[DataStore[Output]]
- Definition Classes
- UnaryPlatform
-
def
persist(): Unit
This is called by the computation's persist step.
This is called by the computation's persist step. In particular, this tends to see how much of the data set actually needs to be re-sent. As of right now, it sends the entirety of the input map to the upstream connector, which handles which things to send/re-send down the pipeline, unless there is a reducer with nothing in the input file, or if there is nothing in the output file. If there is nothing in the output file, the program assumes that a new data store has been created and sends down all data.
- Definition Classes
- UnaryPlatform → Platform
-
var
provenanceCuratorOpt: Option[ProvenanceCurator[Input, Output]]
- Definition Classes
- UnaryPlatform
-
def
run(): Unit
This gets the inputs that need to be computed with getInputs.
This gets the inputs that need to be computed with getInputs. Then, it sends that to the supervisor to compute that asynchronously. All of the workers will end up calling BigActorMapperPlatform.compute_ or BigActorReducerPlatform.compute_ when they get a job to compute depending on the computation of the platform.
- Definition Classes
- BigActorUnaryPlatform → Platform
-
def
setComputation(computation: Computation): Platform
This ends up being called in the init step of the Mapper, Reducer, or PairwiseComposer This links the actual computation and platform together.
This ends up being called in the init step of the Mapper, Reducer, or PairwiseComposer This links the actual computation and platform together.
- computation
The computation that is linked with the platform. (This is usually called through platform.setComputation(this))
- Definition Classes
- Platform
-
var
superActorOpt: Option[ActorRef]
- Definition Classes
- BigActorUnaryPlatform
-
def
supervisor: ActorRef
- Definition Classes
- BigActorUnaryPlatform
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
terminate(): Unit
This function is called when closing down the pipeline.
This function is called when closing down the pipeline. This sends a call to kill the actor system and the workers.
- Definition Classes
- BigActorUnaryPlatform → Platform
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
def
updateConfigAndGetActorNames(conf: Config, name: String): List[String]
- Definition Classes
- BigActor
-
def
updatedVersion(ver: Option[String]): Boolean
- Definition Classes
- UnaryPlatform
-
var
upstreamConnectorOpt: Option[Connector[Input]]
- Definition Classes
- UnaryPlatform
-
var
versionCuratorOpt: Option[VersionCurator[Output]]
- Definition Classes
- UnaryPlatform
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
def
wake(): Unit
This function is called when the upstream connector lets the platform know that there's data to be computed.
This function is called when the upstream connector lets the platform know that there's data to be computed. This allows the supervisor actor to start working on stuff. Without going into the messy details of the ActorSystem, this calls run.
- Definition Classes
- BigActorUnaryPlatform → Platform
-
def
→[B](y: B): (BigActorReducerPlatform[Input, Output], B)
- Implicit
- This member is added by an implicit conversion from BigActorReducerPlatform[Input, Output] to ArrowAssoc[BigActorReducerPlatform[Input, Output]] performed by method ArrowAssoc in scala.Predef.
- Definition Classes
- ArrowAssoc