class Reducer[Input <: Identifiable[Input], Output <: Identifiable[Output]] extends UnaryComputation[Input, Output]
The Reducer Computation, which is a subclass of UnaryComputation In short, this takes an input, groups it together with an output, and then updates the output with the input.
- Input
The type of the data that needs to be computed. In this case, this is bigglue.examples.GitCommitInfo This needs to be an Identifiable type.
- Output
The type of the data that needs to be computed. In this case, this is bigglue.examples.GitCommitGroups This needs to be an Identifiable type.
- Alphabetic
- By Inheritance
- Reducer
- UnaryComputation
- UnaryChecker
- OutputStoreChecker
- InputStoreChecker
- ConfigChecker
- Computation
- ConfigBuildsPlatform
- ConfigBuilds
- AnyRef
- Any
- by any2stringadd
- by StringFormat
- by Ensuring
- by ArrowAssoc
- Hide All
- Show All
- Public
- All
Instance Constructors
-
new
Reducer(groupBy: (Input) ⇒ Identity[Output], fold: (Input) ⇒ (Output) ⇒ Output, zero: Output)
- groupBy
This is the function that has you group an input with an output. You choose the output in this case by it's Identity. In the case of the example, this would be the i => BasicIdentity(i.author) function.
- fold
This is the function that has you update the output with the new input. In the case of the example, this would be the i => o => GitCommitGroups(i.author, o.gitCommitInfos+1) function, where i is the new input and o is the current output.
- zero
This is the default output. This is given when the groupBy function doesn't lead to an actual output, making this the output that all outputs began as. Within the example, this would be GitCommitGroups("", 0).
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
- def +(other: String): String
-
def
+->[End <: Identifiable[End]](p: Pipe[Output, End]): PartialReducerPipe[Input, Output, End]
This is part of the series of calls that make the pipeline.
This is part of the series of calls that make the pipeline. This is called within the FindAuthor()+->authorMap step. This links the reducer to the rest of the pipeline, which in this case is bigglue.pipes.Implicits.DataNode(authorMap).
- End
The type of the final data store within the pipeline. Within the example, this would be bigglue.examples.GitCommitGroups.
- p
The part of the pipeline that follows the reducer computation. In the example, this is simply authorMap.
- returns
This returns a section of the pipe where the reducer is linked together with the pipe that starts with the output data store.
- def ->[B](y: B): (Reducer[Input, Output], B)
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
check(conf: PipeConfig, inputMap: DataStore[Input], outputMap: DataStore[Output]): Unit
Check routine for unary data processors
Check routine for unary data processors
- conf
the config file to check
- inputMap
the input store to check
- outputMap
the output store to check
- Definition Classes
- UnaryChecker
-
def
checkConfig(conf: PipeConfig): Unit
Check that paths in configuration file relevant to host class are well-formed.
Check that paths in configuration file relevant to host class are well-formed. Default does nothing.
- conf
the config file to check
- Definition Classes
- ConfigChecker
-
def
checkInput(inputMap: DataStore[Input]): Unit
Check that input store is compatible with the host class.
Check that input store is compatible with the host class. Default does nothing.
- inputMap
the input store to check
- Definition Classes
- InputStoreChecker
-
def
checkOutput(outputMap: DataStore[Output]): Unit
This allows us to type check the Reducer step; In order for it to work, the output needs to be a DataMap, so we need to make sure that it is before continuing.
This allows us to type check the Reducer step; In order for it to work, the output needs to be a DataMap, so we need to make sure that it is before continuing.
- outputMap
The Output Store to Check
- Definition Classes
- Reducer → OutputStoreChecker
-
def
clone(): AnyRef
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
val
configOpt: Option[PipeConfig]
- Definition Classes
- ConfigBuilds
-
var
configOption: ConfOpt
- Definition Classes
- Computation
-
def
constructBuilder(config: PipeConfig = null): PlatformBuilder
- Definition Classes
- ConfigBuildsPlatform → ConfigBuilds
- def ensuring(cond: (Reducer[Input, Output]) ⇒ Boolean, msg: ⇒ Any): Reducer[Input, Output]
- def ensuring(cond: (Reducer[Input, Output]) ⇒ Boolean): Reducer[Input, Output]
- def ensuring(cond: Boolean, msg: ⇒ Any): Reducer[Input, Output]
- def ensuring(cond: Boolean): Reducer[Input, Output]
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
- def formatted(fmtstr: String): String
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
-
def
getConfig(config: PipeConfig): PipeConfig
- Definition Classes
- ConfigBuilds
-
def
getUnaryPlatform(): UnaryPlatform[Input, Output]
- Definition Classes
- UnaryComputation
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
-
def
init(conf: PipeConfig, inputMap: DataStore[Input], outputMap: DataStore[Output]): Unit
This initializes the Reducer Computation.
This initializes the Reducer Computation. It creates a UnaryPlatform with PlatformBuilder.reducerPlatform, then sets itself as the platform's computation, and then initializes the platform with UnaryPlatform.init.
- conf
The configuration file to build from. Note: If there is a GitCommitInfo+->GitAuthors section in the bigglue section of the configuration file, it will overwrite part of the configuration file with the values within the step's section.
- inputMap
The DataStore that data is being sent in from. In the example, this would be commitInfoMap, which is implemented with a bigglue.store.instances.solr.SolrDataMap.
- outputMap
The DataStore that data is being sent to and updated from. In the example, this would be authorMap, which is implemented with a bigglue.store.instances.solr.SolrDataMap.
-
def
init(conf: PipeConfig, inputMap: DataStore[Input], outputMap: DataStore[Output], platform: UnaryPlatform[Input, Output]): Unit
This connects the newly initialized platform to the Input Data Store by making it so the Input Data Store sends data down to the platform.
This connects the newly initialized platform to the Input Data Store by making it so the Input Data Store sends data down to the platform.
- conf
The configuration file.
- inputMap
The Input Data Store
- outputMap
The Output Data Store
- platform
- Definition Classes
- UnaryComputation
-
def
init(config: PipeConfig, platform: Platform): Unit
- Definition Classes
- Computation
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
def
name(newName: String): UnaryComputation[Input, Output]
- Definition Classes
- UnaryComputation
-
var
name: String
- Definition Classes
- UnaryComputation
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
-
def
persist(): Unit
If the computation has been initialized, it calls Platform.persist on the platform.
If the computation has been initialized, it calls Platform.persist on the platform. This gives the platform the responsibility of what data to send/resend down the pipeline.
- Definition Classes
- Computation
-
var
platformOpt: Option[Platform]
- Definition Classes
- Computation
-
def
run(): Unit
- Definition Classes
- Reducer → Computation
-
def
setConfig(pipeConfig: PipeConfig): Unit
- Definition Classes
- ConfigBuilds
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
terminate(): Unit
- Definition Classes
- Computation
-
def
toStep(conf: PipeConfig, step: String): PipeConfig
- Definition Classes
- Computation
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
def
tryFold(input: Input, output: Output): Option[Output]
Attempts to call fold to get a new output for the Output Data Store If it fails, just returns None.
Attempts to call fold to get a new output for the Output Data Store If it fails, just returns None.
- input
The input document to update the output.
- output
The output document that needs to be updated.
- returns
A new output document based on input and output. Returns None if failed.
-
def
tryGroupBy(input: Input): Option[Identity[Output]]
Attempts to call groupBy to get an Identity to get stuff to group by.
Attempts to call groupBy to get an Identity to get stuff to group by. If that fails, then it just returns None.
- input
The input document to find an output Identity for.
- returns
The output identity to change; None if none.
-
def
tryZero(): Option[Output]
Attempts to call zero to get a starting output set.
Attempts to call zero to get a starting output set.
- returns
The default output. In the case of the example, this would be Some(Counter(0)). Returns None if failed to get a default output.
-
var
unaryPlatformOpt: Option[UnaryPlatform[Input, Output]]
- Definition Classes
- UnaryComputation
-
val
versionOpt: Option[String]
- Definition Classes
- Computation
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
- def withConfig(newConfigOption: ConfOpt): Reducer[Input, Output]
- def →[B](y: B): (Reducer[Input, Output], B)