case class PartialReducerPipe[Input <: Identifiable[Input], Output <: Identifiable[Output], End <: Identifiable[End]](reducer: Reducer[Input, Output], p: Pipe[Output, End]) extends PartialPipe[Input, End] with Product with Serializable
This is a pipe that has no input data store, but has a computation and an ending pipeline. This is generated with the +-> call in the pipeline. You can run this in the case of ParallelPipes, and created a sequence of pipes, but that's out of scope for this example. Within the example, we just use this as a stepping point before MapperPipe and ReducerPipe
- Input
The data type that the computation puts in. bigglue.examples.GitCommitInfo in the case of FindAuthor()+->authorMap.
- End
The type of the data store that shows up at the end of the pipeline; bigglue.examples.GitCommitGroups in this case.
- Alphabetic
- By Inheritance
- PartialReducerPipe
- Serializable
- Serializable
- Product
- Equals
- PartialPipe
- AnyRef
- Any
- by any2stringadd
- by StringFormat
- by Ensuring
- by ArrowAssoc
- Hide All
- Show All
- Public
- All
Instance Constructors
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
def
+(other: String): String
- Implicit
- This member is added by an implicit conversion from PartialReducerPipe[Input, Output, End] to any2stringadd[PartialReducerPipe[Input, Output, End]] performed by method any2stringadd in scala.Predef.
- Definition Classes
- any2stringadd
-
def
->[B](y: B): (PartialReducerPipe[Input, Output, End], B)
- Implicit
- This member is added by an implicit conversion from PartialReducerPipe[Input, Output, End] to ArrowAssoc[PartialReducerPipe[Input, Output, End]] performed by method ArrowAssoc in scala.Predef.
- Definition Classes
- ArrowAssoc
- Annotations
- @inline()
-
def
:-+[Next <: Identifiable[Next], Mid <: Identifiable[Mid]](other: PartialReducerPipe[End, Mid, Next]): PartialPipe[Input, Next]
- Definition Classes
- PartialPipe
-
def
:--[Next <: Identifiable[Next], Mid <: Identifiable[Mid]](other: PartialMapperPipe[End, Mid, Next]): PartialPipe[Input, Next]
- Definition Classes
- PartialPipe
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
check(conf: PipeConfig, input: DataStore[Input]): Unit
In basic terms, this checks to see whether the pipeline that we have created is valid.
In basic terms, this checks to see whether the pipeline that we have created is valid.
- conf
The configuration file that we are checking with.
- input
The input data store to use.
- Definition Classes
- PartialReducerPipe → PartialPipe
-
def
clone(): AnyRef
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
def
end(): DataStore[End]
This is the data store at the end of the pipeline.
This is the data store at the end of the pipeline.
- returns
The data store at the end of the pipeline.
- Definition Classes
- PartialReducerPipe → PartialPipe
-
def
ensuring(cond: (PartialReducerPipe[Input, Output, End]) ⇒ Boolean, msg: ⇒ Any): PartialReducerPipe[Input, Output, End]
- Implicit
- This member is added by an implicit conversion from PartialReducerPipe[Input, Output, End] to Ensuring[PartialReducerPipe[Input, Output, End]] performed by method Ensuring in scala.Predef.
- Definition Classes
- Ensuring
-
def
ensuring(cond: (PartialReducerPipe[Input, Output, End]) ⇒ Boolean): PartialReducerPipe[Input, Output, End]
- Implicit
- This member is added by an implicit conversion from PartialReducerPipe[Input, Output, End] to Ensuring[PartialReducerPipe[Input, Output, End]] performed by method Ensuring in scala.Predef.
- Definition Classes
- Ensuring
-
def
ensuring(cond: Boolean, msg: ⇒ Any): PartialReducerPipe[Input, Output, End]
- Implicit
- This member is added by an implicit conversion from PartialReducerPipe[Input, Output, End] to Ensuring[PartialReducerPipe[Input, Output, End]] performed by method Ensuring in scala.Predef.
- Definition Classes
- Ensuring
-
def
ensuring(cond: Boolean): PartialReducerPipe[Input, Output, End]
- Implicit
- This member is added by an implicit conversion from PartialReducerPipe[Input, Output, End] to Ensuring[PartialReducerPipe[Input, Output, End]] performed by method Ensuring in scala.Predef.
- Definition Classes
- Ensuring
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
finalize(): Unit
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
def
formatted(fmtstr: String): String
- Implicit
- This member is added by an implicit conversion from PartialReducerPipe[Input, Output, End] to StringFormat[PartialReducerPipe[Input, Output, End]] performed by method StringFormat in scala.Predef.
- Definition Classes
- StringFormat
- Annotations
- @inline()
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
-
def
init(conf: PipeConfig, input: DataStore[Input]): Unit
Probably called with input.init.
Probably called with input.init. This calls reducer.init with conf, and the data stores before and after the mapper computation. This also moves the init call along the pipeline, sending it to the parts of the pipeline its connected to.
- conf
The configuration file that we are initializing with. This ideally is the configuration file that is being used to check the pipeline.
- input
The input pipeline used to call the pipeline.
- Definition Classes
- PartialReducerPipe → PartialPipe
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- val p: Pipe[Output, End]
-
def
persist(): Unit
This calls reducer.persist, which will check to see what data to send down the pipeline (again?).
This calls reducer.persist, which will check to see what data to send down the pipeline (again?). Then, it will call p.persist, moving the run call along the pipeline.
- Definition Classes
- PartialReducerPipe → PartialPipe
- val reducer: Reducer[Input, Output]
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
terminate(): Unit
- Definition Classes
- PartialReducerPipe → PartialPipe
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
def
~[Other <: Identifiable[Other]](other: PartialPipe[Input, Other]): PartialPipe[Input, Either[End, Other]]
- Definition Classes
- PartialPipe
-
def
→[B](y: B): (PartialReducerPipe[Input, Output, End], B)
- Implicit
- This member is added by an implicit conversion from PartialReducerPipe[Input, Output, End] to ArrowAssoc[PartialReducerPipe[Input, Output, End]] performed by method ArrowAssoc in scala.Predef.
- Definition Classes
- ArrowAssoc