abstract class Pipe[Head <: Identifiable[Head], End <: Identifiable[End]] extends AnyRef

This is the superclass to MapperPipe and ReducerPipe. See either one of those for further details into what the pipelines end up doing.

Linear Supertypes
AnyRef, Any
Known Subclasses
CompositionPipe, DataNode, TuplePipe, JunctionPipe, MapperPipe, ReducerPipe, St
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Pipe
  2. AnyRef
  3. Any
Implicitly
  1. by any2stringadd
  2. by StringFormat
  3. by Ensuring
  4. by ArrowAssoc
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new Pipe()

Abstract Value Members

  1. abstract def check(conf: PipeConfig): Unit

    This is called within the example with pipe.check(conf).

    This is called within the example with pipe.check(conf). In basic terms, this checks to see whether the pipeline that we have created is valid. See MapperPipe.check() or ReducerPipe.check()

    conf

    The configuration file that we are checking with.

  2. abstract def end(): DataStore[End]

    This is the data store at the end of the pipeline.

    This is the data store at the end of the pipeline.

    returns

    The data store at the end of the pipeline. In all cases, this is authorMap in the example.

  3. abstract def head(): DataStore[Head]

    This is the data store at the start of the pipeline.

    This is the data store at the start of the pipeline.

    returns

    The data store at the start of the pipeline.

  4. abstract def init(conf: PipeConfig): Unit

    This is called with the example with pipe.init(conf).

    This is called with the example with pipe.init(conf). This initializes the pipeline and all of the parts within it. See MapperPipe.init() or ReducerPipe.init()

    conf

    The configuration file that we are initializing with. This ideally is the configuration file that is being used to check the pipeline.

  5. abstract def terminate(): Unit

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. def +(other: String): String
    Implicit
    This member is added by an implicit conversion from Pipe[Head, End] to any2stringadd[Pipe[Head, End]] performed by method any2stringadd in scala.Predef.
    Definition Classes
    any2stringadd
  4. def ->[B](y: B): (Pipe[Head, End], B)
    Implicit
    This member is added by an implicit conversion from Pipe[Head, End] to ArrowAssoc[Pipe[Head, End]] performed by method ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc
    Annotations
    @inline()
  5. def :-+[Next <: Identifiable[Next], Mid <: Identifiable[Mid]](headReducer: PartialReducerPipe[End, Mid, Next]): Pipe[Head, Next]

    This connects the Reducer to the input store, connecting it completely.

    This connects the Reducer to the input store, connecting it completely. This is expected to be called after a bigglue.pipes.Implicits.DataNode, and can be seen after gitID and clonedMap.

    Next

    The type at the end of the new pipeline.

    Mid

    The type that the mapper ends up outputting given the input End.

    headReducer

    The reducer to connect to the input store.

    returns

    A connected ReducerPipe that connects this pipeline to the reducer.

  6. def :--[Next <: Identifiable[Next], Mid <: Identifiable[Mid]](headMapper: PartialMapperPipe[End, Mid, Next]): Pipe[Head, Next]

    This connects the Mapper to the input store, connecting it completely.

    This connects the Mapper to the input store, connecting it completely. This is expected to be called after a bigglue.pipes.Implicits.DataNode, and can be seen after gitID and clonedMap.

    Next

    The type at the end of the new pipeline.

    Mid

    The type that the mapper ends up outputting given the input End.

    headMapper

    The mapper to connect to the input store.

    returns

    A connected MapperPipe that connects this pipeline to the mapper.

  7. def :<[Next <: Identifiable[Next]](parPipes: PartialPipe[End, Next]): Pipe[Head, Next]
  8. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  9. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  10. def clone(): AnyRef
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  11. def ensuring(cond: (Pipe[Head, End]) ⇒ Boolean, msg: ⇒ Any): Pipe[Head, End]
    Implicit
    This member is added by an implicit conversion from Pipe[Head, End] to Ensuring[Pipe[Head, End]] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  12. def ensuring(cond: (Pipe[Head, End]) ⇒ Boolean): Pipe[Head, End]
    Implicit
    This member is added by an implicit conversion from Pipe[Head, End] to Ensuring[Pipe[Head, End]] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  13. def ensuring(cond: Boolean, msg: ⇒ Any): Pipe[Head, End]
    Implicit
    This member is added by an implicit conversion from Pipe[Head, End] to Ensuring[Pipe[Head, End]] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  14. def ensuring(cond: Boolean): Pipe[Head, End]
    Implicit
    This member is added by an implicit conversion from Pipe[Head, End] to Ensuring[Pipe[Head, End]] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  15. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  16. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  17. def finalize(): Unit
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  18. def formatted(fmtstr: String): String
    Implicit
    This member is added by an implicit conversion from Pipe[Head, End] to StringFormat[Pipe[Head, End]] performed by method StringFormat in scala.Predef.
    Definition Classes
    StringFormat
    Annotations
    @inline()
  19. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
  20. def hashCode(): Int
    Definition Classes
    AnyRef → Any
  21. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  22. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  23. final def notify(): Unit
    Definition Classes
    AnyRef
  24. final def notifyAll(): Unit
    Definition Classes
    AnyRef
  25. def persist(): Unit

    This starts/restarts the pipeline.

    This starts/restarts the pipeline. See MapperPipe.persist() or ReducerPipe.persist()

  26. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  27. def toString(): String
    Definition Classes
    AnyRef → Any
  28. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  29. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  30. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  31. def ||[OHead <: Identifiable[OHead], OEnd <: Identifiable[OEnd]](o: Pipe[OHead, OEnd]): PairPipes[Head, OHead, End, OEnd]
  32. def →[B](y: B): (Pipe[Head, End], B)
    Implicit
    This member is added by an implicit conversion from Pipe[Head, End] to ArrowAssoc[Pipe[Head, End]] performed by method ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc

Inherited from AnyRef

Inherited from Any

Inherited by implicit conversion any2stringadd from Pipe[Head, End] to any2stringadd[Pipe[Head, End]]

Inherited by implicit conversion StringFormat from Pipe[Head, End] to StringFormat[Pipe[Head, End]]

Inherited by implicit conversion Ensuring from Pipe[Head, End] to Ensuring[Pipe[Head, End]]

Inherited by implicit conversion ArrowAssoc from Pipe[Head, End] to ArrowAssoc[Pipe[Head, End]]

Ungrouped