Packages

c

bigglue.platforms

UnaryPlatform

abstract class UnaryPlatform[Input <: Identifiable[Input], Output <: Identifiable[Output]] extends Platform with UnaryChecker[Input, Output]

This is a type of platform that only deals with one input. These have a computation of either Mapper or Reducer. Within the example, we use the default of bigglue.platforms.instances.bigactors.BigActorMapperPlatform and bigglue.platforms.instances.bigactors.BigActorReducerPlatform.

Input

The type of the data that's being sent in.

Output

The type of the data that's being sent out.

Linear Supertypes
UnaryChecker[Input, Output], OutputStoreChecker[Output], InputStoreChecker[Input], ConfigChecker, Platform, AnyRef, Any
Known Subclasses
MapperPlatform, BigActorMapperPlatform, BigActorReducerPlatform, BigActorUnaryPlatform, KafkaStreamMapperPlatform, KafkaStreamPlatform, ThinActorMapperPlatform, ThinActorReducerPlatform, ThinActorUnaryPlatform
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. UnaryPlatform
  2. UnaryChecker
  3. OutputStoreChecker
  4. InputStoreChecker
  5. ConfigChecker
  6. Platform
  7. AnyRef
  8. Any
Implicitly
  1. by any2stringadd
  2. by StringFormat
  3. by Ensuring
  4. by ArrowAssoc
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new UnaryPlatform()

Abstract Value Members

  1. abstract def terminate(): Unit
    Definition Classes
    Platform
  2. abstract def wake(): Unit

    This function is called when the upstream connector lets the platform know that there's data to be computed.

    This function is called when the upstream connector lets the platform know that there's data to be computed.

    Definition Classes
    Platform

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. def +(other: String): String
    Implicit
    This member is added by an implicit conversion from UnaryPlatform[Input, Output] to any2stringadd[UnaryPlatform[Input, Output]] performed by method any2stringadd in scala.Predef.
    Definition Classes
    any2stringadd
  4. def ->[B](y: B): (UnaryPlatform[Input, Output], B)
    Implicit
    This member is added by an implicit conversion from UnaryPlatform[Input, Output] to ArrowAssoc[UnaryPlatform[Input, Output]] performed by method ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc
    Annotations
    @inline()
  5. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  6. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  7. def check(conf: PipeConfig, inputMap: DataStore[Input], outputMap: DataStore[Output]): Unit

    Check routine for unary data processors

    Check routine for unary data processors

    conf

    the config file to check

    inputMap

    the input store to check

    outputMap

    the output store to check

    Definition Classes
    UnaryChecker
  8. def checkConfig(conf: PipeConfig): Unit

    Check that paths in configuration file relevant to host class are well-formed.

    Check that paths in configuration file relevant to host class are well-formed. Default does nothing.

    conf

    the config file to check

    Definition Classes
    ConfigChecker
  9. def checkInput(inputMap: DataStore[Input]): Unit

    Check that input store is compatible with the host class.

    Check that input store is compatible with the host class. Default does nothing.

    inputMap

    the input store to check

    Definition Classes
    InputStoreChecker
  10. def checkOutput(outputMap: DataStore[Output]): Unit

    Check that output store is compatible with the host class.

    Check that output store is compatible with the host class. Default does nothing.

    outputMap

    the output store to check

    Definition Classes
    OutputStoreChecker
  11. def clone(): AnyRef
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  12. var computationOpt: Option[Computation]

    What computation this platform ends up running.

    What computation this platform ends up running. (Mapper, Reducer, PairwiseComposer) This ends up being set in init() by both UnaryPlatform and BinaryPlatform.

    Definition Classes
    Platform
  13. def ensuring(cond: (UnaryPlatform[Input, Output]) ⇒ Boolean, msg: ⇒ Any): UnaryPlatform[Input, Output]
    Implicit
    This member is added by an implicit conversion from UnaryPlatform[Input, Output] to Ensuring[UnaryPlatform[Input, Output]] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  14. def ensuring(cond: (UnaryPlatform[Input, Output]) ⇒ Boolean): UnaryPlatform[Input, Output]
    Implicit
    This member is added by an implicit conversion from UnaryPlatform[Input, Output] to Ensuring[UnaryPlatform[Input, Output]] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  15. def ensuring(cond: Boolean, msg: ⇒ Any): UnaryPlatform[Input, Output]
    Implicit
    This member is added by an implicit conversion from UnaryPlatform[Input, Output] to Ensuring[UnaryPlatform[Input, Output]] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  16. def ensuring(cond: Boolean): UnaryPlatform[Input, Output]
    Implicit
    This member is added by an implicit conversion from UnaryPlatform[Input, Output] to Ensuring[UnaryPlatform[Input, Output]] performed by method Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  17. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  18. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  19. var errorCuratorOpt: Option[ErrorCurator[Input]]
  20. def finalize(): Unit
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  21. def formatted(fmtstr: String): String
    Implicit
    This member is added by an implicit conversion from UnaryPlatform[Input, Output] to StringFormat[UnaryPlatform[Input, Output]] performed by method StringFormat in scala.Predef.
    Definition Classes
    StringFormat
    Annotations
    @inline()
  22. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
  23. def getErrorCurator(): ErrorCurator[Input]
  24. def getInputMap(): DataStore[Input]
  25. def getInputs(): Seq[Input]
  26. def getOutputMap(): DataStore[Output]
  27. def getProvenanceCurator(): ProvenanceCurator[Input, Output]
  28. def getUpstreamConnector(): Connector[Input]
  29. def getVersionCurator(): VersionCurator[Output]
  30. def hashCode(): Int
    Definition Classes
    AnyRef → Any
  31. def init(conf: PipeConfig, inputMap: DataStore[Input], outputMap: DataStore[Output], builder: PlatformBuilder): Unit

    This sets up the platform by connecting the Input Map and Output Map to the platform, as well as any other initialization that needs to be done.

    This sets up the platform by connecting the Input Map and Output Map to the platform, as well as any other initialization that needs to be done. In the case of the example, we use bigglue.platforms.instances.bigactors.BigActorUnaryPlatform.init This calls initConnector as well.

    conf

    The configuration file needed to initialize.

    inputMap

    The Map that data gets sent in from.

    outputMap

    The Map that data gets sent out to.

    builder

    The builder that created the platform. This was called with bigglue.computations.Mapper.init or bigglue.computations.Reducer.init

  32. def initConnector(conf: PipeConfig, builder: PlatformBuilder): Unit

    This sets up the connector that sends data down the pipeline.

    This sets up the connector that sends data down the pipeline. This initializes the connector with Connector.init, and then adds the platform to the connector with Connector.registerPlatform. As a default, we use bigglue.connectors.instances.IncrTrackerJobQueue.

    conf

    The configuration file needed to initialize.

    builder

    The builder that created the platform. This was called with bigglue.computations.Mapper.init or bigglue.computations.Reducer.init

  33. var inputMapOpt: Option[DataStore[Input]]
  34. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  35. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  36. final def notify(): Unit
    Definition Classes
    AnyRef
  37. final def notifyAll(): Unit
    Definition Classes
    AnyRef
  38. var outputMapOpt: Option[DataStore[Output]]
  39. def persist(): Unit

    This is called by the computation's persist step.

    This is called by the computation's persist step. In particular, this tends to see how much of the data set actually needs to be re-sent. As of right now, it sends the entirety of the input map to the upstream connector, which handles which things to send/re-send down the pipeline, unless there is a reducer with nothing in the input file, or if there is nothing in the output file. If there is nothing in the output file, the program assumes that a new data store has been created and sends down all data.

    Definition Classes
    UnaryPlatform → Platform
  40. var provenanceCuratorOpt: Option[ProvenanceCurator[Input, Output]]
  41. def run(): Unit
    Definition Classes
    Platform
  42. def setComputation(computation: Computation): Platform

    This ends up being called in the init step of the Mapper, Reducer, or PairwiseComposer This links the actual computation and platform together.

    This ends up being called in the init step of the Mapper, Reducer, or PairwiseComposer This links the actual computation and platform together.

    computation

    The computation that is linked with the platform. (This is usually called through platform.setComputation(this))

    Definition Classes
    Platform
  43. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  44. def toString(): String
    Definition Classes
    AnyRef → Any
  45. def updatedVersion(ver: Option[String]): Boolean
  46. var upstreamConnectorOpt: Option[Connector[Input]]
  47. var versionCuratorOpt: Option[VersionCurator[Output]]
  48. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  49. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  50. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  51. def →[B](y: B): (UnaryPlatform[Input, Output], B)
    Implicit
    This member is added by an implicit conversion from UnaryPlatform[Input, Output] to ArrowAssoc[UnaryPlatform[Input, Output]] performed by method ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc

Inherited from UnaryChecker[Input, Output]

Inherited from OutputStoreChecker[Output]

Inherited from InputStoreChecker[Input]

Inherited from ConfigChecker

Inherited from Platform

Inherited from AnyRef

Inherited from Any

Inherited by implicit conversion any2stringadd from UnaryPlatform[Input, Output] to any2stringadd[UnaryPlatform[Input, Output]]

Inherited by implicit conversion StringFormat from UnaryPlatform[Input, Output] to StringFormat[UnaryPlatform[Input, Output]]

Inherited by implicit conversion Ensuring from UnaryPlatform[Input, Output] to Ensuring[UnaryPlatform[Input, Output]]

Inherited by implicit conversion ArrowAssoc from UnaryPlatform[Input, Output] to ArrowAssoc[UnaryPlatform[Input, Output]]

Ungrouped