Package com.lucidworks.apollo.pipeline

Class Pipeline<M>

java.lang.Object
com.lucidworks.apollo.pipeline.Pipeline<M>
public class Pipeline<M> extends Object
Main runtime support for a pipeline (the actual thing that runs) Controls wiring the stages together, starting, and stopping them. TODO suspending/resuming plugins for stages

  • Field Details

    • PIPELINE_EXECUTION_DURATION_SECS

      public static final io.prometheus.client.Histogram PIPELINE_EXECUTION_DURATION_SECS

    • pipelineId

      protected final String pipelineId

  • Constructor Details

    • Pipeline

      protected Pipeline(String instanceId, String pipelineId, List<com.lucidworks.apollo.pipeline.StageConfig> stageConfigs, StageFactory<M> stageFactory, com.codahale.metrics.MetricRegistry globalMetricRegistry, com.codahale.metrics.MetricRegistry metricRegistry, ConfigurationComponent configurationComponent, Map<String,String> initialProperties)

  • Method Details

    • getInstanceId

      public String getInstanceId()

    • getPipelineId

      public String getPipelineId()

    • getState

      public Pipeline.State getState()

    • getMetrics

      public com.codahale.metrics.MetricRegistry getMetrics()

    • getStartTime

      public Optional<Date> getStartTime()

    • getStages

      public List<RuntimeStage<M>> getStages()

    • getRunningTimeMs

      public long getRunningTimeMs()

    • getErrors

      public Map<String,Collection<RuntimeStage.ThrowableAndMessage<M>>> getErrors()
      Collect errors from the stages
      Returns:

    • stop

      public void stop()

    • maybeStop

      public boolean maybeStop()

    • send

      public void send(M message, com.lucidworks.apollo.pipeline.Context requestContext)

    • send

      public long send(Iterator<M> messages, com.lucidworks.apollo.pipeline.Context requestContext)
      Parameters:
      messages - - Iterator of pipeline messages
      requestContext -
      Returns:
      total number of messages from Iterator sent to a pipeline

    • sendEndOfBatch

      public void sendEndOfBatch(com.lucidworks.apollo.pipeline.Context requestContext)
      for use when messages are being sent through the non iterators method when a downstream consumer needs to know that the last pipeline message has been sent
      Parameters:
      requestContext -

    • zerothStage

      protected void zerothStage(M message, com.lucidworks.apollo.pipeline.Context requestContext)

    • newStageCallback

      protected com.lucidworks.apollo.pipeline.StageCallback<M> newStageCallback()
      Method for generating a new default StageCallback for this pipeline. Overridable by classes extending this class (IndexPipeline extends this at time of writing)

    • addStateListener

      public void addStateListener(Runnable runnable, Pipeline.State... states)

    • isInState

      public boolean isInState(Pipeline.State... states)

    • isTerminalState

      public boolean isTerminalState()