Package com.lucidworks.fusion.connector.plugin.api.fetcher

Interface Fetcher<StartContextT extends FetcherContext.StartContext,FetchContextT extends FetcherContext.FetchContext,StopContextT extends FetcherContext.StopContext>

All Superinterfaces:
Component
All Known Subinterfaces:
ContentFetcher
public interface Fetcher<StartContextT extends FetcherContext.StartContext,FetchContextT extends FetcherContext.FetchContext,StopContextT extends FetcherContext.StopContext> extends Component
The Fetcher is one of the primary interfaces within the Lucidworks Connector SDK. As its name implies, it is responsible for fetching data. Within this interface, there are a variety of methods that can emit data, and all of this is done asynchronously. The Fetcher also follows an explicit lifecycle, which is executed by a FetcherController. The lifecycle is as follows:

start() for-each-phase: startFetcher() for-each-fetch-input: fetch() end-for-each-fetch-input end-for-each-phase stop()

Phases allow a Connector to define processing steps. For example, PhaseA might be responsible for fetching content IDs from a large content repository. PhaseB might then handle fetching the actual binary content for the items emitted from PhaseA. By breaking up the overall fetch into multiple Phases, more flexible crawling strategies are possible.

  • Method Details

    • start

      default StartResult start(StartContextT context)

    • fetch

      default FetchResult fetch(FetchContextT context)
      This is the primary fetch method for a crawler-like connector (for example, a file system or web connector). Messages emitted here can include Documents, which are indexed directly into the associated content collection. There are several other types of messages;
      Parameters:
      context - the Fetch context
      Returns:
      The fetch result

    • stop

      default StopResult stop(StopContextT context)