Package io.github.punchplatform.api.punchline.java

Class Source

java.lang.Object

io.github.punchplatform.api.punchline.Node

io.github.punchplatform.api.punchline.java.Source

public abstract class Source
extends Node

Extend this class to implement a source.

Field Summary

Fields

Modifier and Type	Field	Description
protected MetricContext	metricContext	The metric context to create gauges, histograms and counters.
protected SourceTableCollector	tableCollector	The private table collector in charge of inserting rows into the target tables.

Fields inherited from class io.github.punchplatform.api.punchline.Node

engineSettings, exitConditions, id, kind, load_control, out, punchline, settings, taskId, type

Constructor Summary

Constructors

Constructor	Description
Source()

Method Summary

Modifier and Type	Method	Description
abstract void	ack(Object ctx)	Acknowledge a row.
abstract void	fail(Object ctx)	Fail a row.
protected void	insert(Out out, List<Object> values, Object ctx)	Insert a row into an output.
protected void	insert(List<Out> outs, List<Object> values, Object ctx)	Insert a row toward several destinations.
void	setMetricContext(MetricContext metricContext)
void	setTableCollector(SourceTableCollector tableCollector)	Setter to make it clearer to source implementor to not refer directly to the collector.
abstract void	start()	Start your source.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

metricContext

protected MetricContext metricContext

The metric context to create gauges, histograms and counters.

tableCollector

protected SourceTableCollector tableCollector

The private table collector in charge of inserting rows into the target tables. Not visible from source implementations.

Constructor Detail

Source

public Source()

Method Detail

start

public abstract void start()
                    throws IOException,
                           ConfigurationException

Start your source. This method never return for streaming source, and return or exit for batch source.

This call is expected to never return. You can take ownership of the calling thread in a way you like. Whenever you return, the punchline will stop soon after.

Throws:

IOException - in case of IO error

ConfigurationException - in case of configuration error.

insert

protected void insert(Out out,
                      List<Object> values,
                      Object ctx)

Insert a row into an output. Remember an output is associated to a table.

This call is potentially blocking if the max number of inserted rows has been reached. Eventually it will return because pending rows will eventually be acked or failed.

This blocking behavior is what you want so as to slow down automatically the insertions of new rows in the punchline.

Parameters:

out - the output

values - the column values

ctx - an optional context object

insert

protected void insert(List<Out> outs,
                      List<Object> values,
                      Object ctx)

Insert a row toward several destinations.

Parameters:

outs - the outputs

values - the column values

ctx - an optional context object

ack

public abstract void ack(Object ctx)

Acknowledge a row. What you receive is an arbitrary context object provided at insertion time.

Watch out this is up called from a runtime punchline thread. Do not block it.

Parameters:

ctx - an arbitrary object provide at insert time

fail

public abstract void fail(Object ctx)

Fail a row. What you receive is an arbitrary context object provided at insertion time.

Parameters:

ctx - an arbitrary object provide at insert time

setTableCollector

public void setTableCollector(SourceTableCollector tableCollector)

Setter to make it clearer to source implementor to not refer directly to the collector.

Parameters:

tableCollector - collector to use

setMetricContext

public void setMetricContext(MetricContext metricContext)