airflow.providers.apache.druid.operators.druid

Module Contents

Classes

DruidOperator

Allows to submit a task directly to druid.
class airflow.providers.apache.druid.operators.druid.DruidOperator(*, json_index_file, druid_ingest_conn_id='druid_ingest_default', timeout=1, max_ingestion_time=None, ingestion_type=IngestionType.BATCH, verify_ssl=True, **kwargs)[source]

Bases: airflow.models.BaseOperator

Allows to submit a task directly to druid.

Parameters

  • json_index_file (str) – The filepath to the druid index specification

  • druid_ingest_conn_id (str) – The connection id of the Druid overlord which accepts index jobs

  • timeout (int) – The interval (in seconds) between polling the Druid job for the status of the ingestion job. Must be greater than or equal to 1

  • max_ingestion_time (int | None) – The maximum ingestion time before assuming the job failed

  • ingestion_type (airflow.providers.apache.druid.hooks.druid.IngestionType) – The ingestion type of the job. Could be IngestionType.Batch or IngestionType.MSQ

  • verify_ssl (bool) – Whether to use SSL encryption to submit indexing job. If set to False then checks connection information for path to a CA bundle to use. Defaults to True

template_fields: Sequence[str] = ('json_index_file',)[source]
template_ext: Sequence[str] = ('.json',)[source]
template_fields_renderers[source]
execute(context)[source]

Derive when creating an operator.

Context is the same dictionary used as when rendering jinja templates.

Refer to get_template_context for more context.

Was this entry helpful?