airflow.providers.apache.pig.hooks.pig

Module Contents

Classes

PigCliHook

Simple wrapper around the pig CLI.
class airflow.providers.apache.pig.hooks.pig.PigCliHook(pig_cli_conn_id=default_conn_name)[source]

Bases: airflow.hooks.base.BaseHook

Simple wrapper around the pig CLI.

Note that you can also set default pig CLI properties using the pig_properties to be used in your connection as in {"pig_properties": "-Dpig.tmpfilecompression=true"}

conn_name_attr = pig_cli_conn_id[source]
default_conn_name = pig_cli_default[source]
conn_type = pig_cli[source]
hook_name = Pig Client Wrapper[source]
run_cli(self, pig, pig_opts=None, verbose=True)[source]

Run an pig script using the pig cli

>>> ph = PigCliHook()
>>> result = ph.run_cli("ls /;", pig_opts="-x mapreduce")
>>> ("hdfs://" in result)
True
kill(self)[source]

Kill Pig job

Was this entry helpful?