Apache Pig Operators¶

Apache Pig is a platform for analyzing large data sets that consists of a high-level language for expressing data analysis programs, coupled with infrastructure for evaluating these programs. Pig programs are amenable to substantial parallelization, which in turns enables them to handle very large data sets.

Use the PigOperator to execute a pig script.

tests/system/apache/pig/example_pig.py[source]

    run_this = PigOperator(
        task_id="run_example_pig_script",
        pig="ls /;",
        pig_opts="-x local",
    )