Qubole Check¶
Qubole is an open, simple, and secure data lake platform for machine learning, streaming and adhoc analytics. Qubole delivers a Self-Service Platform for Big Data Analytics built on Amazon Web Services, Microsoft and Google Clouds.
Airflow provides operators to execute tasks (commands) on QDS and perform checks against Qubole Commands.
Perform checks¶
To performs checks against Qubole Commands you can use
QuboleCheckOperator.
Operator expects a command that will be executed on QDS.
By default, each value on first row of the result of this Qubole Command is evaluated using python bool casting.
If any of the values return False, the check is failed and errors out.