apache-airflow-providers-apache-spark

Package apache-airflow-providers-apache-spark

Apache Spark

Release: 4.7.1

Provider package

This is a provider package for apache.spark provider. All classes for this provider package are in airflow.providers.apache.spark python package.

Installation

You can install this package on top of an existing Airflow 2 installation (see Requirements below) for the minimum Airflow version supported) via pip install apache-airflow-providers-apache-spark

Requirements

The minimum Apache Airflow version supported by this provider package is 2.6.0.

PIP package

Version required

apache-airflow

>=2.6.0

pyspark

grpcio-status

>=1.59.0

Cross provider package dependencies

Those are dependencies that might be needed in order to use all the features of the package. You need to install the specified provider packages in order to use them.

You can install such cross-provider dependencies when installing from PyPI. For example:

pip install apache-airflow-providers-apache-spark[cncf.kubernetes]

Dependent package

Extra

apache-airflow-providers-cncf-kubernetes

cncf.kubernetes

Downloading official packages

You can download officially released packages and verify their checksums and signatures from the Official Apache Download site

Was this entry helpful?