FTP to Amazon S3¶
Use the FTPToS3Operator
transfer to copy data from a FTP server to an Amazon Simple Storage Service (S3) file.
Prerequisite Tasks¶
To use these operators, you must do a few things:
Create necessary resources using AWS Console or AWS CLI.
Install API libraries via pip.
pip install 'apache-airflow[amazon]'Detailed information is available Installation of Airflow®
Operators¶
FTP to Amazon S3 transfer operator¶
This operator copies data from a FTP server to an Amazon S3 file.
To get more information about this operator visit:
FTPToS3Operator
Example usage:
tests/system/amazon/aws/example_ftp_to_s3.py
ftp_to_s3_task = FTPToS3Operator(
task_id="ftp_to_s3_task",
ftp_path="/tmp/ftp_path",
s3_bucket=s3_bucket,
s3_key=s3_key,
replace=True,
)