SFTP to Amazon S3¶
Use the SFTPToS3Operator
transfer to copy the data from a SFTP server to an Amazon Simple Storage Service (S3) file.
Prerequisite Tasks¶
To use these operators, you must do a few things:
Create necessary resources using AWS Console or AWS CLI.
Install API libraries via pip.
pip install 'apache-airflow[amazon]'Detailed information is available Installation of Airflow®
Operators¶
SFTP to Amazon S3 transfer operator¶
This operator loads data from a SFTP server to an Amazon S3 file.
To get more information about this operator visit:
SFTPToS3Operator
Example usage:
tests/system/amazon/aws/example_sftp_to_s3.py
sftp_to_s3_job = SFTPToS3Operator(
task_id="sftp_to_s3_job",
sftp_path="/tmp/sftp_path",
s3_bucket=s3_bucket,
s3_key=s3_key,
)