Google Compute Engine SSH Operators¶

Prerequisite Tasks¶

To use these operators, you must do a few things:

Select or create a Cloud Platform project using the Cloud Console.

Enable billing for your project, as described in the Google Cloud documentation.

Enable the API, as described in the Cloud Console documentation.
Install API libraries via pip.
pip install 'apache-airflow[google]'
Detailed information is available for Installation.
Setup a Google Cloud Connection.

ComputeEngineRemoteInstanceSSHOperator¶

Use the SSHOperator together with ComputeEngineSSHHook to execute a command on a remote instance.

This operator uses either the Cloud OS Login or instance metadata to manage SSH keys. To use Cloud OS Login, the service account must have compute.osAdminLogin IAM roles and the instance metadata must have Cloud OS Login enabled. This can be done by setting the instance metadata - enable-oslogin=TRUE

To use instance metadata, make sure to set the Cloud OS Login argument to False in the hook.

Please note that the target instance must allow tcp traffic on port 22.

Below is the code to create the operator:

airflow/providers/google/cloud/example_dags/example_compute_ssh.py[source]

os_login_without_iap_tunnel = SSHOperator(
    task_id="os_login_without_iap_tunnel",
    ssh_hook=ComputeEngineSSHHook(
        instance_name=GCE_INSTANCE,
        zone=GCE_ZONE,
        project_id=GCP_PROJECT_ID,
        use_oslogin=True,
        use_iap_tunnel=False,
    ),
    command="echo os_login_without_iap_tunnel",
)

You can also create the hook without project id - project id will be retrieved from the Google credentials used:

airflow/providers/google/cloud/example_dags/example_compute_ssh.py[source]

metadata_without_iap_tunnel = SSHOperator(
    task_id="metadata_without_iap_tunnel",
    ssh_hook=ComputeEngineSSHHook(
        instance_name=GCE_INSTANCE,
        zone=GCE_ZONE,
        use_oslogin=False,
        use_iap_tunnel=False,
    ),
    command="echo metadata_without_iap_tunnel",
)

More information¶

See Google Compute Engine API documentation and Cloud OS Login API documentation * Google Cloud API Documentation * Google Cloud OS Login API Documentation.