airflow.providers.google.cloud.transfers.mssql_to_gcs

MsSQL to GCS operator.

Module Contents

Classes

MSSQLToGCSOperator

Copy data from Microsoft SQL Server to Google Cloud Storage in JSON, CSV or Parquet format.
class airflow.providers.google.cloud.transfers.mssql_to_gcs.MSSQLToGCSOperator(*, bit_fields=None, mssql_conn_id='mssql_default', **kwargs)[source]

Bases: airflow.providers.google.cloud.transfers.sql_to_gcs.BaseSQLToGCSOperator

Copy data from Microsoft SQL Server to Google Cloud Storage in JSON, CSV or Parquet format.

Parameters

  • bit_fields (Sequence[str] | None) – Sequence of fields names of MSSQL “BIT” data type, to be interpreted in the schema as “BOOLEAN”. “BIT” fields that won’t be included in this sequence, will be interpreted as “INTEGER” by default.

  • mssql_conn_id – Reference to a specific MSSQL hook.

Example:

The following operator will export data from the Customers table within the given MSSQL Database and then upload it to the ‘mssql-export’ GCS bucket (along with a schema file).

export_customers = MSSQLToGCSOperator(
    task_id="export_customers",
    sql="SELECT * FROM dbo.Customers;",
    bit_fields=["some_bit_field", "another_bit_field"],
    bucket="mssql-export",
    filename="data/customers/export.json",
    schema_filename="schemas/export.json",
    mssql_conn_id="mssql_default",
    gcp_conn_id="google_cloud_default",
    dag=dag,
)

See also

For more information on how to use this operator, take a look at the guide: MSSQLToGCSOperator

ui_color = '#e0a98c'[source]
type_map[source]
query()[source]

Query MSSQL and returns a cursor of results.

Returns

mssql cursor

field_to_bigquery(field)[source]

Convert a DBAPI field to BigQuery schema format.

classmethod convert_type(value, schema_type, **kwargs)[source]

Take a value from MSSQL and convert it to a value safe for JSON/Google Cloud Storage/BigQuery.

Datetime, Date and Time are converted to ISO formatted strings.

Was this entry helpful?