airflow.hooks.dbapi

Module Contents

class airflow.hooks.dbapi.ConnectorProtocol[source]

Bases: airflow.typing_compat.Protocol

A protocol where you can connect to a database.

connect(self, host: str, port: int, username: str, schema: str)[source]

Connect to a database.

Parameters

  • host – The database host to connect to.

  • port – The database port to connect to.

  • username – The database username used for the authentication.

  • schema – The database schema to connect to.

Returns

the authorized connection object.

class airflow.hooks.dbapi.DbApiHook(*args, **kwargs)[source]

Bases: airflow.hooks.base.BaseHook

Abstract base class for sql hooks.

conn_name_attr :str[source]
default_conn_name = default_conn_id[source]
supports_autocommit = False[source]
connector :Optional[ConnectorProtocol][source]
get_conn(self)[source]

Returns a connection object

get_uri(self)[source]

Extract the URI from the connection.

Returns

the extracted uri.

get_sqlalchemy_engine(self, engine_kwargs=None)[source]

Get an sqlalchemy_engine object.

Parameters

engine_kwargs – Kwargs used in create_engine().

Returns

the created engine.

get_pandas_df(self, sql, parameters=None, **kwargs)[source]

Executes the sql and returns a pandas dataframe

Parameters

  • sql (str or list) – the sql statement to be executed (str) or a list of sql statements to execute

  • parameters (dict or iterable) – The parameters to render the SQL query with.

  • kwargs (dict) – (optional) passed into pandas.io.sql.read_sql method

get_records(self, sql, parameters=None)[source]

Executes the sql and returns a set of records.

Parameters

  • sql (str or list) – the sql statement to be executed (str) or a list of sql statements to execute

  • parameters (dict or iterable) – The parameters to render the SQL query with.

get_first(self, sql, parameters=None)[source]

Executes the sql and returns the first resulting row.

Parameters

  • sql (str or list) – the sql statement to be executed (str) or a list of sql statements to execute

  • parameters (dict or iterable) – The parameters to render the SQL query with.

run(self, sql, autocommit=False, parameters=None)[source]

Runs a command or a list of commands. Pass a list of sql statements to the sql parameter to get them to execute sequentially

Parameters

  • sql (str or list) – the sql statement to be executed (str) or a list of sql statements to execute

  • autocommit (bool) – What to set the connection’s autocommit setting to before executing the query.

  • parameters (dict or iterable) – The parameters to render the SQL query with.

set_autocommit(self, conn, autocommit)[source]

Sets the autocommit flag on the connection

get_autocommit(self, conn)[source]

Get autocommit setting for the provided connection. Return True if conn.autocommit is set to True. Return False if conn.autocommit is not set or set to False or conn does not support autocommit.

Parameters

conn (connection object.) – Connection to get autocommit setting from.

Returns

connection autocommit setting.

Return type

bool

get_cursor(self)[source]

Returns a cursor

static _generate_insert_sql(table, values, target_fields, replace, **kwargs)[source]

Static helper method that generate the INSERT SQL statement. The REPLACE variant is specific to MySQL syntax.

Parameters

  • table (str) – Name of the target table

  • values (tuple of cell values) – The row to insert into the table

  • target_fields (iterable of strings) – The names of the columns to fill in the table

  • replace (bool) – Whether to replace instead of insert

Returns

The generated INSERT or REPLACE SQL statement

Return type

str

insert_rows(self, table, rows, target_fields=None, commit_every=1000, replace=False, **kwargs)[source]

A generic way to insert a set of tuples into a table, a new transaction is created every commit_every rows

Parameters

  • table (str) – Name of the target table

  • rows (iterable of tuples) – The rows to insert into the table

  • target_fields (iterable of strings) – The names of the columns to fill in the table

  • commit_every (int) – The maximum number of rows to insert in one transaction. Set to 0 to insert all rows in one transaction.

  • replace (bool) – Whether to replace instead of insert

static _serialize_cell(cell, conn=None)[source]

Returns the SQL literal of the cell as a string.

Parameters

  • cell (object) – The cell to insert into the table

  • conn (connection object) – The database connection

Returns

The serialized cell

Return type

str

bulk_dump(self, table, tmp_file)[source]

Dumps a database table into a tab-delimited file

Parameters

  • table (str) – The name of the source table

  • tmp_file (str) – The path of the target file

bulk_load(self, table, tmp_file)[source]

Loads a tab-delimited file into a database table

Parameters

  • table (str) – The name of the target table

  • tmp_file (str) – The path of the file to load into the table

Was this entry helpful?