airflow.contrib.hooks.mongo_hook

Module Contents

class airflow.contrib.hooks.mongo_hook.MongoHook(conn_id='mongo_default', *args, **kwargs)[source]

Bases:airflow.hooks.base_hook.BaseHook

PyMongo Wrapper to Interact With Mongo Database Mongo Connection Documentation https://docs.mongodb.com/manual/reference/connection-string/index.html You can specify connection string options in extra field of your connection https://docs.mongodb.com/manual/reference/connection-string/index.html#connection-string-options ex. {replicaSet: test, ssl: True, connectTimeoutMS: 30000}

conn_type = mongo[source]
__enter__(self)[source]
__exit__(self, exc_type, exc_val, exc_tb)[source]
get_conn(self)[source]

Fetches PyMongo Client

close_conn(self)[source]
get_collection(self, mongo_collection, mongo_db=None)[source]

Fetches a mongo collection object for querying.

Uses connection schema as DB unless specified.

aggregate(self, mongo_collection, aggregate_query, mongo_db=None, **kwargs)[source]

Runs an aggregation pipeline and returns the results https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.aggregate https://api.mongodb.com/python/current/examples/aggregation.html

find(self, mongo_collection, query, find_one=False, mongo_db=None, **kwargs)[source]

Runs a mongo find query and returns the results https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.find

insert_one(self, mongo_collection, doc, mongo_db=None, **kwargs)[source]

Inserts a single document into a mongo collection https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.insert_one

insert_many(self, mongo_collection, docs, mongo_db=None, **kwargs)[source]

Inserts many docs into a mongo collection. https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.insert_many

update_one(self, mongo_collection, filter_doc, update_doc, mongo_db=None, **kwargs)[source]

Updates a single document in a mongo collection. https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.update_one

Parameters

  • mongo_collection (str) – The name of the collection to update.

  • filter_doc (dict) – A query that matches the documents to update.

  • update_doc (dict) – The modifications to apply.

  • mongo_db (str) – The name of the database to use. Can be omitted; then the database from the connection string is used.

update_many(self, mongo_collection, filter_doc, update_doc, mongo_db=None, **kwargs)[source]

Updates one or more documents in a mongo collection. https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.update_many

Parameters

  • mongo_collection (str) – The name of the collection to update.

  • filter_doc (dict) – A query that matches the documents to update.

  • update_doc (dict) – The modifications to apply.

  • mongo_db (str) – The name of the database to use. Can be omitted; then the database from the connection string is used.

replace_one(self, mongo_collection, doc, filter_doc=None, mongo_db=None, **kwargs)[source]

Replaces a single document in a mongo collection. https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.replace_one

Note

If no filter_doc is given, it is assumed that the replacement document contain the _id field which is then used as filters.

Parameters

  • mongo_collection (str) – The name of the collection to update.

  • doc (dict) – The new document.

  • filter_doc (dict) – A query that matches the documents to replace. Can be omitted; then the _id field from doc will be used.

  • mongo_db (str) – The name of the database to use. Can be omitted; then the database from the connection string is used.

replace_many(self, mongo_collection, docs, filter_docs=None, mongo_db=None, upsert=False, collation=None, **kwargs)[source]

Replaces many documents in a mongo collection.

Uses bulk_write with multiple ReplaceOne operations https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.bulk_write

Note

If no filter_docs``are given, it is assumed that all replacement documents contain the ``_id field which are then used as filters.

Parameters

  • mongo_collection (str) – The name of the collection to update.

  • docs (list[dict]) – The new documents.

  • filter_docs (list[dict]) – A list of queries that match the documents to replace. Can be omitted; then the _id fields from docs will be used.

  • mongo_db (str) – The name of the database to use. Can be omitted; then the database from the connection string is used.

  • upsert (bool) – If True, perform an insert if no documents match the filters for the replace operation.

  • collation (pymongo.collation.Collation) – An instance of Collation. This option is only supported on MongoDB 3.4 and above.

delete_one(self, mongo_collection, filter_doc, mongo_db=None, **kwargs)[source]

Deletes a single document in a mongo collection. https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.delete_one

Parameters

  • mongo_collection (str) – The name of the collection to delete from.

  • filter_doc (dict) – A query that matches the document to delete.

  • mongo_db (str) – The name of the database to use. Can be omitted; then the database from the connection string is used.

delete_many(self, mongo_collection, filter_doc, mongo_db=None, **kwargs)[source]

Deletes one or more documents in a mongo collection. https://api.mongodb.com/python/current/api/pymongo/collection.html#pymongo.collection.Collection.delete_many

Parameters

  • mongo_collection (str) – The name of the collection to delete from.

  • filter_doc (dict) – A query that matches the documents to delete.

  • mongo_db (str) – The name of the database to use. Can be omitted; then the database from the connection string is used.