Scroll Search

InsightMaker.scrolled_search(query, dataset=None, fields=None, scroll_size=100,max_size=10000)

Combines the search and scroll methods to perform a scrolled search up to a limit (good for large downloads)

Parameters:

Name	Type	Description
query	String, dict or insight_maker.api. query_builders. SubQuery	InsightMaker Query, can either be a Lucene Syntax query string, Elasticsearch Query DSL as a python dictionary (starting at bool), or a SubQuery (or decedent) object from insight_maker.api.query_builders (wrappers for Query DSL)
dataset	string, Dataset or list	Optional - Dataset object, dataset name string, or list of either
fields	list	Optional – List of field ids of Field objects
scroll_size	integer	Number of results to return per page (default 100)
max_size	integer	Maximum number of documents to acquire (default 10,000)

Response:

A DataSample object, properties:

• DataSample.df A pandas DataFrame object containing all results

• DataSample.scroll_id The scroll ID required for a scroll search

Example:

results = im.scrolled_search("*", dataset = "sharepoint", scroll_size=10, max_size=100)

print(results.df)