Scroll Search

InsightMaker.scrolled_search(query, dataset=None, fields=None, scroll_size=100,max_size=10000)

Combines the search and scroll methods to perform a scrolled search up to a limit (good for large downloads)

Parameters:

NameTypeDescription

query

String, dict or insight_maker.api. query_builders. SubQuery

InsightMaker Query, can either be a Lucene Syntax query string, Elasticsearch Query DSL as a python dictionary (starting at bool), or a SubQuery (or decedent) object from insight_maker.api.query_builders (wrappers for Query DSL)

dataset

string, Dataset or list

Optional - Dataset object, dataset name string, or list of either

fields

list

Optional – List of field ids of Field objects

scroll_size

integer

Number of results to return per page (default 100)

max_size

integer

Maximum number of documents to acquire (default 10,000)

Response:

A DataSample object, properties:

  • DataSample.df A pandas DataFrame object containing all results

  • DataSample.scroll_id The scroll ID required for a scroll search

Example:

results = im.scrolled_search("*", dataset = "sharepoint", scroll_size=10, max_size=100)

print(results.df)