Elasticsearch Aggregations

Aggregated or faceted search dynamically clusters items or search results into categories and allows users to drill into the search results by any value in any field. Each displayed “aggregate” of the search results shows the number of hits within the search that match that category, and users can “drill down” by applying specific constraints to the search results.

Aggregated site search is implemented in dotCMS using Elasticsearch aggregations.

The $estool.search() Method

To perform any Elasticsearch query, including an aggregation query, you must call the Elasticsearch Viewtool search() method (see the examples below).

Arguments

The $estool.search() method takes a single argument that must be a valid ElasticSearch query string (using full Elasticsearch JSON syntax).

When passing the query string to the $estool.search() method, you must render the query string using the Render Viewtool, as in the following example (where $esQuery contains the query string in Elasticsearch JSON format:

#set($rawResults = $estool.search($render.eval($esQuery)))

Search Fields

You can perform an aggregations query using any of the following search terms:

• content
• filename
• host
• mimeType
• title
• description
• keywords
• author
• contentLength
• modified
• score

Results

When you supply an aggregates query to the $estool.search() method, the result is a list of aggregates for the most popular terms across the documents matching your query, or across all documents in the index, each with a term and count.

Parsing the Results

To transform the results into a list which you can iterate across in Velocity, you must call the JSONtool generate() method, as in the following example:

#set($results = $json.generate($rawResults.response))

Once you have transformed the results, you can use a Velocity loop to “drill down” into the results (see the example, below).

Example

The following query performs an aggregations search on the mimeType term.

Note: In all of the following examples, the $searchQ string contains a lucene query string, and the $esQuery string contains an Elasticsearch query string in full Elasticsearch JSON format.

Step 1: Define the aggregations query

First, define the aggregations query using the mimeType term:

#set($esQuery = '{
    "query" : { "query_string" : {"query" : "$searchQ"} },
    "aggs" : {
        "mimeType" : { "terms" : {"field" : "mimeType"} } 
    },
    "size":0
}')

Step 2: Call the $estool.search() method

Next, render the query and call the search method:

#set($aggsQuery = $render.eval($esQuery))
#set($rawResults = $estool.search($aggsQuery))

Step 3: Parse the results and iterate over them in Velocity

Finally, use the JSONtool to parse the results, and iterate over each of the returned aggregations to display both the aggregation term and the count:

#set($results = $json.generate($rawResults.response))
#if($UtilMethods.isSet($results) && $results.size() > 0)
    #foreach($aggregation in $results)
        #foreach($term in $aggregation.entries())
            $term.term ($term.count)
        #end
    #end
#end

Additional Aggregation Implementations

The following examples demonstrate queries that use additional Elasticsearch aggregation implementations, including Range aggregations, Histogram aggregations and Date Histogram aggregations.

For a list of all additional Elasticsearch aggregation implementations, please see the Elasticsearch Bucket Aggregations, Metrics Aggregations, and Pipeline Aggregations documentation.

Range

The following query implements an Elasticsearch Range Aggregation.

#set($esQuery = '  {
    "query" : { "query_string" : {"query" : "$searchQ"} },
    "aggs" : {
        "range1" : {
            "range" : {
                "field" : "contentLength",
                "ranges" : [
                    { "to" : 50 },
                    { "from" : 20, "to" : 70 },
                    { "from" : 70, "to" : 120 },
                    { "from" : 150 }
                ]
            }
        }
    },"size":0
  }')

Histogram

The following query implements an Elasticsearch Histogram Aggregation.

#set($esQuery = '  {
    "query" : { "query_string" : {"query" : "$searchQ"} },
    "aggs" : {
        "histo1" : {
            "histogram" : {
                "field" : "modified",
                "interval" : 100
            }
        }
    },
    "size":0
}')

Date Histogram

The following query implements an Elasticsearch Date Histogram Aggregation.

#set($esQuery = '  {
    "query" : { "query_string" : {"query" : "$searchQ"} },
    "aggs" : {
        "modified" : { "date_histogram" : {"field" : "modified","interval" : "month" }}
    },
    "size":0
}')

For more information on Site search, please see the Site Search documentation.

To preview an example of a site search query on our demo site, please visit http://demo.dotcms.com/home/site-search?q=dotcms&search=Search.

Migrating from Facets to Aggregations

Previous versions of dotCMS used the defunct Elasticsearch Facets feature instead of Elasticsearch aggregations. If you have existing queries that use Elasticsearch Facets, those queries need to be migrated to use aggregations. In most cases, rewriting a facet query as an aggregation query requires only 2 simple steps:

1. Replace the "facet" term in the query with "aggs".
2. Replace the $search.getFacets() call in your Velocity code with a call to $estool.search().

For more information on migrating facet queries to use aggregations, please see the Elasticsearch documentation.

Additional Examples

For additional examples of ElasticSearch queries using aggregations, please see the List of all Content Types and Counts and Tag Cloud examples in the Elasticsearch Examples documentation.