Filters

Normandy filters describe which users a recipe should be executed for. They’re executed locally in the client’s browser and, if they pass, the corresponding recipe is executed. Filters have access to information about the user, such as their location, locale, and Firefox version.

There are two major forms of filters in Normandy. The first is filter objects, which are simpler and more restrictive. They are a good first choice. The second is filter expressions, which are much more expressive, but also more complex. If filter objects can’t do what you want, turn to filter expressions.

Filter Objects

Filter objects are based on a series of simple rules. In order for a recipe to match a user, every part of a filter object must be true. In other words, the parts of a filter expression are AND ed together.

Each filter below defines a rule, and when it matches a user. Filter parameters are represented as JSON. Most users will interact with filters at a higher level, such as a web interface that allows building a filter with a form.

Filter objects are generally specified as a JSON object with at least a “type” field, and other fields determined by that type. For example, ChannelFilter below has a type of “channel” and requires a channels field, so the final JSON would look something like this:

{
    "type": "channel",
    "channels": ["release"]
}
class normandy.recipes.filters.ChannelFilter

Match a user on any of the listed channels.

type

channel

channels
Example:["release", "beta"]
class normandy.recipes.filters.LocaleFilter

Match a user on any of the listed locales.

type

locale

locales

Use full xx-YY locale codes instead of short xx codes.

Example:["en-US", "en-CA"]
class normandy.recipes.filters.CountryFilter

Match a user located in any of the listed countries.

type

country

countries

Use two letter country codes.

Example:["US", "DE"]
class normandy.recipes.filters.BucketSampleFilter

Sample a portion of the users by defining a series of buckets, evenly distributing users into those buckets, and then selecting a range of those buckets.

This is consistent but unpredictable: a given set of inputs will always produce the same answer, but can’t be figured out ahead of time. This makes it appropriate for sampling since it uniformly distributes inputs over the entire sample space, and any variations in the inputs are spread out over the entire space.

The range to check is defined by a start point and length, and can wrap around the input space. For example, if there are 100 buckets, and we ask to check 50 buckets starting from bucket 70, then buckets 70-99 and 0-19 will be checked.

This works by hashing the inputs, and comparing the resulting hash to the possible hash space.

type

bucketSample

input

A list of Context values to consider for the sample.

Example:["normandy.userId", "recipe.id"]
start

The bucket to begin at.

Example:70
count

The number of buckets to include. The size of the included population will be count / total.

Example:50
total

The total number of buckets considered in the space.

Example:100
class normandy.recipes.filters.StableSampleFilter

Sample a portion of users. With a rate of 0.3, 3 out of every 10 users will be selected by this filter.

This is consistent but unpredictable: a given set of inputs will always produce the same answer, but can’t be figured out ahead of time. This makes it appropriate for sampling since it uniformly distributes inputs over the entire sample space, and any variations in the inputs are spread out over the entire space.

This works by hashing the inputs, and then checking if the hash falls above or below the sample point of the hash space.

type

stableSample

input

A list of Context values to consider for the sample.

Example:["normandy.userId", "recipe.id"]
rate

The portion of the sample that should match.

Example:0.5
class normandy.recipes.filters.VersionFilter

Match a user running any of the listed versions. This will include dot releases, and won’t consider channel.

type

version

versions
Example:[59, 61, 62]

Filter Expressions

Filter expressions are written using a language called JEXL. JEXL is an open-source expression language that is given a context (in this case, information about the user’s browser) and evaluates a statement using that context. JEXL stands for “JavaScript Expression Language” and uses JavaScript syntax for several (but not all) of its features.

Note

The rest of this document includes examples of JEXL syntax that has comments inline with the expressions. JEXL does not have any support for comments in statements, but we’re using them to make understanding our examples easier.

JEXL Basics

The JEXL Readme describes the syntax of the language in detail; the following section covers the basics of writing valid JEXL expressions.

Note

Normally, JEXL doesn’t allow newlines or other whitespace besides spaces in expressions, but filter expressions in Normandy allow arbitrary whitespace.

A JEXL expression evaluates down to a single value. JEXL supports several basic types, such as numbers, strings (single or double quoted), and booleans. JEXL also supports several operators for combining values, such as arithmetic, boolean operators, comparisons, and string concatenation.

// Arithmetic
2 + 2 - 3 // == 1

// Numerical comparisons
5 > 7 // == false

// Boolean operators
false || 5 > 4 // == true

// String concatenation
"Mozilla" + " " + "Firefox" // == "Mozilla Firefox"

Expressions can be grouped using parenthesis:

((2 + 3) * 3) - 3 // == 7

JEXL also supports lists and objects (known as dictionaries in other languages) as well as attribute access:

[1, 2, 1].length // == 3
{foo: 1, bar: 2}.foo // == 1

Unlike JavaScript, JEXL supports an in operator for checking if a substring is in a string or if an element is in an array:

"bar" in "foobarbaz" // == true
3 in [1, 2, 3, 4] // == true

The context passed to JEXL can be expressed using identifiers, which also support attribute access:

normandy.locale == 'en-US' // == true if the client's locale is en-US

Another unique feature of JEXL is transforms, which modify the value given to them. Transforms are applied to a value using the | operator, and may take additional arguments passed in the expression:

'1980-01-07'|date // == a date object

Context

This section defines the context passed to filter expressions when they are evaluated. In other words, this is the client information available within filter expressions.

normandy

The normandy object contains general information about the client.

normandy.userId

A v4 UUID uniquely identifying the user. This is uncorrelated with any other unique IDs, such as Telemetry IDs.

normandy.version

Example: '47.0.1'

String containing the user’s Firefox version.

normandy.channel

String containing the update channel. Valid values include, but are not limited to:

  • 'release'
  • 'aurora'
  • 'beta'
  • 'nightly'
  • 'default' (self-built or automated testing builds)
normandy.isDefaultBrowser

Boolean specifying whether Firefox is set as the user’s default browser.

normandy.searchEngine

Example: 'google'

String containing the user’s default search engine identifier. Identifiers are lowercase, and may by locale-specific (Wikipedia, for examnple, often has locale-specific codes like 'wikipedia-es').

The default identifiers included in Firefox are:

  • 'google'
  • 'yahoo'
  • 'amazondotcom'
  • 'bing'
  • 'ddg'
  • 'twitter'
  • 'wikipedia'
normandy.syncSetup

Boolean containing whether the user has set up Firefox Sync.

normandy.syncDesktopDevices

Integer specifying the number of desktop clients the user has added to their Firefox Sync account.

normandy.syncMobileDevices

Integer specifying the number of mobile clients the user has added to their Firefox Sync account.

normandy.syncTotalDevices

Integer specifying the total number of clients the user has added to their Firefox Sync account.

normandy.plugins

An object mapping of plugin names to Plugin() objects describing the plugins installed on the client.

normandy.locale

Example: 'en-US'

String containing the user’s locale.

normandy.country

Example: 'US'

ISO 3166-1 alpha-2 country code for the country that the user is located in. This is determined via IP-based geolocation.

normandy.request_time

Date object set to the time and date that the user requested recipes from Normandy. Useful for comparing against date ranges that a recipe is valid for.

// Do not run recipe after January 1st.
normandy.request_time < '2011-01-01'|date
normandy.distribution

String set to the user’s distribution ID. This is commonly used to target funnelcake builds of Firefox.

On Firefox versions prior to 48.0, this value is set to undefined.

normandy.telemetry

Object containing data for the most recent Telemetry packet of each type. This allows you to target recipes at users based on their Telemetry data.

The object is keyed off the ping type, as documented in the Telemetry data documentation (see the type field in the packet example). The value is the contents of the ping.

// Target clients that are running Firefox on a tablet
normandy.telemetry.main.environment.system.device.isTablet

// Target clients whose last crash had a BuildID of "201403021422"
normandy.telemetry.crash.payload.metadata.BuildID == '201403021422'
normandy.doNotTrack

Boolean specifying whether the user has enabled Do Not Track.

normandy.experiments

Object with several arrays containing the unique slugs for experiments that the user has participated in. Currently, this is limited to preference experiments.

normandy.normandy.experiments.all

Array of experiment slugs for every experiment that the user has enrolled in, whether currently active or expired.

normandy.experiments.active

Array of experiment slugs for active experiments that the user is enrolled in.

normandy.experiments.expired

Array of experiment slugs for expired experiments that the user has enrolled in.

// Target clients that have ever participated in the "australis"
// experiment, including clients that are currently running it
"australis" in normandy.experiments.all

// Target clients that are currently running the "quantum" experiment
"quantum" in normandy.experiments.active

// Target clients that ran the "photon" experiment, and have finished it
"photon" in normandy.experiments.expired
normandy.recipe

Object containing information about the recipe being checked. Only documented attributes are guaranteed to be available.

normandy.normandy.recipe.id

Unique ID number for the recipe.

normandy.recipe.arguments

Object containing the arguments entered for the recipe. The shape of this object varies depending on the recipe, and use of this property is only recommended if you are familiar with the argument schema.

normandy.isFirstRun

Boolean that indicates whether the user has just started Firefox for the first time with their current profile. This is only true once per-profile, and is set to false immediately after the first set of recipes are executed.

Recipes that should not run immediately upon first run should include !normandy.isFirstRun in their filter expression.

normandy.addons

Object containing information about installed add-ons. The keys on this object are add-on IDs. The values contain the following attributes:

normandy.addon.id

String ID of the add-on.

addon.installDate

Date object indicating when the add-on was installed.

addon.isActive

Boolean indicating whether the add-on is active (disabling an add-on but not uninstalling it will set this to false).

addon.name

String containing the user-visible name of the add-on.

addon.type

String indicating the add-on type. Common values are extension, theme, and plugin.

addon.version

String containing the add-on’s version number.

// Target users with a specific add-on installed
normandy.addons["shield-recipe-client@mozilla.org"]

// Target users who have at least one of a group of add-ons installed
normandy.addons|keys intersect [
   "shield-recipe-client@mozilla.org",
   "some-other-addon@example.com"
]

Operators

This section describes the special operators available to filter expressions on top of the standard operators in JEXL. They’re documented as functions, and the parameters correspond to the operands.

intersect(list1, list2)

Returns an array of all values in list1 that are also present in list2. Values are compared using strict equality. If list1 or list2 are not arrays, the returned value is undefined.

Arguments:
  • list1 – The array to the left of the operator.
  • list2 – The array to the right of the operator
// Evaluates to [2, 3]
[1, 2, 3, 4] intersect [5, 6, 2, 7, 3]

Transforms

This section describes the transforms available to filter expressions, and what they do. They’re documented as functions, and the first parameter to each function is the value being transformed.

stableSample(input, rate)

Randomly returns true or false based on the given sample rate. Used to sample over the set of matched users.

Sampling with this transform is stable over the input, meaning that the same input and sample rate will always result in the same return value. The most common use is to pass in a unique user ID and a recipe ID as the input; this means that each user will consistently run or not run a recipe.

Without stable sampling, a user might execute a recipe on Monday, and then not execute it on Tuesday. In addition, without stable sampling, a recipe would be seen by a different percentage of users each day, and over time this would add up such that the recipe is seen by more than the percent sampled.

Arguments:
  • input – A value for the sample to be stable over.
  • rate (number) – A number between 0 and 1 with the sample rate. For example, 0.5 would be a 50% sample rate.
// True 50% of the time, stable per-user per-recipe.
[normandy.userId, normandy.recipe.id]|stableSample(0.5)
bucketSample(input, start, count, total)

Returns true or false if the current user falls within a “bucket” in the given range.

Bucket sampling randomly groups users into a list of “buckets”, in this case based on the input parameter. Then, you specify which range of available buckets you want your sampling to match, and users who fall into a bucket in that range will be matched by this transform. Buckets are stable over the input, meaning that the same input will always result in the same bucket assignment.

Importantly, this means that you can use a recipe-independent input across several recipes to ensure they do not get delivered to the same users. For example, if you have two survey recipes that are variants of each other, you can ensure they are not shown to the same people by using the normandy.userId attribute:

// Half of all users will match the first filter and not the
// second one, while the other half will match the second and not
// the first, _even across multiple recipes_.
[normandy.userId]|bucketSample(0, 5000, 10000)
[normandy.userId]|bucketSample(5000, 5000, 10000)

The range to check wraps around the total bucket range. This means that if you have 100 buckets, and specify a range starting at bucket 70 that is 50 buckets long, this function will check buckets 70-99, and buckets 0-19.

Arguments:
  • input – A value for the bucket sampling to be stable over.
  • start (integer) – The bucket at the start of the range to check. Bucket indexes larger than the total bucket count wrap to the start of the range, e.g. bucket 110 and bucket 10 are the same bucket if the total bucket count is 100.
  • count (integer) – The number of buckets to check, starting at the start bucket. If this is large enough to cause the range to exceed the total number of buckets, the search will wrap to the start of the range again.
  • total (integer) – The number of buckets you want to group users into.
date(dateString)

Parses a string as a date and returns a Date object. Date strings should be in ISO 8601 format.

Arguments:
  • dateString (string) – String to parse as a date.
'2011-10-10T14:48:00'|date // == Date object matching the given date
keys(obj)

Return an array of the given object’s own keys (specifically, its enumerable properties). Similar to Object.keys, except that if given a non-object, keys will return undefined.

Arguments:
  • obj – Object to get the keys for.
// Evaluates to ['foo', 'bar']
{foo: 1, bar:2}|keys

Preference Filters

preferenceValue(prefKey, defaultValue)
Arguments:
  • prefKey (string) – Full dotted-path name of the preference to read.
  • defaultValue – The value to return if the preference does not have a value. Defaults to undefined.
Returns:

The value of the preference.

// Match users with more than 2 content processes
'dom.ipc.processCount'|preferenceValue > 2
preferenceIsUserSet(prefKey)
Arguments:
  • prefKey (string) – Full dotted-path name of the preference to read.
Returns:

true if the preference has a value that is different than its default value, or false if it does not.

// Match users who have modified add-on signature checks
'xpinstall.signatures.required'|preferenceIsUserSet
preferenceExists(prefKey)
Arguments:
  • prefKey (string) – Full dotted-path name of the preference to read.
Returns:

true if the preference has any value (whether it is the default value or a user-set value), or false if it does not.

// Match users with an HTTP proxy
'network.proxy.http'|preferenceExists

Examples

This section lists some examples of commonly-used filter expressions.

// Match users using the en-US locale while located in India
normandy.locale == 'en-US' && normandy.country == 'IN'

// Match 10% of users in the fr locale.
(
   normandy.locale == 'fr'
   && [normandy.userId, normandy.recipe.id]|stableSample(0.1)
)

// Match users in any English locale using Firefox Beta
(
   normandy.locale in ['en-US', 'en-AU', 'en-CA', 'en-GB', 'en-NZ', 'en-ZA']
   && normandy.channel == 'beta'
)

// Only run the recipe between January 1st, 2011 and January 7th, 2011
(
   normandy.request_time > '2011-01-01T00:00:00+00:00'|date
   && normandy.request_time < '2011-01-07T00:00:00+00:00'|date
)

// Match users located in the US who have Firefox as their default browser
normandy.country == 'US' && normandy.isDefaultBrowser

// Match users with the Flash plugin installed. If Flash is missing, the
// plugin list returns `undefined`, which is a falsy value in JavaScript and
// fails the match. Otherwise, it returns a plugin object, which is truthy.
normandy.plugins['Shockwave Flash']

Advanced: Testing Filter Expressions in the Browser Console

  1. Open the DevTools and go into the settings via the Gear Icon

  2. Ensure “Enable browser chrome and add-on debugging toolboxes” is checked.

  3. Open the browser console

    • Tools > Web Developer > Browser Console
    • Cmd + Shift + J
  4. Run the following in the console:

    Cu.import("resource://normandy/lib/RecipeRunner.jsm", {})
      .RecipeRunner.checkFilter({
        id: 1,
        arguments: {},
        filter_expression: FILTER_TO_TEST,
      })
      .then(result => console.log(result))
    

    For example:

    Cu.import("resource://normandy/lib/RecipeRunner.jsm", {})
      .RecipeRunner.checkFilter({
        id: 1,
        arguments: {},
        filter_expression: 'true',
      })
      .then(result => console.log(result))
    

    You can use backticks for multi-line expressions:

    Cu.import("resource://normandy/lib/RecipeRunner.jsm", {})
      .RecipeRunner.checkFilter({
        id: 1,
        arguments: {},
        filter_expression: `
          (
            true &&
            normandy.country == "US"
          )
        `,
      })
      .then(result => console.log(result))
    
  5. The console will output a Promise object, and then log true or false depending on whether the expression passed for your client or not.

Note

  • Certain filters, particular filters that involve normandy.recipe, may not work as expected, as they rely on the id and arguments fields passed in.
  • If you are using time-based or geolocation-based filters, which rely on the Normandy service, this method may fail if you’ve configured Firefox to point towards a local instance of Normandy which is not running.