How to implement autocomplete and advance database queries in firebase

Why firebase?

• Quick and easy setup.
• Realtime database (No need for state management)*.
• Cloud functions support for advance business logic.
• Email/phone and all major third party auth provider support.

To maintain practical context throughout the post, let's consider the use case where we are building a simple match making app and possible data structure can be as followed.

{
  "userkey1": {
    "name": "John Doe",
    "age": 28,
    "gender": "male",
    "city": "mumbai",
    "interests": "Travelling, Cooking",
    "isWillingToShift": true,
    "createdOn": 1533832318           // Unix timestamp at which profile is created.
  },
  "userkey2": {
    "name": "Jane doe",
    "age": 29,
    "gender": "female",
    "city": "mumbai",
    "interests": "Reading, Dancing",
    "isWillingToShift": false,
    "createdOn": 1531153917           // Unix timestamp at which profile is created.
  },
  ...
}

With Firebase as the database, we have a few limitations

• No support for compound queries
• can't search for a match which is both female and willing to shift.
• No way to define sorting order (It's ascending by default).
• can't sort matches by their age from oldest to youngest.
• No way to implement autocomplete that is case insensitive or full text.
• can't search for a match which is interested in dancing.
• Pagination implementation works only when the ordering parameter is unique
• can't implement pagination if matches are sorted by their age.

Workarounds

Database restructuring

dividing users into male, female and all nodes, and then searching for a match willing to relocate in female node works. Number of nodes and in turn redundancy increase as number of search parameters increase.

Firestore

We could use firestore as the database, since it supports compound queries and both sorting orders. Chained queries like

// Finds all matches which are females and willing to relocate
userRef.where('gender', '==', 'female').where('isWillingToShift', '==', 'true');

// Finds all matches which are male and older than 28 years.
userRef.where('gender', '==', 'male').where('age', > , 28);

// Finds all matches which are younger than 35 years and sorts from oldest to youngest
userRef.where('age', '<', '35').orderBy('age', 'desc');

• Still no support for autocomplete.  can't search for a match by it's last name or hobbies.
• Sorting and ordering by different parameter is not allowed.  can't sort men filtered by their age.

The solution I propose, is to use Algolia

Algolia is a hosted full-text, numerical, and faceted search engine capable of delivering realtime results from the first keystroke. Algolia's powerful API lets you quickly and seamlessly implement search within your websites and mobile applications. - Algolia Documentation

Few things I considered before committing to Algolia

• Although algolia is heavily marketed as full text solution, which it is, it has filtering, sorting and autocomplete capabilities as well.
• A quick google search for integrating algolia with firebase, gives complicated answers about writing cloud functions to search algolia indices and saving those results into the firebase database and the reading it from there. Algolia has client side javascript sdk with an easier integration.
• You need at least working knowledge of firebase cloud functions in order to sync firebase lists with algolia indices.
• Algolia is a paid service, free community plan just allows

• storing of 10k records (users in our case)
• 100k operations (counts every records creation, updates and queries).

Setup is pretty straight forward

// Installation 
npm install algoliasearch --save
npm install --save @types/algoliasearch // only for Typescript/Angular

// Usage 
const algoliasearch = require('algoliasearch');
import * as algoliasearch from 'algoliasearch'; // When using TypeScript/Angular

const client = algoliasearch('<your_app_id>', '<your_search_only_api_key>');
const index = client.initIndex('users'); // where users is name of our index.

Algolia in effort to provide relevant results, gives user lots options to customise their query. This can be overwhelming, so let's consider only few relevant things for now.

• Search/Autocomplete
• Pagination
• Ordering (Ranking)
• Compound Queries (Filters)
• Facets

Ranking, filters and facets are words from algolia's vocabulary. Facets and filters, in general, are used for restricting search to a subset of results. Difference between facets and filters

• Filters are used when filtering is done entirely hidden from the end user Serving curated matches for our match making app user's preferences.
• Facets generally used for building a UI, where users can select facets (as categories) to further refine their query  Can provide dropdown to our users to refine matches from specific cities.

Search

Search can be as simple as simple query and can be complex with filters or facets or both as well. There are lots of search parameter options along with query and all documented here.

// Search matches by last name joe
index.search({ 
  query: 'joe'     // Search query
 });

// Search by matches hobby
index.search({
  query: 'dacing'  // Yes even with the typo
}); 

// we can pass addition search parameters 

// attributesToRetrieve => if listing only shows user name and city 
// then we can just request name and city of users.
index.search({
  query: <search_query>,
  attributesToRetrieve: ['name', 'city'] 
}); 

Pagination

// pagination => Returns all matches on page number 3
// with page size of 10 matches/page with name jane.

index.search({
  query: 'jane',   // Search query
  page : 3,        // Page number
  hitsPerPage: 10  // Page size
}); 

// offset => Returns matches starting from 14 to next 25 with name 
index.search({
  query: 'jane',   // Search query
  offset : 14,     // Number of matches to skip
  length: 10       // Number of matches after offset 
}); 

Filters

Filters can be used with or without query parameter.

// filters all male matches
index.search({
  filters: 'gender:male'
});

// filters all matches which are female and will to relocate
index.search({
  filters: 'gender:male AND isWillingToRelocate:true'
});

// filters all maches which stay in mumbai or will to relocate and are females
index.search({
  filters: '(city:mumbai OR isWillingToRelocate:true) AND gender:female'
});

// filters all matches which are female, willing to relocate and older than 30 years.
index.search({
  filters: '(gender:female OR isWillingToRelocate:true) AND age > 30'
});

// More options

// filters: 'attribute:value AND | OR | NOT attribute:value'
// 'numeric_attribute = | != | > | >= | < | <= numeric_value'
// 'attribute:lower_value TO higher_value'

Ranking

Custom ranking or ordering in realtime is not possible when used in client side, index.setSettings method doesn't work with client-side apis, but we can always set default ranking type to whatever order we want from algolia's dashboard.  Also, algolia's dashboard gives an option to seamlessly create replicas of your index (users in our cases) with different ranking strategy.

For our matchmaking app, we can easily create a recommendation engine, using these options. The features I talked about are just the tip of the iceberg, algolia has many more powerful and advanced options. Those can be found here in algolia's documentation.