UPDATE 2/17/24
This article is out-of-date as there have been many changes to Firestore since then. Please see my latest article:
If you have built any Firebase app bigger or more complex than a Todo app, you have realized counting your documents in Cloud Firestore can be a pain. It is a complicated problem. If you come from an SQL background, or you have experience with Graph Databases, you realize that all data is relational. But counting documents is one of the many things in noSQL databases that can cause problems.
Count Aggregation Query
Firestore added the count() function. This function gets the count of any collection or collection query. The Firebase Server scans your documents and sends the aggregation back to you. It is still limited to the basic Firestore querying capabilities. It is also available in your firebase admin in nodejs.
Pricing
You are charged one document read every 1 to 1000 documents that you read.
This is very cheap for small or medium sized databases.
Why this is Important
The last big Cloud Firestore feature update was the not equals feature back in December 2020. For the most part, it has been package updates, and other Firebase features. Thank you Firebase Team for thinking outside the box here!
🤔 Problem: Counting Documents in a Collection
Option # 1 – Count Function
You still have three options for this, and I do NOT recommend the count() function for this. Let's say you have 1,000,000+ documents, and you want to display the total document count on your home page. You will get charged 1000 reads for every visitor, especially if you don't use caching because you have specific dynamic data.
However, if you have a small database, this may be the easiest way if you just want to get up and going. You may want to add a second database for relational data, searching, or more complex counting later.
Option # 2 – Firebase Functions
I retired my package, adv-firestore-functions, because I could not keep up with it anymore. However, most of the code is still good. I made it easy to use Counters with a colCounter function. There are several problems with this method:
-
Firestore Functions can run more than once on a Trigger. Most people are not aware of this. This is usually handled by saving the event ID of the Firebase Function in Cloud Firestore, checking if it exists, and if so DON'T increase the counter. This is called an idempotent function. My package DOES handle this problem, so you don't have to reinvent the wheel.
-
If your site gets so freaking busy, you need to handle sharding. Firebase has written an extension for this, so you should be good there too. However, this gets complicated. In reality, sharding is not a problem you will most likely ever have to deal with, but just in case.
Option # 3 – Batching + Firestore Rules
This is the cheapest and the recommended option for this use case. I wrote two functions identical to set and delete to handle these situations.
npm i j-firebase add a document
await setWithCounter( docRef, data, { merge: true } ); The values are just like the regular set function, except it creates a new document in the counters collection. See this post for more information on how that works.
I also added the ability to count other documents in batch as well. Let's say you want to create a document. In reality, you may want to increase the user's post count, the total posts count, and some other specific count. You could do this with something like this:
import { setWithCounter, deleteWithCounter } from 'j-firebase'; await setWithCounter( docRef, data, { paths: { users: authorId } } ); As you can see from the source code, you can input as many paths as you like. Now you will have your user document count on the user's document as postsCount. I have more options and customization, but that is it in a nutshell. I plan on writing all this up as well as more options on this site soon. If you have questions, feel free to create an issue until I can get the docs up.
The package also has a search index.
Of course, deleting is the same:
await deleteWithCounter( docRef, data, { paths: { users: authorId } } ); 🤔 Problem: Counting Custom Document Queries
There is really no way todo this without the Count Function. You can index specific queries if you know they will exists. For example, you could write a Frontend or Backend counter function that counts all the queries in the posts collection where published equals true, however, it would be nearly impossible to index all the possible counters from all the possible queries. This is where the Count Function really shines.
🤔 Problem: Sorting By a Count
So the Counter Function won't help you there. You absolutely need a custom counter for that. In fact, the custom counter makes this fast and easy. An example of this would be getting all users and sorting by the number of posts they have, or the number of likes they have. Use Batch for creating this.
🤔 Problem: Counting Tree Structures
A complex problem would be counting items within folders within folders. Let's say you have posts within a category. You want that category to display all the posts within that category and its subcategories. Think of how Yahoo used to do this with:
Arts and Entertainment (5,628) This can't be done with batch adds, because you don't know the number of categories to update. This could be solved with a recursive Firebase Function, or you could use the Count Function. If you have 1,000,000 items though, a recursive Firebase Function might be the way to go.
What I hope to See in the Future…
Firebase has said this is the beginning of more aggregation queries to come. We hope to see the ability to have AVG, MAX, MIN and SUM. Of course these functions are VALUE functions instead of Document functions, so it would almost be a whole different thing.
HOWEVER, one thing they could easily do with this is offset. It could return the document (or document ID) at a certain number from any query. This would be great for paging. The infrastructure for this is now there.
So, as you can now see, it is a very complex problem!
Firebase, Keep it Up! Remember Offset!
J