Firestore server-side filtering in TMS WEB Core v1.7
To create a high performance web client application, a lot of things need to be taken in account. But foremost, when you create a modern SPA (single-page-application) architecture web client application, you will want to look at optimizing the communication of the application to your database back-end. In case you use the Google cloud based infrastructure as your database back-end, you will want to perform as little as possible requests and ensure that the data involved in these requests is as compact as possible. The server-side filtering capabilities of Google Firestore allow you to easily achieve this, especially with the new built-in support for it in the TWebFirestoreClientDataSet in TMS WEB Core. Recalling loading and filtering features of TClientDataSet in classic VCL If you have used a TClientDataSet in Delphi VCL, you already know that the component provides an in-memory table that can be manipulated easily and efficiently. But if there are a large number of records in a database, loading all of them in the TClientDataSet needs to be avoided. With traditional Delphi TClientDataSet, if the records are loaded through a DataSetProvider that supports queries then you can change those queries to load limited records based on a criteria. This approach greatly reduces the memory overhead of the TClientDataSet, since fewer records need to be stored in memory. Compare this with another option of local filters, also from standard Delphi VCL, where the number of records visible in the dataset can be reduced by using local filtering options. But that does not reduce the number of records in memory. It only restricts access to the records filtered out. In Web Core, we give a solution similar to DataSetProvider where certain filtering criteria can be applied at the Firestore to reduce the number of records fetched on opening the dataset. New server-side filtering features in TWebFirestoreClientDataSet The new server-side filtering features purpose is to limit the number of records fetched from a Firestore collection. Consider the problem when the Firestore collection has a large number of records. Loading all of them at once is not a good idea due to the memory overhead of an in-memory table in the TClientDataSet. In the new release of TMS WEB Core, the Firestore ClientDataSet component provides new features to apply filters to the data obtained at the Firestore end. Important background information Before we look at the details of each filtering method, it is necessary to understand some core principles. 1) Method names begin with “AddService” There are several Firestore related filtering methods available in the Firestore ClientDataSet component. They are named starting with “AddService” indicating that they are applied at the Firestore Service level. This is to differentiate them from any local filtering methods already available in the standard ClientDataSet. 2) Firestore filters are not as flexible as SQL queries The primary purpose of the Firestore filter methods is to limit the number of records fetched on opening the dataset in order to reduce the memory overhead. Don’t expect the Firestore filters to be as flexible as an SQL query for your data processing requirements. There are many restrictions and limitations as you will discover later. For an exanple, please see “Limitations and Gotchas” under AddServiceFilterCondition method later in this article. If you have advanced filtering requirements then in addition to the […]
