Document splitting

[shopin-dev]

Hello

I have found a missing functionality with the experimental transform function: Document splitting.

I think it would be a interesting feature for the integration

In my case, I am interested in keeping a complete document record in firebase but synchronizing it with algolia as multiple documents splitting the original from firebase.

I have configure my firebase proyect with algolia extension providing a transform function that returns an array of records instead a single record. Returns shapes like that:

{
...data,
objectID: '...'
}

A simple check in the handler create and update functions as follows could add this functionality, supporting the return of an array

if (Array.isArray(data)) {
await index.saveObjects(data);
} else {
await index.saveObject(data);
}

Something similar must be done in extract function for checking size of each element in array instead of whole object size and mapping the payload with the 'timestamp' property

It is now working for me and supports even object or array return

I've also realized that in delete usecase it wont find the correct id because there is a variation between the firebase id ('xxxx') vs algolia objectID ('xxxx-n')

So is needed for deleting to use in the handler the extract function to read the deleted data (before) and check if this document is splitted and, in this case, generate the array of objectIDs corresponding to the original firebase document id and then call deleteObjects(ids) instead deleteObject(id)

So, if in the firebase cloud function env is added a variable that save the key reference of the document key you want to use for splitting or not the document, and another variable with the key reference of the document key you want to use for splitting the documment (must be an array), the transform function, before call the firebase user transform function, splits the document if required and then pass each splitted doc to the transform function and the delete handler can know if this doc is splitted in algolia

The objectIDs can be generated dinamicaly by a function provided by the user that recives the original firebase doc and return an array of ids for the splitted records

This generateIds function must be called before the transform function call and in the delete handler

Changes recap:

• Array checking in handlers and extract function
• Environment variables provided by user in config for 'split:boolean' and 'splitArray:any[ ]'
• User provided function for ids generation

[smomin]

Thanks. I will check this out.

[smomin]

@michaeled when you have some time, can you help me with this addition.

[shopin-dev]

For sure, please let me know how can i help

[shopin-dev]

Please checkout this fork https://github.com/shopin-dev/firestore-algolia-search
If you feel confident with that i can do a pull request

[smomin]

Go ahead with the PR and I will give it a review.

[shopin-dev]

Go ahead with the PR and I will give it a review.

#130

[smomin]

@michaeled @8bittitan Do you have time to look into document splitting? PR #130 has the code but I want to make sure this is something we can deliver and support.