Software Engineering
database search azure
Updated Sun, 22 May 2022 06:27:49 GMT

Does it make sense to use both Cosmos DB and Elasticsearch?


My app uses Cosmos DB to store data. I have a requirement to support full-text search, which Cosmos does not provide out of the box. One of the recommendations was to use Elasticsearch, and I have been trying that out. The more I try it out, the more I feel like I am duplicating data. I literally push the same documents to Cosmos DB and Elasticsearch.

Am I heading down a bad path here? The way I see it, I should do one of these things:

  • use only Cosmos DB, simulating full-text by tokenizing+stemming content myself
  • use only Elasticsearch
  • use Cosmos DB in conjunction with Azure Search, which feels to me like it's not doubling-up on database responsibilities

Does anyone have any thoughts on my direction here?




Solution

Since I don't know everything about your usage, I cannot say what is the correct way for you.
What I can tell you is that mixing elastic with another DB is a legitimate and common course of action.

You can use elasticsearch as your main (or only) storage if you'll invest the time needed. You can also only index your requested fields in elasticsearch (without storing them), and sending a search request to elastic and using the returning ids to query your main storage to get more information of the selected items.





Comments (2)

  • +0 – Interesting point about only indexing without storing. It hadn't even clicked that this was an option, since the API doesn't really lead you down that path. ie. it has you create a document which can be stored/indexed and re-hydrated in full. — Jan 30, 2019 at 01:05  
  • +1 – Note that Elasticsearch doesn't support multi-document ACID transactions, so in most cases I'd advice against using is as a "main" database. Nowadays, there is plenty of databases both RDBMS and NoSQL that support proper transactions. More about transactions in Elasticsearch: elastic.co/guide/en/elasticsearch/reference/7.5/… — Dec 11, 2019 at 12:32