Methodology used for testing
Single InstanceWe conducted the performance testing on an AWS machine with 8 CPUs, 20ECUs, 7GB of memory and 4*420 GB of Storage. We selected a large instance because the scalability also needed to be tested along with the response time performance. Initially we loaded 10 million records into ODE instance and recorded the response times for Dense, Scattered, Sparse, Rare and Empty searches.
Dense Search: 1 in every 100 records
Scattered Search: 1 in every 10,000 records
Sparse Search: 1 in every 100,000 records
Rare Search: 1 in every 1,000,000 records
Empty Search: No records found
We continued performing the same tests after loading 25M, 50M, 100M, 250M, 350M, 450M, 500M and 650M records into the single instance. Response times at each of these steps were recorded, we stopped loading data after 650M records since the response times for all the searches started plateauing.