The stock search engine had to provide screening on different stock types i.e. shares, ETFs, funds, bonds, investment trusts, venture capital trusts, warrants, etc. And each of the stock types had different set of criteria. For e.g., for shares there are earnings per share (EPS), while for bonds there are coupon rates and redemption dates to be searched for.
Solr provides fast and sophisticated text search with scalability. The query response time in case of Solr server is faster and more predictable against any SQL database where query speed depends based on design and use case.
The main challenge for the stock search engine was to deliver real time results and for that all the data needs to be accessed very fast.
All the data was pulled from different data sources like Thomson Reuters, Digital Look, Proquote, MorningStar, InteractiveData, etc. and after merging the data and making a flat structure was injected into different Solr cores based on their respective stock types.
The solution to that was to index all the data into the Solr server and then front-end can query from data from the Solr server. Also, we implemented multiple Solr cores for searching of different criteria based on stock types.
To fetch all the data quickly, we chose Solr for indexing and storing the data over traditional RDBMS databases.