Filtering Process Optimization
Scrawlr is a platform for unconstrained, global interaction with all internet content and users. Scrawlr allows for user evaluation and unconstrained classification of any Scrawlr-hosted or non-Scrawlr content. For non-Scrawlr content, this evaluation and classification allowance will be first at the URL level but will subsequently be provided at the individual content component level.
The research conducted in this project will build an improved programmatic solution to the highly prohibitive database queries currently available when returning a precise result count for large and complex queries, a persistent issue that remains pervasive even with the use of advanced estimator usage tools.
This project will develop general and varying search and filtering optimization approaches for large tightly and loosely coupled datasets to be applied on a variety of technologies (rust/sql/php/lua/redis/redis search). Given current capability limitations, we must determine and optimize solutions for specific database and data storage performance issues at scale.