Distributed Memory Management for Fast Data
High-volume online stream processing, also known as fast data processing, is becoming increasingly important in a number of different commercial sectors. Unlike big data processing in which data is processed asynchronously in batches, fast data processing performs synchronous data analysis that generates actionable results within a specified deadline. One of the key challenges in building a fast data processing system is in scaling with increasing volumes of data. In our proposed research, we plan to build a system to efficiently manage the available memory across the entire deployment. The system will determine which data blocks should remain in memory, where a data block should be placed, and what fault tolerance strategy the system should employ. The objective is to build a scalable processing system that can handle both current and future fast data processing demands. TO BE CONT’D
View Full Project DescriptionBernard Wong
Smash.bi Inc;University of Waterloo
Computer science
Information and cultural industries
University of Waterloo
Accelerate