Distributed Memory Management for Fast Data

High-volume online stream processing, also known as fast data processing, is becoming increasingly important in a number of different commercial sectors. Unlike big data processing in which data is processed asynchronously in batches, fast data processing performs synchronous data analysis that generates actionable results within a specified deadline. One of the key challenges in building a fast data processing system is in scaling with increasing volumes of data. In our proposed research, we plan to build a system to efficiently manage the available memory across the entire deployment. The system will determine which data blocks should remain in memory, where a data block should be placed, and what fault tolerance strategy the system should employ. The objective is to build a scalable processing system that can handle both current and future fast data processing demands. TO BE CONT’D

Faculty Supervisor:

Bernard Wong

Student:

Sajjad Rizvi

Partner:

Smash.bi

Discipline:

Computer science

Sector:

Information and communications technologies

University:

University of Waterloo