Article Preview
TopIntroduction
Enormous information examination is the technique for amassing and arranging huge volumes of information to discover helpful case studies. To measure an enormous volume of information may be tedious (Momani, 2020). Thus, preparing volumes of information on a big scale can be a task. In enormous information investigation, the streaming information is made available by different information sources in a ceaseless way. Enormous information streaming is an interaction where data is quickly handled to extricate continuous instinct. While handing out the big data streams, sufficient resources are needed to complete the task within the time interval (Mauliawaty et al, 2019). Therefore, resource optimization is carried out to find optimal processing units for processing the big data stream (Mallek et al, 2020).
A new predictive scheduling framework was introduced in (Li et al, 2016, Akour et al, 2021) to provide fast stream data processing. Based on the topology of graph and runtime statistics, topology-aware method was developed to effectively calculate the tuple processing time of scheduling process. The effective algorithm was presented in predictive scheduling framework to allocate the tasks to machines for obtaining effective scheduling results. The framework minimizes the tuple processing time, but it failed to use any preprocessing technique for further minimizing the time complexity. Re-Stream was developed in (Sun et al, 2015) to calculate the energy efficiency and time for big data stream computing environments. With the aid of distributed stream computing theories, an essential path was detected through constructed data stream graph. However, memory consumption was not any better.
A dynamic assignment scheduling algorithm was presented in (Liu et al, 2016) for processing the big data stream using a stream query graph. Stream query graph was employed to compute the weight of every edge. The smaller weight edges were chosen to send the tuples. The algorithm does not increase the scheduling efficiency. A Self-Organizing Maps based resource management was introduced in (Kaur et al, 2017) with big data streams. Characteristics of data were considered to generate and assign cluster to big data stream. In order to minimize the waiting time, the topological structure of clusters were produced using Self-Organizing Maps. But the system does not consider multiple resources in the scheduling process.
Acyclic Cyclo-Static Dataflow (CSDF) graphs were introduced in (Bamakhrama et al, 2013) for scheduling the periodic tasks with higher throughput. A framework was presented to determine the periodic task metrics of each actor, and regulating sporadic input streams. In addition, the matched input/output (I/O) rates graphs were applied to use more streaming applications. It failed to predict the optimal processing units for assigning the periodic tasks with minimum scheduling time. An effective work planning measure was planned in (Usama et al, 2017) for handling the huge information with least asset use. Hadoop work booking was introduced to plan the positions with most minimal time in enormous information preparing. The calculation was unsuccessful to utilize the advancement strategy for limiting the inaccurate booking while at the same time handling the huge information.