A Framework for Statistical Analysis of Datasets on Heterogeneous Clusters
Ricolindo Carino, Ioana Banicescu
IEEE International Conference on Cluster
Computing (Cluster 2005) Boston, Massachusetts, USA, September 27 -
30, 2005
Abstract
This paper proposes a framework for the statistical analysis of multiple related datasets on heterogeneous clusters. The set processors of assigned to the framework are partitioned into groups according to rack locations, with the group sizes being chosen to match the degree of concurrency in the analysis procedure. Dynamic loop scheduling is employed to address load imbalance arising from the differences in computational powers of groups, the variability of dataset sizes, and the unpredictable irregularities in the cluster environment. Results of preliminary tests indicate the effectiveness of the framework in fitting gamma-ray burst datasets with vector functional coefficient autoregressive time series models on 64 processors of a heterogeneous general-purpose Linux cluster.