In practical machine learning problems, it is important to understand impact of features on models’ predictions. Such an understanding helps not only better explain the black-box machine learning models but also enables their effective applications in business environment. The general model explanation approaches make an untenable assumption that the model’s features are uncorrelated, which can lead to incorrect or unintelligible explanations. Explanations that consider features’ correlations can help discover their true associations with model predictions, thus they can be more realistically applied in business analysis to questions such as: dimensionality reduction, model performance evaluation metrics and analysis of model drift over time. Since it is computationally challenging to obtain black-box model explanations, this research will also focus on developing efficient computational algorithms.