AutoML is a powerful tool that can often save data scientists a great deal of time by reducing the iterations necessary to find an optimal model. However, the most common techniques to "win" the competition for the best model are ensemble techniques like Random Forest, XGBoost, or other "black-box" techniques. Since transparency is critical in regulated industries and any area where avoiding bias is important, this creates a dilemma. How can we deploy the most powerful algorithms while maintaining transparency? This is where Explainable AI (XAI) comes in. It allows us to use a complex model while providing transparency, for ourselves during model development and to our stakeholders at deployment.