The Challenge
Key to Freddie Mac’s mission and success is being an engaged and informed partner to banks and lenders. Large swings in loan processing volume, high or low, can be an early sign of trouble for Freddie Mac’s sellers. When loan volumes are erratic or outside of historical norms, it often signals that issues in the loan submission process are affecting the customer journey. For example, a server might be down or a system error is keeping the process from advancing. If those roadblocks are not identified and resolved quickly, Freddie Mac risks losing customers and business to its competitors.
With decades of daily loan volume data on hand as well as external data ranging from extreme weather events to monetary policymaking information from the Federal Reserve, Freddie Mac Single Family LQA (Loan Quality Advisor) team knew it had the foundation needed to identify where loan volumes were straying outside of the norm. In 2019, Freddie Mac embarked on a mission to use machine learning and Natural Language Processing (NLP) to design and build automated early warning models for predicting and eliminating challenges along the loan submission journey.
The Solution
Because this early warning system would be one of the very first machine learning models designed to help optimize the customer journey for Freddie Mac, the LQA team partnered with Ventera in order to bring on added machine learning, NLP and big data expertise.
“Our data science team is extremely adept at helping our clients tell the stories held within their data. With the depth of experience and a full tool-box, we worked with several machine learning algorithms before settling on a Gradient Boosting Regressor algorithm,” said Praveen Nedungottil, Ventera’s EVP of Solutions Delivery. “We can predict what is going to happen next, and why with a high degree of accuracy, an imperative for all businesses today.”
The goals for the machine learning proof of concept (POC) system were:
- Determine loan volume outliers: Collect and analyze 10 years of loan data and leverage an existing predictive Splunk tool to evaluate and define loan volume outliers.
- Unleash the predictive capabilities of machine learning: Develop machine learning tools to automatically predict loan volume outliers and identify, with a high degree of confidence, operational risks to the loan processing journey.
- Elevate user experience, enhance the customer journey: Improve ability to identify and proactively eliminate potential loan processing quality or performance issues to help Freddie Mac provide exceptional customer service and an elevated user experience.
- Assess and identify future uses and tools for machine learning: Provide an assessment of existing machine learning technologies and recommend strategies and investments for advancing machine learning capabilities and results.
Streamlining & Optimizing the Data
The Ventera team, made up of data scientists, data engineers and experts in machine learning technologies, collected and assessed 1M+ historical loan records and external data sets using Python-based technologies: Pandas, Scikit-learn, NumPy and Jupyter Notebooks. The team found that the data was extremely noisy and random at first glance. Realizing it would be impossible to predict loan volumes and target outliers from the entire dataset, the Ventera team visualized clusters of loan data trends by sorting lenders and banks into tiers as determined by loan volume. By organizing and streamlining datasets, Ventera was able to improve forecasting accuracy.
Another important element of Ventera’s loan data optimization was the integration of calendar and market data to improve the prediction and detection of data anomalies. Ventera expanded the parameters to include day of the week, weekend and holiday data as well as external influencing factors, such as Federal funds rates, extreme weather events, and Federal Open Market Committee meeting minutes. This inclusion of external data boosted loan volume processing prediction accuracy from 60% to 88%, which made it much easier for Freddie Mac to identify Freddie Mac sellers with irregular loan volumes: outliers.
Early Warning Indicator Model
Using the greatly expanded data sets, Ventera built an Early Warning Indicator (EWI) model that predicts seller volume outliers on an hourly basis. Freddie Mac’s operational team uses the EWI model to proactively identify and eliminate potential issues, ensuring sellers can efficiently move through the loan processing journey.
Change-Incident Predictor
While identifying loan seller outliers was an important step, Freddie Mac also needed to understand the underlying issues affecting these users. Ventera built a Natural Language Processing (NLP) model to analyze change ticket requests in ServiceNow. NLP could rapidly assess and identify which service issues, from IT outages to system updates, were affecting loan processing.
By correlating outage and system error data reported in the change tickets with loan processing volume, the NLP-based change-incident predictor Ventera built can determine whether a system change poses an increased chance of triggering a high-level incident in the week ahead. Based on the risk profile the change-incident predictor delivers, Freddie Mac is able to determine with 82% accuracy whether a system change will result in an outage that would disrupt the loan processing journey. Able to make informed decisions about the risk of given system changes, Freddie Mac proactively optimizes the customer experience for its users.
A Future-ready Foundation
With these two machine learning models in place—the EWI and change-incident predictor—Freddie Mac gained a future-ready foundation for loan processing operations that automatically detects anomalies and assesses risk. As a result, the company is able to respond to loan processing issues faster than ever and has infused proactive service into the customer journey.