Lumin + Snowflake empower healthcare providers control staff attrition

Industry solution powered by the Lumin-Snowflake partnership

Decision Intelligence is one of the most sought-after AI activations by enterprises for the unprecedented advantage it offers businesses. Although there are quite a few decision intelligence solutions in the market, Lumin presents a list of unique attributes that make it stand out from the competition. As a Snowflake-ready technology, Lumin has native connectors to consume Snowflake datasets directly. Lumin also leverages Snowpark for its core AI/ML computations, and integrates with Streamlit, offering close-knit secure solutions right within the Snowflake Data Cloud environment.

In fact, Gartner has recognized Lumin as a Representative Vendor in the Gartner Market Guide for Augmented Analytics 2022, and this positions Lumin as a go-to tool for enterprises to traverse their data-to-decisions journey with ease.

In this blog, we talk about an industry solution for the healthcare sector, combining Lumin as a Decision Intelligence (DI) product, Refract by Fosfor as a Machine Learning (ML) platform, and Snowflake as a Data and Compute Warehouse. The solution addresses the prominent issue of staff attrition, which is a major concern for service providers in the healthcare industry.

The challenge

The US Healthcare system has a severe shortage of registered nurses, and since the pandemic, the problem has only further aggravated. The US Bureau of Labor Statistics projects 194,500 average annual openings for registered nurses between 2020 and 2030, with employment projected to grow at 9%.

There are many reasons for this shortage, such as an aging workforce, inadequate workforce support, and an increased demand for healthcare services. Another prominent reason, which can be attributed to local as well as social factors, is Staff Attrition. Understanding the challenges and factors at play, and retaining the staff has become a matter of paramount importance for service providers. Industry experts see these challenges to present multi-faced implications if not addressed immediately.

Also, since hiring RNs, especially as contractual staff, is expensive, the situation can trigger a perpetual cycle of negative effects as listed below:

• Mounting financial burden as service providers fill open positions with contractual staff.
• Overburdening of current staff, and a consequent possibility of increased attrition rates.
• A negative impact on patient care and customer service.

The Lumin + Snowflake solution

Looking at the historical trend of attrition among nurses, it has emerged as a critical problem that needs to be addressed with speed. Since nurse attrition could be occurring due to multiple reasons, it’s important to zoom in closer to understand not just the probability, but also the actual cause of attrition.

We aggregated sample data from a 50-year-old historical reference (The raw data is extracted from the Snowflake Marketplace, and is provided by Revelio Labs) at the nurse level, capturing specific behaviors that led to attrition. Looking at historical trends, it is important to note that the attrition rates for nurses have increased in the last few years. This blog addresses both the what & why of this phenomenon.

The dataset has been viewed through 2 lenses: one on an overall nurse basis, and the other on monthly churn behavior.

The study attempts to understand the association of nurse behavior with churn probability. Hence, we predict the data at 2 levels:

• Find out who is expected to leave, and
• Find out how overall attrition can be curbed, and retention increased.

Prediction of nurse attrition

Refract by Fosfor, an end-to-end ML platform, has been utilized in this case to predict nurse attrition. We first analyzed the raw data, and then trained a Random Forest ML model for the predictions. This model was trained and tested to accurately predict which nurses were most likely to leave. The raw data was consumed by Refract, and a predictive model was created by utilizing nurse-level attributes as shown in Figure 1 (Solution Architecture) with Refract. The data wrangling, feature engineering, model training, and predicting were performed on Snowpark using Python libraries within Refract. The predictions were then consumed in Lumin and analyzed from different perspectives.

The data was engineered at monthly intervals to understand monthly attrition, and drivers like salary, overtime hours, etc., were measured. This data engineering was done on Snowpark, and the data was saved in Snowflake (as shown in Figure 1 – Solution Architecture) utilizing Snowpark, which was consumed by Lumin for further analysis.

Then, the attrition rates were studied at multiple levels such as distance travelled by staff, type of hospital, etc., and the importance of these variables on attrition were also recorded.

Figure 1 below shows the high-level architecture of Fosfor products working together to provide a tightly coupled comprehensive DSML solution.

Figure 1: Solution Architecture

Breaking down the data

About the data

The data has local as well as social attributes at an individual staff level. It contains the below information:

• About the Employer: Company, hospital type, ownership of the hospital, role of staff, etc.
• Employment details of staff: Employee ID, joining date, salary, job end date, tenure, seniority, overtime hours, etc.
• Social parameters: Educational Degree, school end date, age, ethnicity, sex, distance from work, etc.

As seen in Figure 1, the solution has two facets of analyses – Descriptive and Prescriptive.

For the descriptive analysis, the data is aggregated at a monthly level to understand the dynamics of churn and the trend of attrition. Major metrics such as Active Staff and Staff churned in a particular month are recorded.

For the prescriptive analysis, the output of the ML model trained on Snowpark is ingested and stored in the Snowflake ML schema. It consists of the additional predicted variable of churn prediction as well.

Key performance metrics

A few KPIs have been calculated to gauge the pattern of attrition and compare them across different hospital types, companies, and social parameters.

Attrition and Retention Rate: Although they represent the same data in different perspectives, the goals of the business made it imperative to see both the data points. Also, the distribution of data stipulated that it would make sense to look at retention numbers in alignment with business goals. With Lumin’s forecast capabilities, figuring out the trend of retention rates was simple.

Predicted Rates: Predicted churn percentage is an important factor in making strategic decisions w.r.t a company or hospital facility at a macro level.

The best way to understand the solution and get an understanding of the business use case is to go through a sample analysis.

Solution walkthrough

Descriptive Analytics to understand the attrition behavior

Let’s start with some simple questions that you may want to ask.
Being a service provider administrator, you would like to understand how attrition rates have trended historically. You can ask a descriptive question to Lumin about the trend of churn across the industry.

What is the trend of growth in attrition?

As depicted in Figure 2, Lumin precisely decodes the intent and brings up the column trend chart with easy-to-read variance indicators. The trend shows a strong negative drop of 21% in June 2022. Now let’s understand which type of institutions contributed to this change, and by how much.

Figure 2 is illustrative of Lumin’s capabilities in offering clear and directive insights, with the help of GPT technology, for specific descriptive questions.

Figure 2 : Growth trend of Churn

Contribution to change of growth of attrition by hospital type in June 2022

As shown in Figure 3, we can see that acute care hospitals contribute the most towards attrition. Let’s focus on acute care hospitals and move on to some deep-dive questions.

Figure 3 shows the break-up of the data in the form of a waterfall chart for the specific question.

Figure 3 :Contribution by different hospital types to June 22 drop

Going beyond the what and understanding the why: Unleashing Lumin + Snowpark

Diagnostics has always been Lumin’s strong suit, and with Snowpark, diagnostics is fast and seamless. The transfer of the analytics workload to Snowpark makes heavy diagnostics involving several data profiling and modeling relatively fast. The combination of Lumin and Snowflake dishes out compute-intensive deep dive results within seconds.

As we look at how the numbers look across Hospital types, and you would notice an inflection in the data around June 2022. Now let’s ask Lumin why the inflection happened. Picking up from previous insights, we already know that acute care hospitals were major contributors to churn rates; hence, you would like to know –

Why has churn reduced in June 2022 in acute care hospitals?

The comprehensive analysis shown in Figure 4, gives you actionable insights in terms of key focus areas. Voluntary non-profit, privately owned hospitals contribute majorly to the drop, while we see a drop of 40% in the case of female nurses. Also, Lumin illustrates that key drivers of this change are led by a drop in overtime hours, and active population. It directly tells acute care hospitals where to focus their attention, and what drives employee retention.

Figure 4 illustrates the reasons for the drop in churn rates in June 2022.

Figure 4 : Diagnosing acute care hospitals’ attrition in June 22

Understanding the predictions and developing a retention strategy

The next logical step of the analysis is to understand what the custom ML model prediction looks like. In line with previous findings of churn across hospital types, you can simply ask a question about how prediction looks across hospital types.

To have an unbiased view, you can ask for both the predicted count and percentage metrics in the same question.

What is the Predicted Churn Percentage and Predicted Churn by hospital type?

As shown in Figure 5, acute care hospitals have the highest expected churn count, but their expected percentage of churn is at par with other hospital types. This gives you an idea about the density of the population, and how even a minor change in attrition patterns in these types of hospitals impacts the overall churn numbers..

Figure 5 is a representation of churn data across hospital types.

Figure 5 : Understanding ML predictions.

What are the key drivers of retention in acute care hospitals?

You would also want to understand what makes people stick with an organization. With Snowpark integration for heavy analytics workload, this type of Key Driver Analysis or KDA questions work at lightning speed on Lumin. Lumin can give you information about the key drivers of retention and compare their relative importance on the fly.

Figure 6 illustrates the power of Lumin to offer KDA information.

Figure 6 : Key drivers of retention

As evident from Figure 6, factors such as overtime, age, and salary are the top drivers of retention, with overtime having the highest negative impact. Age is a sensitive feature, and you cannot do much about it, but salary is a regulating parameter. Now you know what is important, and by how much. You may also want to see how much you should tweak these drivers to get the desired retention rate.

Simulate and Measure: You can now run a quick simulation test and change some drivers to see how it may impact the outcome i.e., retention numbers.

Let’s simulate a complex scenario where you are foreseeing a surge in patients and demand for services, but you still want to hold on to your staff (in acute care hospitals). If drivers like overtime is increased by 10%, and to compensate for the overtime, a hike of 10% in salary is provided as a bonus, you would be able to see marked results.

Figure 7 is a representation of the simulation capabilities of Lumin.

Figure 7 :Simulation-Part 1-Changing driver values

Figure 8 illustrates the advanced simulation of the specific scenario provided by Lumin.

Figure 8 : Simulation-Part2-Measuring the impact on Retention numbers.

From the simulation results as shown in Figure 7 & 8, we observe an overall 4.5% hike in retention numbers by simultaneously increasing the overtime and average salary by 10%. This is a good indicator for formulating a strategy to retain employees, despite desperate situations demanding excess overtime.

Forecasting the retention patterns: strategy building

Now to strategize, and increase retention rates in your institution, you would want a peek at the future. So, let’s ask Lumin a prediction of how the simulation is going to work out for acute care hospitals:

What will be Retention in next three months in acute care hospitals?

Figure 9 illustrates the predictive capabilities of Lumin.

Figure 9 :Forecasting retention for acute care hospitals

Forecasting in real-time is heavy on modeling and requires multiple, complex time series model iterations. Since this is accelerated by the power of Snowpark, the overall turn-around time of questions is enhanced by leaps and bounds.

As depicted in Figure 9, Lumin offers you the trend in data for future months, as well as information on the models and factors used for this forecast. You can always click on the “How did I arrive at this insight?” link and delve deeper into the Explainable AI feature of Lumin to understand the data modeling and variable interaction steps.

You can continue your analysis by asking more questions, but some things can always go unnoticed. What if you could monitor your data patterns to the fine-grain level, autonomously? This brings us to another feature of Lumin: Autonomous Nudges.

Keeping a tab on attrition with Autonomous Nudges

With Lumin, the possibilities are endless. Lumin offers a sense of security to stakeholders by autonomously pointing out any overlooked insights. With Lumin Nudges, or Autonomous Anomaly Detection, you can rest assured that you are considering every aspect of your data.

Figure 10 is an illustrative sample of the Nudges function in Lumin, configured on attrition numbers here.

Figure 10 : Unveiling a Nudge on Retention for acute care hospitals.

Lumin monitors the historical retention patterns, and with the help of statistical techniques like moving average in this case, brings you a nudge, informing an unexpected decrease in retention for acute care hospitals circa Aug 22, and salary being a major cause for this drop. This is a great example of how Lumin can help you navigate critical business questions and make informed decisions. With experienced personnel and seasoned healthcare professionals using Lumin, data discovery can be intriguing and reveal much more than just insights.

Summary

Staff attrition in healthcare institutions is a major concern, and not only does it impact the service providers’ operations, but it also impacts the speed and quality of healthcare services. Though there are a lot of macro socio-economic parameters at play, which are difficult to control, with data and decision intelligence, a large part of the problem can be addressed. Lumin, Refract, and Snowflake coming together to provide a comprehensive solution that can monitor, measure, diagnose, foresee, and strategize, can be a game changer, as it can recommend appropriate actions to increase retention, and reduce attrition in any healthcare organization.