Problem statement

Prior to LLMs, sensitivity analysis (aka “what if” analysis) enabled users to interactively explore their data and model different hypothetical scenarios to assess impacts or outcomes.  For example, a banker might ask "what will happen to my total assets under management if there's a recession next quarter?"

This is typically a regression use case; users score a trained model on one or more rows of data and compare the prediction ("y-hat") vs ground truth ("y"). Users can filter, select and partition data; set new data values; selectively change data to simulate different circumstances or outcomes and compare models (we used an ensemble approach based on XGBoost).  This lets them see how changes cause the model predictions to change compared to the original data.

Process and deliverables

I used interviews and participatory design sessions to generate a body of research which I synthesized into PRDs and which informed our target personas. I worked closely with SMEs and Data Scientists to architect the product and translate the user research findings into user journeys, wireframes and other design deliverables for implementation.