-
Event History & Projections
The goal is to provide a simple & clear process such that:
Given the history of your system (number of visitors/signups/conversions each day), you can see a best guess of the conversion/cancellation rates and how they’ve changed over time.
Given hypothetical future conversion/cancellation rates, what will your system look like at some point $t^\prime$ in the future?
(this includes number of free/paid/cancelled users, and revenue totals and rates, etc.)
For the first question, we will use statistics. For the second, we will use simulations.
-
Event History Analysis
It’s usually easy to get a listing of events (“user subscribed to plan A”, “user cancelled from plan C”). We wish to take this list of events, and produce a estimate for the transition rates between each “bucket” at any given time in the past or present.
-
Discrete Event Simulation
This method, DSE, can be used to to simulate, the movement of people across an online business, or to simulate the changing concentration of chemicals in a reaction.
“Discrete” means the objects jump from one type to another, but can’t have values in between. For example, a customer is either a “free trial” customer or a “monthly subscription” customer, nothing in between.
The method works even when the number of objects (people/molecules/etc) is very small, and when their change from one type to another happens randomly over time.
In general, it can simulate any system whose “state” (a list of numbers that completely describe the system) can only change discretely (by whole numbers).
-
Rough Visual of the Goal
Given event data, get a plot like the following for the rate of events as a function of time:
-
Some possible avenues for this solution
nonparametric regression
(locally adaptive) kernel density estimation
smoothing estimators
Bayesian statistics of time series
Inhomogenous Poisson process. (or “time-varying Poisson process”)
spike rate estimation (requires multiple “runs” ?) -
Notes:
- We are interested in how these rates change over time (i.e. improvements lead to higher conversion, or introducing a bug suddenly increases cancellations).
- We probably need to select a time interval over which we assume the rate doesn’t change. For instance, the “visitor → free” conversion rate changes over the course of weeks or months, but we don’t expect it to change every minute.
-
References & People to contact
- Multilevel Discrete-time Event History Analysis
University of Bristol, Fiona Steele - Discrete-Time Fixed-Lag Smoothing Algorithms
Vincent hayward(at)isir.upmc.fr
https://github.com/radarsat1/FOAW - Estimating and Simulating Nonhomogeneous Poisson Processes
Larry Leemis, William & Mary College
leemis@math.wm.edu
- Multilevel Discrete-time Event History Analysis
-
Bayesian Probability
Bayesian probability is a way of updating your best guess distribution of an uncertain result, when you come across new data. There are two main steps:
- Choosing a starting guess for the distribution based on what you know about the possible values it could take.
- When you obtain new data, use Bayes’ Theorem to update this distribution with your new best guess.
-
Nonhomogeneous Poisson Process
So far, the procedure that seems most applicable, and most straightforward.
-
Distribution
A probability distribution is a function that tells you how likely a given result is.
Here are three example distributions for the value $x$. In the blue distribution, $x$ can be anywhere, but is very likely to be at $0$ (the center peak). In the green one, $x$ could still be anywhere, but is likely to be $-2$.