AlaskaTRX wrote: ↑Thu Mar 26, 2020 9:52 pm
Vonz90 wrote: ↑Thu Mar 26, 2020 3:26 pm
Netpackrat wrote: ↑Thu Mar 26, 2020 3:56 am
Behold, the power of wishful thinking. Still, I hope you and they are right.
If you run these kinds of simulations*, the degree of uncertainty drives up the "worst case scenario" without changing the "best case scenario" appreciably because that number is anchored by the numbers you already have. More data will pretty much always drive the simulated worst case down. That is what we are seeing now.
* I do not run simulations for epidemics obviously, but the math is the same for what me (and these days, my staff) does.
That assertion is not true. I, too, crunch data for a living and more data does not always drive toward the worst case; what it usually does is narrow up your uncertainty and improve the simulation’s confidence level. That could be closer to either P10 or P90 cases. We base our studies off of a realistic P50. Maybe some people do lean toward a P10 as a basis, but that’s no way to run a business.
This is true if one starts with a reasonably well defined set of probabilities, and you are right I stated things a bit broadly.
If one is dealing with things that poorly defined rates, the bad side can get very ugly in a hurry. It can be very easy systematically bias the results (just stick a few "conservative" estimates in a row) and you blow your WC out way to the right. Once you have done that, your P50 can easily really be your P90. If it is a well understood process, everyone will just throw the BS flag and go back and fix it, if it is not, then it be easy to run with that estimate. When your real world data is all below the mean, it should kick in that you made a mistake, but that can take a while.
I believe that is what has been happening here and that is how the British estimate can go from over 500k to less than 20k overnight.
Edit here because there is something else I should mention. Depending on what you do, this may be old hat to you or not, but just in case I will mention it. What I said above is primarily an issue when you have a bounded condition on one side. As I do product engineering, this is typical when trying to project failure rates for parts/assemblies. You cannot have less than zero, so any estimate that increases likelihood of any factor contributing to failures drives up your mean number of failures. So if your estimates are conservative because you have a lot of uncertainty about the failure modes (i.e. large in this context) on some of those rates and all of a sudden it looks like you could have huge numbers of bad parts out in the world. This is something I have commonly seen if it is a new failure mode and you have not been able to narrow the root cause at all, it can be hard to give it a sanity check even if the numbers look huge. Get some more data either about the universe of parts that are failing, or the conditions that they fail under, or similar, and then the calculated rate of failure starts getting both more accurate and typically (like every time I have gone through this exercise) significantly lower. If it is a well known failure mode, then you probably have enough history to make accurate estimates of probability up front and more data is not as likely to influence your estimated mean number of failures all that much.
I hope that explains where I am coming from on this.