How much additional data do the Pharmas have to put together before they can release it for peer review?
Is it normal for there to be enough reliable data to release worldwide press released, but not have the complete data available for review? I'd have assumed you need one for the other?
Ultimately the benefit of more data is more precise results, you can narrow down the range of estimates, and so with interim data you have a wider range of estimates, but can still judge the range. In other words you can assess if it is effective enough to meet certain criteria, but more data will help nail down just how effective it is.
Pfizer outlined their interim analysis plans
here on page 103 which essentially says:
- After 32 covid cases they would need the efficacy to be 76.9% to declare it a success
- After 62 cases they would need it to be 68.1%
- After 92 cases they would need it to be 62.7%
- After 120 cases they would need it to be 58.8%
- After 164 cases they would need it to be 52.3%
In other words, the more data you have, the lower the target needs to be because the likelihood that it will be significantly above or below that estimate is gets smaller and smaller with more data.
To declare it a success, it would need to be more than 30% effective - the lowest possible target. On the other hand, if after 62 cases the efficacy was 27.8%, they would know enough to say that the trial was futile, and they should end it there. If the randomised trial was set up properly, it couldn't possibly be effective enough even if they collected data on 3x as many people, so they would just stop. So you don't need final data to make final judgments either way.
Those different case targets were the different stages of interim analysis they outlined - once they hit that target, they'd start analysing the numbers to see if they've passed that minimum threshold. So far, they're reporting this 90% figure based on 94 cases, i.e. beyond the halfway point, and they're well above that 62.7% threshold. So they definitely have enough data to make the claims that the vaccine is successful, not only against their own criteria but against the FDA's approval criteria. In both cases, it looks like they've gone far beyond that.
They expected to finish the trial with 164 cases, but actually they've got even more than that because it was spreading more rapidly in the US than expected. So when they produce the final results, they are expected to have a different number than that 90% - might be higher, might be lower. But what we know already is that it is high enough to get approved, and it will fall somewhere within that known range.
There's two things we're waiting on from Pfizer and co. Publishing the full results of their interim analysis in a scientific journal, which will include that same headline figure, but also dig into some of the details about how that varies across groups, what that translates into in terms of patient outcomes, what side effects come with it, etc. It's not that they don't have the data for it yet, but packaging that into a journal is just a very laborious and precise process.
The second thing is waiting for them to publish the full results of their final analysis. That isn't required for the regulators to know if they can approve it (they can give it an emergency approval on the interim data alone) but it will give us more precise estimates.
