You are correct.
CHMP will look at the OLE data and say that is interesting and categorize it as suggestive. If CHMP wants to follow up on the suggestive nature of that data they would require another trial.
The OLE is open label, that weakens its power since there is no placebo to compare the results to. There is no random assignment.
OLE is self selected, we know nothing about the people who chose not to continue. They could feel the drug did nothing for them, The could have died from other causes. They could have found the demands of the trial too difficult. A caregiver could have left. They could have gotten so much better that they felt they didn't need the drug anymore.
The people that chose to continue might have lived closed to the trial sites. They could be the most desperate for anything. You can think up as many reasons as you like for someone to chose to enter the OLE or to choose not to.
Without random assignment there is no way to control for all those confounding variables.
All of that before you get to the very small number in the OLE compared to the original trial.