Aquis Impeller Shaft Bottom Rubber Stuck

I know externals are not the same as internals, but I can never get the rubber cap from the bottom of the impeller out of my Eheim internals. The impeller shaft with mine has a rubber cap on each end. I insert the impeller shaft complete with impeller into the one that is stuck at the bottom of the impeller well, making sure it is seated firmly, then put the other cap on top of the shaft, and finally clip the cover into place. The cover won't go on if the impeller is not seated properly in both rubber caps.
I don't worry about not cleaning the stuck rubber cap, I just clean the impeller well with it in place.

To be honest, I'd rather it was stuck there, that way I can't lose it