Do your windows have a rubber seal around them, if so, buy a small tube of butyl rubber or black silicone, then while holding the seal (clean out any dirt/dust first) out enough to get the tube spout into the opening run a small bead into the opening. You only need to seal the top and sides.
Clean off excess after it dries an hour for a neat job.