Stop them now and save yourself extra work come spring
Have you looked at your garden lately? When you do, don’t be surprised if you see chickweed, henbit, annual bluegrass, cranesbill, etc. starting to create a green carpet. Those weeds are pretty small now, but if you don’t get out there and control them, they will be much larger next spring.