UPDATE:
When I was driving I could hear the jack rattling around a bit on bumps.
Then I realized the whole pop out was moving up and down and not just the jack.
So I strapped down both sides using the tie-downs and now there's no movement at all!