Yes, that would be a problem if the ruler were rigid. So you've proven that, unless all of relativity breaks down, those rulers can't be rigid. Nothing can. One way to see that is that the parts of the ruler exert forces on each other via electromagnetism, which only propagates at the speed of light. So when you push on one end it takes a while before the other end notices. See some old discussion: https://van.physics.illinois.edu/qa/listing.php?id=1398.
Mike W.
(published on 02/14/2016)