This is a grammar post. I think the sign should say:
Please push your bicycle through the underpass.
I would use “through” because an underpass is basically a tunnel.
Not that prepositions necessarily make any sense, but in my experience, we say you go across things that you are on and we say you go through things you are in.
Thus, if the sign were talking about a bridge, then it could say:
Please push your bicycle across the bridge.
