There is probably not a maximum size for mountains on planets, but there is surely a tendency of planetary surfaces to become smoother with the passage of time. The height of Earth's mountains are the result of tectonic activity driving them up, and erosion by wind and weather smoothing them down. I don't think Mars has enough free water to freeze and thaw, which tends to break up rocks, and Mars has a puny atmosphere" probably not much weather and wind.
It would be unlikely for there to be a cubical planet. We will never know for certain, as the universe is too big to see it all.