For a closed symplectic manifold with compatible Riemannian metric g we study the Sobolev geometry of the group of all diffeomorphisms on M which preserve the symplectic structure. We show that, for sufficiently large s, the metric admits globally defined geodesics and the corresponding exponential map is a non-linear Fredholm map of index zero. Finally, we show that the metric carries conjugate points via some simple examples