Synaptically coupled neurons show in-phase or anti-phase synchrony depending on the chemical and dynamical nature of the synapse. Deterministic theory helps predict the phase differences between two phase-locked oscillators when the coupling is weak. In the presence of noise, however, deterministic theory faces difficulty in the coexistence of multiple stable oscillatory solutions. We show for both excitatory and inhibitory synapses that noise causes important changes in the phase and amplitude dynamics of such coupled neuronal oscillators when multiple oscillatory solutions co-exist. Mixed-mode oscillations occur when distinct bistable solutions are randomly visited. The phase difference between the coupled oscillators in the localized solution that coexists with other in-phase or anti-phase solutions is clearly represented in the stochastic phase dynamics.