Twin experiments were made to compare the reduced rank Kalman filter (RRKF), ensemble Kalman filter (EnKF), and ensemble square-root Kalman filter (EnSKF) for coastal ocean problems in three idealized regimes: a flat bottom circular shelf driven by tidal forcing at the open boundary; an linear slope continental shelf with river discharge; and a rectangular estuary with tidal flushing intertidal zones and freshwater discharge. The hydrodynamics model used in this study is the unstructured grid Finite-Volume Coastal Ocean Model (FVCOM). Comparison results show that the success of the data assimilation method depends on sampling location, assimilation methods (univariate or multivariate covariance approaches), and the nature of the dynamical system. In general, for these applications, EnKF and EnSKF work better than RRKF, especially for time-dependent cases with large perturbations. In EnKF and EnSKF, multivariate covariance approaches should be used in assimilation to avoid the appearance of unrealistic numerical oscillations. Because the coastal ocean features multiscale dynamics in time and space, a case-by-case approach should be used to determine the most effective and most reliable data assimilation method for different dynamical systems.