In this thesis, a Bayesian framework for model selection and parameter estimation is reported to tackle strongly nonlinear systems using noisy observational data. The model selection task involves estimating the posterior probabilities of each proposed model based on observations. To evaluate the probability of a model, the evidence is computed using the Chib-Jeliazkov method that makes use of Metropolis-Hastings samples of the posterior distribution of the parameters. The parameter estimation algorithm entails a state estimation procedure carried out by non-Gaussian filters. When the measurements are sparse, the nonlinearity in the model or measurement operator introduces non-Gaussian features in the system state. The Ensemble Kalman filter (EnKF) and Particle filter (that uses EnKF for the proposal density) handle the state estimation problem. The methodology is illustrated with two numerical examples, namely the Double-Well system and a mass-spring-damper system having multiple types of nonlinearities such as freeplay, cubic stiffness and hysteresis.