Mathematical models are essential for combining data from multiple sources to quantify population endpoints. This is especially true for species, such as marine mammals, for which data on vital rates are difficult to obtain. Since the effects of an environmental disaster are not fixed, we develop time-varying (nonautonomous) matrix population models that account for the eventual recovery of the environment to the pre-disaster state. We use these models to investigate how lethal and sublethal impacts (in the form of reductions in the survival and fecundity, respectively) affect the population's recovery process. We explore two scenarios of the environmental recovery process and include the effect of demographic stochasticity. Our results provide insights into the relationship between the magnitude of the disaster, the duration of the disaster, and the probability that the population recovers to pre-disaster levels or a biologically relevant threshold level. To illustrate this modeling methodology, we provide an application to a sperm whale population. This application was motivated by the 2010 Deepwater Horizon oil rig explosion in the Gulf of Mexico that has impacted a wide variety of species populations including oysters, fish, corals, and whales.