Microtubules are organized by the centrosome, a dynamic organelle that exhibits changes in both size and number during the cell cycle. Here we show that SZY-20, a putative RNA-binding protein, plays a critical role in limiting centrosome size in C. elegans. SZY-20 localizes in part to centrosomes and in its absence centrosomes possess increased levels of centriolar and pericentriolar components including gamma-tubulin and the centriole duplication factors ZYG-1 and SPD-2. These enlarged centrosomes possess normal centrioles, nucleate more microtubules, and fail to properly direct a number of microtubule-dependent processes. Depletion of ZYG-1 restores normal centrosome size and function to szy-20 mutants, whereas loss of szy-20 suppresses the centrosome duplication defects in both zyg-1 and spd-2 mutants. Our results describe a pathway that determines centrosome size and implicate centriole duplication factors in this process.