The cardinal feature of neuronal polarization is the establishment and maintenance of axons and dendrites. How axonal and dendritic proteins are sorted and targeted to different compartments is poorly understood. Here, we identified distinct dileucine motifs that are necessary and sufficient to target transmembrane proteins to either the axon or the dendrite through direct interactions with the clathrin-associated adaptor protein complexes (APs) in C. elegans. Axonal targeting requires AP-3, while dendritic targeting is mediated by AP-1. The axonal dileucine motif binds to AP-3 with higher efficiency than to AP-1. Both AP-3 and AP-1 are localized to the Golgi but occupy adjacent domains. We propose that AP-3 and AP-1 directly select transmembrane proteins and target them to axon and dendrite, respectively, by sorting them into distinct vesicle pools.