Fix an integer N ≥ 2. To each diagram of a link colored by 1,...,N we associate a chain complex of graded matrix factorizations. We prove that the homotopy type of this chain complex is invariant under Reidemeister moves. When every component of the link is colored by 1, this chain complex is isomorphic to the chain complex defined by Khovanov and Rozansky. The homology of this chain complex decategorifies to the Reshetikhin-Turaev 𝔰𝔩(N) polynomial of links colored by exterior powers of the defining representation.