An ordered partition of {1, 2, ..., n} into k blocks B₁, B₂, ..., B_k is a partition where the order of blocks is considered. In the present paper, we consider the case that each block B_i has r_i copies. Using this extension of ordered set partitions, we introduce a new generalization of the Lah and Stirling numbers of both kinds which called multivariate Lah and Stirling numbers, respectively. We study several combinatorial properties such as explicit formulas, recurrence relations, generating functions, and some convolution identities.