Every non-negative integer can be written using the factorial base representation. We explore certain combinatorial structures arising from the arithmetic of these representations. In particular, we investigate the sum-of-digits function, carry sequences, and a partial order referred to as digital dominance. Finally, we describe an analog of a classical theorem due to Kummer that relates the combinatorial objects of interest by constructing a variety of new integer sequences.