In a large bowl, stir the coffee, cinnamon, cardamom, cloves, pepper and 5 cups cold water. Cover and let steep in the refrigerator until flavors infuse, at least 12 hours or overnight.
Line a fine metal sieve with 4 layers of cheesecloth or a kitchen towel (not terrycloth); set over a large bowl. Strain the coffee into the bowl; strain again. Discard the grounds. Serve over ice. Add milk to taste.