Place the potatoes in a large pot; add cold water to cover by 1 inch. Bring to a boil; add salt. Reduce heat to medium; simmer until tender, about 10 minutes.
Meanwhile, in a small saucepan, heat the milk and 2 sticks butter over medium until the butter melts.
Drain the potatoes; return to the pot. Add the milk mixture and horseradish. Mash to the desired consistency; season with salt and pepper. Transfer to a bowl and top with more butter.