Preheat the oven to 350 degrees . In a deep, 10-inch, ovenproof skillet, melt 5 tablespoons butter over medium heat. Add the brown sugar and cook, stirring, until creamy. Remove from the heat and add the plums, skin side down, in a single layer.
In a large bowl, combine the flour, granulated sugar, baking powder and salt. Stir in the remaining 1 stick butter until evenly distributed. Add the eggs, milk and vanilla and stir until smooth. Spread over the plums in an even layer.
Bake the cake until a toothpick inserted in the center comes out clean, 40 to 45 minutes. Let cool in the skillet for about 25 minutes, then run a knife around the edge and invert the cake onto a platter. Let cool for 20 minutes more before cutting into wedges. Serve with whipped cream.