Position racks in the upper and lower thirds of the oven and preheat to 400 degrees . In a saucepan, cook the mangoes, brown sugar, butter, lemon juice, lemon peel, rum and vanilla, stirring, until thickened, 10 minutes. Transfer to a bowl and refrigerate.
Meanwhile, line 2 baking sheets with parchment. On a lightly floured surface, using a 4 1/2-inch round cookie cutter, cut 6 rounds from the puff pastry. Roll out each round to 6 inches in diameter; transfer to the prepared baking sheets. Spoon about 3/4 cup of the cooled mango mixture in the center of each round, leaving a 3/4-inch border. Fold the edges inward to form a 1/2-inch rim; sprinkle with the granulated sugar.
Bake the tartlets, switching and rotating the pans halfway through, until golden, 20 to 25 minutes. Serve with the ice cream.