Preheat the oven to 350 degrees . In a small bowl, stir together the flour and baking powder. Set aside.
In the bowl of a mixer, blend 3/4 cup of the peanut butter with the butter on medium speed. Add the granulated sugar and brown sugar and beat until smooth, about 5 minutes. Beat in the egg until incorporated. Stir in the reserved flour mixture until just combined.
Using a small (1 1/2-inch) ice cream scoop or a tablespoon, drop the dough 2 inches apart on 2 ungreased, parchment-lined baking sheets. Using the back of a melon baller or your thumb, gently dent the center of each cookie. Spoon about 1/2 teaspoon of jelly into each center. Bake until lightly golden, about 12 minutes. Remove to a rack and cool.
In a small bowl, beat together the remaining 1/4 cup of peanut butter and the confectioners sugar. Whisk in the milk, a little at a time, until the frosting reaches a good consistency for piping. Using a resealable sandwich bag with a tiny corner snipped off, pipe the frosting over the cookies.