Preheat a grill or grill pan to medium-high. In a large bowl, combine 1/4 cup EVOO and the garlic, Worcestershire sauce, mustard, paprika and hot pepper sauce. Add the chicken, turn to coat evenly and let marinate for 15 minutes. Grill, turning once, until the juices run clear when the chicken is pierced, about 5 minutes on each side.
Drizzle the stone fruits with EVOO and season with salt and pepper. Place on a grill, without touching the chicken, and cook until grill marks appear, 5 to 6 minutes.
In a small saucepan, bring the vinegar, brown sugar and rosemary to a boil. Lower the heat and simmer until reduced by about half, 6 to 7 minutes (watch carefully to avoid burning). Serve the chicken with the fruit alongside and drizzle the sauce over both.