Preheat the oven to 350 degrees . Rub the peppers with 2 tablespoons olive oil; season with salt and pepper. Stand in a baking dish, cover with the tops and roast for 30 minutes.
Meanwhile, in a medium skillet, heat 1/2 tablespoon olive oil over medium heat. Add the bacon and cook until browned, about 4 minutes; transfer to a medium bowl. In the same skillet, heat the remaining 1/2 tablespoon olive oil. Cook the garlic and oregano until fragrant, about 2 minutes; stir into the bacon along with the beans, tomatoes, mozzarella and breadcrumbs. Season with salt and pepper.
Fill the peppers with the bean mixture; return to the oven and heat through, about 15 minutes. Set the tops into place.