Spread the peanut butter on 4 bread slices and the jelly on the other 4 slices, and combine to make 4 sammies.
In a wide, medium bowl, whisk together the eggs, milk and vanilla.
In a large nonstick skillet, heat 2 tablespoons butter over medium heat. When the butter is almost melted, dunk two of the sammies into the french toast batter and let the excess drip off. Place in the pan and cook until golden brown, 3 to 4 minutes for each side. Repeat with the remaining 2 tablespoons butter and 2 sammies. Let cool slightly and cut into sticks; serve with the warm maple syrup.