In a large saucepan over medium heat, cook the bacon, stirring occasionally, until the fat renders, about 6 minutes; discard the fat. Add the olive oil and garlic and cook, stirring, until golden, 2 to 3 minutes. Add the onion and celery stalks and cook, stirring, until softened, about 6 minutes. Stir in the potato, the tomatoes and their juice and the chicken broth and bring to a boil. Reduce the heat and simmer until the potato is tender, 15 to 20 minutes.
Stir in the clams with their liquid and cook to warm through. Stir in the chopped celery leaves and season with salt and pepper.