Preheat a grill to high. In a small bowl, stir together the mustard and Cajun spices, then mix in 2 tablespoons olive oil. Brush the mixture on the chicken; sprinkle with salt. Cover and grill, turning once, until an instant-read thermometer inserted in the thigh registers 155 degrees , about 1 hour.
Meanwhile, in a medium saucepan, cover the potatoes with salted water. Bring to a boil, then lower the heat and simmer until fork-tender, about 10 minutes; drain.
About 15 minutes before the chicken is done, toss the potatoes with the remaining 2 tablespoons olive oil; season with salt and pepper. Grill, turning, until crisp.
Transfer the chicken to a cutting board; let rest for 10 minutes before carving. Serve with the potatoes.
