Preheat a grill to medium-high. Rub the steak with the paprika and 1/2 teaspoon each salt and pepper; let sit. Meanwhile, in a large pot, combine the potatoes and enough salted water to cover. Bring to a boil and cook until crisp-tender, about 8 minutes; drain and transfer to a large bowl. Add the olive oil, garlic and lemon juice; season with salt and pepper and toss to coat.
Arrange the potatoes on the grill and cook, turning once, until crisp, about 5 minutes. Transfer the potatoes back to the bowl and let cool. Toss in the arugula.
Grill the steak, turning once, for 4 to 6 minutes for medium-rare. Slice into 1/4-inch-thick slices and serve over the potato salad.