In a small saucepan, stir together 2 cups milk and the chocolate pudding mix over medium heat. Bring to a boil, stirring constantly. Let cool for 5 minutes.
Meanwhile, in another small saucepan, stir together the remaining 2 cups milk and the vanilla pudding mix over medium heat. Bring to a boil, stirring constantly. Let cool for 5 minutes.
Pour the coffee into a shallow dish next to an 8-inch pie pan. Dip the ladyfingers into the coffee, turning to coat, and arrange in a single layer in the pie pan. Sprinkle with half of the cognac. Spread the chocolate pudding on top, then cover with another layer of coffee-dipped ladyfingers. Sprinkle the remaining cognac over the ladyfingers, then spread the vanilla pudding on top. Press a sheet of plastic wrap onto the top of the pudding and refrigerate until set and chilled, about 3 hours. Dust with the cocoa powder and serve with the whipped cream.