Preheat the oven to 375°F (190°C) and line two 9×13 inch baking pans with parchment paper on the bottom, spraying the sides with non-stick spray. In a medium-sized bowl, whisk together the all-purpose flour, unsweetened cocoa powder, baking powder, and salt until well combined. Set this dry mixture aside.
In a large bowl, beat together the unsalted butter and sugar just until the mixture is smooth and combined. Add in the egg and milk, mixing until well incorporated. Gradually add the dry ingredient mixture from Step 1 and mix just until a smooth batter forms. Be careful not to overmix.
Divide the batter evenly between the two prepared pans, using a food scale or measuring cups to ensure both layers are equal and spread the batter thinly and evenly. Bake the cake layers for 5–8 minutes, or until a toothpick inserted in the center comes out clean. Let the cakes cool mostly in the pans, then turn out onto a parchment-lined cooling rack sprayed lightly with non-stick spray to prevent sticking. Allow cakes to cool completely. I always make sure the 'tops' look nice since they'll be the outside of the sandwiches.
Once the cake layers are completely cool, place them in the freezer for about 20–30 minutes or longer if you're working ahead. At the same time, remove the first flavor of ice cream (vanilla) from the freezer and let it sit at room temperature for 15–20 minutes until it's soft enough to spread. This helps ensure smooth assembly later.
Line one of the clean 9×13-inch pans with parchment paper, leaving extra on the long sides for easy lifting later. Place one frozen cake layer at the bottom with the 'pretty side' facing down. Scoop about half of the softened vanilla ice cream into a large bowl and stir until smooth and spreadable. Evenly spread the vanilla ice cream over the cake layer. Freeze the pan for 30 minutes to an hour, until the ice cream is firm enough for the next layer.
About 15–20 minutes before adding the next ice cream flavor, set the strawberry ice cream out to soften. Stir about half of the softened strawberry ice cream in a bowl until smooth and spreadable. Gently spread the strawberry ice cream over the firm vanilla layer in the pan. Top with the second frozen cake layer, 'pretty side' up, pressing lightly into the ice cream to help it adhere and fill in any gaps. Freeze the assembled sandwich for at least 2 hours, or until completely set. For even more flavor, I sometimes add a thin layer of fresh sliced strawberries between the ice cream layers, but that's optional.
Using the parchment paper overhang, lift the frozen ice cream sandwich out of the pan. Use a long knife to trim the rough edges, then slice into bars of your desired size. Return the bars to the freezer until you’re ready to serve. I like to wrap individual bars in parchment for easy grab-and-go treats.