Preheat your oven to 350°F. In a bowl, combine the all-purpose flour, granulated sugar, light brown sugar, 1/2 teaspoon cinnamon, 1/4 teaspoon nutmeg, and salt. Add the cold cubed butter and mix with your fingertips until the mixture resembles coarse breadcrumbs and just begins to hold together. Spread the mixture evenly on a baking sheet and bake for 15 minutes, stirring twice during baking to ensure even toasting. Once golden and fragrant, remove from the oven and let cool completely, then break into bite-sized crumbles. This can be done while you prepare other components.

Peel both peaches and prepare them differently: slice one peach into thin, even slices and toss with 3/4 teaspoon cinnamon, 1/4 teaspoon nutmeg, and bourbon if using; dice the second peach and toss separately with the same spices. Grease and flour a 9x13 inch baking dish, then arrange the sliced peaches in a single layer on the bottom. In a saucepan, combine 1 cup sugar and 1/4 cup water, then heat over medium-high heat without stirring until the mixture turns a deep amber color (about 8-10 minutes). Carefully pour the hot caramel over the peach slices—I like to work quickly here as caramel sets fast, and the heat will gently cook the peaches.

In a medium bowl, whisk together the cake flour, baking powder, salt, and 1/2 teaspoon cinnamon; set aside. In a large bowl, beat the softened butter and 2 cups sugar together for 2 minutes until light and fluffy. Add the room-temperature eggs one at a time, beating well after each addition, then mix in the vanilla extract. Alternate adding the dry ingredient mixture and buttermilk to the butter mixture in three additions (dry-wet-dry-wet-dry), beating on low speed after each addition just until combined. I find that using room-temperature eggs ensures a smoother, more homogeneous batter that incorporates evenly.

Pour two-thirds of the pound cake batter from Step 3 into the prepared baking dish over the caramel and peach slices, smoothing the top gently. Scatter the diced peaches from Step 2 over the batter, then sprinkle 1 cup of the cooled crumble topping from Step 1 evenly over them. Pour the remaining batter on top, smooth it gently, and finish with the remaining crumble topping. Bake at 350°F for approximately 1 hour, until a toothpick inserted into the cake portion comes out clean and the top is golden brown.

Remove the cake from the oven and let it cool in the pan for 30 minutes—this cooling time allows the caramel to set slightly so the cake holds together when inverted. While the cake cools, prepare the glaze by beating together the room-temperature cream cheese, softened butter, sifted powdered sugar, milk, and vanilla extract until smooth and pourable. Carefully invert the cooled cake onto a serving plate so the caramel and peach layer is now on top. Drizzle the glaze over the warm cake and top with any remaining crumbles.