Heat your oven to 350°F. In a bowl, combine 1 cup all-purpose flour, 1/3 cup granulated sugar, 1/3 cup brown sugar, 3/4 tsp cinnamon, 1/4 tsp nutmeg, and 1/8 tsp salt. Cut in 6 tbsp cold cubed butter until the mixture resembles coarse crumbs and just begins to hold together when squeezed. Spread this mixture onto a baking sheet and bake for 15 minutes, stirring twice during baking, until golden and fragrant. Remove and set aside to cool completely—you'll use this crumble topping throughout the cake. While the crumble bakes, prepare your peaches: boil them for 1-2 minutes, then peel and slice one peach and dice the other. Toss the sliced peach with 1/2 tsp cinnamon and 1/4 tsp nutmeg, and toss the diced peach separately with the same spices. This separate preparation allows you to layer the peaches for better texture and flavor distribution in the cake.

While the crumble finishes, grease and flour a 9x13 inch baking dish (or similar size cake pan). In a saucepan, combine 1 cup granulated sugar and 1/3 cup water, then cook over medium-high heat without stirring until the mixture turns a deep amber color (about 8-10 minutes)—swirl the pan gently if needed to ensure even cooking. Carefully pour this caramel over the bottom of your prepared pan, then arrange the sliced peaches in a single layer over the caramel. The caramel will set slightly as it cools, creating a flavorful base for the pound cake batter.

In a medium bowl, whisk together 3 cups sifted cake flour, 2 1/4 tsp baking powder, 3/4 tsp salt, and 3/4 tsp cinnamon. Set this dry mixture aside—I like to sift the flour twice for pound cake to ensure a tender, delicate crumb that doesn't compete with the peach layers.

In a large mixing bowl, beat 1 1/2 cups softened butter for 2 minutes until pale and fluffy. Add 2 cups granulated sugar and beat for another 2 minutes until the mixture is light and creamy—this creaming process incorporates air, which is crucial for pound cake's tender structure. Add 6 room-temperature eggs one at a time, beating well after each addition, then mix in 1 tbsp vanilla extract and 1/4 tsp almond extract. Now alternate adding the flour mixture from Step 3: add half of it and mix until just combined, then pour in 3/4 cup buttermilk and mix gently, then add the remaining flour mixture and fold until no streaks remain. Be careful not to overmix once you add the flour, as this can toughen the cake.

Pour two-thirds of the pound cake batter from Step 4 over the caramel and peach slices in your prepared pan, spreading gently to create an even layer. Scatter the diced peaches from Step 1 over this layer, then sprinkle 1 cup of the cooled crumble topping from Step 1 over the peaches. Pour the remaining batter over the crumble and top with the final 1 cup of crumble. Bake at 350°F for 1 hour, or until a toothpick inserted into the cake portion (not through the peaches) comes out clean. The cake should be golden brown and the crumble topping should be dark and crispy.

Remove the cake from the oven and let it cool in the pan for 30 minutes—this allows the structure to set while the caramel is still warm enough to act as a "glue" when you flip. Carefully invert the cake onto a serving platter, letting gravity help release it from the pan. Some caramel and peaches may stick to the bottom of the pan; simply spoon these over the top of the cake. Let the cake cool completely before adding the glaze.

In a small bowl, beat 1/4 cup softened cream cheese and 2 tbsp softened butter together until smooth and combined. Add 1/2 cup powdered sugar, 1/2 tsp vanilla extract, and 2-3 tbsp milk (adding milk gradually until you reach a pourable consistency), then whisk until the glaze is smooth and silky. Drizzle the glaze over the cooled cake and sprinkle the remaining crumbs from Step 1 over the top for a beautiful finish and extra texture contrast.