Preheat your oven to 375°F and generously spray a 9×13 inch baking dish with nonstick cooking spray, making sure to coat the bottom and sides evenly. This prevents sticking and makes cleanup much easier.
Cut the peeled sweet potatoes into 1-inch pieces and place them in a large pot, then cover with cold water. Bring to a boil over high heat and cook for 15-20 minutes until the potatoes are fork-tender and easily pierced. Drain well in a colander and let cool for about 5 minutes so they’re easier to handle and won’t break apart when mashing.
Transfer the cooled sweet potatoes to a large mixing bowl and add the brown sugar, unsalted butter, salt, and vanilla extract. Mash everything together until you reach your desired consistency—I like to leave it slightly chunky rather than completely smooth for better texture. Fold in the first 1/4 cup of finely chopped pecans, distributing them evenly throughout the mixture.
Spread the sweet potato mixture evenly into the prepared baking dish, smoothing the top with a spatula. Sprinkle the remaining 1/4 cup of chopped pecans over the top in an even layer, then arrange the mini marshmallows on top in a single layer, leaving small gaps between them so they can puff up and brown beautifully during baking.
Place the assembled casserole in the preheated 375°F oven and bake for 20 minutes until the marshmallows are puffed and golden brown on top. Watch carefully during the last few minutes so the marshmallows don’t burn—I find they go from perfectly golden to charred pretty quickly! Let the casserole rest for 5 minutes before serving so it sets slightly and becomes easier to portion.