In a medium-sized mixing bowl, combine the flour, granulated sugar, baking powder, cinnamon, ginger, salt, and nutmeg, and whisk until well blended.
In a separate bowl, beat together the heavy cream, vanilla yogurt, and egg until smooth. Add this mixture to the dry ingredients, stirring gently until just combined. Fold in the diced peaches.
Pour oil into a frying pan until halfway full. Heat the oil over medium heat until it reaches 375°F, checking the temperature with a candy thermometer.
Gently drop ¼ cup of batter into the heated oil. Fry for 2-4 minutes on each side until golden brown, adjusting time based on the oil’s temperature. Flip and fry the other side until golden. To ensure doneness, insert a toothpick into the center; if batter clings to the toothpick, cook a bit longer. Transfer each fritter to a paper towel-lined rack to cool. Repeat with remaining batter, maintaining the oil temperature.
For the glaze, whisk together confectioners' sugar, vanilla, and 3 teaspoons of reserved peach juice. Add additional peach juice, 1 teaspoon at a time, until the glaze reaches a pourable consistency.
Drizzle the glaze over the cooled fritters, allowing it to set before serving.