Preheat oven to 350°F. Grease and flour a 9x13 inch baking pan or line with parchment.
Whisk flour, baking powder, baking soda, and salt in a medium bowl.
In a large bowl, cream butter and sugar until light and fluffy, about 2 to 3 minutes. Beat in eggs one at a time, then mix in vanilla.
Add dry ingredients to the butter mixture in 2 additions, alternating with the buttermilk. Mix just until combined, then fold in diced rhubarb and orange zest.
Spread batter into the prepared pan. Sprinkle coarse sugar on top if using. Bake until golden and a toothpick comes out clean, 35 to 40 minutes.
While the cake bakes, make the tangy sauce: In a saucepan, combine sliced rhubarb, sugar, orange juice, and water. Bring to a simmer over medium heat and cook until rhubarb softens, 6 to 8 minutes.
Stir cornstarch with 2 tablespoons cold water to make a slurry. Whisk into the simmering sauce and cook until thickened and glossy, 1 to 2 minutes. Remove from heat and stir in lemon juice, vanilla, and a pinch of salt. Keep warm.
Cool cake in the pan for 10 minutes. Slice and serve warm with spoonfuls of tangy rhubarb sauce.
Notes
For deeper flavor, swap half the sauce sugar for light brown sugar. The cake keeps covered at room temperature for 2 days; refrigerate the sauce separately for up to 1 week. Stir in a pinch of ground ginger to the sauce for a zesty twist.