It is known that adding bounded-error randomization to PSPACE doesn't add power. That is, BPPSAPCE=PSPACE.
It is famously unknown whether P=BPP, but it is known that $BPP\subseteq \Sigma_2\cap \Pi_2$.
Thus, it is possible (while conjectured to be false) that adding probability to P adds expressive power.
My question is whether we know (or have evidence of) the border between P and PSPACE where adding randomization no longer adds power.
Specifically,
Are there any problems that are known to be in $BP\Sigma_i$ (resp. $BP\Pi_i$) that are not known to be in $\Sigma_i$ (resp. $\Pi_i$)? And similarly for $BPPH$ v.s. $PH$?
There is a difficulty with the premise of your question — "when does randomization stops helping within $\mathrm{PSPACE}$ — because it suggests that the computational classes $\mathrm{X}$ such that $\mathrm{P \subseteq X \subseteq PSPACE}$ form some sort of linear hierarchy when this is not evident.
We can illustrate this by comparisons between the polynomial hierarchy and counting classes. As Emil Jebek indicates in the comments, $$\begin{align*} \mathrm{BP\cdot \Sigma_i^p \subseteq \Pi_{i+1}^p} &&\text{and}&& \mathrm{BP\cdot \Pi_i^p \subseteq \Sigma_{i+1}^p} \end{align*}$$ by relativisation of $\mathrm{AM \subseteq \Pi_2^p}\,$; and therefore $\mathrm{BP \cdot PH = PH}$. On the other hand, Toda's Theorem shows that $$\mathrm{PH \subseteq BP\cdot\oplus P}.$$ If you suppose that "randomization has stopped adding power by the time you ascend to $\mathrm{PH}$", then you will be tempted to suspect that because $\mathrm{PH \subseteq BP \cdot \oplus P}$, perhaps in fact $\mathrm{BP \cdot \oplus P = \oplus P}$. But I don't know that anyone conjectures this, or even that $\mathrm{PH \subseteq \oplus P}$ (which would be a necessary consequence); I think that any result of this sort would be considered a major breakthrough.
Of course, if you only care about the polynomial hierarchy, and more generally (to scale up to $\mathrm{PSPACE}$) quantified boolean formulas, then you can extract some sort of linear answer to your question — in which case Emil's comments are about as complete an answer as you are likely to get.
External links referenced by this document: