Massive uverse latency

My internet is extremely slow in practical usage.  The problem seems to be the latency.  Below I copied from the gateway speed test page.  Look at the latency values.  It makes the internet practically unusable.  It is not like this all the time, but whenever my internet is going slow and I go test it, the latency is huge.  Then it goes back down to 30-50 without doing anything on my side.  How can I fix it, short of switching to another provider (which I am close to doing).

WARNING: During this Speed Test, services that are using the network (e.g., web browsers, voice calls, video) may have their performance impacted. In addition, the results of this Speed Test will be lower when these services are in use during the test.

Test History

Test Completion Time Direction Throughput(Mbps) Overhead(Kbps) Average Latency(ms)

06/14/19 4:16:39 PM	downstream	24.261	1369	807.348
06/14/19 4:16:20 PM	upstream	1.525	87	1055.542
05/22/19 2:58:00 PM	downstream	25.131	1418	1644.831
05/22/19 2:56:57 PM	downstream	24.825	1401	1142.826
05/22/19 2:56:35 PM	upstream	1.767	100	1568.214