Slow Voice despite decent Hardware

The live mode voice takes a little too long to start talking for my taste. I am aware that usually, this would boil down to getting faster hardware, but it seems my PC is not even really using its resources. my 4070 for example hovers around 10%. Is this an error on my end or is there no way for me to make it faster?

specs:
CPU: Ryzen 7 3800x
RAM: 32gb ddr4 3200mhz
GPU: RTX 4070 12GB