I spent $50 testing LLM serving settings 23% more throughput on the same GPU

While I was surfing through vllm offical slack someone asked a question that why does vllm do not support gtx 1080 that is when I thought what if the hardware had the capabilties but we are not really optimizing them I saw few methods online tried to combine them and created this result Same GPU, same model 23% more throughput just by changing the serving settings. While working on my personal project, I wondered how much performance I was leaving behind with default settings. So I built PolyServe and spent around $50 on GPU experiments to investigate.(too much for me as I'm currently a studen