Memory-efficient attention for decoding. It supports page size >= 1. It supports FP8 quantized KV cache with on-the-fly dequantization.
Memory-efficient attention for decoding. It supports page size >= 1. It supports FP8 quantized KV cache with on-the-fly dequantization.