모종의 음모/ai 프로그램
LLM / 1080 ti 모드 별 벤치마크
구차니
2026. 9. 17. 10:36
궁금한 것 을 해소하기 위해 대충 5시간에 걸쳐 GPU 갈굼
- MTP 가 정말 속도에 영향을 주냐
- GPU 2개인데 pcie3.0 x8과 x4에 영향을 받는가
- 컨텍스트 최대 길이가 성능에 영향을 미치는가
- 입력 토큰길이에 따라 성능 저하는 어느정도 발생하는가
- 단일과 -sm layer로 나누었을때 성능은 어떠한가
결과만 요약하면
- GPU 0 / 1이 우선권 문제인지 lane 폭 문제인진 모르겠으나 GPU0이 3~10% 정도 이상 성능이 좋게 나옴
- MTP는 켤수 있으면 켜라. 조건에 따라 다르지만 MTP 1이 1.3~1.5배 향상됨
- 컨텍스트 최대 길이는 의외로 생성 성능에 영향을 안준다.
- 한쪽에 가능하지만, 억지로 레이어 쪼개서 양쪽으로 올리면 1.1~1.4배 성능 차이가 난다. 되도록이면 1대로 돌리자
아래는 테스트용 스크립트들. 추후 쓰려면 바이너리 경로등을 손봐야 함
아래는 원본 내용
llama.cpp Benchmark Report
Short conversation (3-turn, context accumulates)
Questions: see questions.txt. Columns show prompt_t/s / generation_t/s per turn.
gemma4-12b
quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status
| gemma-4-12b-it-Q4_0 | none | 0 | 131072 | off | 139.2/34.8 | 143.4/34.6 | 270.3/34.4 | 38.4 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 131072 | on(n=1) | 123.9/57.5 | 141.8/56.1 | 246.3/51.2 | 21.5 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 131072 | on(n=2) | 123.3/61.5 | 187.3/61.7 | 238.3/59.1 | 18.6 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 131072 | on(n=3) | 118.4/65.5 | 227.8/59.7 | 231.2/51.6 | 18.9 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 131072 | on(n=4) | 123.2/65.5 | 181.6/57.1 | 240.6/45.5 | 20.0 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 65536 | off | 140.0/33.5 | 146.5/33.2 | 266.5/32.8 | 27.0 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 65536 | on(n=1) | 118.5/54.5 | 138.2/55.1 | 245.5/51.2 | 20.0 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 65536 | on(n=2) | 121.3/59.5 | 226.9/62.4 | 229.2/51.9 | 19.0 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 65536 | on(n=3) | 122.9/66.3 | 243.9/52.7 | 222.2/52.1 | 19.0 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 65536 | on(n=4) | 118.3/65.2 | 139.6/60.8 | 245.8/48.4 | 19.1 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 32768 | off | 136.6/33.6 | 146.1/33.2 | 263.9/32.7 | 27.0 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 32768 | on(n=1) | 125.1/57.3 | 140.9/55.5 | 242.6/51.8 | 19.7 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 32768 | on(n=2) | 120.7/65.7 | 139.8/58.8 | 243.7/57.5 | 18.3 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 32768 | on(n=3) | 125.6/63.6 | 139.2/60.2 | 244.5/53.6 | 18.7 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 32768 | on(n=4) | 113.1/60.4 | 182.1/63.9 | 242.7/43.8 | 19.8 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 16384 | off | 139.6/33.5 | 146.0/32.9 | 267.0/32.7 | 27.1 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 16384 | on(n=1) | 122.8/55.8 | 213.0/56.2 | 230.9/53.9 | 19.6 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 16384 | on(n=2) | 124.2/62.4 | 139.8/60.7 | 243.0/54.1 | 18.7 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 16384 | on(n=3) | 122.5/60.7 | 140.1/63.2 | 244.8/53.5 | 18.7 | OK |
| gemma-4-12b-it-Q4_0 | none | 0 | 16384 | on(n=4) | 120.8/60.6 | 141.5/58.4 | 245.8/50.0 | 19.4 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 131072 | off | 135.6/32.8 | 249.3/32.7 | 225.6/32.5 | 27.0 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 131072 | on(n=1) | 106.7/52.1 | 130.0/51.5 | 223.8/49.9 | 22.8 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 131072 | on(n=2) | 117.4/59.5 | 129.0/57.8 | 224.2/53.2 | 21.3 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 131072 | on(n=3) | 117.7/61.5 | 155.4/54.7 | 218.6/52.5 | 21.5 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 131072 | on(n=4) | 112.3/62.0 | 128.8/56.6 | 223.6/45.7 | 22.1 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 65536 | off | 130.6/31.5 | 243.5/30.9 | 223.0/30.6 | 29.4 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 65536 | on(n=1) | 117.7/50.5 | 130.1/50.2 | 225.2/48.2 | 23.2 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 65536 | on(n=2) | 117.1/57.9 | 128.0/56.5 | 222.8/49.6 | 21.9 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 65536 | on(n=3) | 117.0/51.0 | 129.5/54.2 | 222.5/51.0 | 22.5 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 65536 | on(n=4) | 117.0/61.7 | 225.9/59.6 | 200.0/45.0 | 21.5 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 32768 | off | 133.9/31.7 | 212.7/31.0 | 227.7/30.7 | 30.3 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 32768 | on(n=1) | 119.1/49.3 | 130.3/51.8 | 222.2/48.9 | 22.9 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 32768 | on(n=2) | 119.2/61.1 | 130.8/57.4 | 224.5/48.8 | 21.4 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 32768 | on(n=3) | 119.4/58.1 | 127.6/53.1 | 216.0/45.0 | 22.4 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 32768 | on(n=4) | 118.7/60.7 | 130.8/53.0 | 225.1/45.8 | 22.2 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 16384 | off | 132.8/31.8 | 137.5/31.0 | 244.6/30.7 | 30.1 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 16384 | on(n=1) | 107.6/52.2 | 227.2/51.0 | 200.4/48.9 | 21.2 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 16384 | on(n=2) | 110.5/57.1 | 209.0/56.6 | 204.2/51.1 | 21.1 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 16384 | on(n=3) | 117.0/59.2 | 129.6/57.1 | 208.3/46.9 | 21.8 | OK |
| gemma-4-12b-it-Q4_0 | none | 1 | 16384 | on(n=4) | 117.1/63.0 | 193.5/54.3 | 208.2/43.3 | 22.3 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 131072 | off | 138.0/33.4 | 145.7/33.3 | 260.4/33.2 | 28.0 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 131072 | on(n=1) | 123.1/49.2 | 136.9/48.5 | 238.2/47.3 | 23.3 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 131072 | on(n=2) | 122.3/55.9 | 239.8/50.8 | 213.4/45.0 | 22.8 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 131072 | on(n=3) | 122.4/49.1 | 135.9/48.8 | 236.3/43.6 | 23.7 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 131072 | on(n=4) | 118.0/53.8 | 136.3/47.4 | 237.4/38.9 | 24.4 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 65536 | off | 136.4/33.1 | 143.0/33.0 | 255.2/33.0 | 28.1 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 65536 | on(n=1) | 122.1/48.4 | 225.8/49.4 | 218.4/47.4 | 22.5 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 65536 | on(n=2) | 122.3/54.4 | 136.8/51.9 | 237.7/44.2 | 22.5 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 65536 | on(n=3) | 122.4/45.0 | 136.5/47.1 | 237.8/44.8 | 24.0 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 65536 | on(n=4) | 122.4/53.7 | 136.4/47.1 | 238.4/42.6 | 23.3 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 32768 | off | 140.6/33.1 | 145.9/33.1 | 260.2/33.0 | 27.9 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 32768 | on(n=1) | 122.9/48.3 | 137.3/49.0 | 239.4/47.7 | 22.8 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 32768 | on(n=2) | 109.1/52.3 | 137.7/52.0 | 239.2/46.8 | 22.2 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 32768 | on(n=3) | 113.3/53.9 | 138.1/47.0 | 239.0/45.8 | 22.7 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 32768 | on(n=4) | 124.3/52.7 | 237.4/48.5 | 217.2/38.7 | 23.7 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 16384 | off | 135.4/32.8 | 143.2/32.8 | 257.2/32.8 | 28.1 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 16384 | on(n=1) | 118.5/50.7 | 135.7/49.0 | 237.0/47.7 | 22.5 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 16384 | on(n=2) | 120.0/53.6 | 216.3/52.3 | 219.0/46.5 | 21.9 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 16384 | on(n=3) | 119.0/53.8 | 235.9/49.9 | 214.2/46.3 | 21.2 | OK |
| gemma-4-12b-it-Q4_0 | layer | both | 16384 | on(n=4) | 98.1/51.2 | 236.1/46.7 | 214.0/40.8 | 23.8 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 131072 | off | 85.4/31.3 | 93.9/31.1 | 203.3/30.5 | 36.9 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=1) | 78.3/42.7 | 166.5/42.6 | 181.6/40.2 | 24.0 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=2) | 78.5/45.1 | 92.4/43.7 | 184.5/34.0 | 24.7 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=3) | 78.6/40.4 | 91.7/41.9 | 189.3/31.6 | 26.2 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=4) | 78.9/42.5 | 153.1/39.7 | 183.6/26.9 | 27.7 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 65536 | off | 85.2/30.8 | 92.6/29.7 | 203.0/29.5 | 29.2 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=1) | 78.9/45.4 | 163.3/42.7 | 181.7/40.6 | 22.6 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=2) | 78.0/43.3 | 91.8/42.6 | 190.4/35.2 | 24.6 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=3) | 79.2/41.9 | 131.7/42.2 | 185.5/32.4 | 25.5 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=4) | 78.1/42.2 | 92.5/41.4 | 191.5/35.2 | 25.1 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 32768 | off | 84.8/30.6 | 93.7/29.6 | 203.1/29.4 | 29.4 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=1) | 73.8/44.8 | 157.0/42.1 | 182.4/39.7 | 23.7 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=2) | 70.6/43.6 | 155.4/46.0 | 181.4/36.8 | 23.9 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=3) | 77.4/40.6 | 93.5/43.4 | 189.3/29.6 | 26.3 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=4) | 78.5/43.6 | 92.2/36.1 | 188.0/30.9 | 26.7 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 16384 | off | 83.1/30.8 | 96.2/29.6 | 198.6/29.4 | 29.3 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=1) | 78.3/44.4 | 93.2/43.3 | 191.1/40.6 | 23.4 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=2) | 76.5/45.0 | 92.9/44.7 | 187.7/36.5 | 23.9 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=3) | 78.8/43.9 | 92.3/44.5 | 188.9/33.0 | 24.8 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=4) | 68.9/45.3 | 173.1/37.3 | 181.7/31.7 | 25.1 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 131072 | off | 84.3/30.6 | 93.4/30.5 | 192.8/30.2 | 30.1 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=1) | 76.0/43.4 | 178.7/40.8 | 164.0/37.0 | 26.3 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=2) | 76.1/44.7 | 90.1/42.6 | 174.1/36.0 | 26.0 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=3) | 71.8/38.8 | 177.5/38.2 | 163.8/37.9 | 27.1 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=4) | 75.7/38.1 | 88.6/40.9 | 178.3/28.7 | 29.1 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 65536 | off | 81.6/28.6 | 87.9/28.2 | 188.6/28.0 | 32.1 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=1) | 70.9/43.4 | 81.7/39.7 | 176.3/37.0 | 26.2 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=2) | 75.9/42.5 | 178.2/38.9 | 164.5/37.8 | 25.7 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=3) | 76.2/40.1 | 123.6/36.9 | 173.4/29.7 | 28.8 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=4) | 73.4/43.0 | 87.9/36.1 | 170.6/27.2 | 29.4 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 32768 | off | 82.1/28.5 | 185.9/28.1 | 176.6/28.1 | 32.1 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=1) | 75.6/42.5 | 87.4/42.0 | 175.6/36.3 | 26.1 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=2) | 76.1/42.0 | 160.3/44.1 | 166.9/35.7 | 24.6 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=3) | 66.4/36.6 | 88.3/35.5 | 171.7/30.3 | 29.6 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=4) | 76.0/46.1 | 148.8/46.1 | 162.2/26.7 | 27.3 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 16384 | off | 81.2/28.1 | 91.4/28.0 | 187.5/28.0 | 32.3 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=1) | 76.3/42.1 | 88.9/42.3 | 174.9/35.7 | 26.2 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=2) | 76.3/39.3 | 174.5/41.0 | 164.8/35.1 | 26.9 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=3) | 74.7/40.4 | 123.5/38.3 | 170.6/31.1 | 28.2 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=4) | 76.3/40.4 | 89.3/40.0 | 178.1/30.9 | 27.9 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 131072 | off | 81.7/31.8 | 92.9/31.7 | 196.6/31.6 | 28.8 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 131072 | on(n=1) | 75.9/42.6 | 159.0/41.0 | 174.5/40.2 | 25.7 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 131072 | on(n=2) | 73.9/44.2 | 90.3/38.6 | 184.9/35.4 | 26.7 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 131072 | on(n=3) | 73.9/39.7 | 90.3/39.6 | 185.0/29.4 | 28.6 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 131072 | on(n=4) | 71.8/35.7 | 87.9/35.7 | 182.8/26.0 | 31.2 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 65536 | off | 81.6/31.5 | 90.1/31.2 | 196.0/31.3 | 28.8 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 65536 | on(n=1) | 72.9/43.2 | 90.6/41.7 | 184.6/37.1 | 25.7 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 65536 | on(n=2) | 75.7/41.7 | 90.3/41.2 | 184.0/36.1 | 26.2 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 65536 | on(n=3) | 75.7/38.8 | 90.0/37.7 | 184.7/30.1 | 28.6 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 65536 | on(n=4) | 64.3/37.0 | 88.7/40.3 | 184.7/28.0 | 29.2 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 32768 | off | 81.7/31.5 | 129.4/31.3 | 186.1/30.8 | 28.9 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 32768 | on(n=1) | 73.2/42.8 | 138.4/41.9 | 175.3/39.9 | 25.1 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 32768 | on(n=2) | 75.6/42.0 | 90.6/38.4 | 185.3/37.5 | 26.1 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 32768 | on(n=3) | 71.1/37.6 | 167.8/38.1 | 173.2/27.0 | 29.6 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 32768 | on(n=4) | 76.0/34.6 | 178.5/30.4 | 174.5/27.6 | 31.6 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 16384 | off | 80.9/31.6 | 91.1/31.3 | 190.5/31.0 | 28.8 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 16384 | on(n=1) | 75.9/41.5 | 90.1/43.4 | 184.0/37.2 | 25.2 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 16384 | on(n=2) | 73.9/41.2 | 102.5/40.8 | 183.7/36.6 | 25.8 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 16384 | on(n=3) | 73.8/38.1 | 90.5/40.6 | 185.0/31.5 | 27.4 | OK |
| gemma-4-12b-it-UD-Q2_K_XL | layer | both | 16384 | on(n=4) | 72.1/34.2 | 90.2/36.8 | 184.3/30.5 | 29.3 | OK |
gemma4-26b
quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status
| gemma-4-26B-A4B-it-UD-IQ2_M | none | 0 | 131072 | off | - | - | - | 21.8 | OOM |
| gemma-4-26B-A4B-it-UD-IQ2_M | none | 0 | 65536 | off | - | - | - | 3.6 | OOM |
| gemma-4-26B-A4B-it-UD-IQ2_M | none | 0 | 32768 | off | 92.5/65.4 | 175.2/65.2 | 341.6/64.8 | 16.1 | OK |
| gemma-4-26B-A4B-it-UD-IQ2_M | none | 0 | 16384 | off | 135.9/64.9 | 196.1/64.4 | 338.4/63.2 | 16.2 | OK |
| gemma-4-26B-A4B-it-UD-IQ2_M | none | 1 | 131072 | off | - | - | - | 6.1 | OOM |
| gemma-4-26B-A4B-it-UD-IQ2_M | none | 1 | 65536 | off | - | - | - | 6.1 | OOM |
| gemma-4-26B-A4B-it-UD-IQ2_M | none | 1 | 32768 | off | 147.6/60.0 | 177.3/59.9 | 317.9/59.4 | 19.5 | OK |
| gemma-4-26B-A4B-it-UD-IQ2_M | none | 1 | 16384 | off | 149.8/59.8 | 176.1/59.6 | 314.3/58.2 | 19.7 | OK |
| gemma-4-26B-A4B-it-UD-IQ2_M | layer | both | 131072 | off | 148.0/61.8 | 196.3/61.8 | 333.4/61.7 | 18.1 | OK |
| gemma-4-26B-A4B-it-UD-IQ2_M | layer | both | 65536 | off | 147.7/61.7 | 196.1/61.6 | 332.7/61.5 | 18.1 | OK |
| gemma-4-26B-A4B-it-UD-IQ2_M | layer | both | 32768 | off | 149.5/61.5 | 194.9/61.5 | 330.8/61.4 | 17.9 | OK |
| gemma-4-26B-A4B-it-UD-IQ2_M | layer | both | 16384 | off | 149.1/61.5 | 195.4/61.5 | 331.4/61.3 | 17.8 | OK |
gemma4-31b
quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 131072 | off | - | - | - | 19.0 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 131072 | on(n=1) | - | - | - | 4.1 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 131072 | on(n=2) | - | - | - | 4.1 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 131072 | on(n=3) | - | - | - | 4.2 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 131072 | on(n=4) | - | - | - | 4.1 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 65536 | off | - | - | - | 3.4 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 65536 | on(n=1) | - | - | - | 4.1 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 65536 | on(n=2) | - | - | - | 4.1 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 65536 | on(n=3) | - | - | - | 4.1 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 65536 | on(n=4) | - | - | - | 4.1 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 32768 | off | - | - | - | 3.4 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 32768 | on(n=1) | - | - | - | 4.2 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 32768 | on(n=2) | - | - | - | 4.2 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 32768 | on(n=3) | - | - | - | 4.2 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 32768 | on(n=4) | - | - | - | 4.2 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 16384 | off | 43.5/15.0 | 48.2/14.9 | 96.1/14.2 | 57.6 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 16384 | on(n=1) | - | - | - | 6.0 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 16384 | on(n=2) | - | - | - | 5.1 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 16384 | on(n=3) | - | - | - | 5.1 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 16384 | on(n=4) | - | - | - | 5.1 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 131072 | off | - | - | - | 5.5 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 131072 | on(n=1) | - | - | - | 6.3 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 131072 | on(n=2) | - | - | - | 6.2 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 131072 | on(n=3) | - | - | - | 6.2 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 131072 | on(n=4) | - | - | - | 6.2 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 65536 | off | - | - | - | 5.5 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 65536 | on(n=1) | - | - | - | 6.3 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 65536 | on(n=2) | - | - | - | 6.4 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 65536 | on(n=3) | - | - | - | 6.3 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 65536 | on(n=4) | - | - | - | 6.3 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 32768 | off | - | - | - | 5.5 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 32768 | on(n=1) | - | - | - | 6.3 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 32768 | on(n=2) | - | - | - | 6.4 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 32768 | on(n=3) | - | - | - | 6.3 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 32768 | on(n=4) | - | - | - | 6.4 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 16384 | off | 43.2/14.8 | 46.4/14.0 | 89.6/13.3 | 62.3 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 16384 | on(n=1) | - | - | - | 7.2 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 16384 | on(n=2) | - | - | - | 7.2 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 16384 | on(n=3) | - | - | - | 7.1 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 16384 | on(n=4) | - | - | - | 7.1 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 131072 | off | 41.9/15.3 | 60.1/15.2 | 94.0/15.1 | 57.2 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 131072 | on(n=1) | - | - | - | 6.6 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 131072 | on(n=2) | - | - | - | 6.7 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 131072 | on(n=3) | - | - | - | 6.6 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 131072 | on(n=4) | - | - | - | 6.6 | OOM |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 65536 | off | 42.1/15.2 | 79.4/15.1 | 92.2/15.0 | 57.5 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 65536 | on(n=1) | 39.3/22.3 | 67.4/21.9 | 85.3/21.0 | 44.1 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 65536 | on(n=2) | 39.2/23.4 | 91.9/20.7 | 81.8/21.2 | 44.0 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 65536 | on(n=3) | 37.0/23.9 | 138.9/16.4 | 49.6/15.9 | 52.9 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 65536 | on(n=4) | 39.2/20.4 | 45.8/19.0 | 88.5/18.2 | 48.7 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 32768 | off | 41.3/15.1 | 85.4/15.0 | 133.1/14.8 | 57.8 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 32768 | on(n=1) | 38.7/22.9 | 92.4/20.9 | 80.9/20.6 | 42.9 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 32768 | on(n=2) | 37.8/22.7 | 67.6/21.7 | 85.1/19.9 | 42.0 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 32768 | on(n=3) | 39.2/20.7 | 45.9/19.4 | 88.9/17.5 | 48.6 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 32768 | on(n=4) | 39.2/20.8 | 45.7/20.3 | 87.5/17.2 | 48.2 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 16384 | off | 41.6/15.1 | 47.9/14.8 | 95.1/14.6 | 58.1 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 16384 | on(n=1) | 39.3/22.6 | 66.8/21.9 | 86.5/20.7 | 43.8 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 16384 | on(n=2) | 39.3/24.3 | 90.5/20.5 | 81.3/17.9 | 46.0 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 16384 | on(n=3) | 39.4/20.3 | 52.6/19.9 | 88.4/17.8 | 48.3 | OK |
| gemma-4-31B-it-UD-IQ2_XXS | layer | both | 16384 | on(n=4) | 39.3/19.7 | 145.8/17.8 | 53.0/15.5 | 54.2 | OK |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 131072 | off | - | - | - | 24.0 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=1) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=2) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=3) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=4) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 65536 | off | - | - | - | 2.9 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=1) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=2) | - | - | - | 3.7 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=3) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=4) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 32768 | off | - | - | - | 2.9 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=1) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=2) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=3) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=4) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 16384 | off | - | - | - | 2.9 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=1) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=2) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=3) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=4) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 131072 | off | - | - | - | 2.9 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=1) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=2) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=3) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=4) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 65536 | off | - | - | - | 2.9 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=1) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=2) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=3) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=4) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 32768 | off | - | - | - | 2.9 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=1) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=2) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=3) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=4) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 16384 | off | - | - | - | 2.9 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=1) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=2) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=3) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=4) | - | - | - | 3.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 131072 | off | - | - | - | 5.5 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 131072 | on(n=1) | - | - | - | 6.2 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 131072 | on(n=2) | - | - | - | 6.3 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 131072 | on(n=3) | - | - | - | 6.3 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 131072 | on(n=4) | - | - | - | 6.2 | OOM |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 65536 | off | 35.3/13.8 | 40.7/13.7 | 86.1/13.6 | 63.6 | OK |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 65536 | on(n=1) | 32.7/20.5 | 40.3/20.7 | 79.4/20.2 | 47.5 | OK |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 65536 | on(n=2) | 32.7/22.4 | 64.2/21.0 | 76.9/18.7 | 47.3 | OK |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 65536 | on(n=3) | 32.9/21.9 | 64.0/20.4 | 76.9/18.2 | 46.8 | OK |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 65536 | on(n=4) | 31.0/21.0 | 39.6/20.1 | 80.4/17.7 | 49.4 | OK |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 32768 | off | 34.1/13.6 | 68.3/13.4 | 82.1/13.3 | 60.9 | OK |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 32768 | on(n=1) | 30.9/20.1 | 48.1/20.7 | 78.7/19.4 | 48.0 | OK |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 32768 | on(n=2) | 30.9/20.9 | 40.1/21.4 | 79.6/18.7 | 47.6 | OK |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 32768 | on(n=3) | 31.2/20.9 | 40.0/20.6 | 80.0/17.5 | 49.0 | OK |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 32768 | on(n=4) | 32.6/20.2 | 64.3/22.5 | 75.7/18.2 | 46.0 | OK |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 16384 | off | 34.3/13.6 | 67.6/13.4 | 81.0/13.4 | 64.6 | OK |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 16384 | on(n=1) | 30.8/20.3 | 63.9/20.1 | 76.7/19.3 | 48.4 | OK |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 16384 | on(n=2) | 32.6/20.8 | 40.0/21.9 | 79.9/18.8 | 47.3 | OK |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 16384 | on(n=3) | 32.8/19.9 | 39.9/17.8 | 79.8/17.7 | 51.3 | OK |
| gemma-4-31B-it-UD-Q2_K_XL | layer | both | 16384 | on(n=4) | 32.8/22.5 | 40.2/21.1 | 79.5/16.9 | 48.3 | OK |
gemma4-e2b
quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status
| gemma-4-E2B-it-Q4_K_M | none | 0 | 131072 | off | 251.7/92.4 | 268.8/92.4 | 827.5/92.3 | 16.6 | OK |
| gemma-4-E2B-it-Q4_K_M | none | 0 | 65536 | off | 177.7/92.2 | 380.4/92.2 | 847.8/92.1 | 11.0 | OK |
| gemma-4-E2B-it-Q4_K_M | none | 0 | 32768 | off | 178.4/91.9 | 892.3/92.1 | 837.1/92.0 | 8.7 | OK |
| gemma-4-E2B-it-Q4_K_M | none | 0 | 16384 | off | 262.1/88.7 | 430.0/92.2 | 846.8/92.1 | 8.3 | OK |
| gemma-4-E2B-it-Q4_K_M | none | 1 | 131072 | off | 243.8/86.7 | 358.6/86.7 | 817.0/86.6 | 12.0 | OK |
| gemma-4-E2B-it-Q4_K_M | none | 1 | 65536 | off | 258.9/85.4 | 352.0/85.3 | 803.6/85.4 | 12.1 | OK |
| gemma-4-E2B-it-Q4_K_M | none | 1 | 32768 | off | 262.6/81.3 | 408.6/86.0 | 804.0/85.8 | 8.9 | OK |
| gemma-4-E2B-it-Q4_K_M | none | 1 | 16384 | off | 263.2/82.2 | 406.5/85.9 | 799.4/85.0 | 8.9 | OK |
| gemma-4-E2B-it-Q4_K_M | layer | both | 131072 | off | 205.5/65.2 | 290.7/64.6 | 660.5/62.9 | 15.2 | OK |
| gemma-4-E2B-it-Q4_K_M | layer | both | 65536 | off | 215.5/63.5 | 340.7/65.2 | 672.0/63.5 | 11.0 | OK |
| gemma-4-E2B-it-Q4_K_M | layer | both | 32768 | off | 163.0/66.0 | 355.8/67.8 | 707.0/65.9 | 10.7 | OK |
| gemma-4-E2B-it-Q4_K_M | layer | both | 16384 | off | 167.1/62.6 | 337.5/64.7 | 664.8/63.0 | 11.0 | OK |
gemma4-e4b
quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status
| gemma-4-E4B-it-Q4_K_M | none | 0 | 131072 | off | 137.3/61.6 | 439.9/61.5 | 510.4/61.2 | 24.2 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 131072 | on(n=1) | 162.3/81.3 | 557.5/87.7 | 486.8/88.6 | 11.5 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 131072 | on(n=2) | 154.5/91.4 | 550.3/92.6 | 488.2/88.9 | 10.9 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 131072 | on(n=3) | 163.7/89.2 | 338.7/95.8 | 496.6/91.6 | 10.5 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 131072 | on(n=4) | 167.1/69.7 | 559.5/89.2 | 475.0/84.6 | 11.3 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 65536 | off | 180.4/60.7 | 517.8/60.9 | 497.2/60.0 | 12.0 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 65536 | on(n=1) | 163.0/88.7 | 232.9/92.9 | 490.9/85.8 | 13.4 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 65536 | on(n=2) | 163.9/87.5 | 548.8/91.6 | 490.6/87.5 | 10.9 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 65536 | on(n=3) | 123.7/66.3 | 544.5/101.7 | 484.8/76.2 | 11.3 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 65536 | on(n=4) | 161.4/63.4 | 547.0/86.8 | 486.2/76.4 | 11.7 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 32768 | off | 137.9/60.7 | 527.3/60.9 | 503.6/59.6 | 12.1 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 32768 | on(n=1) | 163.6/78.8 | 600.7/93.1 | 475.7/87.3 | 11.1 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 32768 | on(n=2) | 165.2/74.1 | 399.6/99.9 | 495.0/90.8 | 10.4 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 32768 | on(n=3) | 164.3/74.0 | 589.4/93.0 | 469.5/88.0 | 11.1 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 32768 | on(n=4) | 125.3/59.3 | 477.0/85.9 | 489.8/72.6 | 11.8 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 16384 | off | 158.4/60.9 | 576.5/60.8 | 495.3/59.6 | 12.3 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 16384 | on(n=1) | 155.8/79.5 | 510.5/86.7 | 484.5/87.8 | 11.0 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 16384 | on(n=2) | 160.8/73.3 | 565.4/97.5 | 484.7/86.0 | 10.9 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 16384 | on(n=3) | 165.7/86.4 | 232.9/105.1 | 492.4/94.8 | 12.8 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 0 | 16384 | on(n=4) | 166.7/65.1 | 567.5/91.5 | 483.8/83.0 | 11.4 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 131072 | off | 163.1/57.4 | 225.8/57.4 | 492.2/57.1 | 17.2 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 131072 | on(n=1) | 155.3/73.1 | 368.1/87.5 | 467.6/80.4 | 11.9 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 131072 | on(n=2) | 136.0/75.6 | 476.1/91.2 | 457.9/83.2 | 11.9 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 131072 | on(n=3) | 139.9/76.5 | 570.5/90.8 | 441.5/83.2 | 12.2 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 131072 | on(n=4) | 143.7/67.1 | 533.1/78.7 | 447.0/77.4 | 12.8 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 65536 | off | 134.1/56.9 | 581.6/55.2 | 444.0/54.5 | 14.1 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 65536 | on(n=1) | 133.0/83.1 | 575.9/81.1 | 438.6/83.2 | 12.6 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 65536 | on(n=2) | 120.5/80.7 | 530.4/88.3 | 442.6/81.0 | 12.2 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 65536 | on(n=3) | 159.8/63.8 | 429.5/92.1 | 445.3/87.8 | 11.7 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 65536 | on(n=4) | 157.2/76.2 | 504.1/77.3 | 448.0/67.6 | 13.2 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 32768 | off | 177.3/57.0 | 630.6/55.3 | 454.5/54.9 | 14.1 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 32768 | on(n=1) | 151.4/84.0 | 517.6/82.6 | 437.5/79.5 | 12.4 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 32768 | on(n=2) | 162.7/66.7 | 562.5/86.3 | 432.2/75.0 | 12.5 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 32768 | on(n=3) | 153.3/66.4 | 533.8/92.9 | 442.7/77.1 | 12.1 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 32768 | on(n=4) | 158.0/58.9 | 512.7/93.8 | 437.6/68.2 | 12.8 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 16384 | off | 169.7/55.6 | 316.9/54.5 | 443.6/54.5 | 13.4 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 16384 | on(n=1) | 160.8/77.1 | 323.0/83.8 | 457.0/81.1 | 11.7 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 16384 | on(n=2) | 159.0/75.8 | 554.9/87.0 | 438.5/80.5 | 12.3 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 16384 | on(n=3) | 121.6/87.3 | 224.7/88.1 | 475.7/76.6 | 14.8 | OK |
| gemma-4-E4B-it-Q4_K_M | none | 1 | 16384 | on(n=4) | 159.6/88.0 | 225.4/91.6 | 476.4/76.9 | 14.6 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 131072 | off | 171.6/57.4 | 559.6/57.7 | 494.4/57.6 | 13.3 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 131072 | on(n=1) | 150.0/76.6 | 455.2/80.8 | 478.5/76.7 | 12.9 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 131072 | on(n=2) | 156.5/64.9 | 533.9/82.7 | 472.6/81.3 | 12.9 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 131072 | on(n=3) | 157.8/78.4 | 559.7/92.1 | 473.4/77.8 | 12.6 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 131072 | on(n=4) | 152.5/62.7 | 628.6/87.3 | 463.4/61.2 | 14.0 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 65536 | off | 174.8/57.5 | 548.8/57.6 | 492.1/57.5 | 13.3 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 65536 | on(n=1) | 154.3/76.4 | 227.1/83.8 | 487.5/79.0 | 15.3 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 65536 | on(n=2) | 158.9/65.6 | 572.5/85.0 | 465.8/80.8 | 12.6 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 65536 | on(n=3) | 152.1/61.2 | 540.7/86.7 | 474.2/78.9 | 12.5 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 65536 | on(n=4) | 162.1/66.4 | 470.2/75.9 | 478.2/68.5 | 13.0 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 32768 | off | 160.8/57.3 | 439.7/57.5 | 494.9/57.4 | 12.8 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 32768 | on(n=1) | 154.6/77.8 | 627.5/80.9 | 464.0/80.2 | 12.5 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 32768 | on(n=2) | 159.5/74.9 | 567.8/84.8 | 468.9/77.7 | 12.6 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 32768 | on(n=3) | 153.4/84.3 | 432.2/79.2 | 477.6/73.6 | 12.4 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 32768 | on(n=4) | 159.2/54.2 | 489.7/70.3 | 474.7/69.8 | 13.4 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 16384 | off | 172.1/57.1 | 476.7/57.5 | 495.0/57.4 | 12.9 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 16384 | on(n=1) | 161.2/82.0 | 226.4/85.5 | 489.0/79.1 | 14.5 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 16384 | on(n=2) | 159.3/82.0 | 226.6/89.0 | 486.9/80.5 | 14.3 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 16384 | on(n=3) | 128.7/73.9 | 531.0/83.8 | 474.7/70.0 | 12.5 | OK |
| gemma-4-E4B-it-Q4_K_M | layer | both | 16384 | on(n=4) | 127.7/69.2 | 597.7/89.2 | 464.9/67.4 | 12.7 | OK |
gemma4-e4b-qat
quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status
| gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 0 | 131072 | off | 225.8/43.5 | 338.4/43.3 | 547.1/43.0 | 26.3 | OK |
| gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 0 | 65536 | off | 254.4/42.6 | 205.3/43.4 | 536.7/42.7 | 20.6 | OK |
| gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 0 | 32768 | off | 234.9/41.0 | 290.8/43.4 | 532.0/43.2 | 14.8 | OK |
| gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 0 | 16384 | off | 226.2/43.0 | 361.1/42.8 | 537.0/42.9 | 18.1 | OK |
| gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 1 | 131072 | off | 243.3/42.9 | 244.4/41.9 | 931.4/41.9 | 16.0 | OK |
| gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 1 | 65536 | off | 217.4/41.7 | 211.8/42.1 | 520.5/42.1 | 21.6 | OK |
| gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 1 | 32768 | off | 222.5/42.1 | 339.9/41.7 | 512.0/41.2 | 16.1 | OK |
| gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 1 | 16384 | off | 219.6/42.5 | 339.9/42.1 | 499.4/42.0 | 16.0 | OK |
| gemma-4-E4B-it-qat-UD-Q2_K_XL | layer | both | 131072 | off | 204.2/39.2 | 319.1/39.3 | 480.3/39.5 | 22.9 | OK |
| gemma-4-E4B-it-qat-UD-Q2_K_XL | layer | both | 65536 | off | 218.9/42.2 | 286.2/40.6 | 488.3/38.5 | 16.4 | OK |
| gemma-4-E4B-it-qat-UD-Q2_K_XL | layer | both | 32768 | off | 176.8/34.6 | 284.8/42.1 | 528.9/42.0 | 15.6 | OK |
| gemma-4-E4B-it-qat-UD-Q2_K_XL | layer | both | 16384 | off | 212.6/41.5 | 389.0/42.0 | 522.5/41.6 | 16.0 | OK |
| gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 0 | 131072 | off | 263.9/69.5 | 420.4/70.1 | 641.8/70.0 | 17.8 | OK |
| gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 0 | 65536 | off | 265.9/69.4 | 686.1/69.3 | 632.7/69.2 | 10.9 | OK |
| gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 0 | 32768 | off | 305.0/69.5 | 510.7/69.7 | 636.5/69.3 | 10.4 | OK |
| gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 0 | 16384 | off | 302.9/69.5 | 314.8/69.2 | 632.7/68.1 | 13.8 | OK |
| gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 1 | 131072 | off | 280.7/65.8 | 298.1/65.7 | 602.2/65.4 | 15.2 | OK |
| gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 1 | 65536 | off | 284.6/65.5 | 642.7/65.5 | 576.5/65.4 | 12.0 | OK |
| gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 1 | 32768 | off | 280.9/65.3 | 652.1/64.6 | 569.5/63.6 | 12.2 | OK |
| gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 1 | 16384 | off | 288.0/65.3 | 690.6/63.7 | 557.8/63.3 | 12.4 | OK |
| gemma-4-E4B-it-qat-UD-Q4_K_XL | layer | both | 131072 | off | 262.2/63.4 | 658.0/65.6 | 608.9/65.5 | 11.9 | OK |
| gemma-4-E4B-it-qat-UD-Q4_K_XL | layer | both | 65536 | off | 289.4/65.6 | 305.1/65.6 | 870.7/65.4 | 15.1 | OK |
| gemma-4-E4B-it-qat-UD-Q4_K_XL | layer | both | 32768 | off | 289.7/65.2 | 619.3/65.2 | 606.4/65.2 | 11.5 | OK |
| gemma-4-E4B-it-qat-UD-Q4_K_XL | layer | both | 16384 | off | 289.4/64.9 | 430.1/65.4 | 612.2/65.3 | 11.1 | OK |
qwen3.5
quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status
| Qwen3.5-0.8B-UD-IQ2_M | none | 0 | 131072 | off | 140.5/175.7 | 346.4/175.9 | 690.4/174.6 | 7.1 | OK |
| Qwen3.5-0.8B-UD-IQ2_M | none | 0 | 65536 | off | 349.3/175.1 | 1051.4/175.9 | 1812.4/175.8 | 3.8 | OK |
| Qwen3.5-0.8B-UD-IQ2_M | none | 0 | 32768 | off | 353.0/175.5 | 1439.3/175.9 | 2042.0/175.7 | 4.1 | OK |
| Qwen3.5-0.8B-UD-IQ2_M | none | 0 | 16384 | off | 343.2/173.7 | 570.3/175.7 | 3438.5/175.4 | 4.9 | OK |
| Qwen3.5-0.8B-UD-IQ2_M | none | 1 | 131072 | off | 293.2/160.5 | 1865.5/161.2 | 685.3/161.3 | 6.3 | OK |
| Qwen3.5-0.8B-UD-IQ2_M | none | 1 | 65536 | off | 296.6/160.8 | 2372.6/0.0 | 681.2/160.9 | 6.6 | OK |
| Qwen3.5-0.8B-UD-IQ2_M | none | 1 | 32768 | off | 267.1/161.8 | 256.9/162.1 | 678.0/162.0 | 6.6 | OK |
| Qwen3.5-0.8B-UD-IQ2_M | none | 1 | 16384 | off | 233.3/159.8 | 819.6/160.8 | 684.7/160.8 | 5.4 | OK |
| Qwen3.5-0.8B-UD-IQ2_M | layer | both | 131072 | off | 238.6/152.9 | 660.8/160.3 | 792.1/163.3 | 5.6 | OK |
| Qwen3.5-0.8B-UD-IQ2_M | layer | both | 65536 | off | 276.1/163.9 | 298.6/163.9 | 802.6/164.1 | 6.8 | OK |
| Qwen3.5-0.8B-UD-IQ2_M | layer | both | 32768 | off | 287.7/162.3 | 298.6/164.4 | 3081.2/164.1 | 6.6 | OK |
| Qwen3.5-0.8B-UD-IQ2_M | layer | both | 16384 | off | 213.0/163.5 | 1300.6/164.4 | 804.1/164.0 | 5.5 | OK |
| Qwen3.5-0.8B-UD-Q4_K_XL | none | 0 | 131072 | off | 238.6/161.0 | 359.9/163.9 | 965.5/163.6 | 7.6 | OK |
| Qwen3.5-0.8B-UD-Q4_K_XL | none | 0 | 65536 | off | 251.8/162.7 | 536.5/162.8 | 951.2/162.7 | 6.6 | OK |
| Qwen3.5-0.8B-UD-Q4_K_XL | none | 0 | 32768 | off | 142.0/162.8 | 357.6/162.8 | 990.8/162.7 | 6.6 | OK |
| Qwen3.5-0.8B-UD-Q4_K_XL | none | 0 | 16384 | off | 152.1/162.7 | 358.2/162.8 | 990.2/162.7 | 6.6 | OK |
| Qwen3.5-0.8B-UD-Q4_K_XL | none | 1 | 131072 | off | 205.4/150.2 | 260.6/151.3 | 678.7/151.1 | 7.2 | OK |
| Qwen3.5-0.8B-UD-Q4_K_XL | none | 1 | 65536 | off | 208.9/149.9 | 262.3/149.9 | 681.2/149.8 | 7.2 | OK |
| Qwen3.5-0.8B-UD-Q4_K_XL | none | 1 | 32768 | off | 150.8/150.1 | 513.2/150.1 | 678.1/150.1 | 7.2 | OK |
| Qwen3.5-0.8B-UD-Q4_K_XL | none | 1 | 16384 | off | 183.8/149.8 | 261.0/149.9 | 675.6/149.9 | 7.2 | OK |
| Qwen3.5-0.8B-UD-Q4_K_XL | layer | both | 131072 | off | 166.1/143.6 | 298.3/146.0 | 778.9/145.7 | 7.7 | OK |
| Qwen3.5-0.8B-UD-Q4_K_XL | layer | both | 65536 | off | 169.0/152.9 | 301.5/153.5 | 802.8/153.1 | 7.2 | OK |
| Qwen3.5-0.8B-UD-Q4_K_XL | layer | both | 32768 | off | 171.6/152.9 | 304.3/153.7 | 805.5/153.2 | 7.1 | OK |
| Qwen3.5-0.8B-UD-Q4_K_XL | layer | both | 16384 | off | 164.4/152.0 | 299.7/152.5 | 2021.4/152.0 | 7.0 | OK |
| Qwen3.5-0.8B-UD-Q8_K_XL | none | 0 | 131072 | off | 266.9/134.6 | 336.2/135.8 | 2332.9/135.7 | 9.8 | OK |
| Qwen3.5-0.8B-UD-Q8_K_XL | none | 0 | 65536 | off | 267.2/134.9 | 337.0/135.1 | 899.7/134.9 | 7.7 | OK |
| Qwen3.5-0.8B-UD-Q8_K_XL | none | 0 | 32768 | off | 153.0/135.3 | 337.3/135.2 | 897.0/135.1 | 7.8 | OK |
| Qwen3.5-0.8B-UD-Q8_K_XL | none | 0 | 16384 | off | 278.5/135.0 | 342.8/135.0 | 883.3/134.9 | 7.5 | OK |
| Qwen3.5-0.8B-UD-Q8_K_XL | none | 1 | 131072 | off | 180.9/127.2 | 248.8/126.9 | 629.6/126.9 | 8.4 | OK |
| Qwen3.5-0.8B-UD-Q8_K_XL | none | 1 | 65536 | off | 202.9/125.7 | 376.4/125.8 | 628.9/125.7 | 7.5 | OK |
| Qwen3.5-0.8B-UD-Q8_K_XL | none | 1 | 32768 | off | 222.9/125.2 | 713.0/125.3 | 643.7/125.2 | 7.6 | OK |
| Qwen3.5-0.8B-UD-Q8_K_XL | none | 1 | 16384 | off | 235.2/125.6 | 249.6/125.6 | 641.1/125.5 | 8.4 | OK |
| Qwen3.5-0.8B-UD-Q8_K_XL | layer | both | 131072 | off | 177.1/120.8 | 542.3/121.8 | 726.5/121.8 | 8.9 | OK |
| Qwen3.5-0.8B-UD-Q8_K_XL | layer | both | 65536 | off | 163.5/125.7 | 282.3/124.7 | 733.4/124.5 | 8.5 | OK |
| Qwen3.5-0.8B-UD-Q8_K_XL | layer | both | 32768 | off | 163.1/128.6 | 287.0/128.7 | 750.1/128.7 | 8.3 | OK |
| Qwen3.5-0.8B-UD-Q8_K_XL | layer | both | 16384 | off | 176.6/128.1 | 578.8/128.2 | 752.3/128.0 | 8.3 | OK |
qwen3.6_27B
quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status
| Qwen3.6-27B-Q4_K_S | none | 0 | 131072 | off | - | - | - | 31.0 | OOM |
| Qwen3.6-27B-Q4_K_S | none | 0 | 65536 | off | - | - | - | 2.7 | OOM |
| Qwen3.6-27B-Q4_K_S | none | 0 | 32768 | off | - | - | - | 2.7 | OOM |
| Qwen3.6-27B-Q4_K_S | none | 0 | 16384 | off | - | - | - | 2.7 | OOM |
| Qwen3.6-27B-Q4_K_S | none | 1 | 131072 | off | - | - | - | 2.7 | OOM |
| Qwen3.6-27B-Q4_K_S | none | 1 | 65536 | off | - | - | - | 2.7 | OOM |
| Qwen3.6-27B-Q4_K_S | none | 1 | 32768 | off | - | - | - | 2.7 | OOM |
| Qwen3.6-27B-Q4_K_S | none | 1 | 16384 | off | - | - | - | 2.7 | OOM |
| Qwen3.6-27B-Q4_K_S | layer | both | 131072 | off | - | - | - | 6.0 | OOM |
| Qwen3.6-27B-Q4_K_S | layer | both | 65536 | off | 40.7/15.8 | 38.0/15.8 | 82.1/15.7 | 56.3 | OK |
| Qwen3.6-27B-Q4_K_S | layer | both | 32768 | off | 43.6/15.8 | 41.4/15.7 | 81.7/15.6 | 56.6 | OK |
| Qwen3.6-27B-Q4_K_S | layer | both | 16384 | off | 41.8/15.7 | 37.8/15.7 | 82.6/15.6 | 56.6 | OK |
qwen3.6_35B
quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status
| Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 0 | 131072 | off | - | - | - | 24.6 | OOM |
| Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 0 | 65536 | off | - | - | - | 2.3 | OOM |
| Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 0 | 32768 | off | - | - | - | 2.3 | OOM |
| Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 0 | 16384 | off | - | - | - | 2.3 | OOM |
| Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 1 | 131072 | off | - | - | - | 2.3 | OOM |
| Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 1 | 65536 | off | - | - | - | 2.3 | OOM |
| Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 1 | 32768 | off | - | - | - | 2.3 | OOM |
| Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 1 | 16384 | off | - | - | - | 2.3 | OOM |
| Qwen3.6-35B-A3B-UD-Q2_K_XL | layer | both | 131072 | off | 52.2/56.9 | 92.9/57.2 | 202.7/57.1 | 17.8 | OK |
| Qwen3.6-35B-A3B-UD-Q2_K_XL | layer | both | 65536 | off | 69.6/56.7 | 108.3/56.7 | 207.9/56.5 | 19.3 | OK |
| Qwen3.6-35B-A3B-UD-Q2_K_XL | layer | both | 32768 | off | 68.5/56.4 | 107.3/56.6 | 208.1/56.4 | 19.3 | OK |
| Qwen3.6-35B-A3B-UD-Q2_K_XL | layer | both | 16384 | off | 75.2/56.6 | 108.1/56.6 | 208.5/56.4 | 19.2 | OK |
qwen3.8_27B
quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status
| Qwen3.8-27B-UD-IQ2_XXS | none | 0 | 131072 | off | - | - | - | 19.4 | OOM |
| Qwen3.8-27B-UD-IQ2_XXS | none | 0 | 65536 | off | - | - | - | 3.0 | OOM |
| Qwen3.8-27B-UD-IQ2_XXS | none | 0 | 32768 | off | 98.4/14.2 | 37.4/14.1 | 82.0/13.5 | 42.7 | OK |
| Qwen3.8-27B-UD-IQ2_XXS | none | 0 | 16384 | off | 99.5/13.7 | 37.4/13.1 | 81.8/12.9 | 49.0 | OK |
| Qwen3.8-27B-UD-IQ2_XXS | none | 1 | 131072 | off | - | - | - | 5.0 | OOM |
| Qwen3.8-27B-UD-IQ2_XXS | none | 1 | 65536 | off | - | - | - | 5.0 | OOM |
| Qwen3.8-27B-UD-IQ2_XXS | none | 1 | 32768 | off | 90.0/14.3 | 31.3/14.1 | 70.1/13.6 | 49.0 | OK |
| Qwen3.8-27B-UD-IQ2_XXS | none | 1 | 16384 | off | 89.1/13.3 | 30.8/12.5 | 69.6/12.3 | 52.5 | OK |
| Qwen3.8-27B-UD-IQ2_XXS | layer | both | 131072 | off | 86.0/15.6 | 34.7/15.4 | 76.3/15.2 | 45.4 | OK |
| Qwen3.8-27B-UD-IQ2_XXS | layer | both | 65536 | off | 91.9/14.7 | 34.3/14.7 | 76.3/14.7 | 44.3 | OK |
| Qwen3.8-27B-UD-IQ2_XXS | layer | both | 32768 | off | 86.0/14.9 | 34.1/14.8 | 75.2/14.7 | 45.9 | OK |
| Qwen3.8-27B-UD-IQ2_XXS | layer | both | 16384 | off | 92.3/15.5 | 34.2/15.4 | 75.5/15.3 | 43.3 | OK |
Long context (single turn, >10k tokens)
Prompt: see long_prompt.txt (needle-in-haystack recall test).
familyquantsmgpuctxmtpprompt t/sgen t/swall(s)status
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 131072 | off | 415.9 | 30.7 | 37.6 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 131072 | on(n=1) | 405.2 | 48.8 | 38.7 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 131072 | on(n=2) | 403.5 | 52.4 | 38.6 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 131072 | on(n=3) | 402.4 | 53.1 | 38.7 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 131072 | on(n=4) | 401.9 | 43.0 | 39.3 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 65536 | off | 401.0 | 28.4 | 38.9 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 65536 | on(n=1) | 398.3 | 46.7 | 39.1 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 65536 | on(n=2) | 398.7 | 51.7 | 38.8 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 65536 | on(n=3) | 400.7 | 53.7 | 38.5 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 65536 | on(n=4) | 399.5 | 50.1 | 38.8 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 32768 | off | 397.4 | 29.7 | 38.9 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 32768 | on(n=1) | 395.0 | 47.8 | 39.3 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 32768 | on(n=2) | 397.6 | 50.6 | 38.9 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 32768 | on(n=3) | 397.0 | 48.8 | 39.0 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 32768 | on(n=4) | 397.5 | 43.2 | 39.3 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 16384 | off | 396.9 | 29.4 | 39.0 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 16384 | on(n=1) | 395.7 | 47.6 | 39.2 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 16384 | on(n=2) | 397.1 | 47.7 | 39.1 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 16384 | on(n=3) | 399.0 | 49.2 | 38.8 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 0 | 16384 | on(n=4) | 400.2 | 48.1 | 38.8 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 131072 | off | 412.5 | 28.7 | 39.8 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 131072 | on(n=1) | 393.7 | 46.3 | 41.5 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 131072 | on(n=2) | 386.5 | 41.0 | 42.5 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 131072 | on(n=3) | 387.7 | 45.0 | 42.1 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 131072 | on(n=4) | 389.2 | 43.3 | 42.1 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 65536 | off | 394.8 | 26.9 | 41.2 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 65536 | on(n=1) | 387.5 | 42.8 | 42.2 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 65536 | on(n=2) | 387.1 | 43.6 | 42.2 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 65536 | on(n=3) | 387.8 | 39.8 | 42.4 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 65536 | on(n=4) | 387.5 | 48.4 | 41.9 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 32768 | off | 391.2 | 27.1 | 41.5 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 32768 | on(n=1) | 392.1 | 42.3 | 41.7 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 32768 | on(n=2) | 387.5 | 48.8 | 41.6 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 32768 | on(n=3) | 387.5 | 47.0 | 41.7 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 32768 | on(n=4) | 386.7 | 45.3 | 41.9 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 16384 | off | 394.7 | 26.8 | 41.2 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 16384 | on(n=1) | 389.4 | 42.5 | 42.1 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 16384 | on(n=2) | 388.9 | 45.8 | 41.6 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 16384 | on(n=3) | 386.2 | 43.2 | 42.0 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | none | 1 | 16384 | on(n=4) | 388.6 | 48.0 | 41.5 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 131072 | off | 726.7 | 29.9 | 25.8 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 131072 | on(n=1) | 605.9 | 35.3 | 31.0 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 131072 | on(n=2) | 603.5 | 29.9 | 31.7 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 131072 | on(n=3) | 599.3 | 31.4 | 31.6 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 131072 | on(n=4) | 602.7 | 21.1 | 33.5 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 65536 | off | 713.9 | 29.7 | 26.1 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 65536 | on(n=1) | 598.1 | 36.1 | 30.9 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 65536 | on(n=2) | 599.1 | 35.0 | 31.0 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 65536 | on(n=3) | 601.8 | 31.4 | 31.3 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 65536 | on(n=4) | 599.8 | 27.5 | 31.9 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 32768 | off | 706.8 | 29.6 | 26.0 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 32768 | on(n=1) | 598.9 | 35.8 | 30.6 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 32768 | on(n=2) | 597.8 | 31.0 | 31.2 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 32768 | on(n=3) | 595.9 | 29.8 | 31.5 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 32768 | on(n=4) | 596.5 | 31.0 | 31.3 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 16384 | off | 708.1 | 29.5 | 26.0 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 16384 | on(n=1) | 597.9 | 36.2 | 30.4 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 16384 | on(n=2) | 593.4 | 34.6 | 30.7 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 16384 | on(n=3) | 596.3 | 26.8 | 31.9 | OK |
| gemma4-12b | gemma-4-12b-it-Q4_0 | layer | both | 16384 | on(n=4) | 596.9 | 25.6 | 32.1 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 131072 | off | 352.9 | 26.0 | 43.1 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=1) | 353.3 | 34.6 | 43.8 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=2) | 351.9 | 35.4 | 43.8 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=3) | 352.7 | 31.9 | 44.2 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=4) | 352.7 | 34.5 | 43.9 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 65536 | off | 354.9 | 26.1 | 42.9 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=1) | 351.5 | 37.7 | 43.4 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=2) | 351.3 | 34.7 | 43.8 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=3) | 351.3 | 35.3 | 43.9 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=4) | 353.2 | 26.9 | 44.8 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 32768 | off | 355.2 | 25.9 | 42.9 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=1) | 352.1 | 40.1 | 43.2 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=2) | 352.1 | 37.6 | 43.4 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=3) | 350.8 | 30.1 | 44.4 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=4) | 351.1 | 26.7 | 44.9 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 16384 | off | 355.5 | 26.5 | 42.7 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=1) | 350.9 | 36.3 | 43.6 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=2) | 352.8 | 34.4 | 43.6 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=3) | 352.0 | 37.2 | 43.5 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=4) | 351.5 | 27.8 | 44.7 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 131072 | off | 366.7 | 25.5 | 43.1 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=1) | 355.2 | 33.6 | 45.2 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=2) | 356.2 | 33.5 | 45.1 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=3) | 351.6 | 29.5 | 46.1 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=4) | 353.7 | 26.5 | 46.3 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 65536 | off | 359.3 | 24.6 | 43.9 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=1) | 355.1 | 31.8 | 45.1 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=2) | 352.1 | 35.9 | 45.0 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=3) | 352.6 | 32.0 | 45.3 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=4) | 354.5 | 31.7 | 45.2 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 32768 | off | 357.9 | 24.7 | 44.1 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=1) | 353.7 | 33.6 | 45.0 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=2) | 354.3 | 32.9 | 45.1 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=3) | 349.9 | 31.1 | 45.8 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=4) | 353.1 | 28.7 | 45.8 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 16384 | off | 359.8 | 24.6 | 43.9 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=1) | 354.0 | 36.5 | 44.7 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=2) | 351.5 | 31.0 | 45.6 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=3) | 351.9 | 30.2 | 45.6 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=4) | 352.0 | 34.7 | 45.1 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 131072 | off | 639.1 | 27.8 | 27.9 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 131072 | on(n=1) | 531.0 | 29.5 | 34.0 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 131072 | on(n=2) | 525.5 | 25.6 | 34.9 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 131072 | on(n=3) | 525.3 | 23.5 | 35.3 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 131072 | on(n=4) | 522.1 | 19.6 | 36.5 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 65536 | off | 618.7 | 27.6 | 28.5 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 65536 | on(n=1) | 528.3 | 31.8 | 33.5 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 65536 | on(n=2) | 528.9 | 26.3 | 34.3 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 65536 | on(n=3) | 525.8 | 22.5 | 35.3 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 65536 | on(n=4) | 526.0 | 20.8 | 35.7 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 32768 | off | 614.3 | 27.9 | 28.4 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 32768 | on(n=1) | 525.6 | 31.0 | 33.5 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 32768 | on(n=2) | 524.3 | 32.0 | 33.4 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 32768 | on(n=3) | 524.7 | 23.9 | 34.8 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 32768 | on(n=4) | 525.1 | 22.1 | 35.1 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 16384 | off | 612.1 | 27.8 | 28.5 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 16384 | on(n=1) | 524.9 | 31.9 | 33.2 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 16384 | on(n=2) | 525.0 | 25.9 | 34.1 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 16384 | on(n=3) | 528.6 | 22.2 | 34.7 | OK |
| gemma4-12b | gemma-4-12b-it-UD-Q2_K_XL | layer | both | 16384 | on(n=4) | 523.7 | 27.2 | 34.1 | OK |
| gemma4-26b | gemma-4-26B-A4B-it-UD-IQ2_M | none | 0 | 131072 | off | - | - | 3.7 | OOM |
| gemma4-26b | gemma-4-26B-A4B-it-UD-IQ2_M | none | 0 | 65536 | off | - | - | 3.6 | OOM |
| gemma4-26b | gemma-4-26B-A4B-it-UD-IQ2_M | none | 0 | 32768 | off | 727.3 | 57.4 | 23.1 | OK |
| gemma4-26b | gemma-4-26B-A4B-it-UD-IQ2_M | none | 0 | 16384 | off | 709.7 | 51.5 | 23.8 | OK |
| gemma4-26b | gemma-4-26B-A4B-it-UD-IQ2_M | none | 1 | 131072 | off | - | - | 6.1 | OOM |
| gemma4-26b | gemma-4-26B-A4B-it-UD-IQ2_M | none | 1 | 65536 | off | - | - | 6.1 | OOM |
| gemma4-26b | gemma-4-26B-A4B-it-UD-IQ2_M | none | 1 | 32768 | off | 724.9 | 52.5 | 26.0 | OK |
| gemma4-26b | gemma-4-26B-A4B-it-UD-IQ2_M | none | 1 | 16384 | off | 708.0 | 48.5 | 26.4 | OK |
| gemma4-26b | gemma-4-26B-A4B-it-UD-IQ2_M | layer | both | 131072 | off | 1293.5 | 54.7 | 17.2 | OK |
| gemma4-26b | gemma-4-26B-A4B-it-UD-IQ2_M | layer | both | 65536 | off | 1293.4 | 54.6 | 17.1 | OK |
| gemma4-26b | gemma-4-26B-A4B-it-UD-IQ2_M | layer | both | 32768 | off | 1282.3 | 54.8 | 17.0 | OK |
| gemma4-26b | gemma-4-26B-A4B-it-UD-IQ2_M | layer | both | 16384 | off | 1282.9 | 54.8 | 16.9 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 131072 | off | - | - | 3.4 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 131072 | on(n=1) | - | - | 4.1 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 131072 | on(n=2) | - | - | 4.1 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 131072 | on(n=3) | - | - | 4.1 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 131072 | on(n=4) | - | - | 4.1 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 65536 | off | - | - | 3.4 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 65536 | on(n=1) | - | - | 4.1 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 65536 | on(n=2) | - | - | 4.1 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 65536 | on(n=3) | - | - | 4.1 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 65536 | on(n=4) | - | - | 4.1 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 32768 | off | - | - | 3.4 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 32768 | on(n=1) | - | - | 4.2 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 32768 | on(n=2) | - | - | 4.2 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 32768 | on(n=3) | - | - | 4.2 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 32768 | on(n=4) | - | - | 4.2 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 16384 | off | - | - | 4.6 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 16384 | on(n=1) | - | - | 5.0 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 16384 | on(n=2) | - | - | 5.1 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 16384 | on(n=3) | - | - | 5.1 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 0 | 16384 | on(n=4) | - | - | 5.0 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 131072 | off | - | - | 5.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 131072 | on(n=1) | - | - | 6.2 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 131072 | on(n=2) | - | - | 6.2 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 131072 | on(n=3) | - | - | 6.3 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 131072 | on(n=4) | - | - | 6.3 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 65536 | off | - | - | 5.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 65536 | on(n=1) | - | - | 6.4 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 65536 | on(n=2) | - | - | 6.3 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 65536 | on(n=3) | - | - | 6.3 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 65536 | on(n=4) | - | - | 6.2 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 32768 | off | - | - | 5.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 32768 | on(n=1) | - | - | 6.3 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 32768 | on(n=2) | - | - | 6.3 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 32768 | on(n=3) | - | - | 6.3 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 32768 | on(n=4) | - | - | 6.3 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 16384 | off | 149.8 | 12.5 | 99.1 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 16384 | on(n=1) | - | - | 7.2 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 16384 | on(n=2) | - | - | 7.2 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 16384 | on(n=3) | - | - | 7.1 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | none | 1 | 16384 | on(n=4) | - | - | 7.2 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 131072 | off | 262.7 | 13.3 | 62.0 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 131072 | on(n=1) | - | - | 6.6 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 131072 | on(n=2) | - | - | 6.6 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 131072 | on(n=3) | - | - | 6.6 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 131072 | on(n=4) | - | - | 6.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 65536 | off | 258.9 | 13.2 | 62.8 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 65536 | on(n=1) | 220.2 | 16.0 | 71.5 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 65536 | on(n=2) | 220.0 | 15.3 | 71.9 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 65536 | on(n=3) | 220.0 | 13.2 | 73.2 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 65536 | on(n=4) | 220.7 | 13.7 | 72.7 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 32768 | off | 256.0 | 13.3 | 63.0 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 32768 | on(n=1) | 219.9 | 16.0 | 71.3 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 32768 | on(n=2) | 219.9 | 13.8 | 72.6 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 32768 | on(n=3) | 219.5 | 14.4 | 72.3 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 32768 | on(n=4) | 220.0 | 13.2 | 73.0 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 16384 | off | 254.6 | 13.4 | 63.2 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 16384 | on(n=1) | 218.9 | 16.2 | 71.5 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 16384 | on(n=2) | 219.6 | 15.9 | 71.4 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 16384 | on(n=3) | 220.2 | 14.1 | 72.3 | OK |
| gemma4-31b | gemma-4-31B-it-UD-IQ2_XXS | layer | both | 16384 | on(n=4) | 219.7 | 10.7 | 75.3 | OK |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 131072 | off | - | - | 2.9 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=1) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=2) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=3) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 131072 | on(n=4) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 65536 | off | - | - | 2.9 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=1) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=2) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=3) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 65536 | on(n=4) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 32768 | off | - | - | 2.9 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=1) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=2) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=3) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 32768 | on(n=4) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 16384 | off | - | - | 2.9 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=1) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=2) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=3) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 0 | 16384 | on(n=4) | - | - | 3.7 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 131072 | off | - | - | 2.9 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=1) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=2) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=3) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 131072 | on(n=4) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 65536 | off | - | - | 2.9 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=1) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=2) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=3) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 65536 | on(n=4) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 32768 | off | - | - | 2.9 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=1) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=2) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=3) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 32768 | on(n=4) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 16384 | off | - | - | 2.9 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=1) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=2) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=3) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | none | 1 | 16384 | on(n=4) | - | - | 3.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 131072 | off | - | - | 5.5 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 131072 | on(n=1) | - | - | 6.3 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 131072 | on(n=2) | - | - | 6.3 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 131072 | on(n=3) | - | - | 6.3 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 131072 | on(n=4) | - | - | 6.3 | OOM |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 65536 | off | 245.3 | 12.3 | 67.1 | OK |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 65536 | on(n=1) | 207.3 | 15.1 | 76.6 | OK |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 65536 | on(n=2) | 207.2 | 13.8 | 77.3 | OK |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 65536 | on(n=3) | 207.3 | 14.5 | 76.8 | OK |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 65536 | on(n=4) | 207.0 | 14.0 | 77.2 | OK |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 32768 | off | 241.0 | 12.2 | 67.9 | OK |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 32768 | on(n=1) | 206.6 | 14.9 | 76.5 | OK |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 32768 | on(n=2) | 206.7 | 14.9 | 76.6 | OK |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 32768 | on(n=3) | 207.0 | 14.3 | 76.8 | OK |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 32768 | on(n=4) | 207.1 | 11.2 | 79.3 | OK |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 16384 | off | 241.0 | 12.3 | 67.8 | OK |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 16384 | on(n=1) | 207.0 | 15.6 | 76.1 | OK |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 16384 | on(n=2) | 207.1 | 14.3 | 76.8 | OK |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 16384 | on(n=3) | 207.7 | 14.4 | 76.5 | OK |
| gemma4-31b | gemma-4-31B-it-UD-Q2_K_XL | layer | both | 16384 | on(n=4) | 207.5 | 10.4 | 80.0 | OK |
| gemma4-e2b | gemma-4-E2B-it-Q4_K_M | none | 0 | 131072 | off | 1424.6 | 85.0 | 12.8 | OK |
| gemma4-e2b | gemma-4-E2B-it-Q4_K_M | none | 0 | 65536 | off | 1419.1 | 85.0 | 12.8 | OK |
| gemma4-e2b | gemma-4-E2B-it-Q4_K_M | none | 0 | 32768 | off | 1415.2 | 82.6 | 12.9 | OK |
| gemma4-e2b | gemma-4-E2B-it-Q4_K_M | none | 0 | 16384 | off | 1409.1 | 84.8 | 12.8 | OK |
| gemma4-e2b | gemma-4-E2B-it-Q4_K_M | none | 1 | 131072 | off | 1420.8 | 79.4 | 13.3 | OK |
| gemma4-e2b | gemma-4-E2B-it-Q4_K_M | none | 1 | 65536 | off | 1395.5 | 78.1 | 13.5 | OK |
| gemma4-e2b | gemma-4-E2B-it-Q4_K_M | none | 1 | 32768 | off | 1393.3 | 74.5 | 13.5 | OK |
| gemma4-e2b | gemma-4-E2B-it-Q4_K_M | none | 1 | 16384 | off | 1334.5 | 76.5 | 13.9 | OK |
| gemma4-e2b | gemma-4-E2B-it-Q4_K_M | layer | both | 131072 | off | 2339.7 | 18.3 | 15.3 | OK |
| gemma4-e2b | gemma-4-E2B-it-Q4_K_M | layer | both | 65536 | off | 2333.8 | 18.5 | 15.0 | OK |
| gemma4-e2b | gemma-4-E2B-it-Q4_K_M | layer | both | 32768 | off | 2351.6 | 18.3 | 15.0 | OK |
| gemma4-e2b | gemma-4-E2B-it-Q4_K_M | layer | both | 16384 | off | 2348.7 | 18.5 | 14.9 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 131072 | off | 958.7 | 56.4 | 18.2 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 131072 | on(n=1) | 948.6 | 77.3 | 17.9 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 131072 | on(n=2) | 948.3 | 75.5 | 17.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 131072 | on(n=3) | 948.5 | 86.9 | 19.1 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 131072 | on(n=4) | 947.0 | 82.5 | 19.2 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 65536 | off | 942.2 | 55.0 | 16.1 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 65536 | on(n=1) | 942.1 | 72.0 | 19.4 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 65536 | on(n=2) | 945.1 | 76.4 | 17.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 65536 | on(n=3) | 946.9 | 80.1 | 19.2 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 65536 | on(n=4) | 945.3 | 80.6 | 17.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 32768 | off | 937.2 | 54.0 | 18.4 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 32768 | on(n=1) | 943.9 | 76.4 | 17.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 32768 | on(n=2) | 946.2 | 76.4 | 19.3 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 32768 | on(n=3) | 946.8 | 97.9 | 17.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 32768 | on(n=4) | 945.9 | 80.9 | 17.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 16384 | off | 931.8 | 55.0 | 18.4 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 16384 | on(n=1) | 943.3 | 75.0 | 17.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 16384 | on(n=2) | 946.7 | 75.9 | 17.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 16384 | on(n=3) | 944.4 | 95.5 | 17.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 0 | 16384 | on(n=4) | 947.5 | 85.0 | 17.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 131072 | off | 964.7 | 51.7 | 16.6 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 131072 | on(n=1) | 945.2 | 72.0 | 18.6 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 131072 | on(n=2) | 930.7 | 68.0 | 18.8 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 131072 | on(n=3) | 924.8 | 87.6 | 18.8 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 131072 | on(n=4) | 916.5 | 77.7 | 19.0 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 65536 | off | 921.0 | 47.8 | 19.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 65536 | on(n=1) | 923.9 | 67.8 | 20.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 65536 | on(n=2) | 937.7 | 84.9 | 20.0 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 65536 | on(n=3) | 918.3 | 67.3 | 20.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 65536 | on(n=4) | 911.5 | 65.6 | 20.9 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 32768 | off | 925.4 | 46.7 | 17.2 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 32768 | on(n=1) | 929.2 | 67.6 | 18.8 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 32768 | on(n=2) | 914.2 | 71.3 | 19.0 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 32768 | on(n=3) | 931.0 | 92.9 | 18.5 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 32768 | on(n=4) | 921.1 | 76.4 | 20.4 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 16384 | off | 922.1 | 47.6 | 19.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 16384 | on(n=1) | 918.5 | 68.7 | 20.4 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 16384 | on(n=2) | 921.6 | 66.1 | 20.5 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 16384 | on(n=3) | 924.5 | 85.4 | 18.6 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | none | 1 | 16384 | on(n=4) | 901.6 | 64.5 | 21.0 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 131072 | off | 1697.8 | 53.3 | 13.2 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 131072 | on(n=1) | 1432.4 | 66.3 | 14.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 131072 | on(n=2) | 1429.3 | 65.2 | 16.3 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 131072 | on(n=3) | 1430.5 | 54.5 | 16.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 131072 | on(n=4) | 1430.8 | 64.6 | 16.3 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 65536 | off | 1695.8 | 52.5 | 10.9 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 65536 | on(n=1) | 1428.5 | 65.9 | 16.0 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 65536 | on(n=2) | 1426.1 | 65.4 | 16.0 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 65536 | on(n=3) | 1426.6 | 78.4 | 14.0 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 65536 | on(n=4) | 1428.6 | 65.6 | 14.2 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 32768 | off | 1692.2 | 53.2 | 12.9 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 32768 | on(n=1) | 1424.0 | 67.9 | 14.0 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 32768 | on(n=2) | 1423.1 | 64.9 | 15.8 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 32768 | on(n=3) | 1425.7 | 60.2 | 15.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 32768 | on(n=4) | 1427.5 | 65.6 | 13.9 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 16384 | off | 1650.6 | 51.8 | 10.9 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 16384 | on(n=1) | 1423.1 | 68.0 | 13.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 16384 | on(n=2) | 1423.0 | 62.9 | 13.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 16384 | on(n=3) | 1423.9 | 78.7 | 13.7 | OK |
| gemma4-e4b | gemma-4-E4B-it-Q4_K_M | layer | both | 16384 | on(n=4) | 1420.6 | 62.1 | 15.7 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 0 | 131072 | off | 1024.0 | 40.8 | 17.8 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 0 | 65536 | off | 1022.1 | 40.7 | 17.8 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 0 | 32768 | off | 1022.5 | 40.7 | 17.8 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 0 | 16384 | off | 1019.5 | 40.8 | 17.8 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 1 | 131072 | off | 1024.6 | 39.9 | 18.5 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 1 | 65536 | off | 1018.0 | 38.6 | 18.7 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 1 | 32768 | off | 1014.1 | 37.7 | 18.8 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q2_K_XL | none | 1 | 16384 | off | 1003.5 | 37.7 | 18.9 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q2_K_XL | layer | both | 131072 | off | 1037.1 | 39.9 | 18.2 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q2_K_XL | layer | both | 65536 | off | 1033.6 | 39.6 | 18.2 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q2_K_XL | layer | both | 32768 | off | 1032.2 | 38.6 | 18.3 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q2_K_XL | layer | both | 16384 | off | 1031.6 | 39.5 | 18.0 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 0 | 131072 | off | 1051.2 | 63.5 | 16.7 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 0 | 65536 | off | 1041.5 | 60.0 | 14.6 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 0 | 32768 | off | 1025.4 | 58.2 | 16.8 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 0 | 16384 | off | 1034.2 | 58.7 | 16.7 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 1 | 131072 | off | 1046.0 | 59.7 | 17.4 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 1 | 65536 | off | 1026.5 | 57.1 | 17.7 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 1 | 32768 | off | 1022.3 | 55.6 | 17.8 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q4_K_XL | none | 1 | 16384 | off | 1017.2 | 58.1 | 15.7 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q4_K_XL | layer | both | 131072 | off | 1850.1 | 59.4 | 10.1 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q4_K_XL | layer | both | 65536 | off | 1855.4 | 59.9 | 12.0 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q4_K_XL | layer | both | 32768 | off | 1848.5 | 59.3 | 9.8 | OK |
| gemma4-e4b-qat | gemma-4-E4B-it-qat-UD-Q4_K_XL | layer | both | 16384 | off | 1849.1 | 59.9 | 11.8 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-IQ2_M | none | 0 | 131072 | off | 4451.0 | 161.5 | 5.0 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-IQ2_M | none | 0 | 65536 | off | 4453.3 | 155.3 | 4.3 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-IQ2_M | none | 0 | 32768 | off | 4448.8 | 161.6 | 5.0 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-IQ2_M | none | 0 | 16384 | off | 4449.4 | 161.7 | 5.0 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-IQ2_M | none | 1 | 131072 | off | 4420.6 | 148.2 | 5.3 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-IQ2_M | none | 1 | 65536 | off | 4407.3 | 149.0 | 5.1 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-IQ2_M | none | 1 | 32768 | off | 4393.5 | 149.1 | 5.1 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-IQ2_M | none | 1 | 16384 | off | 4388.0 | 147.9 | 5.1 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-IQ2_M | layer | both | 131072 | off | 7569.6 | 151.9 | 4.3 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-IQ2_M | layer | both | 65536 | off | 7531.3 | 150.5 | 4.3 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-IQ2_M | layer | both | 32768 | off | 7533.3 | 151.2 | 4.0 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-IQ2_M | layer | both | 16384 | off | 7511.7 | 150.3 | 4.0 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q4_K_XL | none | 0 | 131072 | off | 4460.4 | 151.0 | 5.1 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q4_K_XL | none | 0 | 65536 | off | 4436.3 | 150.7 | 5.1 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q4_K_XL | none | 0 | 32768 | off | 4422.0 | 150.1 | 5.1 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q4_K_XL | none | 0 | 16384 | off | 4415.1 | 150.3 | 5.1 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q4_K_XL | none | 1 | 131072 | off | 4387.4 | 139.5 | 5.4 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q4_K_XL | none | 1 | 65536 | off | 4375.3 | 139.4 | 5.4 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q4_K_XL | none | 1 | 32768 | off | 4353.9 | 139.5 | 5.4 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q4_K_XL | none | 1 | 16384 | off | 4349.8 | 139.3 | 5.4 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q4_K_XL | layer | both | 131072 | off | 7328.9 | 142.1 | 4.4 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q4_K_XL | layer | both | 65536 | off | 7368.6 | 142.4 | 4.4 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q4_K_XL | layer | both | 32768 | off | 7343.5 | 141.4 | 4.3 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q4_K_XL | layer | both | 16384 | off | 7363.9 | 140.2 | 4.3 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q8_K_XL | none | 0 | 131072 | off | 4128.0 | 126.5 | 5.7 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q8_K_XL | none | 0 | 65536 | off | 4109.7 | 126.4 | 5.7 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q8_K_XL | none | 0 | 32768 | off | 4084.1 | 126.3 | 5.5 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q8_K_XL | none | 0 | 16384 | off | 4090.5 | 125.8 | 5.7 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q8_K_XL | none | 1 | 131072 | off | 4092.0 | 118.4 | 5.9 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q8_K_XL | none | 1 | 65536 | off | 4062.2 | 118.1 | 6.0 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q8_K_XL | none | 1 | 32768 | off | 4054.0 | 117.9 | 6.0 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q8_K_XL | none | 1 | 16384 | off | 4026.7 | 113.0 | 6.0 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q8_K_XL | layer | both | 131072 | off | 6803.8 | 120.2 | 5.1 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q8_K_XL | layer | both | 65536 | off | 6906.1 | 120.4 | 4.8 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q8_K_XL | layer | both | 32768 | off | 6870.5 | 120.5 | 4.8 | OK |
| qwen3.5 | Qwen3.5-0.8B-UD-Q8_K_XL | layer | both | 16384 | off | 6862.7 | 119.3 | 4.8 | OK |
| qwen3.6_27B | Qwen3.6-27B-Q4_K_S | none | 0 | 131072 | off | - | - | 2.7 | OOM |
| qwen3.6_27B | Qwen3.6-27B-Q4_K_S | none | 0 | 65536 | off | - | - | 2.7 | OOM |
| qwen3.6_27B | Qwen3.6-27B-Q4_K_S | none | 0 | 32768 | off | - | - | 2.7 | OOM |
| qwen3.6_27B | Qwen3.6-27B-Q4_K_S | none | 0 | 16384 | off | - | - | 2.5 | OOM |
| qwen3.6_27B | Qwen3.6-27B-Q4_K_S | none | 1 | 131072 | off | - | - | 2.5 | OOM |
| qwen3.6_27B | Qwen3.6-27B-Q4_K_S | none | 1 | 65536 | off | - | - | 2.7 | OOM |
| qwen3.6_27B | Qwen3.6-27B-Q4_K_S | none | 1 | 32768 | off | - | - | 2.5 | OOM |
| qwen3.6_27B | Qwen3.6-27B-Q4_K_S | none | 1 | 16384 | off | - | - | 2.7 | OOM |
| qwen3.6_27B | Qwen3.6-27B-Q4_K_S | layer | both | 131072 | off | - | - | 5.9 | OOM |
| qwen3.6_27B | Qwen3.6-27B-Q4_K_S | layer | both | 65536 | off | 397.0 | 14.7 | 44.1 | OK |
| qwen3.6_27B | Qwen3.6-27B-Q4_K_S | layer | both | 32768 | off | 386.4 | 14.6 | 44.9 | OK |
| qwen3.6_27B | Qwen3.6-27B-Q4_K_S | layer | both | 16384 | off | 386.7 | 14.7 | 44.8 | OK |
| qwen3.6_35B | Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 0 | 131072 | off | - | - | 2.4 | OOM |
| qwen3.6_35B | Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 0 | 65536 | off | - | - | 2.3 | OOM |
| qwen3.6_35B | Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 0 | 32768 | off | - | - | 2.3 | OOM |
| qwen3.6_35B | Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 0 | 16384 | off | - | - | 2.3 | OOM |
| qwen3.6_35B | Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 1 | 131072 | off | - | - | 2.3 | OOM |
| qwen3.6_35B | Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 1 | 65536 | off | - | - | 2.3 | OOM |
| qwen3.6_35B | Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 1 | 32768 | off | - | - | 2.3 | OOM |
| qwen3.6_35B | Qwen3.6-35B-A3B-UD-Q2_K_XL | none | 1 | 16384 | off | - | - | 2.3 | OOM |
| qwen3.6_35B | Qwen3.6-35B-A3B-UD-Q2_K_XL | layer | both | 131072 | off | 1415.3 | 53.2 | 15.9 | OK |
| qwen3.6_35B | Qwen3.6-35B-A3B-UD-Q2_K_XL | layer | both | 65536 | off | 1406.3 | 53.1 | 15.7 | OK |
| qwen3.6_35B | Qwen3.6-35B-A3B-UD-Q2_K_XL | layer | both | 32768 | off | 1388.6 | 51.4 | 13.8 | OK |
| qwen3.6_35B | Qwen3.6-35B-A3B-UD-Q2_K_XL | layer | both | 16384 | off | 1401.5 | 53.1 | 15.7 | OK |
| qwen3.8_27B | Qwen3.8-27B-UD-IQ2_XXS | none | 0 | 131072 | off | - | - | 3.0 | OOM |
| qwen3.8_27B | Qwen3.8-27B-UD-IQ2_XXS | none | 0 | 65536 | off | - | - | 2.9 | OOM |
| qwen3.8_27B | Qwen3.8-27B-UD-IQ2_XXS | none | 0 | 32768 | off | 207.2 | 11.9 | 64.0 | OK |
| qwen3.8_27B | Qwen3.8-27B-UD-IQ2_XXS | none | 0 | 16384 | off | 207.1 | 12.3 | 69.3 | OK |
| qwen3.8_27B | Qwen3.8-27B-UD-IQ2_XXS | none | 1 | 131072 | off | - | - | 5.0 | OOM |
| qwen3.8_27B | Qwen3.8-27B-UD-IQ2_XXS | none | 1 | 65536 | off | - | - | 5.0 | OOM |
| qwen3.8_27B | Qwen3.8-27B-UD-IQ2_XXS | none | 1 | 32768 | off | 206.2 | 11.6 | 68.0 | OK |
| qwen3.8_27B | Qwen3.8-27B-UD-IQ2_XXS | none | 1 | 16384 | off | 204.5 | 11.4 | 73.0 | OK |
| qwen3.8_27B | Qwen3.8-27B-UD-IQ2_XXS | layer | both | 131072 | off | 379.1 | 14.1 | 44.1 | OK |
| qwen3.8_27B | Qwen3.8-27B-UD-IQ2_XXS | layer | both | 65536 | off | 371.6 | 13.3 | 40.7 | OK |
| qwen3.8_27B | Qwen3.8-27B-UD-IQ2_XXS | layer | both | 32768 | off | 367.4 | 13.7 | 45.0 | OK |
| qwen3.8_27B | Qwen3.8-27B-UD-IQ2_XXS | layer | both | 16384 | off | 366.7 | 13.9 | 41.2 | OK |