모종의 음모/ai 프로그램2026. 9. 17. 10:36

LLM / 1080 ti 모드 별 벤치마크

궁금한 것 을 해소하기 위해 대충 5시간에 걸쳐 GPU 갈굼

  • MTP 가 정말 속도에 영향을 주냐
  • GPU 2개인데 pcie3.0 x8과 x4에 영향을 받는가
  • 컨텍스트 최대 길이가 성능에 영향을 미치는가
  • 입력 토큰길이에 따라 성능 저하는 어느정도 발생하는가
  • 단일과 -sm layer로 나누었을때 성능은 어떠한가

 

결과만 요약하면

  • GPU 0 / 1이 우선권 문제인지 lane 폭 문제인진 모르겠으나 GPU0이 3~10% 정도 이상 성능이 좋게 나옴
  • MTP는 켤수 있으면 켜라. 조건에 따라 다르지만 MTP 1이 1.3~1.5배 향상됨
  • 컨텍스트 최대 길이는 의외로 생성 성능에 영향을 안준다.
  • 한쪽에 가능하지만, 억지로 레이어 쪼개서 양쪽으로 올리면 1.1~1.4배 성능 차이가 난다. 되도록이면 1대로 돌리자

 

아래는 테스트용 스크립트들. 추후 쓰려면 바이너리 경로등을 손봐야 함

bench.zip
0.01MB

 

 

 

아래는 원본 내용

report.md
0.08MB

 

 

llama.cpp Benchmark Report

Short conversation (3-turn, context accumulates)

Questions: see questions.txt. Columns show prompt_t/s / generation_t/s per turn.

gemma4-12b

quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status

gemma-4-12b-it-Q4_0 none 0 131072 off 139.2/34.8 143.4/34.6 270.3/34.4 38.4 OK
gemma-4-12b-it-Q4_0 none 0 131072 on(n=1) 123.9/57.5 141.8/56.1 246.3/51.2 21.5 OK
gemma-4-12b-it-Q4_0 none 0 131072 on(n=2) 123.3/61.5 187.3/61.7 238.3/59.1 18.6 OK
gemma-4-12b-it-Q4_0 none 0 131072 on(n=3) 118.4/65.5 227.8/59.7 231.2/51.6 18.9 OK
gemma-4-12b-it-Q4_0 none 0 131072 on(n=4) 123.2/65.5 181.6/57.1 240.6/45.5 20.0 OK
gemma-4-12b-it-Q4_0 none 0 65536 off 140.0/33.5 146.5/33.2 266.5/32.8 27.0 OK
gemma-4-12b-it-Q4_0 none 0 65536 on(n=1) 118.5/54.5 138.2/55.1 245.5/51.2 20.0 OK
gemma-4-12b-it-Q4_0 none 0 65536 on(n=2) 121.3/59.5 226.9/62.4 229.2/51.9 19.0 OK
gemma-4-12b-it-Q4_0 none 0 65536 on(n=3) 122.9/66.3 243.9/52.7 222.2/52.1 19.0 OK
gemma-4-12b-it-Q4_0 none 0 65536 on(n=4) 118.3/65.2 139.6/60.8 245.8/48.4 19.1 OK
gemma-4-12b-it-Q4_0 none 0 32768 off 136.6/33.6 146.1/33.2 263.9/32.7 27.0 OK
gemma-4-12b-it-Q4_0 none 0 32768 on(n=1) 125.1/57.3 140.9/55.5 242.6/51.8 19.7 OK
gemma-4-12b-it-Q4_0 none 0 32768 on(n=2) 120.7/65.7 139.8/58.8 243.7/57.5 18.3 OK
gemma-4-12b-it-Q4_0 none 0 32768 on(n=3) 125.6/63.6 139.2/60.2 244.5/53.6 18.7 OK
gemma-4-12b-it-Q4_0 none 0 32768 on(n=4) 113.1/60.4 182.1/63.9 242.7/43.8 19.8 OK
gemma-4-12b-it-Q4_0 none 0 16384 off 139.6/33.5 146.0/32.9 267.0/32.7 27.1 OK
gemma-4-12b-it-Q4_0 none 0 16384 on(n=1) 122.8/55.8 213.0/56.2 230.9/53.9 19.6 OK
gemma-4-12b-it-Q4_0 none 0 16384 on(n=2) 124.2/62.4 139.8/60.7 243.0/54.1 18.7 OK
gemma-4-12b-it-Q4_0 none 0 16384 on(n=3) 122.5/60.7 140.1/63.2 244.8/53.5 18.7 OK
gemma-4-12b-it-Q4_0 none 0 16384 on(n=4) 120.8/60.6 141.5/58.4 245.8/50.0 19.4 OK
gemma-4-12b-it-Q4_0 none 1 131072 off 135.6/32.8 249.3/32.7 225.6/32.5 27.0 OK
gemma-4-12b-it-Q4_0 none 1 131072 on(n=1) 106.7/52.1 130.0/51.5 223.8/49.9 22.8 OK
gemma-4-12b-it-Q4_0 none 1 131072 on(n=2) 117.4/59.5 129.0/57.8 224.2/53.2 21.3 OK
gemma-4-12b-it-Q4_0 none 1 131072 on(n=3) 117.7/61.5 155.4/54.7 218.6/52.5 21.5 OK
gemma-4-12b-it-Q4_0 none 1 131072 on(n=4) 112.3/62.0 128.8/56.6 223.6/45.7 22.1 OK
gemma-4-12b-it-Q4_0 none 1 65536 off 130.6/31.5 243.5/30.9 223.0/30.6 29.4 OK
gemma-4-12b-it-Q4_0 none 1 65536 on(n=1) 117.7/50.5 130.1/50.2 225.2/48.2 23.2 OK
gemma-4-12b-it-Q4_0 none 1 65536 on(n=2) 117.1/57.9 128.0/56.5 222.8/49.6 21.9 OK
gemma-4-12b-it-Q4_0 none 1 65536 on(n=3) 117.0/51.0 129.5/54.2 222.5/51.0 22.5 OK
gemma-4-12b-it-Q4_0 none 1 65536 on(n=4) 117.0/61.7 225.9/59.6 200.0/45.0 21.5 OK
gemma-4-12b-it-Q4_0 none 1 32768 off 133.9/31.7 212.7/31.0 227.7/30.7 30.3 OK
gemma-4-12b-it-Q4_0 none 1 32768 on(n=1) 119.1/49.3 130.3/51.8 222.2/48.9 22.9 OK
gemma-4-12b-it-Q4_0 none 1 32768 on(n=2) 119.2/61.1 130.8/57.4 224.5/48.8 21.4 OK
gemma-4-12b-it-Q4_0 none 1 32768 on(n=3) 119.4/58.1 127.6/53.1 216.0/45.0 22.4 OK
gemma-4-12b-it-Q4_0 none 1 32768 on(n=4) 118.7/60.7 130.8/53.0 225.1/45.8 22.2 OK
gemma-4-12b-it-Q4_0 none 1 16384 off 132.8/31.8 137.5/31.0 244.6/30.7 30.1 OK
gemma-4-12b-it-Q4_0 none 1 16384 on(n=1) 107.6/52.2 227.2/51.0 200.4/48.9 21.2 OK
gemma-4-12b-it-Q4_0 none 1 16384 on(n=2) 110.5/57.1 209.0/56.6 204.2/51.1 21.1 OK
gemma-4-12b-it-Q4_0 none 1 16384 on(n=3) 117.0/59.2 129.6/57.1 208.3/46.9 21.8 OK
gemma-4-12b-it-Q4_0 none 1 16384 on(n=4) 117.1/63.0 193.5/54.3 208.2/43.3 22.3 OK
gemma-4-12b-it-Q4_0 layer both 131072 off 138.0/33.4 145.7/33.3 260.4/33.2 28.0 OK
gemma-4-12b-it-Q4_0 layer both 131072 on(n=1) 123.1/49.2 136.9/48.5 238.2/47.3 23.3 OK
gemma-4-12b-it-Q4_0 layer both 131072 on(n=2) 122.3/55.9 239.8/50.8 213.4/45.0 22.8 OK
gemma-4-12b-it-Q4_0 layer both 131072 on(n=3) 122.4/49.1 135.9/48.8 236.3/43.6 23.7 OK
gemma-4-12b-it-Q4_0 layer both 131072 on(n=4) 118.0/53.8 136.3/47.4 237.4/38.9 24.4 OK
gemma-4-12b-it-Q4_0 layer both 65536 off 136.4/33.1 143.0/33.0 255.2/33.0 28.1 OK
gemma-4-12b-it-Q4_0 layer both 65536 on(n=1) 122.1/48.4 225.8/49.4 218.4/47.4 22.5 OK
gemma-4-12b-it-Q4_0 layer both 65536 on(n=2) 122.3/54.4 136.8/51.9 237.7/44.2 22.5 OK
gemma-4-12b-it-Q4_0 layer both 65536 on(n=3) 122.4/45.0 136.5/47.1 237.8/44.8 24.0 OK
gemma-4-12b-it-Q4_0 layer both 65536 on(n=4) 122.4/53.7 136.4/47.1 238.4/42.6 23.3 OK
gemma-4-12b-it-Q4_0 layer both 32768 off 140.6/33.1 145.9/33.1 260.2/33.0 27.9 OK
gemma-4-12b-it-Q4_0 layer both 32768 on(n=1) 122.9/48.3 137.3/49.0 239.4/47.7 22.8 OK
gemma-4-12b-it-Q4_0 layer both 32768 on(n=2) 109.1/52.3 137.7/52.0 239.2/46.8 22.2 OK
gemma-4-12b-it-Q4_0 layer both 32768 on(n=3) 113.3/53.9 138.1/47.0 239.0/45.8 22.7 OK
gemma-4-12b-it-Q4_0 layer both 32768 on(n=4) 124.3/52.7 237.4/48.5 217.2/38.7 23.7 OK
gemma-4-12b-it-Q4_0 layer both 16384 off 135.4/32.8 143.2/32.8 257.2/32.8 28.1 OK
gemma-4-12b-it-Q4_0 layer both 16384 on(n=1) 118.5/50.7 135.7/49.0 237.0/47.7 22.5 OK
gemma-4-12b-it-Q4_0 layer both 16384 on(n=2) 120.0/53.6 216.3/52.3 219.0/46.5 21.9 OK
gemma-4-12b-it-Q4_0 layer both 16384 on(n=3) 119.0/53.8 235.9/49.9 214.2/46.3 21.2 OK
gemma-4-12b-it-Q4_0 layer both 16384 on(n=4) 98.1/51.2 236.1/46.7 214.0/40.8 23.8 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 131072 off 85.4/31.3 93.9/31.1 203.3/30.5 36.9 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 131072 on(n=1) 78.3/42.7 166.5/42.6 181.6/40.2 24.0 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 131072 on(n=2) 78.5/45.1 92.4/43.7 184.5/34.0 24.7 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 131072 on(n=3) 78.6/40.4 91.7/41.9 189.3/31.6 26.2 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 131072 on(n=4) 78.9/42.5 153.1/39.7 183.6/26.9 27.7 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 65536 off 85.2/30.8 92.6/29.7 203.0/29.5 29.2 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 65536 on(n=1) 78.9/45.4 163.3/42.7 181.7/40.6 22.6 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 65536 on(n=2) 78.0/43.3 91.8/42.6 190.4/35.2 24.6 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 65536 on(n=3) 79.2/41.9 131.7/42.2 185.5/32.4 25.5 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 65536 on(n=4) 78.1/42.2 92.5/41.4 191.5/35.2 25.1 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 32768 off 84.8/30.6 93.7/29.6 203.1/29.4 29.4 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 32768 on(n=1) 73.8/44.8 157.0/42.1 182.4/39.7 23.7 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 32768 on(n=2) 70.6/43.6 155.4/46.0 181.4/36.8 23.9 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 32768 on(n=3) 77.4/40.6 93.5/43.4 189.3/29.6 26.3 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 32768 on(n=4) 78.5/43.6 92.2/36.1 188.0/30.9 26.7 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 16384 off 83.1/30.8 96.2/29.6 198.6/29.4 29.3 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 16384 on(n=1) 78.3/44.4 93.2/43.3 191.1/40.6 23.4 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 16384 on(n=2) 76.5/45.0 92.9/44.7 187.7/36.5 23.9 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 16384 on(n=3) 78.8/43.9 92.3/44.5 188.9/33.0 24.8 OK
gemma-4-12b-it-UD-Q2_K_XL none 0 16384 on(n=4) 68.9/45.3 173.1/37.3 181.7/31.7 25.1 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 131072 off 84.3/30.6 93.4/30.5 192.8/30.2 30.1 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 131072 on(n=1) 76.0/43.4 178.7/40.8 164.0/37.0 26.3 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 131072 on(n=2) 76.1/44.7 90.1/42.6 174.1/36.0 26.0 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 131072 on(n=3) 71.8/38.8 177.5/38.2 163.8/37.9 27.1 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 131072 on(n=4) 75.7/38.1 88.6/40.9 178.3/28.7 29.1 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 65536 off 81.6/28.6 87.9/28.2 188.6/28.0 32.1 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 65536 on(n=1) 70.9/43.4 81.7/39.7 176.3/37.0 26.2 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 65536 on(n=2) 75.9/42.5 178.2/38.9 164.5/37.8 25.7 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 65536 on(n=3) 76.2/40.1 123.6/36.9 173.4/29.7 28.8 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 65536 on(n=4) 73.4/43.0 87.9/36.1 170.6/27.2 29.4 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 32768 off 82.1/28.5 185.9/28.1 176.6/28.1 32.1 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 32768 on(n=1) 75.6/42.5 87.4/42.0 175.6/36.3 26.1 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 32768 on(n=2) 76.1/42.0 160.3/44.1 166.9/35.7 24.6 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 32768 on(n=3) 66.4/36.6 88.3/35.5 171.7/30.3 29.6 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 32768 on(n=4) 76.0/46.1 148.8/46.1 162.2/26.7 27.3 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 16384 off 81.2/28.1 91.4/28.0 187.5/28.0 32.3 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 16384 on(n=1) 76.3/42.1 88.9/42.3 174.9/35.7 26.2 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 16384 on(n=2) 76.3/39.3 174.5/41.0 164.8/35.1 26.9 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 16384 on(n=3) 74.7/40.4 123.5/38.3 170.6/31.1 28.2 OK
gemma-4-12b-it-UD-Q2_K_XL none 1 16384 on(n=4) 76.3/40.4 89.3/40.0 178.1/30.9 27.9 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 131072 off 81.7/31.8 92.9/31.7 196.6/31.6 28.8 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 131072 on(n=1) 75.9/42.6 159.0/41.0 174.5/40.2 25.7 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 131072 on(n=2) 73.9/44.2 90.3/38.6 184.9/35.4 26.7 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 131072 on(n=3) 73.9/39.7 90.3/39.6 185.0/29.4 28.6 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 131072 on(n=4) 71.8/35.7 87.9/35.7 182.8/26.0 31.2 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 65536 off 81.6/31.5 90.1/31.2 196.0/31.3 28.8 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 65536 on(n=1) 72.9/43.2 90.6/41.7 184.6/37.1 25.7 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 65536 on(n=2) 75.7/41.7 90.3/41.2 184.0/36.1 26.2 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 65536 on(n=3) 75.7/38.8 90.0/37.7 184.7/30.1 28.6 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 65536 on(n=4) 64.3/37.0 88.7/40.3 184.7/28.0 29.2 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 32768 off 81.7/31.5 129.4/31.3 186.1/30.8 28.9 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 32768 on(n=1) 73.2/42.8 138.4/41.9 175.3/39.9 25.1 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 32768 on(n=2) 75.6/42.0 90.6/38.4 185.3/37.5 26.1 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 32768 on(n=3) 71.1/37.6 167.8/38.1 173.2/27.0 29.6 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 32768 on(n=4) 76.0/34.6 178.5/30.4 174.5/27.6 31.6 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 16384 off 80.9/31.6 91.1/31.3 190.5/31.0 28.8 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 16384 on(n=1) 75.9/41.5 90.1/43.4 184.0/37.2 25.2 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 16384 on(n=2) 73.9/41.2 102.5/40.8 183.7/36.6 25.8 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 16384 on(n=3) 73.8/38.1 90.5/40.6 185.0/31.5 27.4 OK
gemma-4-12b-it-UD-Q2_K_XL layer both 16384 on(n=4) 72.1/34.2 90.2/36.8 184.3/30.5 29.3 OK

gemma4-26b

quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status

gemma-4-26B-A4B-it-UD-IQ2_M none 0 131072 off - - - 21.8 OOM
gemma-4-26B-A4B-it-UD-IQ2_M none 0 65536 off - - - 3.6 OOM
gemma-4-26B-A4B-it-UD-IQ2_M none 0 32768 off 92.5/65.4 175.2/65.2 341.6/64.8 16.1 OK
gemma-4-26B-A4B-it-UD-IQ2_M none 0 16384 off 135.9/64.9 196.1/64.4 338.4/63.2 16.2 OK
gemma-4-26B-A4B-it-UD-IQ2_M none 1 131072 off - - - 6.1 OOM
gemma-4-26B-A4B-it-UD-IQ2_M none 1 65536 off - - - 6.1 OOM
gemma-4-26B-A4B-it-UD-IQ2_M none 1 32768 off 147.6/60.0 177.3/59.9 317.9/59.4 19.5 OK
gemma-4-26B-A4B-it-UD-IQ2_M none 1 16384 off 149.8/59.8 176.1/59.6 314.3/58.2 19.7 OK
gemma-4-26B-A4B-it-UD-IQ2_M layer both 131072 off 148.0/61.8 196.3/61.8 333.4/61.7 18.1 OK
gemma-4-26B-A4B-it-UD-IQ2_M layer both 65536 off 147.7/61.7 196.1/61.6 332.7/61.5 18.1 OK
gemma-4-26B-A4B-it-UD-IQ2_M layer both 32768 off 149.5/61.5 194.9/61.5 330.8/61.4 17.9 OK
gemma-4-26B-A4B-it-UD-IQ2_M layer both 16384 off 149.1/61.5 195.4/61.5 331.4/61.3 17.8 OK

gemma4-31b

quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status

gemma-4-31B-it-UD-IQ2_XXS none 0 131072 off - - - 19.0 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 131072 on(n=1) - - - 4.1 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 131072 on(n=2) - - - 4.1 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 131072 on(n=3) - - - 4.2 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 131072 on(n=4) - - - 4.1 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 65536 off - - - 3.4 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 65536 on(n=1) - - - 4.1 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 65536 on(n=2) - - - 4.1 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 65536 on(n=3) - - - 4.1 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 65536 on(n=4) - - - 4.1 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 32768 off - - - 3.4 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 32768 on(n=1) - - - 4.2 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 32768 on(n=2) - - - 4.2 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 32768 on(n=3) - - - 4.2 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 32768 on(n=4) - - - 4.2 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 16384 off 43.5/15.0 48.2/14.9 96.1/14.2 57.6 OK
gemma-4-31B-it-UD-IQ2_XXS none 0 16384 on(n=1) - - - 6.0 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 16384 on(n=2) - - - 5.1 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 16384 on(n=3) - - - 5.1 OOM
gemma-4-31B-it-UD-IQ2_XXS none 0 16384 on(n=4) - - - 5.1 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 131072 off - - - 5.5 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 131072 on(n=1) - - - 6.3 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 131072 on(n=2) - - - 6.2 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 131072 on(n=3) - - - 6.2 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 131072 on(n=4) - - - 6.2 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 65536 off - - - 5.5 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 65536 on(n=1) - - - 6.3 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 65536 on(n=2) - - - 6.4 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 65536 on(n=3) - - - 6.3 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 65536 on(n=4) - - - 6.3 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 32768 off - - - 5.5 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 32768 on(n=1) - - - 6.3 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 32768 on(n=2) - - - 6.4 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 32768 on(n=3) - - - 6.3 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 32768 on(n=4) - - - 6.4 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 16384 off 43.2/14.8 46.4/14.0 89.6/13.3 62.3 OK
gemma-4-31B-it-UD-IQ2_XXS none 1 16384 on(n=1) - - - 7.2 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 16384 on(n=2) - - - 7.2 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 16384 on(n=3) - - - 7.1 OOM
gemma-4-31B-it-UD-IQ2_XXS none 1 16384 on(n=4) - - - 7.1 OOM
gemma-4-31B-it-UD-IQ2_XXS layer both 131072 off 41.9/15.3 60.1/15.2 94.0/15.1 57.2 OK
gemma-4-31B-it-UD-IQ2_XXS layer both 131072 on(n=1) - - - 6.6 OOM
gemma-4-31B-it-UD-IQ2_XXS layer both 131072 on(n=2) - - - 6.7 OOM
gemma-4-31B-it-UD-IQ2_XXS layer both 131072 on(n=3) - - - 6.6 OOM
gemma-4-31B-it-UD-IQ2_XXS layer both 131072 on(n=4) - - - 6.6 OOM
gemma-4-31B-it-UD-IQ2_XXS layer both 65536 off 42.1/15.2 79.4/15.1 92.2/15.0 57.5 OK
gemma-4-31B-it-UD-IQ2_XXS layer both 65536 on(n=1) 39.3/22.3 67.4/21.9 85.3/21.0 44.1 OK
gemma-4-31B-it-UD-IQ2_XXS layer both 65536 on(n=2) 39.2/23.4 91.9/20.7 81.8/21.2 44.0 OK
gemma-4-31B-it-UD-IQ2_XXS layer both 65536 on(n=3) 37.0/23.9 138.9/16.4 49.6/15.9 52.9 OK
gemma-4-31B-it-UD-IQ2_XXS layer both 65536 on(n=4) 39.2/20.4 45.8/19.0 88.5/18.2 48.7 OK
gemma-4-31B-it-UD-IQ2_XXS layer both 32768 off 41.3/15.1 85.4/15.0 133.1/14.8 57.8 OK
gemma-4-31B-it-UD-IQ2_XXS layer both 32768 on(n=1) 38.7/22.9 92.4/20.9 80.9/20.6 42.9 OK
gemma-4-31B-it-UD-IQ2_XXS layer both 32768 on(n=2) 37.8/22.7 67.6/21.7 85.1/19.9 42.0 OK
gemma-4-31B-it-UD-IQ2_XXS layer both 32768 on(n=3) 39.2/20.7 45.9/19.4 88.9/17.5 48.6 OK
gemma-4-31B-it-UD-IQ2_XXS layer both 32768 on(n=4) 39.2/20.8 45.7/20.3 87.5/17.2 48.2 OK
gemma-4-31B-it-UD-IQ2_XXS layer both 16384 off 41.6/15.1 47.9/14.8 95.1/14.6 58.1 OK
gemma-4-31B-it-UD-IQ2_XXS layer both 16384 on(n=1) 39.3/22.6 66.8/21.9 86.5/20.7 43.8 OK
gemma-4-31B-it-UD-IQ2_XXS layer both 16384 on(n=2) 39.3/24.3 90.5/20.5 81.3/17.9 46.0 OK
gemma-4-31B-it-UD-IQ2_XXS layer both 16384 on(n=3) 39.4/20.3 52.6/19.9 88.4/17.8 48.3 OK
gemma-4-31B-it-UD-IQ2_XXS layer both 16384 on(n=4) 39.3/19.7 145.8/17.8 53.0/15.5 54.2 OK
gemma-4-31B-it-UD-Q2_K_XL none 0 131072 off - - - 24.0 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 131072 on(n=1) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 131072 on(n=2) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 131072 on(n=3) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 131072 on(n=4) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 65536 off - - - 2.9 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 65536 on(n=1) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 65536 on(n=2) - - - 3.7 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 65536 on(n=3) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 65536 on(n=4) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 32768 off - - - 2.9 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 32768 on(n=1) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 32768 on(n=2) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 32768 on(n=3) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 32768 on(n=4) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 16384 off - - - 2.9 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 16384 on(n=1) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 16384 on(n=2) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 16384 on(n=3) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 0 16384 on(n=4) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 131072 off - - - 2.9 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 131072 on(n=1) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 131072 on(n=2) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 131072 on(n=3) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 131072 on(n=4) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 65536 off - - - 2.9 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 65536 on(n=1) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 65536 on(n=2) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 65536 on(n=3) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 65536 on(n=4) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 32768 off - - - 2.9 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 32768 on(n=1) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 32768 on(n=2) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 32768 on(n=3) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 32768 on(n=4) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 16384 off - - - 2.9 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 16384 on(n=1) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 16384 on(n=2) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 16384 on(n=3) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL none 1 16384 on(n=4) - - - 3.5 OOM
gemma-4-31B-it-UD-Q2_K_XL layer both 131072 off - - - 5.5 OOM
gemma-4-31B-it-UD-Q2_K_XL layer both 131072 on(n=1) - - - 6.2 OOM
gemma-4-31B-it-UD-Q2_K_XL layer both 131072 on(n=2) - - - 6.3 OOM
gemma-4-31B-it-UD-Q2_K_XL layer both 131072 on(n=3) - - - 6.3 OOM
gemma-4-31B-it-UD-Q2_K_XL layer both 131072 on(n=4) - - - 6.2 OOM
gemma-4-31B-it-UD-Q2_K_XL layer both 65536 off 35.3/13.8 40.7/13.7 86.1/13.6 63.6 OK
gemma-4-31B-it-UD-Q2_K_XL layer both 65536 on(n=1) 32.7/20.5 40.3/20.7 79.4/20.2 47.5 OK
gemma-4-31B-it-UD-Q2_K_XL layer both 65536 on(n=2) 32.7/22.4 64.2/21.0 76.9/18.7 47.3 OK
gemma-4-31B-it-UD-Q2_K_XL layer both 65536 on(n=3) 32.9/21.9 64.0/20.4 76.9/18.2 46.8 OK
gemma-4-31B-it-UD-Q2_K_XL layer both 65536 on(n=4) 31.0/21.0 39.6/20.1 80.4/17.7 49.4 OK
gemma-4-31B-it-UD-Q2_K_XL layer both 32768 off 34.1/13.6 68.3/13.4 82.1/13.3 60.9 OK
gemma-4-31B-it-UD-Q2_K_XL layer both 32768 on(n=1) 30.9/20.1 48.1/20.7 78.7/19.4 48.0 OK
gemma-4-31B-it-UD-Q2_K_XL layer both 32768 on(n=2) 30.9/20.9 40.1/21.4 79.6/18.7 47.6 OK
gemma-4-31B-it-UD-Q2_K_XL layer both 32768 on(n=3) 31.2/20.9 40.0/20.6 80.0/17.5 49.0 OK
gemma-4-31B-it-UD-Q2_K_XL layer both 32768 on(n=4) 32.6/20.2 64.3/22.5 75.7/18.2 46.0 OK
gemma-4-31B-it-UD-Q2_K_XL layer both 16384 off 34.3/13.6 67.6/13.4 81.0/13.4 64.6 OK
gemma-4-31B-it-UD-Q2_K_XL layer both 16384 on(n=1) 30.8/20.3 63.9/20.1 76.7/19.3 48.4 OK
gemma-4-31B-it-UD-Q2_K_XL layer both 16384 on(n=2) 32.6/20.8 40.0/21.9 79.9/18.8 47.3 OK
gemma-4-31B-it-UD-Q2_K_XL layer both 16384 on(n=3) 32.8/19.9 39.9/17.8 79.8/17.7 51.3 OK
gemma-4-31B-it-UD-Q2_K_XL layer both 16384 on(n=4) 32.8/22.5 40.2/21.1 79.5/16.9 48.3 OK

gemma4-e2b

quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status

gemma-4-E2B-it-Q4_K_M none 0 131072 off 251.7/92.4 268.8/92.4 827.5/92.3 16.6 OK
gemma-4-E2B-it-Q4_K_M none 0 65536 off 177.7/92.2 380.4/92.2 847.8/92.1 11.0 OK
gemma-4-E2B-it-Q4_K_M none 0 32768 off 178.4/91.9 892.3/92.1 837.1/92.0 8.7 OK
gemma-4-E2B-it-Q4_K_M none 0 16384 off 262.1/88.7 430.0/92.2 846.8/92.1 8.3 OK
gemma-4-E2B-it-Q4_K_M none 1 131072 off 243.8/86.7 358.6/86.7 817.0/86.6 12.0 OK
gemma-4-E2B-it-Q4_K_M none 1 65536 off 258.9/85.4 352.0/85.3 803.6/85.4 12.1 OK
gemma-4-E2B-it-Q4_K_M none 1 32768 off 262.6/81.3 408.6/86.0 804.0/85.8 8.9 OK
gemma-4-E2B-it-Q4_K_M none 1 16384 off 263.2/82.2 406.5/85.9 799.4/85.0 8.9 OK
gemma-4-E2B-it-Q4_K_M layer both 131072 off 205.5/65.2 290.7/64.6 660.5/62.9 15.2 OK
gemma-4-E2B-it-Q4_K_M layer both 65536 off 215.5/63.5 340.7/65.2 672.0/63.5 11.0 OK
gemma-4-E2B-it-Q4_K_M layer both 32768 off 163.0/66.0 355.8/67.8 707.0/65.9 10.7 OK
gemma-4-E2B-it-Q4_K_M layer both 16384 off 167.1/62.6 337.5/64.7 664.8/63.0 11.0 OK

gemma4-e4b

quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status

gemma-4-E4B-it-Q4_K_M none 0 131072 off 137.3/61.6 439.9/61.5 510.4/61.2 24.2 OK
gemma-4-E4B-it-Q4_K_M none 0 131072 on(n=1) 162.3/81.3 557.5/87.7 486.8/88.6 11.5 OK
gemma-4-E4B-it-Q4_K_M none 0 131072 on(n=2) 154.5/91.4 550.3/92.6 488.2/88.9 10.9 OK
gemma-4-E4B-it-Q4_K_M none 0 131072 on(n=3) 163.7/89.2 338.7/95.8 496.6/91.6 10.5 OK
gemma-4-E4B-it-Q4_K_M none 0 131072 on(n=4) 167.1/69.7 559.5/89.2 475.0/84.6 11.3 OK
gemma-4-E4B-it-Q4_K_M none 0 65536 off 180.4/60.7 517.8/60.9 497.2/60.0 12.0 OK
gemma-4-E4B-it-Q4_K_M none 0 65536 on(n=1) 163.0/88.7 232.9/92.9 490.9/85.8 13.4 OK
gemma-4-E4B-it-Q4_K_M none 0 65536 on(n=2) 163.9/87.5 548.8/91.6 490.6/87.5 10.9 OK
gemma-4-E4B-it-Q4_K_M none 0 65536 on(n=3) 123.7/66.3 544.5/101.7 484.8/76.2 11.3 OK
gemma-4-E4B-it-Q4_K_M none 0 65536 on(n=4) 161.4/63.4 547.0/86.8 486.2/76.4 11.7 OK
gemma-4-E4B-it-Q4_K_M none 0 32768 off 137.9/60.7 527.3/60.9 503.6/59.6 12.1 OK
gemma-4-E4B-it-Q4_K_M none 0 32768 on(n=1) 163.6/78.8 600.7/93.1 475.7/87.3 11.1 OK
gemma-4-E4B-it-Q4_K_M none 0 32768 on(n=2) 165.2/74.1 399.6/99.9 495.0/90.8 10.4 OK
gemma-4-E4B-it-Q4_K_M none 0 32768 on(n=3) 164.3/74.0 589.4/93.0 469.5/88.0 11.1 OK
gemma-4-E4B-it-Q4_K_M none 0 32768 on(n=4) 125.3/59.3 477.0/85.9 489.8/72.6 11.8 OK
gemma-4-E4B-it-Q4_K_M none 0 16384 off 158.4/60.9 576.5/60.8 495.3/59.6 12.3 OK
gemma-4-E4B-it-Q4_K_M none 0 16384 on(n=1) 155.8/79.5 510.5/86.7 484.5/87.8 11.0 OK
gemma-4-E4B-it-Q4_K_M none 0 16384 on(n=2) 160.8/73.3 565.4/97.5 484.7/86.0 10.9 OK
gemma-4-E4B-it-Q4_K_M none 0 16384 on(n=3) 165.7/86.4 232.9/105.1 492.4/94.8 12.8 OK
gemma-4-E4B-it-Q4_K_M none 0 16384 on(n=4) 166.7/65.1 567.5/91.5 483.8/83.0 11.4 OK
gemma-4-E4B-it-Q4_K_M none 1 131072 off 163.1/57.4 225.8/57.4 492.2/57.1 17.2 OK
gemma-4-E4B-it-Q4_K_M none 1 131072 on(n=1) 155.3/73.1 368.1/87.5 467.6/80.4 11.9 OK
gemma-4-E4B-it-Q4_K_M none 1 131072 on(n=2) 136.0/75.6 476.1/91.2 457.9/83.2 11.9 OK
gemma-4-E4B-it-Q4_K_M none 1 131072 on(n=3) 139.9/76.5 570.5/90.8 441.5/83.2 12.2 OK
gemma-4-E4B-it-Q4_K_M none 1 131072 on(n=4) 143.7/67.1 533.1/78.7 447.0/77.4 12.8 OK
gemma-4-E4B-it-Q4_K_M none 1 65536 off 134.1/56.9 581.6/55.2 444.0/54.5 14.1 OK
gemma-4-E4B-it-Q4_K_M none 1 65536 on(n=1) 133.0/83.1 575.9/81.1 438.6/83.2 12.6 OK
gemma-4-E4B-it-Q4_K_M none 1 65536 on(n=2) 120.5/80.7 530.4/88.3 442.6/81.0 12.2 OK
gemma-4-E4B-it-Q4_K_M none 1 65536 on(n=3) 159.8/63.8 429.5/92.1 445.3/87.8 11.7 OK
gemma-4-E4B-it-Q4_K_M none 1 65536 on(n=4) 157.2/76.2 504.1/77.3 448.0/67.6 13.2 OK
gemma-4-E4B-it-Q4_K_M none 1 32768 off 177.3/57.0 630.6/55.3 454.5/54.9 14.1 OK
gemma-4-E4B-it-Q4_K_M none 1 32768 on(n=1) 151.4/84.0 517.6/82.6 437.5/79.5 12.4 OK
gemma-4-E4B-it-Q4_K_M none 1 32768 on(n=2) 162.7/66.7 562.5/86.3 432.2/75.0 12.5 OK
gemma-4-E4B-it-Q4_K_M none 1 32768 on(n=3) 153.3/66.4 533.8/92.9 442.7/77.1 12.1 OK
gemma-4-E4B-it-Q4_K_M none 1 32768 on(n=4) 158.0/58.9 512.7/93.8 437.6/68.2 12.8 OK
gemma-4-E4B-it-Q4_K_M none 1 16384 off 169.7/55.6 316.9/54.5 443.6/54.5 13.4 OK
gemma-4-E4B-it-Q4_K_M none 1 16384 on(n=1) 160.8/77.1 323.0/83.8 457.0/81.1 11.7 OK
gemma-4-E4B-it-Q4_K_M none 1 16384 on(n=2) 159.0/75.8 554.9/87.0 438.5/80.5 12.3 OK
gemma-4-E4B-it-Q4_K_M none 1 16384 on(n=3) 121.6/87.3 224.7/88.1 475.7/76.6 14.8 OK
gemma-4-E4B-it-Q4_K_M none 1 16384 on(n=4) 159.6/88.0 225.4/91.6 476.4/76.9 14.6 OK
gemma-4-E4B-it-Q4_K_M layer both 131072 off 171.6/57.4 559.6/57.7 494.4/57.6 13.3 OK
gemma-4-E4B-it-Q4_K_M layer both 131072 on(n=1) 150.0/76.6 455.2/80.8 478.5/76.7 12.9 OK
gemma-4-E4B-it-Q4_K_M layer both 131072 on(n=2) 156.5/64.9 533.9/82.7 472.6/81.3 12.9 OK
gemma-4-E4B-it-Q4_K_M layer both 131072 on(n=3) 157.8/78.4 559.7/92.1 473.4/77.8 12.6 OK
gemma-4-E4B-it-Q4_K_M layer both 131072 on(n=4) 152.5/62.7 628.6/87.3 463.4/61.2 14.0 OK
gemma-4-E4B-it-Q4_K_M layer both 65536 off 174.8/57.5 548.8/57.6 492.1/57.5 13.3 OK
gemma-4-E4B-it-Q4_K_M layer both 65536 on(n=1) 154.3/76.4 227.1/83.8 487.5/79.0 15.3 OK
gemma-4-E4B-it-Q4_K_M layer both 65536 on(n=2) 158.9/65.6 572.5/85.0 465.8/80.8 12.6 OK
gemma-4-E4B-it-Q4_K_M layer both 65536 on(n=3) 152.1/61.2 540.7/86.7 474.2/78.9 12.5 OK
gemma-4-E4B-it-Q4_K_M layer both 65536 on(n=4) 162.1/66.4 470.2/75.9 478.2/68.5 13.0 OK
gemma-4-E4B-it-Q4_K_M layer both 32768 off 160.8/57.3 439.7/57.5 494.9/57.4 12.8 OK
gemma-4-E4B-it-Q4_K_M layer both 32768 on(n=1) 154.6/77.8 627.5/80.9 464.0/80.2 12.5 OK
gemma-4-E4B-it-Q4_K_M layer both 32768 on(n=2) 159.5/74.9 567.8/84.8 468.9/77.7 12.6 OK
gemma-4-E4B-it-Q4_K_M layer both 32768 on(n=3) 153.4/84.3 432.2/79.2 477.6/73.6 12.4 OK
gemma-4-E4B-it-Q4_K_M layer both 32768 on(n=4) 159.2/54.2 489.7/70.3 474.7/69.8 13.4 OK
gemma-4-E4B-it-Q4_K_M layer both 16384 off 172.1/57.1 476.7/57.5 495.0/57.4 12.9 OK
gemma-4-E4B-it-Q4_K_M layer both 16384 on(n=1) 161.2/82.0 226.4/85.5 489.0/79.1 14.5 OK
gemma-4-E4B-it-Q4_K_M layer both 16384 on(n=2) 159.3/82.0 226.6/89.0 486.9/80.5 14.3 OK
gemma-4-E4B-it-Q4_K_M layer both 16384 on(n=3) 128.7/73.9 531.0/83.8 474.7/70.0 12.5 OK
gemma-4-E4B-it-Q4_K_M layer both 16384 on(n=4) 127.7/69.2 597.7/89.2 464.9/67.4 12.7 OK

gemma4-e4b-qat

quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status

gemma-4-E4B-it-qat-UD-Q2_K_XL none 0 131072 off 225.8/43.5 338.4/43.3 547.1/43.0 26.3 OK
gemma-4-E4B-it-qat-UD-Q2_K_XL none 0 65536 off 254.4/42.6 205.3/43.4 536.7/42.7 20.6 OK
gemma-4-E4B-it-qat-UD-Q2_K_XL none 0 32768 off 234.9/41.0 290.8/43.4 532.0/43.2 14.8 OK
gemma-4-E4B-it-qat-UD-Q2_K_XL none 0 16384 off 226.2/43.0 361.1/42.8 537.0/42.9 18.1 OK
gemma-4-E4B-it-qat-UD-Q2_K_XL none 1 131072 off 243.3/42.9 244.4/41.9 931.4/41.9 16.0 OK
gemma-4-E4B-it-qat-UD-Q2_K_XL none 1 65536 off 217.4/41.7 211.8/42.1 520.5/42.1 21.6 OK
gemma-4-E4B-it-qat-UD-Q2_K_XL none 1 32768 off 222.5/42.1 339.9/41.7 512.0/41.2 16.1 OK
gemma-4-E4B-it-qat-UD-Q2_K_XL none 1 16384 off 219.6/42.5 339.9/42.1 499.4/42.0 16.0 OK
gemma-4-E4B-it-qat-UD-Q2_K_XL layer both 131072 off 204.2/39.2 319.1/39.3 480.3/39.5 22.9 OK
gemma-4-E4B-it-qat-UD-Q2_K_XL layer both 65536 off 218.9/42.2 286.2/40.6 488.3/38.5 16.4 OK
gemma-4-E4B-it-qat-UD-Q2_K_XL layer both 32768 off 176.8/34.6 284.8/42.1 528.9/42.0 15.6 OK
gemma-4-E4B-it-qat-UD-Q2_K_XL layer both 16384 off 212.6/41.5 389.0/42.0 522.5/41.6 16.0 OK
gemma-4-E4B-it-qat-UD-Q4_K_XL none 0 131072 off 263.9/69.5 420.4/70.1 641.8/70.0 17.8 OK
gemma-4-E4B-it-qat-UD-Q4_K_XL none 0 65536 off 265.9/69.4 686.1/69.3 632.7/69.2 10.9 OK
gemma-4-E4B-it-qat-UD-Q4_K_XL none 0 32768 off 305.0/69.5 510.7/69.7 636.5/69.3 10.4 OK
gemma-4-E4B-it-qat-UD-Q4_K_XL none 0 16384 off 302.9/69.5 314.8/69.2 632.7/68.1 13.8 OK
gemma-4-E4B-it-qat-UD-Q4_K_XL none 1 131072 off 280.7/65.8 298.1/65.7 602.2/65.4 15.2 OK
gemma-4-E4B-it-qat-UD-Q4_K_XL none 1 65536 off 284.6/65.5 642.7/65.5 576.5/65.4 12.0 OK
gemma-4-E4B-it-qat-UD-Q4_K_XL none 1 32768 off 280.9/65.3 652.1/64.6 569.5/63.6 12.2 OK
gemma-4-E4B-it-qat-UD-Q4_K_XL none 1 16384 off 288.0/65.3 690.6/63.7 557.8/63.3 12.4 OK
gemma-4-E4B-it-qat-UD-Q4_K_XL layer both 131072 off 262.2/63.4 658.0/65.6 608.9/65.5 11.9 OK
gemma-4-E4B-it-qat-UD-Q4_K_XL layer both 65536 off 289.4/65.6 305.1/65.6 870.7/65.4 15.1 OK
gemma-4-E4B-it-qat-UD-Q4_K_XL layer both 32768 off 289.7/65.2 619.3/65.2 606.4/65.2 11.5 OK
gemma-4-E4B-it-qat-UD-Q4_K_XL layer both 16384 off 289.4/64.9 430.1/65.4 612.2/65.3 11.1 OK

qwen3.5

quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status

Qwen3.5-0.8B-UD-IQ2_M none 0 131072 off 140.5/175.7 346.4/175.9 690.4/174.6 7.1 OK
Qwen3.5-0.8B-UD-IQ2_M none 0 65536 off 349.3/175.1 1051.4/175.9 1812.4/175.8 3.8 OK
Qwen3.5-0.8B-UD-IQ2_M none 0 32768 off 353.0/175.5 1439.3/175.9 2042.0/175.7 4.1 OK
Qwen3.5-0.8B-UD-IQ2_M none 0 16384 off 343.2/173.7 570.3/175.7 3438.5/175.4 4.9 OK
Qwen3.5-0.8B-UD-IQ2_M none 1 131072 off 293.2/160.5 1865.5/161.2 685.3/161.3 6.3 OK
Qwen3.5-0.8B-UD-IQ2_M none 1 65536 off 296.6/160.8 2372.6/0.0 681.2/160.9 6.6 OK
Qwen3.5-0.8B-UD-IQ2_M none 1 32768 off 267.1/161.8 256.9/162.1 678.0/162.0 6.6 OK
Qwen3.5-0.8B-UD-IQ2_M none 1 16384 off 233.3/159.8 819.6/160.8 684.7/160.8 5.4 OK
Qwen3.5-0.8B-UD-IQ2_M layer both 131072 off 238.6/152.9 660.8/160.3 792.1/163.3 5.6 OK
Qwen3.5-0.8B-UD-IQ2_M layer both 65536 off 276.1/163.9 298.6/163.9 802.6/164.1 6.8 OK
Qwen3.5-0.8B-UD-IQ2_M layer both 32768 off 287.7/162.3 298.6/164.4 3081.2/164.1 6.6 OK
Qwen3.5-0.8B-UD-IQ2_M layer both 16384 off 213.0/163.5 1300.6/164.4 804.1/164.0 5.5 OK
Qwen3.5-0.8B-UD-Q4_K_XL none 0 131072 off 238.6/161.0 359.9/163.9 965.5/163.6 7.6 OK
Qwen3.5-0.8B-UD-Q4_K_XL none 0 65536 off 251.8/162.7 536.5/162.8 951.2/162.7 6.6 OK
Qwen3.5-0.8B-UD-Q4_K_XL none 0 32768 off 142.0/162.8 357.6/162.8 990.8/162.7 6.6 OK
Qwen3.5-0.8B-UD-Q4_K_XL none 0 16384 off 152.1/162.7 358.2/162.8 990.2/162.7 6.6 OK
Qwen3.5-0.8B-UD-Q4_K_XL none 1 131072 off 205.4/150.2 260.6/151.3 678.7/151.1 7.2 OK
Qwen3.5-0.8B-UD-Q4_K_XL none 1 65536 off 208.9/149.9 262.3/149.9 681.2/149.8 7.2 OK
Qwen3.5-0.8B-UD-Q4_K_XL none 1 32768 off 150.8/150.1 513.2/150.1 678.1/150.1 7.2 OK
Qwen3.5-0.8B-UD-Q4_K_XL none 1 16384 off 183.8/149.8 261.0/149.9 675.6/149.9 7.2 OK
Qwen3.5-0.8B-UD-Q4_K_XL layer both 131072 off 166.1/143.6 298.3/146.0 778.9/145.7 7.7 OK
Qwen3.5-0.8B-UD-Q4_K_XL layer both 65536 off 169.0/152.9 301.5/153.5 802.8/153.1 7.2 OK
Qwen3.5-0.8B-UD-Q4_K_XL layer both 32768 off 171.6/152.9 304.3/153.7 805.5/153.2 7.1 OK
Qwen3.5-0.8B-UD-Q4_K_XL layer both 16384 off 164.4/152.0 299.7/152.5 2021.4/152.0 7.0 OK
Qwen3.5-0.8B-UD-Q8_K_XL none 0 131072 off 266.9/134.6 336.2/135.8 2332.9/135.7 9.8 OK
Qwen3.5-0.8B-UD-Q8_K_XL none 0 65536 off 267.2/134.9 337.0/135.1 899.7/134.9 7.7 OK
Qwen3.5-0.8B-UD-Q8_K_XL none 0 32768 off 153.0/135.3 337.3/135.2 897.0/135.1 7.8 OK
Qwen3.5-0.8B-UD-Q8_K_XL none 0 16384 off 278.5/135.0 342.8/135.0 883.3/134.9 7.5 OK
Qwen3.5-0.8B-UD-Q8_K_XL none 1 131072 off 180.9/127.2 248.8/126.9 629.6/126.9 8.4 OK
Qwen3.5-0.8B-UD-Q8_K_XL none 1 65536 off 202.9/125.7 376.4/125.8 628.9/125.7 7.5 OK
Qwen3.5-0.8B-UD-Q8_K_XL none 1 32768 off 222.9/125.2 713.0/125.3 643.7/125.2 7.6 OK
Qwen3.5-0.8B-UD-Q8_K_XL none 1 16384 off 235.2/125.6 249.6/125.6 641.1/125.5 8.4 OK
Qwen3.5-0.8B-UD-Q8_K_XL layer both 131072 off 177.1/120.8 542.3/121.8 726.5/121.8 8.9 OK
Qwen3.5-0.8B-UD-Q8_K_XL layer both 65536 off 163.5/125.7 282.3/124.7 733.4/124.5 8.5 OK
Qwen3.5-0.8B-UD-Q8_K_XL layer both 32768 off 163.1/128.6 287.0/128.7 750.1/128.7 8.3 OK
Qwen3.5-0.8B-UD-Q8_K_XL layer both 16384 off 176.6/128.1 578.8/128.2 752.3/128.0 8.3 OK

qwen3.6_27B

quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status

Qwen3.6-27B-Q4_K_S none 0 131072 off - - - 31.0 OOM
Qwen3.6-27B-Q4_K_S none 0 65536 off - - - 2.7 OOM
Qwen3.6-27B-Q4_K_S none 0 32768 off - - - 2.7 OOM
Qwen3.6-27B-Q4_K_S none 0 16384 off - - - 2.7 OOM
Qwen3.6-27B-Q4_K_S none 1 131072 off - - - 2.7 OOM
Qwen3.6-27B-Q4_K_S none 1 65536 off - - - 2.7 OOM
Qwen3.6-27B-Q4_K_S none 1 32768 off - - - 2.7 OOM
Qwen3.6-27B-Q4_K_S none 1 16384 off - - - 2.7 OOM
Qwen3.6-27B-Q4_K_S layer both 131072 off - - - 6.0 OOM
Qwen3.6-27B-Q4_K_S layer both 65536 off 40.7/15.8 38.0/15.8 82.1/15.7 56.3 OK
Qwen3.6-27B-Q4_K_S layer both 32768 off 43.6/15.8 41.4/15.7 81.7/15.6 56.6 OK
Qwen3.6-27B-Q4_K_S layer both 16384 off 41.8/15.7 37.8/15.7 82.6/15.6 56.6 OK

qwen3.6_35B

quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status

Qwen3.6-35B-A3B-UD-Q2_K_XL none 0 131072 off - - - 24.6 OOM
Qwen3.6-35B-A3B-UD-Q2_K_XL none 0 65536 off - - - 2.3 OOM
Qwen3.6-35B-A3B-UD-Q2_K_XL none 0 32768 off - - - 2.3 OOM
Qwen3.6-35B-A3B-UD-Q2_K_XL none 0 16384 off - - - 2.3 OOM
Qwen3.6-35B-A3B-UD-Q2_K_XL none 1 131072 off - - - 2.3 OOM
Qwen3.6-35B-A3B-UD-Q2_K_XL none 1 65536 off - - - 2.3 OOM
Qwen3.6-35B-A3B-UD-Q2_K_XL none 1 32768 off - - - 2.3 OOM
Qwen3.6-35B-A3B-UD-Q2_K_XL none 1 16384 off - - - 2.3 OOM
Qwen3.6-35B-A3B-UD-Q2_K_XL layer both 131072 off 52.2/56.9 92.9/57.2 202.7/57.1 17.8 OK
Qwen3.6-35B-A3B-UD-Q2_K_XL layer both 65536 off 69.6/56.7 108.3/56.7 207.9/56.5 19.3 OK
Qwen3.6-35B-A3B-UD-Q2_K_XL layer both 32768 off 68.5/56.4 107.3/56.6 208.1/56.4 19.3 OK
Qwen3.6-35B-A3B-UD-Q2_K_XL layer both 16384 off 75.2/56.6 108.1/56.6 208.5/56.4 19.2 OK

qwen3.8_27B

quantsmgpuctxmtpturn1 (p/g t/s)turn2 (p/g t/s)turn3 (p/g t/s)wall(s)status

Qwen3.8-27B-UD-IQ2_XXS none 0 131072 off - - - 19.4 OOM
Qwen3.8-27B-UD-IQ2_XXS none 0 65536 off - - - 3.0 OOM
Qwen3.8-27B-UD-IQ2_XXS none 0 32768 off 98.4/14.2 37.4/14.1 82.0/13.5 42.7 OK
Qwen3.8-27B-UD-IQ2_XXS none 0 16384 off 99.5/13.7 37.4/13.1 81.8/12.9 49.0 OK
Qwen3.8-27B-UD-IQ2_XXS none 1 131072 off - - - 5.0 OOM
Qwen3.8-27B-UD-IQ2_XXS none 1 65536 off - - - 5.0 OOM
Qwen3.8-27B-UD-IQ2_XXS none 1 32768 off 90.0/14.3 31.3/14.1 70.1/13.6 49.0 OK
Qwen3.8-27B-UD-IQ2_XXS none 1 16384 off 89.1/13.3 30.8/12.5 69.6/12.3 52.5 OK
Qwen3.8-27B-UD-IQ2_XXS layer both 131072 off 86.0/15.6 34.7/15.4 76.3/15.2 45.4 OK
Qwen3.8-27B-UD-IQ2_XXS layer both 65536 off 91.9/14.7 34.3/14.7 76.3/14.7 44.3 OK
Qwen3.8-27B-UD-IQ2_XXS layer both 32768 off 86.0/14.9 34.1/14.8 75.2/14.7 45.9 OK
Qwen3.8-27B-UD-IQ2_XXS layer both 16384 off 92.3/15.5 34.2/15.4 75.5/15.3 43.3 OK

Long context (single turn, >10k tokens)

Prompt: see long_prompt.txt (needle-in-haystack recall test).

familyquantsmgpuctxmtpprompt t/sgen t/swall(s)status

gemma4-12b gemma-4-12b-it-Q4_0 none 0 131072 off 415.9 30.7 37.6 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 131072 on(n=1) 405.2 48.8 38.7 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 131072 on(n=2) 403.5 52.4 38.6 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 131072 on(n=3) 402.4 53.1 38.7 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 131072 on(n=4) 401.9 43.0 39.3 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 65536 off 401.0 28.4 38.9 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 65536 on(n=1) 398.3 46.7 39.1 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 65536 on(n=2) 398.7 51.7 38.8 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 65536 on(n=3) 400.7 53.7 38.5 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 65536 on(n=4) 399.5 50.1 38.8 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 32768 off 397.4 29.7 38.9 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 32768 on(n=1) 395.0 47.8 39.3 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 32768 on(n=2) 397.6 50.6 38.9 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 32768 on(n=3) 397.0 48.8 39.0 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 32768 on(n=4) 397.5 43.2 39.3 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 16384 off 396.9 29.4 39.0 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 16384 on(n=1) 395.7 47.6 39.2 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 16384 on(n=2) 397.1 47.7 39.1 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 16384 on(n=3) 399.0 49.2 38.8 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 0 16384 on(n=4) 400.2 48.1 38.8 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 131072 off 412.5 28.7 39.8 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 131072 on(n=1) 393.7 46.3 41.5 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 131072 on(n=2) 386.5 41.0 42.5 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 131072 on(n=3) 387.7 45.0 42.1 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 131072 on(n=4) 389.2 43.3 42.1 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 65536 off 394.8 26.9 41.2 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 65536 on(n=1) 387.5 42.8 42.2 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 65536 on(n=2) 387.1 43.6 42.2 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 65536 on(n=3) 387.8 39.8 42.4 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 65536 on(n=4) 387.5 48.4 41.9 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 32768 off 391.2 27.1 41.5 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 32768 on(n=1) 392.1 42.3 41.7 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 32768 on(n=2) 387.5 48.8 41.6 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 32768 on(n=3) 387.5 47.0 41.7 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 32768 on(n=4) 386.7 45.3 41.9 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 16384 off 394.7 26.8 41.2 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 16384 on(n=1) 389.4 42.5 42.1 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 16384 on(n=2) 388.9 45.8 41.6 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 16384 on(n=3) 386.2 43.2 42.0 OK
gemma4-12b gemma-4-12b-it-Q4_0 none 1 16384 on(n=4) 388.6 48.0 41.5 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 131072 off 726.7 29.9 25.8 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 131072 on(n=1) 605.9 35.3 31.0 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 131072 on(n=2) 603.5 29.9 31.7 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 131072 on(n=3) 599.3 31.4 31.6 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 131072 on(n=4) 602.7 21.1 33.5 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 65536 off 713.9 29.7 26.1 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 65536 on(n=1) 598.1 36.1 30.9 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 65536 on(n=2) 599.1 35.0 31.0 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 65536 on(n=3) 601.8 31.4 31.3 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 65536 on(n=4) 599.8 27.5 31.9 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 32768 off 706.8 29.6 26.0 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 32768 on(n=1) 598.9 35.8 30.6 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 32768 on(n=2) 597.8 31.0 31.2 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 32768 on(n=3) 595.9 29.8 31.5 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 32768 on(n=4) 596.5 31.0 31.3 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 16384 off 708.1 29.5 26.0 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 16384 on(n=1) 597.9 36.2 30.4 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 16384 on(n=2) 593.4 34.6 30.7 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 16384 on(n=3) 596.3 26.8 31.9 OK
gemma4-12b gemma-4-12b-it-Q4_0 layer both 16384 on(n=4) 596.9 25.6 32.1 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 131072 off 352.9 26.0 43.1 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 131072 on(n=1) 353.3 34.6 43.8 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 131072 on(n=2) 351.9 35.4 43.8 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 131072 on(n=3) 352.7 31.9 44.2 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 131072 on(n=4) 352.7 34.5 43.9 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 65536 off 354.9 26.1 42.9 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 65536 on(n=1) 351.5 37.7 43.4 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 65536 on(n=2) 351.3 34.7 43.8 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 65536 on(n=3) 351.3 35.3 43.9 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 65536 on(n=4) 353.2 26.9 44.8 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 32768 off 355.2 25.9 42.9 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 32768 on(n=1) 352.1 40.1 43.2 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 32768 on(n=2) 352.1 37.6 43.4 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 32768 on(n=3) 350.8 30.1 44.4 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 32768 on(n=4) 351.1 26.7 44.9 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 16384 off 355.5 26.5 42.7 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 16384 on(n=1) 350.9 36.3 43.6 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 16384 on(n=2) 352.8 34.4 43.6 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 16384 on(n=3) 352.0 37.2 43.5 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 0 16384 on(n=4) 351.5 27.8 44.7 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 131072 off 366.7 25.5 43.1 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 131072 on(n=1) 355.2 33.6 45.2 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 131072 on(n=2) 356.2 33.5 45.1 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 131072 on(n=3) 351.6 29.5 46.1 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 131072 on(n=4) 353.7 26.5 46.3 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 65536 off 359.3 24.6 43.9 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 65536 on(n=1) 355.1 31.8 45.1 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 65536 on(n=2) 352.1 35.9 45.0 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 65536 on(n=3) 352.6 32.0 45.3 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 65536 on(n=4) 354.5 31.7 45.2 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 32768 off 357.9 24.7 44.1 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 32768 on(n=1) 353.7 33.6 45.0 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 32768 on(n=2) 354.3 32.9 45.1 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 32768 on(n=3) 349.9 31.1 45.8 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 32768 on(n=4) 353.1 28.7 45.8 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 16384 off 359.8 24.6 43.9 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 16384 on(n=1) 354.0 36.5 44.7 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 16384 on(n=2) 351.5 31.0 45.6 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 16384 on(n=3) 351.9 30.2 45.6 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL none 1 16384 on(n=4) 352.0 34.7 45.1 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 131072 off 639.1 27.8 27.9 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 131072 on(n=1) 531.0 29.5 34.0 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 131072 on(n=2) 525.5 25.6 34.9 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 131072 on(n=3) 525.3 23.5 35.3 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 131072 on(n=4) 522.1 19.6 36.5 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 65536 off 618.7 27.6 28.5 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 65536 on(n=1) 528.3 31.8 33.5 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 65536 on(n=2) 528.9 26.3 34.3 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 65536 on(n=3) 525.8 22.5 35.3 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 65536 on(n=4) 526.0 20.8 35.7 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 32768 off 614.3 27.9 28.4 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 32768 on(n=1) 525.6 31.0 33.5 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 32768 on(n=2) 524.3 32.0 33.4 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 32768 on(n=3) 524.7 23.9 34.8 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 32768 on(n=4) 525.1 22.1 35.1 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 16384 off 612.1 27.8 28.5 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 16384 on(n=1) 524.9 31.9 33.2 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 16384 on(n=2) 525.0 25.9 34.1 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 16384 on(n=3) 528.6 22.2 34.7 OK
gemma4-12b gemma-4-12b-it-UD-Q2_K_XL layer both 16384 on(n=4) 523.7 27.2 34.1 OK
gemma4-26b gemma-4-26B-A4B-it-UD-IQ2_M none 0 131072 off - - 3.7 OOM
gemma4-26b gemma-4-26B-A4B-it-UD-IQ2_M none 0 65536 off - - 3.6 OOM
gemma4-26b gemma-4-26B-A4B-it-UD-IQ2_M none 0 32768 off 727.3 57.4 23.1 OK
gemma4-26b gemma-4-26B-A4B-it-UD-IQ2_M none 0 16384 off 709.7 51.5 23.8 OK
gemma4-26b gemma-4-26B-A4B-it-UD-IQ2_M none 1 131072 off - - 6.1 OOM
gemma4-26b gemma-4-26B-A4B-it-UD-IQ2_M none 1 65536 off - - 6.1 OOM
gemma4-26b gemma-4-26B-A4B-it-UD-IQ2_M none 1 32768 off 724.9 52.5 26.0 OK
gemma4-26b gemma-4-26B-A4B-it-UD-IQ2_M none 1 16384 off 708.0 48.5 26.4 OK
gemma4-26b gemma-4-26B-A4B-it-UD-IQ2_M layer both 131072 off 1293.5 54.7 17.2 OK
gemma4-26b gemma-4-26B-A4B-it-UD-IQ2_M layer both 65536 off 1293.4 54.6 17.1 OK
gemma4-26b gemma-4-26B-A4B-it-UD-IQ2_M layer both 32768 off 1282.3 54.8 17.0 OK
gemma4-26b gemma-4-26B-A4B-it-UD-IQ2_M layer both 16384 off 1282.9 54.8 16.9 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 131072 off - - 3.4 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 131072 on(n=1) - - 4.1 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 131072 on(n=2) - - 4.1 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 131072 on(n=3) - - 4.1 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 131072 on(n=4) - - 4.1 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 65536 off - - 3.4 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 65536 on(n=1) - - 4.1 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 65536 on(n=2) - - 4.1 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 65536 on(n=3) - - 4.1 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 65536 on(n=4) - - 4.1 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 32768 off - - 3.4 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 32768 on(n=1) - - 4.2 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 32768 on(n=2) - - 4.2 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 32768 on(n=3) - - 4.2 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 32768 on(n=4) - - 4.2 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 16384 off - - 4.6 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 16384 on(n=1) - - 5.0 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 16384 on(n=2) - - 5.1 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 16384 on(n=3) - - 5.1 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 0 16384 on(n=4) - - 5.0 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 131072 off - - 5.5 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 131072 on(n=1) - - 6.2 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 131072 on(n=2) - - 6.2 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 131072 on(n=3) - - 6.3 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 131072 on(n=4) - - 6.3 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 65536 off - - 5.5 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 65536 on(n=1) - - 6.4 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 65536 on(n=2) - - 6.3 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 65536 on(n=3) - - 6.3 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 65536 on(n=4) - - 6.2 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 32768 off - - 5.5 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 32768 on(n=1) - - 6.3 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 32768 on(n=2) - - 6.3 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 32768 on(n=3) - - 6.3 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 32768 on(n=4) - - 6.3 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 16384 off 149.8 12.5 99.1 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 16384 on(n=1) - - 7.2 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 16384 on(n=2) - - 7.2 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 16384 on(n=3) - - 7.1 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS none 1 16384 on(n=4) - - 7.2 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 131072 off 262.7 13.3 62.0 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 131072 on(n=1) - - 6.6 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 131072 on(n=2) - - 6.6 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 131072 on(n=3) - - 6.6 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 131072 on(n=4) - - 6.5 OOM
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 65536 off 258.9 13.2 62.8 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 65536 on(n=1) 220.2 16.0 71.5 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 65536 on(n=2) 220.0 15.3 71.9 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 65536 on(n=3) 220.0 13.2 73.2 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 65536 on(n=4) 220.7 13.7 72.7 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 32768 off 256.0 13.3 63.0 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 32768 on(n=1) 219.9 16.0 71.3 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 32768 on(n=2) 219.9 13.8 72.6 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 32768 on(n=3) 219.5 14.4 72.3 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 32768 on(n=4) 220.0 13.2 73.0 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 16384 off 254.6 13.4 63.2 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 16384 on(n=1) 218.9 16.2 71.5 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 16384 on(n=2) 219.6 15.9 71.4 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 16384 on(n=3) 220.2 14.1 72.3 OK
gemma4-31b gemma-4-31B-it-UD-IQ2_XXS layer both 16384 on(n=4) 219.7 10.7 75.3 OK
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 131072 off - - 2.9 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 131072 on(n=1) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 131072 on(n=2) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 131072 on(n=3) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 131072 on(n=4) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 65536 off - - 2.9 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 65536 on(n=1) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 65536 on(n=2) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 65536 on(n=3) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 65536 on(n=4) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 32768 off - - 2.9 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 32768 on(n=1) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 32768 on(n=2) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 32768 on(n=3) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 32768 on(n=4) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 16384 off - - 2.9 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 16384 on(n=1) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 16384 on(n=2) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 16384 on(n=3) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 0 16384 on(n=4) - - 3.7 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 131072 off - - 2.9 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 131072 on(n=1) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 131072 on(n=2) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 131072 on(n=3) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 131072 on(n=4) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 65536 off - - 2.9 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 65536 on(n=1) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 65536 on(n=2) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 65536 on(n=3) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 65536 on(n=4) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 32768 off - - 2.9 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 32768 on(n=1) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 32768 on(n=2) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 32768 on(n=3) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 32768 on(n=4) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 16384 off - - 2.9 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 16384 on(n=1) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 16384 on(n=2) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 16384 on(n=3) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL none 1 16384 on(n=4) - - 3.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 131072 off - - 5.5 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 131072 on(n=1) - - 6.3 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 131072 on(n=2) - - 6.3 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 131072 on(n=3) - - 6.3 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 131072 on(n=4) - - 6.3 OOM
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 65536 off 245.3 12.3 67.1 OK
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 65536 on(n=1) 207.3 15.1 76.6 OK
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 65536 on(n=2) 207.2 13.8 77.3 OK
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 65536 on(n=3) 207.3 14.5 76.8 OK
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 65536 on(n=4) 207.0 14.0 77.2 OK
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 32768 off 241.0 12.2 67.9 OK
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 32768 on(n=1) 206.6 14.9 76.5 OK
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 32768 on(n=2) 206.7 14.9 76.6 OK
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 32768 on(n=3) 207.0 14.3 76.8 OK
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 32768 on(n=4) 207.1 11.2 79.3 OK
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 16384 off 241.0 12.3 67.8 OK
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 16384 on(n=1) 207.0 15.6 76.1 OK
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 16384 on(n=2) 207.1 14.3 76.8 OK
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 16384 on(n=3) 207.7 14.4 76.5 OK
gemma4-31b gemma-4-31B-it-UD-Q2_K_XL layer both 16384 on(n=4) 207.5 10.4 80.0 OK
gemma4-e2b gemma-4-E2B-it-Q4_K_M none 0 131072 off 1424.6 85.0 12.8 OK
gemma4-e2b gemma-4-E2B-it-Q4_K_M none 0 65536 off 1419.1 85.0 12.8 OK
gemma4-e2b gemma-4-E2B-it-Q4_K_M none 0 32768 off 1415.2 82.6 12.9 OK
gemma4-e2b gemma-4-E2B-it-Q4_K_M none 0 16384 off 1409.1 84.8 12.8 OK
gemma4-e2b gemma-4-E2B-it-Q4_K_M none 1 131072 off 1420.8 79.4 13.3 OK
gemma4-e2b gemma-4-E2B-it-Q4_K_M none 1 65536 off 1395.5 78.1 13.5 OK
gemma4-e2b gemma-4-E2B-it-Q4_K_M none 1 32768 off 1393.3 74.5 13.5 OK
gemma4-e2b gemma-4-E2B-it-Q4_K_M none 1 16384 off 1334.5 76.5 13.9 OK
gemma4-e2b gemma-4-E2B-it-Q4_K_M layer both 131072 off 2339.7 18.3 15.3 OK
gemma4-e2b gemma-4-E2B-it-Q4_K_M layer both 65536 off 2333.8 18.5 15.0 OK
gemma4-e2b gemma-4-E2B-it-Q4_K_M layer both 32768 off 2351.6 18.3 15.0 OK
gemma4-e2b gemma-4-E2B-it-Q4_K_M layer both 16384 off 2348.7 18.5 14.9 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 131072 off 958.7 56.4 18.2 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 131072 on(n=1) 948.6 77.3 17.9 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 131072 on(n=2) 948.3 75.5 17.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 131072 on(n=3) 948.5 86.9 19.1 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 131072 on(n=4) 947.0 82.5 19.2 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 65536 off 942.2 55.0 16.1 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 65536 on(n=1) 942.1 72.0 19.4 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 65536 on(n=2) 945.1 76.4 17.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 65536 on(n=3) 946.9 80.1 19.2 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 65536 on(n=4) 945.3 80.6 17.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 32768 off 937.2 54.0 18.4 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 32768 on(n=1) 943.9 76.4 17.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 32768 on(n=2) 946.2 76.4 19.3 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 32768 on(n=3) 946.8 97.9 17.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 32768 on(n=4) 945.9 80.9 17.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 16384 off 931.8 55.0 18.4 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 16384 on(n=1) 943.3 75.0 17.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 16384 on(n=2) 946.7 75.9 17.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 16384 on(n=3) 944.4 95.5 17.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 0 16384 on(n=4) 947.5 85.0 17.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 131072 off 964.7 51.7 16.6 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 131072 on(n=1) 945.2 72.0 18.6 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 131072 on(n=2) 930.7 68.0 18.8 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 131072 on(n=3) 924.8 87.6 18.8 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 131072 on(n=4) 916.5 77.7 19.0 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 65536 off 921.0 47.8 19.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 65536 on(n=1) 923.9 67.8 20.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 65536 on(n=2) 937.7 84.9 20.0 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 65536 on(n=3) 918.3 67.3 20.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 65536 on(n=4) 911.5 65.6 20.9 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 32768 off 925.4 46.7 17.2 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 32768 on(n=1) 929.2 67.6 18.8 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 32768 on(n=2) 914.2 71.3 19.0 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 32768 on(n=3) 931.0 92.9 18.5 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 32768 on(n=4) 921.1 76.4 20.4 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 16384 off 922.1 47.6 19.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 16384 on(n=1) 918.5 68.7 20.4 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 16384 on(n=2) 921.6 66.1 20.5 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 16384 on(n=3) 924.5 85.4 18.6 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M none 1 16384 on(n=4) 901.6 64.5 21.0 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 131072 off 1697.8 53.3 13.2 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 131072 on(n=1) 1432.4 66.3 14.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 131072 on(n=2) 1429.3 65.2 16.3 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 131072 on(n=3) 1430.5 54.5 16.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 131072 on(n=4) 1430.8 64.6 16.3 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 65536 off 1695.8 52.5 10.9 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 65536 on(n=1) 1428.5 65.9 16.0 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 65536 on(n=2) 1426.1 65.4 16.0 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 65536 on(n=3) 1426.6 78.4 14.0 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 65536 on(n=4) 1428.6 65.6 14.2 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 32768 off 1692.2 53.2 12.9 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 32768 on(n=1) 1424.0 67.9 14.0 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 32768 on(n=2) 1423.1 64.9 15.8 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 32768 on(n=3) 1425.7 60.2 15.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 32768 on(n=4) 1427.5 65.6 13.9 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 16384 off 1650.6 51.8 10.9 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 16384 on(n=1) 1423.1 68.0 13.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 16384 on(n=2) 1423.0 62.9 13.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 16384 on(n=3) 1423.9 78.7 13.7 OK
gemma4-e4b gemma-4-E4B-it-Q4_K_M layer both 16384 on(n=4) 1420.6 62.1 15.7 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q2_K_XL none 0 131072 off 1024.0 40.8 17.8 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q2_K_XL none 0 65536 off 1022.1 40.7 17.8 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q2_K_XL none 0 32768 off 1022.5 40.7 17.8 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q2_K_XL none 0 16384 off 1019.5 40.8 17.8 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q2_K_XL none 1 131072 off 1024.6 39.9 18.5 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q2_K_XL none 1 65536 off 1018.0 38.6 18.7 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q2_K_XL none 1 32768 off 1014.1 37.7 18.8 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q2_K_XL none 1 16384 off 1003.5 37.7 18.9 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q2_K_XL layer both 131072 off 1037.1 39.9 18.2 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q2_K_XL layer both 65536 off 1033.6 39.6 18.2 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q2_K_XL layer both 32768 off 1032.2 38.6 18.3 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q2_K_XL layer both 16384 off 1031.6 39.5 18.0 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q4_K_XL none 0 131072 off 1051.2 63.5 16.7 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q4_K_XL none 0 65536 off 1041.5 60.0 14.6 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q4_K_XL none 0 32768 off 1025.4 58.2 16.8 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q4_K_XL none 0 16384 off 1034.2 58.7 16.7 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q4_K_XL none 1 131072 off 1046.0 59.7 17.4 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q4_K_XL none 1 65536 off 1026.5 57.1 17.7 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q4_K_XL none 1 32768 off 1022.3 55.6 17.8 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q4_K_XL none 1 16384 off 1017.2 58.1 15.7 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q4_K_XL layer both 131072 off 1850.1 59.4 10.1 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q4_K_XL layer both 65536 off 1855.4 59.9 12.0 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q4_K_XL layer both 32768 off 1848.5 59.3 9.8 OK
gemma4-e4b-qat gemma-4-E4B-it-qat-UD-Q4_K_XL layer both 16384 off 1849.1 59.9 11.8 OK
qwen3.5 Qwen3.5-0.8B-UD-IQ2_M none 0 131072 off 4451.0 161.5 5.0 OK
qwen3.5 Qwen3.5-0.8B-UD-IQ2_M none 0 65536 off 4453.3 155.3 4.3 OK
qwen3.5 Qwen3.5-0.8B-UD-IQ2_M none 0 32768 off 4448.8 161.6 5.0 OK
qwen3.5 Qwen3.5-0.8B-UD-IQ2_M none 0 16384 off 4449.4 161.7 5.0 OK
qwen3.5 Qwen3.5-0.8B-UD-IQ2_M none 1 131072 off 4420.6 148.2 5.3 OK
qwen3.5 Qwen3.5-0.8B-UD-IQ2_M none 1 65536 off 4407.3 149.0 5.1 OK
qwen3.5 Qwen3.5-0.8B-UD-IQ2_M none 1 32768 off 4393.5 149.1 5.1 OK
qwen3.5 Qwen3.5-0.8B-UD-IQ2_M none 1 16384 off 4388.0 147.9 5.1 OK
qwen3.5 Qwen3.5-0.8B-UD-IQ2_M layer both 131072 off 7569.6 151.9 4.3 OK
qwen3.5 Qwen3.5-0.8B-UD-IQ2_M layer both 65536 off 7531.3 150.5 4.3 OK
qwen3.5 Qwen3.5-0.8B-UD-IQ2_M layer both 32768 off 7533.3 151.2 4.0 OK
qwen3.5 Qwen3.5-0.8B-UD-IQ2_M layer both 16384 off 7511.7 150.3 4.0 OK
qwen3.5 Qwen3.5-0.8B-UD-Q4_K_XL none 0 131072 off 4460.4 151.0 5.1 OK
qwen3.5 Qwen3.5-0.8B-UD-Q4_K_XL none 0 65536 off 4436.3 150.7 5.1 OK
qwen3.5 Qwen3.5-0.8B-UD-Q4_K_XL none 0 32768 off 4422.0 150.1 5.1 OK
qwen3.5 Qwen3.5-0.8B-UD-Q4_K_XL none 0 16384 off 4415.1 150.3 5.1 OK
qwen3.5 Qwen3.5-0.8B-UD-Q4_K_XL none 1 131072 off 4387.4 139.5 5.4 OK
qwen3.5 Qwen3.5-0.8B-UD-Q4_K_XL none 1 65536 off 4375.3 139.4 5.4 OK
qwen3.5 Qwen3.5-0.8B-UD-Q4_K_XL none 1 32768 off 4353.9 139.5 5.4 OK
qwen3.5 Qwen3.5-0.8B-UD-Q4_K_XL none 1 16384 off 4349.8 139.3 5.4 OK
qwen3.5 Qwen3.5-0.8B-UD-Q4_K_XL layer both 131072 off 7328.9 142.1 4.4 OK
qwen3.5 Qwen3.5-0.8B-UD-Q4_K_XL layer both 65536 off 7368.6 142.4 4.4 OK
qwen3.5 Qwen3.5-0.8B-UD-Q4_K_XL layer both 32768 off 7343.5 141.4 4.3 OK
qwen3.5 Qwen3.5-0.8B-UD-Q4_K_XL layer both 16384 off 7363.9 140.2 4.3 OK
qwen3.5 Qwen3.5-0.8B-UD-Q8_K_XL none 0 131072 off 4128.0 126.5 5.7 OK
qwen3.5 Qwen3.5-0.8B-UD-Q8_K_XL none 0 65536 off 4109.7 126.4 5.7 OK
qwen3.5 Qwen3.5-0.8B-UD-Q8_K_XL none 0 32768 off 4084.1 126.3 5.5 OK
qwen3.5 Qwen3.5-0.8B-UD-Q8_K_XL none 0 16384 off 4090.5 125.8 5.7 OK
qwen3.5 Qwen3.5-0.8B-UD-Q8_K_XL none 1 131072 off 4092.0 118.4 5.9 OK
qwen3.5 Qwen3.5-0.8B-UD-Q8_K_XL none 1 65536 off 4062.2 118.1 6.0 OK
qwen3.5 Qwen3.5-0.8B-UD-Q8_K_XL none 1 32768 off 4054.0 117.9 6.0 OK
qwen3.5 Qwen3.5-0.8B-UD-Q8_K_XL none 1 16384 off 4026.7 113.0 6.0 OK
qwen3.5 Qwen3.5-0.8B-UD-Q8_K_XL layer both 131072 off 6803.8 120.2 5.1 OK
qwen3.5 Qwen3.5-0.8B-UD-Q8_K_XL layer both 65536 off 6906.1 120.4 4.8 OK
qwen3.5 Qwen3.5-0.8B-UD-Q8_K_XL layer both 32768 off 6870.5 120.5 4.8 OK
qwen3.5 Qwen3.5-0.8B-UD-Q8_K_XL layer both 16384 off 6862.7 119.3 4.8 OK
qwen3.6_27B Qwen3.6-27B-Q4_K_S none 0 131072 off - - 2.7 OOM
qwen3.6_27B Qwen3.6-27B-Q4_K_S none 0 65536 off - - 2.7 OOM
qwen3.6_27B Qwen3.6-27B-Q4_K_S none 0 32768 off - - 2.7 OOM
qwen3.6_27B Qwen3.6-27B-Q4_K_S none 0 16384 off - - 2.5 OOM
qwen3.6_27B Qwen3.6-27B-Q4_K_S none 1 131072 off - - 2.5 OOM
qwen3.6_27B Qwen3.6-27B-Q4_K_S none 1 65536 off - - 2.7 OOM
qwen3.6_27B Qwen3.6-27B-Q4_K_S none 1 32768 off - - 2.5 OOM
qwen3.6_27B Qwen3.6-27B-Q4_K_S none 1 16384 off - - 2.7 OOM
qwen3.6_27B Qwen3.6-27B-Q4_K_S layer both 131072 off - - 5.9 OOM
qwen3.6_27B Qwen3.6-27B-Q4_K_S layer both 65536 off 397.0 14.7 44.1 OK
qwen3.6_27B Qwen3.6-27B-Q4_K_S layer both 32768 off 386.4 14.6 44.9 OK
qwen3.6_27B Qwen3.6-27B-Q4_K_S layer both 16384 off 386.7 14.7 44.8 OK
qwen3.6_35B Qwen3.6-35B-A3B-UD-Q2_K_XL none 0 131072 off - - 2.4 OOM
qwen3.6_35B Qwen3.6-35B-A3B-UD-Q2_K_XL none 0 65536 off - - 2.3 OOM
qwen3.6_35B Qwen3.6-35B-A3B-UD-Q2_K_XL none 0 32768 off - - 2.3 OOM
qwen3.6_35B Qwen3.6-35B-A3B-UD-Q2_K_XL none 0 16384 off - - 2.3 OOM
qwen3.6_35B Qwen3.6-35B-A3B-UD-Q2_K_XL none 1 131072 off - - 2.3 OOM
qwen3.6_35B Qwen3.6-35B-A3B-UD-Q2_K_XL none 1 65536 off - - 2.3 OOM
qwen3.6_35B Qwen3.6-35B-A3B-UD-Q2_K_XL none 1 32768 off - - 2.3 OOM
qwen3.6_35B Qwen3.6-35B-A3B-UD-Q2_K_XL none 1 16384 off - - 2.3 OOM
qwen3.6_35B Qwen3.6-35B-A3B-UD-Q2_K_XL layer both 131072 off 1415.3 53.2 15.9 OK
qwen3.6_35B Qwen3.6-35B-A3B-UD-Q2_K_XL layer both 65536 off 1406.3 53.1 15.7 OK
qwen3.6_35B Qwen3.6-35B-A3B-UD-Q2_K_XL layer both 32768 off 1388.6 51.4 13.8 OK
qwen3.6_35B Qwen3.6-35B-A3B-UD-Q2_K_XL layer both 16384 off 1401.5 53.1 15.7 OK
qwen3.8_27B Qwen3.8-27B-UD-IQ2_XXS none 0 131072 off - - 3.0 OOM
qwen3.8_27B Qwen3.8-27B-UD-IQ2_XXS none 0 65536 off - - 2.9 OOM
qwen3.8_27B Qwen3.8-27B-UD-IQ2_XXS none 0 32768 off 207.2 11.9 64.0 OK
qwen3.8_27B Qwen3.8-27B-UD-IQ2_XXS none 0 16384 off 207.1 12.3 69.3 OK
qwen3.8_27B Qwen3.8-27B-UD-IQ2_XXS none 1 131072 off - - 5.0 OOM
qwen3.8_27B Qwen3.8-27B-UD-IQ2_XXS none 1 65536 off - - 5.0 OOM
qwen3.8_27B Qwen3.8-27B-UD-IQ2_XXS none 1 32768 off 206.2 11.6 68.0 OK
qwen3.8_27B Qwen3.8-27B-UD-IQ2_XXS none 1 16384 off 204.5 11.4 73.0 OK
qwen3.8_27B Qwen3.8-27B-UD-IQ2_XXS layer both 131072 off 379.1 14.1 44.1 OK
qwen3.8_27B Qwen3.8-27B-UD-IQ2_XXS layer both 65536 off 371.6 13.3 40.7 OK
qwen3.8_27B Qwen3.8-27B-UD-IQ2_XXS layer both 32768 off 367.4 13.7 45.0 OK
qwen3.8_27B Qwen3.8-27B-UD-IQ2_XXS layer both 16384 off 366.7 13.9 41.2 OK
저작자표시 (새창열림)

'모종의 음모 > ai 프로그램' 카테고리의 다른 글

v100 흐음...  (0) 2026.09.16
llama.cpp pascal 지원 빌드 / 성능 테스트  (0) 2026.09.16
1080 ti / pcie3.0 설정후 재 테스트  (0) 2026.09.15
auroc - area under the ROC curve  (0) 2026.09.01
llama.cpp vulkan / intel uhd 630  (0) 2026.09.01
Posted by 구차니
모종의 음모/CAN2026. 9. 16. 17:28

cansniffer

딱 바뀐 값만 보여준다고. 리버싱 용

 

[링크 : https://manpages.debian.org/testing/can-utils/cansniffer.1.en.html]

[링크 : https://makersweb.net/linux/23696]  socketCAN / linux

[링크 : https://sourceforge.net/projects/cansniffer/]

[링크 : https://github.com/Balth-D/CANSniffer]

저작자표시 (새창열림)

'모종의 음모 > CAN' 카테고리의 다른 글

can / 상위 프로토콜(?)  (0) 2026.09.14
MCP2515와 CAN2.0B  (0) 2026.09.14
can.. 개조?  (0) 2026.08.31
can 트랜시버 - SN65HVD230  (0) 2026.08.20
MCP2515 on rpi .. 2번째  (0) 2026.07.24
Posted by 구차니
모종의 음모/ai 프로그램2026. 9. 16. 17:08

v100 흐음...

v100 16GB * 4 = 64GB

돈도 돈이지만 전기도 전기라

이 걸 업어오냐 마냐가 고민

 

112token/s 나오는거 보면 끌리긴 한단 말이지.. (1080 ti의 2~3배 사이)

[링크 : https://willitrunai.com/zh/can-run/gemma-4-e4b-on-v100-32gb]

저작자표시 (새창열림)

'모종의 음모 > ai 프로그램' 카테고리의 다른 글

LLM / 1080 ti 모드 별 벤치마크  (0) 2026.09.17
llama.cpp pascal 지원 빌드 / 성능 테스트  (0) 2026.09.16
1080 ti / pcie3.0 설정후 재 테스트  (0) 2026.09.15
auroc - area under the ROC curve  (0) 2026.09.01
llama.cpp vulkan / intel uhd 630  (0) 2026.09.01
Posted by 구차니
이론 관련/전기 전자2026. 9. 16. 14:20

open drain

솔찍히 봐도 먼소리인지 모르겠다 -ㅁ-

 

요약만 하자면 i2c 같은 경우 버스를 공유 하기 때문에 다른 애들이 버스의 전압레벨을 흔들수 있어야 하는데 그렇기에

모든 장치가 open drain 으로 하고 버스 라인을 pull up 해두면 장치들이 알아서 내릴수 있으니 해결~ 이라는 의미

[링크 : https://gdnn.tistory.com/281]

 

결국에 open drain 은 끌어 내리는것만 담당하고, pull up을 외부에 맡기는 구조라고 한다.

push - pull open drain

[링크 : https://chatgpt.com/share/6aaa2675-ad58-83e8-ae1f-70acf9d324ee]

저작자표시 (새창열림)

'이론 관련 > 전기 전자' 카테고리의 다른 글

analog discovery - scope 와 logic / protocol  (0) 2026.08.19
ocs (Optical Circuit Switching)  (0) 2026.07.21
최적 PID 튜닝 Ziegler–Nichols method  (0) 2026.07.02
xgboost, CALCE  (0) 2026.04.27
CPO - Co Packaged Optics  (0) 2026.04.22
Posted by 구차니
모종의 음모/ai 프로그램2026. 9. 16. 11:55

llama.cpp pascal 지원 빌드 / 성능 테스트

그냥 귀찮아서 가장 최신 받아서

claude 에서 빌드해달라고 하고 해버림

여전히 -sm row, -sm tensor 는 안되는데 머가 문제이려나?

 

cuda로 해서 그런가.. 아니면 가장 최신버전 이라서 성능향상이 생긴건가?

commit 930e2fa5995789efbf249a8bf61325bb626e417b (HEAD -> master, tag: b10991, origin/master, origin/HEAD)
Author: Jhen-Jie Hong <iainst0409@gmail.com>
Date:   Wed Sep 16 07:02:06 2026 +0800

    hexagon: add back missing contiguous fast-path and hvx_copy_uu for each run (#28886)

 

11 정도였던것 같은데 약간 상승한듯?

/mnt/Downloads/llama.cpp/build/bin$ ./llama-cli -m /mnt/Downloads/model/gemma4-31b/gemma-4-31B-it-UD-Q2_K_XL.gguf -sm layer


> 안녕?
[ Prompt: 35.4 t/s | Generation: 14.3 t/s ]

> 너에 대해 설명해줘
[ Prompt: 63.5 t/s | Generation: 13.9 t/s ]

> 파이썬으로 셀레니움을 통해 웹을 서칭하고 텍스트만 추출하고 makrdown 으로 변환후  md 파일과 pdf로 저장하는 기능을 구현해줘

[ Prompt: 162.4 t/s | Generation: 13.3 t/s ]

 

pcie3.0 은 별 차이 없었으니 일단 이전 기록 끌어오면 2token/s 정도씩 상승한듯?

$ ../../llama-b9692/llama-cli -m ./gemma-4-31B-it-UD-Q2_K_XL.gguf 
안녕?
[ Prompt: 8.9 t/s | Generation: 12.2 t/s ]

너에 대해 설명해줘
[ Prompt: 33.3 t/s | Generation: 11.8 t/s ]

파이썬으로 셀레니움을 통해 웹을 서칭하고 텍스트만 추출하고 makrdown 으로 변환후  md 파일과 pdf로 저장하는 기능을 구현해줘
[ Prompt: 36.1 t/s | Generation: 11.4 t/s ]

2026.07.30 - [모종의 음모/ai 프로그램] - llama.cpp / 1080 ti 11GB * 2/ gemma4-31B

 

 

 

+

클로드가 왜 row 안되는지 보고 있는데 split buffer 모드가 멀까?

● -sm row이 실패하는 원인을 정확히 잡았습니다: GTX 1080 Ti가 CUDA의 "split buffer"를 지원하지 않는다는 에러입니다.
  소스에서 어떤 조건인지 확인하겠습니다.

 

구글 검색 ai 답변

volta 사라고 등 떠미는 느낌이지 왜? ㅠㅠ

 
  • 지원 CUDA 버전: 기술적으로는 멀티 GPU 메모리 접근 및 P2P(Peer-to-Peer) 전송이 가능한 CUDA 9.x / 10.x 이상의 모든 최신 버전에서 지원합니다. [1]
 
  • 지원 하드웨어 SM: GPU 간 데이터 통신(NVLink 또는 PCIe 클리크)이 지원되는 하드웨어라면 대부분 구동됩니다.
    • SM 7.0 (Volta), SM 7.5 (Turing)
    • SM 8.0 / 8.6 (Ampere)
    • SM 9.0 (Hopper) 등 대부분의 메인스트림 아키텍처 [1, 2, 3, 4]
저작자표시 (새창열림)

'모종의 음모 > ai 프로그램' 카테고리의 다른 글

LLM / 1080 ti 모드 별 벤치마크  (0) 2026.09.17
v100 흐음...  (0) 2026.09.16
1080 ti / pcie3.0 설정후 재 테스트  (0) 2026.09.15
auroc - area under the ROC curve  (0) 2026.09.01
llama.cpp vulkan / intel uhd 630  (0) 2026.09.01
Posted by 구차니
모종의 음모/USB 자작 키보드2026. 9. 16. 11:05

홀 타이트 , 퀵스왑 소켓

음.. 비용도 비용이고 그냥 이건 포기할까?

 

[링크 : https://quasarzone.com/bbs/qc_user/views/74460]

[링크 : https://coolenjoy.net/bbs/34/1940426]

[링크 : https://m.blog.naver.com/PostView.naver?blogId=psychiclove&logNo=222228563689&proxyReferer=]

저작자표시 (새창열림)

'모종의 음모 > USB 자작 키보드' 카테고리의 다른 글

stm32f103 / bluepill 키보드 스캔 관련 part..2?  (0) 2026.09.14
키보드 스캔 매트릭스 - 고스팅? 블로킹?  (0) 2026.09.10
kicad로 키보드 설계하기  (0) 2026.09.10
커스텀 키보드 자작 아이디어  (2) 2026.09.01
Posted by 구차니
모종의 음모/midi 피아노2026. 9. 15. 23:59

영창/커즈와일 m110 보면대

주워온거라 고장나서 고정이 안되나? 생각하면서 뒤에 먼가 두고 기대놓고 썼는데

사진을 찾아보니 어..? 먼가 걸리는게 있었어?

[링크 : https://m.blog.naver.com/khses90/221313472983]

 

 

원래는 이렇게 서있어서 아래 먼가 걸쇠가 있네

경첩이 약해져서 못서나 했는데

 

아이씨 이게 꺽이는 거였어?!?!?!? (멘붕)

 

 

(5초뒤) 아싸 돈 굳었다 ㅋㅋㅋㅋ

저작자표시 (새창열림)

'모종의 음모 > midi 피아노' 카테고리의 다른 글

신기하네  (0) 2026.09.15
ardour6 midi 연결  (0) 2026.09.13
악보 - 보표와 옥타브  (0) 2026.09.09
lmms 악기 교체(프로그램)  (0) 2026.09.09
피아노 운지법  (0) 2026.09.08
Posted by 구차니
모종의 음모/midi 피아노2026. 9. 15. 23:42

신기하네

바이엘 왼손 연습하는데

유튜브를 따라야 하나 일단 책을 따라야 하나 고민중

이전에 왼손 도는 위치가 다른 악보라고 했는데

바이엘에서 그냥 도는 도위치에 있고 5번으로 누르게 되어있다.

 

조금은 지루해서

213141514131  213141514131 2

453525152535 453525152535 4

를 눌렀는데 그럴싸한 음악이 나와서 기분이 좋아짐 ㅋㅋ

저작자표시 (새창열림)

'모종의 음모 > midi 피아노' 카테고리의 다른 글

영창/커즈와일 m110 보면대  (0) 2026.09.15
ardour6 midi 연결  (0) 2026.09.13
악보 - 보표와 옥타브  (0) 2026.09.09
lmms 악기 교체(프로그램)  (0) 2026.09.09
피아노 운지법  (0) 2026.09.08
Posted by 구차니
모종의 음모/ai 프로그램2026. 9. 15. 10:40

1080 ti / pcie3.0 설정후 재 테스트

두번째 녀석이 4,1로 떠서 pcie1.0 x4 lane으로 작동하고 있었는데

$ nvidia-smi --query-gpu=pcie.link.width.current,pcie.link.gen.current 
pcie.link.width.current, pcie.link.gen.current
16, 3
4, 1

 

bios 에서 pcie3.0으로 설정해서 3.0 으로라도 잡히게 일단 하고 시도

 

그냥은 1로 뜨고(절전모드) 부하를 걸어주면 이렇게 올라온다.

$ nvidia-smi --query-gpu=pcie.link.width.current,pcie.link.gen.current 
pcie.link.width.current, pcie.link.gen.current
16, 3
4, 3

 

llama.cpp 에서 -sm row, tensor 모두 실패.

layer만 가능한데 돌려보니 이전에 pcie1.0 일것으로 추정되는 때랑 차이가 없다.

$ ../../llama-b10488/llama-cli -m gemma-4-31B-it-UD-Q2_K_XL.gguf -sm layer

> 안녕?
[ Prompt: 1.3 t/s | Generation: 12.0 t/s ]

> 너에 대해 설명해줘
[ Prompt: 20.4 t/s | Generation: 11.8 t/s ]

> 파이썬으로 셀레니움을 통해 웹을 서칭하고 텍스트만 추출하고 makrdown 으로 변환후  md 파일과 pdf로 저장하는 기능을 구현해줘
[ Prompt: 30.6 t/s | Generation: 11.5 t/s ]



 

이건 이전 기록.. 흐음..

$ ../../llama-b9692/llama-cli -m ./gemma-4-31B-it-UD-Q2_K_XL.gguf 
안녕?
[ Prompt: 8.9 t/s | Generation: 12.2 t/s ]

너에 대해 설명해줘
[ Prompt: 33.3 t/s | Generation: 11.8 t/s ]

파이썬으로 셀레니움을 통해 웹을 서칭하고 텍스트만 추출하고 makrdown 으로 변환후  md 파일과 pdf로 저장하는 기능을 구현해줘
[ Prompt: 36.1 t/s | Generation: 11.4 t/s ]

2026.07.30 - [모종의 음모/ai 프로그램] - llama.cpp / 1080 ti 11GB * 2/ gemma4-31B

 

저작자표시 (새창열림)

'모종의 음모 > ai 프로그램' 카테고리의 다른 글

v100 흐음...  (0) 2026.09.16
llama.cpp pascal 지원 빌드 / 성능 테스트  (0) 2026.09.16
auroc - area under the ROC curve  (0) 2026.09.01
llama.cpp vulkan / intel uhd 630  (0) 2026.09.01
stable diffusion model 이란  (0) 2026.08.28
Posted by 구차니
모종의 음모/USB 자작 키보드2026. 9. 14. 18:13

stm32f103 / bluepill 키보드 스캔 관련 part..2?

좀더 hal 코드를 써서 고생을 해봐야하나..

아무튼, 간단하게  float input 으로 바꾸기 위해 input mode로 바꾸고

output mode로 바꾸고 나서는 출력을 써줘야 된다.

#define BUF_SIZE 50
uint8_t usb_buf[BUF_SIZE];
uint32_t count = 0;
#if 1
int prev_val = 0;

// STM32F1: 4 bits per pin (CNF[1:0] MODE[1:0]) in CRL (pins 0-7) / CRH (pins 8-15)
static inline void pin_set_cfg(GPIO_TypeDef *port, uint16_t pin, uint32_t cfg)
{
    uint32_t num = __builtin_ctz(pin);
    uint32_t *reg = (num < 8U) ? &port->CRL : &port->CRH;
    uint32_t pos = (num % 8U) * 4U;
    *reg = (*reg & ~(0xFU << pos)) | (cfg << pos);
}

static inline void pin_hiz(GPIO_TypeDef *port, uint16_t pin)
{
    pin_set_cfg(port, pin, 0x4U);  // CNF=01 MODE=00: floating input (Hi-Z)
}

static inline void pin_output(GPIO_TypeDef *port, uint16_t pin)
{
    pin_set_cfg(port, pin, 0x2U);  // CNF=00 MODE=10: push-pull output, 2 MHz
}

void gpio_proc()
{
  int val = 0;
  // row
  // val |= (HAL_GPIO_ReadPin(GPIOA, GPIO_PIN_4) << 4);
  // val |= (HAL_GPIO_ReadPin(GPIOA, GPIO_PIN_5) << 5);
  // val |= (HAL_GPIO_ReadPin(GPIOA, GPIO_PIN_6) << 6);
  // val |= (HAL_GPIO_ReadPin(GPIOA, GPIO_PIN_7) << 7);

  char *swname[4][4] = {
    {"S1", "S2", "S3", "S4"},
    {"S5", "S6", "S7", "S8"},
    {"S9", "S10", "S11", "S12"},
    {"S13", "S14", "S15", "S16"}
  };

  for(uint16_t row_idx = 0; row_idx < 4; row_idx ++)
  {
    uint16_t pin_arr[] = {GPIO_PIN_4, GPIO_PIN_5, GPIO_PIN_6, GPIO_PIN_7};
    pin_output(GPIOA, pin_arr[row_idx]);
    HAL_GPIO_WritePin(GPIOA, pin_arr[row_idx], GPIO_PIN_SET);
    
    HAL_Delay(1);

    // column
    val |= (HAL_GPIO_ReadPin(GPIOA, GPIO_PIN_0) << 0);
    val |= (HAL_GPIO_ReadPin(GPIOA, GPIO_PIN_1) << 1);
    val |= (HAL_GPIO_ReadPin(GPIOA, GPIO_PIN_2) << 2);
    val |= (HAL_GPIO_ReadPin(GPIOA, GPIO_PIN_3) << 3);

    // HAL_GPIO_WritePin(GPIOA, pin_arr[row_idx], GPIO_PIN_RESET);
    pin_hiz(GPIOA, pin_arr[row_idx]);

    int col_idx = 0;
    for(int col_idx = 0; col_idx < 4; col_idx++)
    {
      if((val >> col_idx) & 0x01)
      {
        char *name = swname[row_idx][col_idx];
        sprintf((char*)usb_buf + strlen(usb_buf), "%s ", name);
      }
    }

    val = 0;
  }

  sprintf((char*)usb_buf + strlen(usb_buf), " --- %d %X\r\n", count, val);
  CDC_Transmit_FS(usb_buf, BUF_SIZE);
  memset(usb_buf,0,BUF_SIZE);
  count++;

}
#endif

 

전에는 이건 되었는데

 

요건 안되었었는데 input mode로 바꾸고 나서 잘 된다.

 

 

+

S1 / S4 / S16을 눌렀는데 S13 가 눌린걸로 인식되는건 다이오드 달아야지 머..

 

 

+

2026.09.15

클로드 님의 말

ra4~7(row) 는  open drain / level high

pa0~3(col) 은 input / pull up 설정

row 에서 set - hiz, reset - low 구동으로 된다고

대신 pull up  이라서 col 을 읽을때 논리가 반대가 된다는데

레지스터 바꾸면서 바쁠것 보다는 이게 낫지 않나 생각이 된다. (바꿀게 적으면 클럭이 낮아도 빠른 스캔을 유지할테니까)

근데.. od랑 input pull up이랑 어떻게 되는지 좀 공부 해야 할듯..

 

+

2026.09.16

gpt 조언

col 쪽에 전부 pull up/

row쪽은 Open drain output 으로 설정

 

저작자표시 (새창열림)

'모종의 음모 > USB 자작 키보드' 카테고리의 다른 글

홀 타이트 , 퀵스왑 소켓  (0) 2026.09.16
키보드 스캔 매트릭스 - 고스팅? 블로킹?  (0) 2026.09.10
kicad로 키보드 설계하기  (0) 2026.09.10
커스텀 키보드 자작 아이디어  (2) 2026.09.01
Posted by 구차니

티스토리툴바