Data Science Wire

Agnes-AI/Agnes-3.0-Flash 33B Multimodal, AA score: 36

Reddit r/LocalLLaMA3d4 min read

I find this new model at HF: "Built for demanding work. A 262 144-token context window, adjustable reasoning effort, tool calling, and text, image and video understanding. Architecture Agnes-3.0-Flash is a hybrid-attention decoder: three of every four layers run a gated delta rule (recurrent, with per-layer state independent of sequence length), and the fourth runs standard global attention. Only 18 of the 72 layers therefore hold a KV cache that grows with context." Context length 262 144 tokens Decoder layers 72 = 54 delta-rule recurrent + 18 global attention, alternating 3 : 1 Hidden size 5

Read the full story at Reddit r/LocalLLaMA

More in AI