On the Depth Scalability of Logic Gate Networks

arXiv cs.LG5d4 min read

arXiv:2607.21633v1 Announce Type: new Abstract: Logic Gate Networks (LGNs) implement computation through compositions of Boolean operations, yet unlike classical Boolean circuits, existing LGNs do not reliably benefit from increased depth. We identify two distinct causes: optimization collapse in deep relaxed LGNs and a topology-induced limitation that persists even when skip-biased initialization and straight-through estimation stabilize training. Thus, trainability alone is insufficient; deeper layers must also receive information that supports useful computation. We introduce Input-Anchored

Read the full story at arXiv cs.LG →

On the Depth Scalability of Logic Gate Networks

More in Machine Learning

Reducing Human Annotation with ML Active Learning

Learning What Matters: Supervising Sparse Attention Routing with Causal Evidence Sets

Pretraining EHR Foundation Models with Patient-Aware Sampling

From Perturbation Correction to Geometry-Aware Sampling: Sharpness-Guided Equilibrium Sampling for Balanced Flat Minima in Long-Tailed Learning

Parameter-free Adaptive Sparse Attention via Compression-Based Content Selection

Cross-Domain Off-Policy Evaluation and Learning for Contextual Bandits