We present a general method for discovering symbolic local learning rules that can replace backpropagation, applicable, in principle, to any deep learning architecture and task. The method automatically discovers local loss functions specific to each layer by (1) constructing a dictionary of monomials of local tensor variables, inspired by Penrose diagram notation, (2) regressing over coefficients weighting these monomials to build layerwise polynomial losses whose gradients together minimize a target global loss. We test the method on reconstruction, self-supervised learning, and classification tasks using small multilayer networks on minimal datasets.

Across all tasks, the discovered rules induce non-trivial interactions between layers that lead to significant reductions in the corresponding global losses, e.g., reaching classification accuracies comparable to backpropagation. Furthermore, the method selects sparse combinations of trace monomials, finding local rules that are more interpretable than those generated by MLP-based alternatives. The potential implications of our method range from energy-efficient deep learning, to the discovery of biologically plausible learning mechanisms.