Links for 2025-01-15
AI:
1. Google presents the successor to the Transformer architecture: Titans marks a significant step in neural network architecture by integrating a bio-inspired long-term memory mechanism that complements the short-term context modeling of traditional attention mechanisms. A key innovation is that the memory module is trained to learn how to memorize and forget during test time. This allows the model to adapt to new, unseen data distributions, which is crucial for real-world applications. The way Titans decides what to memorize is inspired by how the human brain prioritizes surprising or unexpected events. The authors introduce the concept of "momentary surprise" (how much a new input deviates from the model's current understanding) and "past surprise" (a decaying record of past surprises) to guide the memory module's updates. This mirrors the human tendency to remember events that stand out from the norm. https://arxiv.org/abs/2501.00663
2. Transformer^2: Self-adaptive LLMs — dynamically adapts to new tasks in real-time, using smart "expert" vectors to fine-tune performance. https://sakana.ai/transformer-squared/
3. Multiagent Finetuning: Self Improvement with Diverse Reasoning Chains https://llm-multiagent-ft.github.io/
4. Imagine while Reasoning in Space: Multimodal Visualization-of-Thought — MVoT moves beyond Chain-of-Thought (CoT) to enable AI to imagine what it thinks with generated visual images. By blending verbal and visual reasoning, MVoT makes tackling complex problems more intuitive, interpretable, and powerful. https://arxiv.org/abs/2501.07542
5. VideoRAG: A framework that enhances RAG by leveraging video content as an external knowledge source. https://arxiv.org/abs/2501.05874
6. O1 Replication Journey -- Part 3: Inference-time Scaling for Medical Reasoning https://arxiv.org/abs/2501.06458
7. The Lessons of Developing Process Reward Models in Mathematical Reasoning https://arxiv.org/abs/2501.07301
8. Exploring the Potential of Large Concept Models https://arxiv.org/abs/2501.05487
9. UC Berkeley releases a $450 open-source reasoning model that matches o1-preview https://novasky-ai.github.io/posts/sky-t1/
11. MatchAnything: Universal Cross-Modality Image Matching with Large-Scale Pre-Training https://zju3dv.github.io/MatchAnything/
AI economics:
2. “…even though standard measures of AI quality scale poorly as a function of resources, the financial returns might still scale very well as a function of resources. Indeed, if they scale better than linearly, that would create a paradigm of increasing marginal returns…” https://www.tobyord.com/writing/the-scaling-paradox
3. Applying traditional economic thinking to AGI: a trilemma https://www.lesswrong.com/posts/TkWCKzWjcbfGzdNK5/applying-traditional-economic-thinking-to-agi-a-trilemma
Bio(tech):
1. Nanocarrier imaging at single-cell resolution across entire mouse bodies with deep learning https://www.nature.com/articles/s41587-024-02528-1
2. New computational chemistry techniques accelerate the prediction of molecules and materials https://news.mit.edu/2025/new-computational-chemistry-techniques-accelerate-prediction-molecules-materials-0114
3. ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning https://arxiv.org/abs/2501.06590
4. About 5% of cyanobacteria fished from the ocean are connected via nanotubes. https://www.quantamagazine.org/the-ocean-teems-with-networks-of-interconnected-bacteria-20250106/
5. The use of genetically engineered bacteria to recover or recycle chemicals and turn them into useful products is progressing fast https://www.bbc.com/news/articles/cz6pje1z5dqo
6. Heritability: what is it, what do we know about it, and how we should think about it? https://www.lesswrong.com/posts/xXtDCeYLBR88QWebJ/heritability-five-battles
7. Synchron to Advance Implantable Brain-Computer Interface Technology with NVIDIA Holoscan https://www.businesswire.com/news/home/20250113376337/en/Synchron-to-Advance-Implantable-Brain-Computer-Interface-Technology-with-NVIDIA-Holoscan
AI:
1. Google presents the successor to the Transformer architecture: Titans marks a significant step in neural network architecture by integrating a bio-inspired long-term memory mechanism that complements the short-term context modeling of traditional attention mechanisms. A key innovation is that the memory module is trained to learn how to memorize and forget during test time. This allows the model to adapt to new, unseen data distributions, which is crucial for real-world applications. The way Titans decides what to memorize is inspired by how the human brain prioritizes surprising or unexpected events. The authors introduce the concept of "momentary surprise" (how much a new input deviates from the model's current understanding) and "past surprise" (a decaying record of past surprises) to guide the memory module's updates. This mirrors the human tendency to remember events that stand out from the norm. https://arxiv.org/abs/2501.00663
2. Transformer^2: Self-adaptive LLMs — dynamically adapts to new tasks in real-time, using smart "expert" vectors to fine-tune performance. https://sakana.ai/transformer-squared/
3. Multiagent Finetuning: Self Improvement with Diverse Reasoning Chains https://llm-multiagent-ft.github.io/
4. Imagine while Reasoning in Space: Multimodal Visualization-of-Thought — MVoT moves beyond Chain-of-Thought (CoT) to enable AI to imagine what it thinks with generated visual images. By blending verbal and visual reasoning, MVoT makes tackling complex problems more intuitive, interpretable, and powerful. https://arxiv.org/abs/2501.07542
5. VideoRAG: A framework that enhances RAG by leveraging video content as an external knowledge source. https://arxiv.org/abs/2501.05874
6. O1 Replication Journey -- Part 3: Inference-time Scaling for Medical Reasoning https://arxiv.org/abs/2501.06458
7. The Lessons of Developing Process Reward Models in Mathematical Reasoning https://arxiv.org/abs/2501.07301
8. Exploring the Potential of Large Concept Models https://arxiv.org/abs/2501.05487
9. UC Berkeley releases a $450 open-source reasoning model that matches o1-preview https://novasky-ai.github.io/posts/sky-t1/
11. MatchAnything: Universal Cross-Modality Image Matching with Large-Scale Pre-Training https://zju3dv.github.io/MatchAnything/
AI economics:
2. “…even though standard measures of AI quality scale poorly as a function of resources, the financial returns might still scale very well as a function of resources. Indeed, if they scale better than linearly, that would create a paradigm of increasing marginal returns…” https://www.tobyord.com/writing/the-scaling-paradox
3. Applying traditional economic thinking to AGI: a trilemma https://www.lesswrong.com/posts/TkWCKzWjcbfGzdNK5/applying-traditional-economic-thinking-to-agi-a-trilemma
Bio(tech):
1. Nanocarrier imaging at single-cell resolution across entire mouse bodies with deep learning https://www.nature.com/articles/s41587-024-02528-1
2. New computational chemistry techniques accelerate the prediction of molecules and materials https://news.mit.edu/2025/new-computational-chemistry-techniques-accelerate-prediction-molecules-materials-0114
3. ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning https://arxiv.org/abs/2501.06590
4. About 5% of cyanobacteria fished from the ocean are connected via nanotubes. https://www.quantamagazine.org/the-ocean-teems-with-networks-of-interconnected-bacteria-20250106/
5. The use of genetically engineered bacteria to recover or recycle chemicals and turn them into useful products is progressing fast https://www.bbc.com/news/articles/cz6pje1z5dqo
6. Heritability: what is it, what do we know about it, and how we should think about it? https://www.lesswrong.com/posts/xXtDCeYLBR88QWebJ/heritability-five-battles
7. Synchron to Advance Implantable Brain-Computer Interface Technology with NVIDIA Holoscan https://www.businesswire.com/news/home/20250113376337/en/Synchron-to-Advance-Implantable-Brain-Computer-Interface-Technology-with-NVIDIA-Holoscan