Axis of Ordinary

Links for 2025-01-15

AI:

1. Google presents the successor to the Transformer architecture: Titans marks a significant step in neural network architecture by integrating a bio-inspired long-term memory mechanism that complements the short-term context modeling of traditional attention mechanisms. A key innovation is that the memory module is trained to learn how to memorize and forget during test time. This allows the model to adapt to new, unseen data distributions, which is crucial for real-world applications. The way Titans decides what to memorize is inspired by how the human brain prioritizes surprising or unexpected events. The authors introduce the concept of "momentary surprise" (how much a new input deviates from the model's current understanding) and "past surprise" (a decaying record of past surprises) to guide the memory module's updates. This mirrors the human tendency to remember events that stand out from the norm. https://arxiv.org/abs/2501.00663

2. Transformer^2: Self-adaptive LLMs — dynamically adapts to new tasks in real-time, using smart "expert" vectors to fine-tune performance. https://sakana.ai/transformer-squared/

3. Multiagent Finetuning: Self Improvement with Diverse Reasoning Chains https://llm-multiagent-ft.github.io/

4. Imagine while Reasoning in Space: Multimodal Visualization-of-Thought — MVoT moves beyond Chain-of-Thought (CoT) to enable AI to imagine what it thinks with generated visual images. By blending verbal and visual reasoning, MVoT makes tackling complex problems more intuitive, interpretable, and powerful. https://arxiv.org/abs/2501.07542

5. VideoRAG: A framework that enhances RAG by leveraging video content as an external knowledge source. https://arxiv.org/abs/2501.05874

6. O1 Replication Journey -- Part 3: Inference-time Scaling for Medical Reasoning https://arxiv.org/abs/2501.06458

7. The Lessons of Developing Process Reward Models in Mathematical Reasoning https://arxiv.org/abs/2501.07301

8. Exploring the Potential of Large Concept Models https://arxiv.org/abs/2501.05487

9. UC Berkeley releases a $450 open-source reasoning model that matches o1-preview https://novasky-ai.github.io/posts/sky-t1/

11. MatchAnything: Universal Cross-Modality Image Matching with Large-Scale Pre-Training https://zju3dv.github.io/MatchAnything/

AI economics:

2. “…even though standard measures of AI quality scale poorly as a function of resources, the financial returns might still scale very well as a function of resources. Indeed, if they scale better than linearly, that would create a paradigm of increasing marginal returns…” https://www.tobyord.com/writing/the-scaling-paradox

3. Applying traditional economic thinking to AGI: a trilemma https://www.lesswrong.com/posts/TkWCKzWjcbfGzdNK5/applying-traditional-economic-thinking-to-agi-a-trilemma

Bio(tech):

1. Nanocarrier imaging at single-cell resolution across entire mouse bodies with deep learning https://www.nature.com/articles/s41587-024-02528-1

2. New computational chemistry techniques accelerate the prediction of molecules and materials https://news.mit.edu/2025/new-computational-chemistry-techniques-accelerate-prediction-molecules-materials-0114

3. ChemAgent: Self-updating Library in Large Language Models Improves Chemical Reasoning https://arxiv.org/abs/2501.06590

4. About 5% of cyanobacteria fished from the ocean are connected via nanotubes. https://www.quantamagazine.org/the-ocean-teems-with-networks-of-interconnected-bacteria-20250106/

5. The use of genetically engineered bacteria to recover or recycle chemicals and turn them into useful products is progressing fast https://www.bbc.com/news/articles/cz6pje1z5dqo

6. Heritability: what is it, what do we know about it, and how we should think about it? https://www.lesswrong.com/posts/xXtDCeYLBR88QWebJ/heritability-five-battles

7. Synchron to Advance Implantable Brain-Computer Interface Technology with NVIDIA Holoscan https://www.businesswire.com/news/home/20250113376337/en/Synchron-to-Advance-Implantable-Brain-Computer-Interface-Technology-with-NVIDIA-Holoscan

Links for 2025-01-13

AI:

1. MIDAS speeds up language model training by up to 40%. While MIDAS-trained models may have similar or slightly worse perplexity compared to traditional training methods, they perform significantly better on downstream reasoning tasks. https://arxiv.org/abs/2409.19044

2. Building AI Research Fleets https://www.lesswrong.com/posts/WJ7y8S9WdKRvrzJmR/building-ai-research-fleets

3. Superhuman forecaster seems reachable in 2025 https://arxiv.org/abs/2412.18544

4. Training Transformers for simple next token prediction on videos leads to competitive performance across all benchmarks. https://arxiv.org/abs/2501.05453

5. Optimizing LLM Test-Time Compute Involves Solving a Meta-RL Problem https://blog.ml.cmu.edu/2025/01/08/optimizing-llm-test-time-compute-involves-solving-a-meta-rl-problem/

6. Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning https://arxiv.org/abs/2412.15797

7. Creating a LLM-as-a-Judge That Drives Business Results https://hamel.dev/blog/posts/llm-judge/

8. Grokking at the Edge of Numerical Stability https://arxiv.org/abs/2501.04697

9. Can transformers be scaled up to AGI? Ilya Sutskever: Obviously, yes https://youtu.be/Ft0gTO2K85A?si=ab3ADAzLoUr4n5Ns&t=1680

10. Web UI for interacting with Qwen (Alibaba) models including their reasoning model https://chat.qwenlm.ai/

AI politics:

1. What would happen if remote work were fully automated? Matthew Barnett argues the economic impact would be massive—with the economy doubling in size even in the most conservative scenario. https://epoch.ai/gradient-updates/consequences-of-automating-remote-work

2. Once robots can do physical jobs, how quickly could they scale up? Converting car factories might produce 1 billion robots annually in under 5 years. Here are some maths for rapid robot deployment. https://www.lesswrong.com/posts/6Jo4oCzPuXYgmB45q/how-quickly-could-robots-scale-up

3. David Dalrymple on Safeguarded, Transformative AI https://www.youtube.com/watch?v=MPrU69sFQiE

4. Human takeover might be worse than AI takeover https://www.lesswrong.com/posts/FEcw6JQ8surwxvRfr/human-takeover-might-be-worse-than-ai-takeover

5. NVIDIA CEO Jensen Huang: "the critical technologies necessary to build general humanoid robotics is just around the corner" and an aging population and declining birthrate makes this imperative as the world needs more workers https://youtu.be/Z_DR1_zhmCU?si=3-yePRXlqzQtTHeX&t=65

Health:

1. “Yet more evidence that Alzheimer's is caused by human herpesvirus variants. The HHV family of viruses is almost certainly responsible for a very wide variety of horrifying human illnesses. (EBV, for example, is the root cause of Multiple Sclerosis.)” (via Perry E. Metzger) https://www.science.org/doi/10.1126/scisignal.ado6430

2. Heritable polygenic editing: the next frontier in genomic medicine? Very large potential gains in long-term health from completely removing certain bad alleles present in our collective gene pool. https://www.nature.com/articles/s41586-024-08300-4

Psychology:

1. Are the average genetic scores for intelligence decreasing between birth cohorts? https://www.emilkirkegaard.com/p/dysgenics-within-and-between

2. New study finds enhanced creativity in autistic adults is linked to co-occurring ADHD rather than autism itself (N=352). https://psycnet.apa.org/fulltext/2025-66159-001.html

Computer science:

1. “Above my pay grade: Jensen Huang and the quantum computing stock market crash” https://scottaaronson.blog/?p=8567

2.“The single axiom ((a•b)•c)•(a•((a•c)•a))=c is a complete axiom system for Boolean algebra” https://writings.stephenwolfram.com/2025/01/who-can-understand-the-proof-a-window-on-formalized-mathematics/

3. The purposeful drunkard https://www.lesswrong.com/posts/s39XbvtzzmusHxgky/the-purposeful-drunkard

4. The Dilithium implementation in Google and Microsoft's Caliptra root of trust just got hacked by measuring the switching power consumption of internal pipeline registers to extract keys https://eprint.iacr.org/2025/009.pdf

Links for 2025-01-09

AI:

1. Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought https://arxiv.org/abs/2501.04682

2. Inference time MCTS can boost Qwen *1.5B* to o1-preview levels on MATH and AIME '24. Microsoft presents rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking https://arxiv.org/abs/2501.04519

3. Smaller, Weaker, Yet Better: “Our findings reveal that models fine-tuned on weaker & cheaper generated data consistently outperform those trained on stronger & more-expensive generated data across multiple benchmarks…” https://arxiv.org/abs/2408.16737

4. Agent Laboratory: Using LLM Agents as Research Assistants https://agentlaboratory.github.io/

5. AI unveils strange chip designs, while discovering new functionalities https://www.nature.com/articles/s41467-024-54178-1

6. A foundation model of transcription across human cell types https://www.nature.com/articles/s41586-024-08391-z

7. AI helps doctors detect more breast cancer in the largest real-world study https://www.nature.com/articles/s41591-024-03408-6

8. Ovarian cancer diagnosis improved by AI https://healthcare-in-europe.com/en/news/ovarian-cancer-diagnosis-ai.html

9. Microsoft CEO Satya Nadella: "we fundamentally believe the scaling laws are absolutely still great and will work and continue to work" https://www.youtube.com/live/bYgP-tC5BFU?si=kASx6DIOk6ioBpUl&t=932

10. Beyond Sight Finetuning Generalist Robot Policies with Heterogeneous Sensors via Language Grounding https://fuse-model.github.io/

11. NVIDIA Redefines Game AI With ACE Autonomous Game Characters https://www.nvidia.com/en-us/geforce/news/nvidia-ace-autonomous-ai-companions-pubg-naraka-bladepoint/

12. While everyday users still encounter hallucinating chatbots and the media declares an AI slowdown, behind the scenes, AI is rapidly advancing in technical domains. https://time.com/7205359/why-ai-progress-is-increasingly-invisible/

13.Large language models for artificial general intelligence (AGI): A survey of foundational principles and approaches https://arxiv.org/abs/2501.03151v1

14. Google is building its own ‘world modeling’ AI team for games and robot training https://www.theverge.com/2025/1/7/24338053/google-deepmind-world-modeling-ai-team-gaming-robot-training

15. Tyler Cowen - The #1 Bottleneck to AI progress Is Humans https://www.dwarkeshpatel.com/p/tyler-cowen-4

Biotech:

1. “What I saw would cause any biotech leader to sit up and take notice. I saw science parks many multiples larger than Kendall Square or South SF, filled with startups. Integrated biology, chemistry, biochem and structural biology, and vivarium labs were running at scale. Even smaller biotechs were running vivariums processing tens of thousands of in vivo mouse experiments monthly. Programs which went from standing start to registering for human clinical trials within 18 months(!) were not uncommon.” https://timmermanreport.com/2025/01/china-is-here-to-stay-as-a-leader-on-the-global-biotech-stage/

2. Sana Biotechnology Announces Positive Clinical Results from Type 1 Diabetes Study of Islet Cell Transplantation Without Immunosuppression https://www.globenewswire.com/news-release/2025/01/07/3005841/0/en/Sana-Biotechnology-Announces-Positive-Clinical-Results-from-Type-1-Diabetes-Study-of-Islet-Cell-Transplantation-Without-Immunosuppression.html

Miscellaneous:

1. Particle that only has mass when moving in one direction observed for first time https://www.psu.edu/news/research/story/particle-only-has-mass-when-moving-one-direction-observed-first-time

2. On Eating the Sun https://www.lesswrong.com/posts/6Fo8fjvpL7pwCTz3t/on-eating-the-sun

3. NVIDIA Announces World’s Smallest AI Supercomputer https://www.nvidia.com/en-us/project-digits/

4. New study says the Romans suffered "widespread cognitive decline including an estimated 2.5-to-3 point reduction" in IQ in their European territories because of industrialized silver mining releasing lead into the atmosphere https://pnas.org/doi/10.1073/pnas.2419630121

Links for 2025-01-07

AI:

1. Language agents backed by open-source, non-frontier LLMs can match and exceed both frontier LLM agents and human experts on multiple scientific tasks at up to 100x lower inference cost. https://arxiv.org/abs/2412.21154

2. A LLM agent for multi-agent settings that generates hypotheses about other agents' latent states in natural language, adapting to diverse agents across collaborative, competitive, and mixed-motive domains https://arxiv.org/abs/2407.07086

3. Microsoft's Charles Lamanna: "By this time next year, you'll have a team of [AI] agents working for you" https://www.fastcompany.com/91254053/25-experts-predict-how-ai-will-change-business-and-life-in-2025

4. Google demonstrates scalability of AI agents, enabling complex workflow automation. https://www.kaggle.com/whitepaper-agents

5. "AI agents is likely to be a multi-trillion dollar opportunity"— Jensen Huang, Nvidia CEO https://www.youtube.com/live/k82RwXqZHY8?si=lVdJlwg51hKwIyU_&t=2638

6. Given sufficient context, LLMs can suddenly shift from their concept representations to 'in-context representations' that align with the task structure https://arxiv.org/abs/2501.00070

7. B-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasoners https://arxiv.org/abs/2412.17256

8. Transformer takes 22 seconds of brain state vectors, and outputs the next 5 seconds of human neural activity with decent accuracy. https://www.arxiv.org/abs/2412.19814

9. Generating All-Atom Protein Structure from Sequence-Only Training Data https://amyxlu.github.io/plaid/

10. A metagenomic foundation model to help detect and prevent the next pandemic early https://metagene.ai/metagene-1-paper.pdf

11. Turn smartphones into pocket laboratories for farmers https://ai.meta.com/blog/inarix-agricultural-supply-chain-meta-dino-v2/

12. Self-driving tractors and trucks https://www.theverge.com/2025/1/6/24334357/john-deere-autonomous-tractor-truck-orchard-mow-ces

13. NVIDIA Cosmos: World Foundation Model Platform for Physical AI https://github.com/NVIDIA/Cosmos

14. Google: "we believe scaling on video and multimodal data is on the critical path to artificial general intelligence" https://techcrunch.com/2025/01/06/google-is-forming-a-new-team-to-build-ai-that-can-simulate-the-physical-world/

15. Test-time Computing: from System-1 Thinking to System-2 Thinking https://arxiv.org/abs/2501.02497

Health:

1. The mission to restore sight takes a big leap forward! 🚀 🌟 Large-scale RF mapping without visual input for neuroprostheses https://www.medrxiv.org/content/10.1101/2024.12.22.24319047v1.full

2. Transplanting young Hematopoietic Stem Cells into old mice rejuvenates their blood's epigenetic age and boosts physical performance https://www.nature.com/articles/s41422-024-01057-5

Tech:

1. Why China Is Building a Thorium Molten-Salt Reactor https://spectrum.ieee.org/chinas-thorium-molten-salt-reactor

2. Storing Thousands of Terabytes in a Single Gram of DNA https://www.nature.com/articles/s41586-024-08040-5

Math:

1. “I have two children, at least one of whom is a boy born on a day that I'll tell you in 5 minutes. What is the chance that both are boys, and what will the chance be after I tell you the day?” https://www.lesswrong.com/posts/7i4qTDCxf5QBYWqvg/practicing-bayesian-epistemology-with-two-boys-probability

2. How can it be feasible to find proofs? https://drive.google.com/file/d/1-FFa6nMVg18m1zPtoAQrFalwpx2YaGK4/view

Science:

1. Any living creature in our universe, natural or artificial, will inevitably view light's speed in empty space as extremely quick. https://profmattstrassler.com/2024/10/03/why-is-the-speed-of-light-so-fast-part-2/

2. “The penguins nodded off >10,000 times per day, engaging in bouts of bihemispheric and unihemispheric slow-wave sleep lasting on average only 4 seconds, but resulting in the accumulation of >11 hours of sleep for each hemisphere.” https://www.mpg.de/21169426/1127-orni-penguins-nesting-in-a-dangerous-environment-obtain-large-quantities-of-sleep-via-seconds-long-microsleeps-154562-x