The AI Canvas Newsletter #16

Evolving New Foundation Models: Unleashing the Power of Automating Model Development – Sakana AI

“The core research focus of Sakana AI is in applying nature-inspired ideas, such as evolution and collective intelligence, to create new foundation models. We are currently developing technology that makes use of evolution with the goal of automating the development of foundation models with particular abilities suitable for user-specified application domains. Our goal isn’t to just train any particular individual model. We want to create the machinery to automatically generate foundation models for us!.”

Beyond Self-Attention: How a Small Language Model Predicts the Next Token - Shyam Pather

“I trained a small (~10 million parameter) transformer following Andrej Karpathy’s excellent tutorial, Let’s build GPT: from scratch, in code, spelled out. After getting it working, I wanted to understand, as deeply as possible, what it was doing internally and how it produced its results.”

What I learned from looking at 900 most popular open source AI tools – Chip Huyen

“Four years ago, I did an analysis of the open source ML ecosystem. Since then, the landscape has changed, so I revisited the topic..”

You can now train a 70b language model at home – Answer.ai

“We’re releasing an open source system, based on FSDP and QLoRA, that can train a 70b model on two 24GB GPUs.”

ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs – Fengqing Jiang et al

“In this paper, we propose a novel ASCII art-based jailbreak attack and introduce a comprehensive benchmark Vision-in-Text Challenge (ViTC) to evaluate the capabilities of LLMs in recognizing prompts that cannot be solely interpreted by semantics.”

Demystifying how GPT works: From Architecture to...Excel!?! 🚀

“Think AI is too complex? Think again! If you're spreadsheet-savvy, you're ready to grasp modern AI.”

Skyvern

“Automate browser-based workflows with LLMs and Computer Vision”

Ask HN: If you've used GPT-4-Turbo and Claude Opus, which do you prefer?

An interesting discussion about GPT4 vs Claude 3 Opus. It seems that Opus may have the leading edge!

Will GPT-4 Run DOOM?

“We show that GPT-4's reasoning and planning capabilities extend to the 1993 first-person shooter Doom.”

Paper-Piano

“Now we don't need to buy a Piano if we want to play music. We can play piano on paper, although it may not give a feeling of pressing keys on the piano but it gets work done!”

Monte-Carlo Graph Search from First Principles

This page offers a practical insight into Monte-Carlo Graph Search, an advanced AI technique crucial for optimizing search algorithms in complex state spaces with transpositions. It is essential reading for AI practitioners and researchers aiming to enhance their understanding of search algorithms and their applications in cutting-edge AI systems.

Dive into 'The AI Canvas', our podcast exploring the transformative potential of generative AI. Engage in fireside chats, case studies, and innovative discussions on AI’s impact on industries and creativity.