The AI Canvas Newsletter #15

Building Meta’s GenAI Infrastructure – Meta

“To lead in developing AI means leading investments in hardware infrastructure. Hardware infrastructure plays an important role in AI’s future. Today, we’re sharing details on two versions of our 24,576-GPU data centre scale cluster at Meta.

Beyond Self-Attention: How a Small Language Model Predicts the Next Token - Shyam Pather

“I trained a small (~10 million parameter) transformer following Andrej Karpathy’s excellent tutorial, Let’s build GPT: from scratch, in code, spelled out. After getting it working, I wanted to understand, as deeply as possible, what it was doing internally and how it produced its results.”

AI Watermarking 101: Tools and Techniques – Hugging Face

“In this blog post, we will describe approaches to carry out watermarking of AI-generated content, discuss their pros and cons, and present some of the tools available on the Hugging Face Hub for adding/detecting watermarks.”

Don't Mock Machine Learning Models In Unit Tests – Eugene Yan

“I’ve been applying typical unit testing practices to machine learning code and it hasn’t been straightforward. In software, units are small, isolated pieces of logic that we can test independently and quickly. In machine learning, models are blobs of logic learned from data, and machine learning code is the logic to learn and use these derived blobs of logic. This difference makes it necessary to rethink how we unit test machine learning code.

Training great LLMs entirely from ground up in the wilderness as a startup – Yi Tay

Given that we’ve successfully trained pretty strong multimodal language models at Reka, many people have been particularly curious about the experiences of building infrastructure and training large language & multimodal models from scratch from a completely clean slate.

The State of Competitive Machine Learning – ML Contests

“We summarise the state of the competitive landscape and analyse the 300+ competitions that took place in 2023. Plus a deep dive analysis of 60+ winning solutions to figure out the best strategies to win at competitive ML.”

Feature Engineering A-Z

“Welcome to “Feature Engineering A-Z”! This book is written to be used as a reference guide to nearly all feature engineering methods you will encounter. This is reflected in the chapter structure. Any question a practitioner is having should be answered by looking at the index and finding the right chapter.”

Causal ML Book

“The book introduces ideas from classical structural equation models (SEMs) and their modern AI equivalent, directed a cyclical graphs (DAGs) and structural causal models (SCMs), and presents Double/Debiased Machine Learning methods to do inference in such models using modern predictive tools..”

Dive into 'The AI Canvas', our podcast exploring the transformative potential of generative AI. Engage in fireside chats, case studies, and innovative discussions on AI’s impact on industries and creativity.