Derivativ Roundup: 5 Interesting Reads from Around the Gen AI World

AI Trends from Sapphire VC, a comparison of agentive tech and agentic AI, innovation in chain-of-thought prompting, thoughts on transforming transformers, and more.

Welcome to the first Derivativ Gaggle: a curated selection of articles and research showcasing some of the latest trends and advancements in generative AI. We’ve got a roundup of market insights in Sapphire VC’s Market Memo, a comparison of agentive tech and agentic AI, new innovative research in chain-of-thought prompting, and thoughts on transforming transformers. And don't miss the groundbreaking paper from Meta AI that's cracked the code on tuning-free personalized image generation!

Sapphire VC – July 2024 Market Memo

If the “Sapphire VC Market Memo” isn’t on your regular reading list, it should be. The firm’s market coverage includes an insightful roundup of a number of macrotrends in AI, including where we are in the AI HypeCycle (likely past the peak of inflated expectations, in Sapphire’s view); where we are in the AI CapEx cycle with a forecast for Cloud Capex growth of 52% in 2024, to $182B (with a roundup of related quotes from CxO’s of Microsoft, Amazon, and Meta); what’s driving the “extract-hire” trend by hyper-scalers who have collectively done multi-billion dollar talent takeouts from leading AI startups like Inflection, Adept, and Character.AI; a useful summary of publicly available Generative AI revenue figures and ARR growth from top Gen AI players, including hyperscalers and unicorns; and a roundup of notable Generative AI funding raises from July 2024, including Hebbia, Sentient, Contextual.AI, Fireworks AI, and Ema.

"Where we are in the AI Hype Cycle", Sapphire VC, Image Credit: Sapphire VC

Best of all, in case you don’t get enough interesting reads from this post, Sapphire includes its own must-read list which includes a timely article from MIT Technology Review about the notion of ‘Addictive Intelligence’, and the inherent dangers of AI companions built to consume our attention.

Agentive Tech and Agentic AI: Cousins not adversaries – by Chris Noessel

I’ve been spending a lot of time trying to build my first intelligent agents using Langchain and Langflow… I want so badly want to be agentic like all the cool AI kids!  In surveying the topic of intelligent agents, I came across this great piece, “Agentive Tech and Agentic AI: Cousins not adversaries”, by Chris Noessel. Chris opines on the distinction between Agentive Tech (think old school AI like your Roomba vacuum cleaner that automates a particular task on your behalf) and Agentic AI which automates tasks on behalf of YOUR AI. If you hear these terms, it’s important to know the difference because they are, as Chris says, cousins but not necessarily interchangeable. It’s probably better if I let Chris explain this in his own words, and then you should go read his piece:

“One way to think about the difference is to ask, for each framework, who is boss of the agents? In the case of agentic AI, the agents are working on behalf of the primary AI, the thing coordinating all of their efforts and consolidating them for the user. In agentive technology, the agents are working on behalf of the person, the user.”
- Chris Noessel

Strategic Chain-of-Thought: Guiding Accurate Reasoning in LLMs through Strategy Elicitation

Speaking of agentic, one of the more useful mechanisms for creating an intelligent agent is chain-of-thought prompting (CoT). Using CoT you can design agents to reason through a particular task in a stepwise fashion. As a simple example, suppose you want your agent to write a paper for you. With Chain of thought, your agent might be designed to first prompt the LLM for an outline of the paper, then prompt to request a summary for each topic in the outline, then prompt for a paragraph or page for each topic, iterating until the agent reaches the desired level of detail for the paper. (If you want to learn more about Chain-of-Thought reasoning, Lance Elliot has a great explainer here at Forbes)

CoT methods can, however, be unstable and inconsistent, generating varying levels of quality in reasoning paths for agents to follow. The research team behind the just released research paper, “Strategic Chain-of-Thought: Guiding Accurate Reasoning in LLMs through Strategy Elicitation” has devised a novel technique to help LLMs autonomously generate an optimal Chain-of-Thought path. In their research, they describe an approach that uses a prompt consisting of five components: Role, Workflow, Rule, Initialization, and Task Input. The prompt design is intended to first elicit strategic knowledge from the LLM for solving the problem at hand. Clever! In the paper, they test this novel approach, which they call S-CoT, or Strategic Chain-of-Thought on mathematical reasoning tasks and found that it improved accuracy from 1.11% to 24.13% across three different data sets evaluated. If you are designing Intelligent Agents, I highly recommend giving the Strategic Chain-of-Thought paper a read.

Strategic Chain-of-Thought Accuracy vs CoT, Strategic Chain-of-Thought: Guiding Accurate Reasoning in LLMs through Strategy Elicitation, Yu Wang et. al. 2024

Is the Next Frontier in Generative AI transforming Transformers?

In his article “Is the Next Frontier in Generative AI transforming Transformers?”, Ashish Kakran, asks what’s next after transformers? It’s an important question, particularly as we march towards agentic AI, because, as Kakran mentions in his piece, “computation complexity increases with long sequences (ex- DNA), leading to slow performance and high-memory consumption.” As we ask agents to solve increasingly complex tasks, which require long input sequences, transformer performance will have difficulty keeping up with the demands. Kakran explores several possible avenues to solve this long-sequence problem, including FlashAttention (managing read and write performance on the GPU), Aproximate Attention (reducing computation complexity for self-attention mechanisms), and State Space Models, an alternative to Transformers altogether. While there are no clear answers yet as to what comes next, Kakran points to a few new examples of Mixture of Experts and Composition of Expert models (each supporting larger context windows) from Databricks, SambaNova, and A121 Labs as potential near-term avenues for overcoming the performance limitations of transformer models.

Imagine yourself: Tuning-Free Personalized Image Generation

Lastly, the “Imagine yourself: Tuning-Free Personalized Image Generation” paper from Meta AI caught my eye because researchers there have discovered a novel way to fine-tune, or what they call “personalize” images without, well, actually fine-tuning. I’ve spent the last two years perfecting the art of fine-tuning images using diffusion models like SDXL and FLUX.1 only to learn that I probably won’t need to do image fine-tuning for much longer. Meta AI’s got it all figured out! Wow, this space really moves fast.

Kudos to the Meta AI research team. Unlike other fine-tuning approaches, which require creating a new model for each subject that you wish to personalize an image for, “Imagine Yourself” users share a single, universal underlying model for all image personalization. To use the model, you just need a single image of yourself and the Imagine Yourself model can transform your likeness into anything you can imagine. Want to be a superhero? An elf in middle earth? A villain in a galaxy far, far, away? Imagine Yourself can do that. And, once more, this isn’t all academic. Meta AI has already put the feature into production in Whatsapp and Instagram through the Meta AI tool. Simply click Meta AI next time you’re in the app and type “Imagine me” followed by whatever scene you want to see yourself in, and Meta AI will pull from your recent photos to create a personalized image for you. Read the paper for the academic explanation of how the one-model-for-all-personalization approach works but suffice it to say Meta AI has cracked a really hard nut here and made a major advancement in the field of diffusion modeling.

Conclusion

And that's a wrap for our first Derivativ gaggle. The velocity of change in this space is bonkers and is accelerating more every day. These research papers and articles are just a few examples of the progress and rapid change in AI that is having such a profound impact on so many aspects of our lives. Which did you find most interesting? What’s next on your Gen AI reading list? Leave your comments below!