According to open ai's o1 documentation prompt engineering for o1 models is different from earlier models: - Keep prompts simple and direct and do not guide the model too much because it understands instructions well - Avoid chain of thought prompts since o1 models already reasons internally - Use delimiters like triple quotation markets, XML tags and section titles so the model can get clarity on which sections it is interpreting - Limit additional context for retrieval augmented generation (RAG) because OpenAI said adding more context or documents when using the models for RAG tasks could overcomplicate its response

Salesforce is now creating Agentforce—a new AI agent platform that will make a debut at the company’s annual Dreamforce conference this month. Agentforce lets users quickly build and deploy autonomous AI-powered agents—pieces of software that can make decisions and act on information—that run on top of Salesforce’s existing apps for businesses and automate customer service tasks.

Quite interesting coverage of Nvidia's hardware choices: "Instead of one big piece of silicon, Blackwell consists of two advanced new Nvidia processors and numerous memory components joined in a single, delicate mesh of silicon, metal and plastic. The manufacturing of each chip has to be close to perfect: Serious defects in any one part can spell disaster, and with more components involved, there is a greater chance of that happening. What’s more, the heat generated by all those pieces risks warping different materials in the package at different rates." https://www.wsj.com/tech/nvidias-future-relies-on-chips-that-push-technologys-limits-bd3839fc

I wrote a piece on how AI-generated data leads to declining model performance and strategies on how to mitigate/address it. Please comment your thoughts and if you found it useful please share and subscribe! https://open.substack.com/pub/bockster/p/model-collapse-the-self-destructive?r=ionsk&utm_campaign=post&utm_medium=web&showWelcomeOnShare=true

Anthropic has recently launched a significant feature called **Prompt Caching**, aimed at enhancing the performance and cost-effectiveness of its large language models (LLMs), specifically Claude 3.5 Sonnet and Claude 3 Haiku, with support for Claude 3 Opus coming soon. This feature allows users to cache frequently used contextual information, which can be reused in future API calls, thereby reducing both latency and operational costs. ## Benefits of Prompt Caching 1. **Cost Reduction**: Prompt caching can reduce costs by up to **90%**. While caching an input token costs 25% more than the base input token price, utilizing cached content is **10% cheaper** than the base price. This makes it a financially attractive option for businesses that frequently interact with the same data. 2. **Latency Improvement**: Users can expect latency reductions of up to **85%** for long prompts. For example, chatting with a book that has 100,000 tokens cached can take only **2.4 seconds**, compared to **11.5 seconds** without caching, resulting in a **79% reduction** in response time. 3. **Enhanced User Experience**: By enabling the storage of extensive background knowledge and example outputs, prompt caching allows for more efficient interactions. This is particularly beneficial for applications such as conversational agents, coding assistants, and document processing, where maintaining context over multiple queries is essential. 4. **Versatile Use Cases**: The feature is ideal for various scenarios, including: - **Conversational agents**: Reducing costs and latency in extended dialogues. - **Coding assistants**: Improving autocomplete and codebase Q&A capabilities. - **Document processing**: Handling long-form materials without increasing response times. - **Detailed instruction sets**: Providing comprehensive examples and instructions to fine-tune responses. ## Potential of Prompt Caching By adopting prompt caching, users can leverage the full power of Claude, ensuring that their interactions with AI are not only efficient but also economically viable. For those interested in maximizing their AI capabilities, prompt caching is a must-try feature that promises to deliver substantial benefits in real-world applications.