@Jackie Seem — Not really. The prompt framework is more geared towards a thought partner. LLMs are pretty good at summarization at this point. To your point, what you focus on may be different. In those cases, I’d give the LLM a role (I.e., product manager with 20 years of experience with a focus on go to market strategy) and then ask it to summarize. Once summarized, validate the output. The role you give an LLM will determine what it focuses on with the summary.

@Jackie Seem — TL;DR: Hallucinations cannot be avoided, merely mitigated. Context: I tend to use perplexity to measure accuracy in an LLM. To manage hallucinations, it will depend on your data, and the model. I’ll provide two examples. 1. Let’s assume you have a structured data set in a csv. You could fine tune 3 different roBERTa models with different hyperparameters, and feed those into another roBERTa model. Multimodal in this case helps reduce perplexity and leads to less hallucinations (they won’t be zero, but far less common). 2. You have a ton of unstructured data in the form of PDFs. You could use Llama 3.1 (LLM) and fine tune it. Or, if that data set is changing often enough, you can implement RAG. As it stands, RAG is pretty tough to implement in a production environment. Something to note, this is a GenAI issue and not really an ML issue. Here’s a link going over different measurement methods. https://research.aimultiple.com/large-language-model-evaluation/

Great idea, @Jackie Seem! I'll work on finding an updating market map for Gen AI and do a video on it. The TL;DR version is this: It can be daunting to keep up with the latest and greatest. Unless you enjoy doing so, here are a few of my go-to's that I use the most. Chat GPT & Claude: I use these two arguably the most. Chat GPT for thought partnership and Claude for peer review and code. You can throw Chat PRD in this group if you have that since it's using Chat GPT as the model, just with custom instructions. Perplexity/Arc Search: I don't use Google Search anymore, outside of "near me" searches. Other than that, I use Perplexity (which is also the backend for Arc Search). It summarizes the content for my ask, provides sources, and I can ask follow up questions directly related to my search. Google attempted this and it didn't go so well. Everything else I use is more use case specific. For example, Make I use for automations when it's something that is duplicative (and eventually pass to an AI Agent when they get more sophisticated). Miro can create boards from AI prompts if you use Miro a lot (think brainstorming sessions or process related items). Figma has AI in their Figjam app, which is similar to Miro's functionality. CoPilot I'll use if I'm in the Office ecosystem, and Gemini in the Google Workspace world. Github actions if I'm in Github, or Supergrow if I'm looking to create v.1 draft of content. That's the short version, but I'll do some digging on the market map and look at creating something for that. Thanks for the idea!