Gemini 1.5's groundbreaking 1M token context window is a remarkable advancement in LLMs, providing capabilities unlike any other currently available model. With its 1M context window, Gemini 1.5 can ingest the equivalent of 10 Harry Potter books in one go. However, this enormous context window is not without its limitations. In my experience, Gemini 1.5 often struggles to retrieve the most relevant information from the vast amount of contextual data it has access to. The "Needle in a Haystack" benchmark is a well-known challenge for LLMs, which tests their ability to find specific information within a large corpus of text. This benchmark is particularly relevant for models with large context windows, as they must efficiently search through vast amounts of data to locate the most pertinent information. To address this issue, I have developed a novel prompting technique that I call "Multi-Candidate Needle Prompting." This approach aims to improve the model's ability to accurately retrieve key information from within its large context window. The technique involves prompting the LLM to identify 10 relevant sentences from different parts of the input text, and then asking it to consider which of these sentences (i.e. candidate needles) is the most pertinent to the question at hand before providing the final answer. This process bears some resemblance to Retrieval Augmented Generation (RAG), but the key difference is that the entire process is carried out by the LLM itself, without relying on a separate retrieval mechanism. By prompting the model to consider multiple relevant sentences from various parts of the text, "Multi-Candidate Needle Prompting" promotes a more thorough search of the available information and minimizes the chances of overlooking crucial details. Moreover, requiring the model to explicitly write out the relevant sentences serves as a form of intermediate reasoning, providing insights into the model's thought process. The attached screenshot anecdotally demonstrates the effectiveness of my approach.

Anthropic was formed by a group of ex-AI employees who didn't think OpenAI was safe enough. They previously promised to never release a state-of-the-art LLM. However, with Claude 3 Opus, they appear to have gone back on their word. Opus is now the strongest model that is publicly available. Did Anthropic sell out? Personally, I'm an accelerationist, so I'm a winner here, but I can't help but wonder about the drama that is unfolding behind the scenes.

In Celebration of our first member I have setup a space for ComfyUI to be used by our members. ComfyUI is an image generator. If you don't know how to use it, don't worry. It has this great feature that lets you load the state that an image was generated in by extracting the data from the image and rebuilding the pipeline. Long story short, just drag and drop the image attached to this post into ComfyUI (make sure it gets dropped on the back ground). and the program will do the set for you. Now just hit Queue and you're generating art. Parameters to play with: - Prompt: the text fields on the board, they are often inside the nodes classified as a "Text Encoder" and are what the model uses to interpret your request. Focus on the the positives and break your concept into smaller chunks separated by a comma. Ex. "Beautiful sunset, blue skies, sandy beach, ocean lapping into shore, award wining illustration, very detailed, bold linework, bright saturated colors" - CFG: you can find this on the node called "Sampler" often K Sampler. This controls how closely the model tries to adhere to your request. Lower number means it will stick to your prompt, high number means it will be more creative. There is a diminishing returns though. After a certain point the image will look "burnt" or "overcooked." Basically over exposed and ugly. 7-9 is a good starting range. - Denoise: This is also on the sampler, it controls how much noise (think of static on a television but the static is the starting image) is taken away during the process. 1(or 100% they use 1 as the representation of the percent, just add two zeros to the number to figure out the percent) means no noise left over, 0 means nothing but noise. I usually go for 80-90% on an image generated from text and 10-30% for image to image. - Steps: You can select the number of passes that the model takes over the image. Each pass it removes a bit more white noise. To more passes the more clear the image gets, but like CFG it has diminishing returns. If it passes over an image with no noise it will start damaging the image removing something that is not there. I like 15-25 with regular stable diffusion models and 35-50 with XL models. - Sampler: Think of these as kind of like brushes, they give different textures and other characteristics that are not easily definable. i like dimm, euler and dpmm_3m_sde_gpu(the name rolls of the tongue doesn't it?), but play around see what works, what doesnt. - Scheduler: These are the algorithms that decide when and how much noise to take out. I personally almost always stick to exponential but you can get good results with any of them. It depends on your other settings. Best way to figure it out is to play with the know and hit the button to see what comes out.