I haven't used ClaudDev myself, but from what I can see, it's an agentic application. Agent workflows typically involve a chain-of-thought process, which means a lot of back-and-forth over API. This can get expensive quickly. Every time Claude needs to access a file or directory, it requires another API call, which doesn't seem very efficient to me. Instead, I've been using the VS Code plugin called Continue, and it's been fantastic. I highly recommend it. One of its great features is that you can configure different models for different tasks. For example, I use local models for autocompletion and embeddings, while I use Claude 3.5 for chat functionality. You should definitely check it out. https://github.com/continuedev/continue docs: https://docs.continue.dev/how-to-use-continue

Oh, I see. The "tokens in" are actually the ones eating up your limit. The "tokens out" are Claude's responses. I'd recommend trying out Perplexity with Claude 3.5 Sonnet. It has an excellent workflow and cleverly carries over the pasted files throughout the session. The UI is user-friendly too. If you'd like, give me a prompt and I can run it with Perplexity Pro and share a screenshot with you here. Here's how I use it with Claude sometimes.

When you build and intelligent RAG system is a good practice to consider the following aspects: Dynamic Session Management efficiently handles multiple concurrent conversations, tracking both active and historical sessions. It optimizes resource allocation while maintaining crucial contextual information. The Dual-Storage approach implements a two-tiered system that stores raw conversation data as individual document pairs, each associated with a unique session ID, while also creating and maintaining quick-access summaries for efficient information retrieval. Intelligent Summarization generates concise summaries of past conversations, enabling rapid context recall without the need to parse entire chat logs. Context-aware retrieval functions as an attentive system that recalls past interactions and surfaces relevant information, enhancing the coherence and personalization of each interaction. Asynchronous processing handles resource-intensive tasks in the background, ensuring responsive conversations even during peak system activity. Implementation Highlights The initialization process prepares the conversational environment by setting up necessary components, including user ID, session ID, and other critical parameters. Periodic processing employs a background task to identify and process inactive sessions, creating summaries and clearing processed sessions from active memory. The summarization engine utilizes an agent-based approach to distill the essence of lengthy conversations into concise, informative summaries. Vector-based retrieval implements a high-performance information retrieval system using vector representations for quick and relevant data access. Optimization Strategies Memory-Index integration creates a unified reference system that integrates personal interaction history with general knowledge for rapid information access. Hierarchical indexing organizes information in a structured, tree-like format for efficient navigation from broad categories to specific details.

I forgot to mention something. In my setup, I've created an enhanced retrieval mechanism that extends the base retrieval. The EnhancedRetriever integrates multiple data sources: knowledgebase, research documents from trusted sources, medical database, and conversation history. It queries these sources sequentially, triggering a LangGraph-based workflow for recent research if initial results from research documents relevance score falls below a threshold. The retriever then combines and scores all results, ensuring rich, contextual information for complex conversations. You can apply similar approach to enhance the results fetching data from other external or internal sources