CellStrat Gen AI Course Class 6 - LLM Appl Devl with LangChain and LlamaIndex
B Banik 10:11 What happens to token size everytime passing chat history Indrajit CS 10:12 @Banik LLMs rely solely on the information provided in the current prompt to generate a response. This means that every time you interact with an LLM, it treats the prompt (including the chat history) as a fresh input. Since the entire chat history needs to be included in the prompt for context, the total token size increases with every new message. This can become a concern if your conversations tend to be lengthy, as exceeding the maximum token limit might prevent the LLM from processing the entire prompt. Sudhakar Reddy 10:16 this is a UI but in case I want to use the same during my application development can I do the same with python apis B Banik 10:16 Thanks Indrajit Yes concern is exceeding max token size and we had similar use case setting history in context every prompt Indrajit CS 10:18 @Shudhakar, While LangChain offers a user interface (UI) for initial exploration and experimen...