TA Session 11 July: LLM APi Setup and Use#

Duration: 2h 48m

Here’s an FAQ based on the provided transcript, summarizing the student questions and instructor’s answers.

Q1: What is the content for today’s session?

A1: Today’s session is on Week 5 content. We’ll be discussing the first two videos in depth.

Q2: I haven’t watched the Week 5 lectures yet. Should I have watched them already?

A2: Yes, it’s quite disappointing that many haven’t watched them. These are new videos recorded by Anand Sir, and they contain up-to-date content that’s quite new even to me. They cover hand-on practice with some complex but not impossible concepts.

Q3: Was this content covered in yesterday’s instructor session?

A3: No, yesterday’s session focused on operational parts, like API emails, not educational content. So, we’ll start discussing from the very basics.

Q4: Can you explain what LLM stands for and what it means?

A4: LLM stands for Large Language Model. It’s a type of artificial intelligence program that processes a vast amount of data, hence “Large.” It can communicate in human languages, like Hindi, English, and many other major languages, which is why it’s a “Language” model. It uses neural networks and deep learning, which are advanced machine learning models.

Q5: How many of us have used ChatGPT?

A5: Let’s do a quick poll! [Poll initiated and results showed that most attendees had used ChatGPT]. You’ve used ChatGPT, but you can’t explain what an LLM is? You could even ask ChatGPT “What is an LLM?” and recite the answer!

Q6: How can ChatGPT understand different languages without being told what language I’m using?

A6: That’s the beauty of Large Language Models! They are trained on such a vast amount of data that they can differentiate between languages and understand the context and meaning of the sequence of words, even colloquialisms or slang. This allows them to respond appropriately in the language you use.

Q7: Can you explain the difference between a “view” and a “fact” when talking about language?

A7: A “view” is a personal opinion or feeling, like “I don’t like sun.” People might agree or disagree with it. A “fact,” on the other hand, is objectively true and generally undeniable, like “the sun is bright.” When discussing LLMs, we call this “subjectivity.” A statement with zero subjectivity is a fact, while high subjectivity means it’s an opinion.

Q8: What is sentiment analysis and how does it relate to LLMs?

A8: Sentiment analysis involves determining the emotional tone behind a piece of text—whether it’s positive, negative, or neutral. This is a huge business application of LLMs. For example, a politician might post about improving education and get a million comments. An LLM can analyze these comments to gauge public sentiment, helping the politician decide whether to push that agenda or not. This is done by analyzing words that convey positive (e.g., “very important,” “sure vote”) or negative (e.g., “I don’t like”) sentiment.

Q9: My TextBlob output for “I like cars” is giving a subjectivity of zero. Is that right?

A9: Not all machine learning models are 100% accurate. TextBlob is a relatively simple model. It can struggle with slang or context-dependent phrases, like calling a friend “stupid” as a term of endearment. More advanced LLMs are better at handling such nuances. The content of Week 5 used to cover these nuances, but it’s now moved forward in the syllabus.

Q10: What is the TextBlob polarity score?

A10: Polarity is TextBlob’s way of measuring sentiment, ranging from -1 (very negative) to +1 (very positive), with 0 being neutral. This range is continuous. A score closer to -1 indicates strong negative sentiment, while closer to +1 means strong positive sentiment.

Q11: How do LLMs distinguish between factual statements and opinions, even with similar-sounding words?

A11: LLMs consider the sequence of words and context. For example, “Aam Aadmi Party” is a political party. If I say “Aam Aadmi Party kaise ho?” (How are you, Aam Aadmi Party?), an LLM understands that “Aam Aadmi” in this context refers to a person, not the party, because asking a party “How are you?” doesn’t make sense. It must consider all possibilities to understand the true meaning.

Q12: Is it okay to post our full API key directly in the notebook?

A12: No, it’s not. Your API key is a secret. While I might share my screen during a session, you should never directly paste your key into the code where others can see it. Always use the Google Colab “Secrets” feature to securely store and access your API key. This helps protect your account and usage.

Q13: My notebook is giving an error related to “header =” when using TextBlob. What’s wrong?

A13: [After some back-and-forth, the instructor identifies the error: a typo in the Python keyword from (written as form in the student’s code).]

Q14: I’m facing an error when running my code. What should I do?

A14: Make sure you’re running your code in Google Colab, as TextBlob is an in-built library there. If you’re still encountering errors, we can discuss it.

Q15: What is the purpose of the API key, and how does it relate to tokens?

A15: The API key allows your Python code to access external services, like ChatGPT’s powerful LLM. These services are not free; they charge based on “tokens,” which represent chunks of text processed (not just individual words). Anand Sir’s company (TDS team) has purchased tokens and provided each of us with a 50-cent credit. This is how we can use the API without direct payment.

Q16: How can I check how many tokens I have left or how much a query costs?

A16: If you open the email with the subject “Large Language Model AI Proxy” (sent around July 9th), you’ll see details about the tokens. The API calls show the number of tokens consumed and the cost per query. Be mindful: using more tokens or purchasing additional ones costs money. You can also monitor your remaining token balance through the Open AI platform if you purchase tokens directly.

Q17: Is it possible to use a custom role name instead of “system” or “user”?

A17: No, the roles like “system,” “user,” and “assistant” are predefined by OpenAI. They have specific purposes in guiding the LLM’s behavior and context. You cannot change these keys in the JSON body, but you can change the content associated with them.

Q18: Will running the get_sentiment function multiple times waste tokens if the prompt is already defined?

A18: Yes, it will. Every time you run the get_sentiment function, it makes a new API call and consumes tokens. This is why it’s important to refine your prompt carefully. Once you get a satisfactory response, assign it to a local variable. Then you can work with that variable locally without incurring additional token costs.

Q19: How can I optimize my prompts to save tokens?

A19: Prompt engineering is key here. Define your instructions clearly and concisely. For example, instead of asking for “positive and negative” sentiment and then asking for “score on the scale of -1 to +1,” you can combine these into a single, specific request like “give me result as positive and negative, also give me sentiment score on the scale of -1 to +1.” This saves tokens by reducing redundancy. Also, thinking carefully about how to define the request in natural language is important.

Q20: Can ChatGPT understand numerical ranges like [-1, 1] if I use them in the prompt?

A20: While ChatGPT is advanced, using human-like language for instructions is generally preferred. While it might understand symbols like square brackets, it’s safer and clearer to explicitly state “on the scale of -1 to +1” to avoid any ambiguity, as it aims for human-like interaction.

Q21: Is it worth it to write Python code for these calls instead of using the ChatGPT web interface directly?

A21: Yes, absolutely! The web interface is user-friendly but less efficient for bulk processing. With Python, you can integrate ChatGPT’s power into your own scripts. Imagine having a CSV file with thousands of movie reviews. You can run sentiment analysis on all of them automatically using a loop in Python. This allows for scalability and automation that the web interface doesn’t offer.

Q22: My laptop battery just died, so I can’t share my screen. What should I do about the code?

A22: No worries, you can carry on. Just try to open the Colab notebook on your desktop. If you can share your screen there, we can troubleshoot.

Q23: I’m still struggling with the code. Can someone help me?

A23: Yes, of course. Please share your screen, and we’ll go through it step by step. We’re all here to learn from each other.

Q24: The balance of my tokens is decreasing quickly. Should I be worried?

A24: Yes, be careful. Tokens are a limited resource provided by Anand Sir. While 50 cents might seem like a lot for one query, it can be depleted quickly with inefficient prompting or repeated runs. Avoid unnecessary runs, reuse processed results by storing them in local variables, and focus on precise prompt engineering. Remember, spending a few extra minutes perfecting a prompt can save many tokens.

Q25: Can we use any model besides GPT-3.5 Turbo for our assignments?

A25: For graded assignments and to avoid depleting your token balance too quickly, it’s strongly recommended to stick to GPT-3.5 Turbo. Other models, like GPT-4, are much more expensive and will consume your credit very fast. So, use GPT-3.5 Turbo as specified in the instructions.

Q26: I missed some parts of the session. Will a recording be available?

A26: Yes, we always try to provide recordings as soon as possible after the session concludes, typically within two days.

Q27: Is it okay to use the existing notebook’s API key for my assignments?

A27: No, please do not use the API key provided directly in the notebook’s code, especially if it’s visible. You should use the individual API key that was shared with you via email. Each student has a unique key for their token balance.

Q28: Is the API key provided in the notebook valid for everyone?

A28: No, the API key visible in the original notebook is specific to its creator. For your personal use and assignments, you must use your own unique API key, which was shared with you via email.

Q29: What is this “AI Proxy” and how does it relate to our API keys?

A29: The AI Proxy is a system set up by Anand Sir’s company. OpenAI charges for API access, and Anand Sir has purchased access in bulk through his company. This AI Proxy acts as an intermediary, distributing API keys to us. So, when we make an API call, it goes through the AI Proxy’s server, which then interacts with OpenAI. This allows us to use OpenAI’s infrastructure while benefiting from the bulk purchase. That’s why we change the endpoint URL to point to the AI Proxy.