TA’s Session TDS – 2024 11 04 16 56 IST – Recording#

TA’s Session TDS – 2024 11 04 16 56 IST – Recording

Duration: 1h 55m

Here’s an FAQ based on the TDS live tutorial:

Q1: What will be covered in this session?

A1: We will be covering the content from Week 5 and addressing any doubts or issues you may have regarding the course material or the project. Jivraj, the other TA, is also available to help.

Q2: I haven’t received my API token yet. How do I get it?

A2: I sent an email with the subject “TDS API” that contains a link to the AI proxy. Access that link, and you will be able to get your API token. Every student has been given a certain allowance for experimentation, but you can also purchase your own tokens if needed.

Q3: What is the purpose of the AI proxy and its API?

A3: The AI proxy acts as a bridge for accessing OpenAI’s keys, routing requests through our server. The main thing to remember is the URL for the API. It allows you to make GET and POST requests to interact with Language Learning Models (LLMs). The API also provides useful information in the headers of its responses, such as the cost of your request and your monthly limit.

Q4: What is the monthly usage limit for the API token?

A4: Your monthly limit is 50 cents. While it might not sound like a lot, students barely used their dollar allowance last term, so 50 cents should be plenty for experimentation.

Q5: What exactly are LLMs?

A5: LLMs like ChatGPT are not “magic boxes.” They use sophisticated statistical modeling and mathematical transformations. When you send a query, it’s broken down into “tokens,” represented by high-dimensional vectors with weights. The LLM then uses the relationships between these tokens to predict and generate a response. Essentially, they are predictive text mappings that rely on massive matrices and significant computational power.

Q6: What is sentiment analysis, and why is it useful?

A6: Sentiment analysis is the process of determining the emotional tone behind a sentence or text – whether it’s positive, negative, or neutral. This is incredibly useful for various applications:

  • Product Reviews: Understanding customer sentiment from reviews (e.g., Google reviews) to gauge product reception.
  • Social Media Monitoring: Tracking public opinion on political campaigns, brands, or current events to calibrate strategies.
  • Advertising & Marketing: Assessing the effectiveness of campaigns and adapting messaging based on public reaction.

Q7: Can we perform sentiment analysis without using expensive LLM tokens?

A7: Yes, you can! A library called TextBlob can perform sentiment analysis. While LLMs can do this, TextBlob is often a good alternative because it’s very fast, requires minimal resources, and doesn’t incur costs for token usage. It might not be 100% accurate every time, but it provides a reasonably good idea of the sentiment in large datasets.

Q8: How does TextBlob classify sentiment?

A8: TextBlob provides two main metrics:

  • Polarity: Indicates whether the sentiment is positive (+1), negative (-1), or neutral (0).
  • Subjectivity: Ranges from 0 (objective) to 1 (subjective), indicating how much the statement is based on personal feelings or opinions.

For example, “I like football” or “I hate football” would likely have high subjectivity, while “This is a laptop” would be objective.

Q9: Why are LLMs considered an improvement over traditional methods like TextBlob?

A9: While TextBlob is fast and free, LLMs offer significantly more sophisticated and context-aware sentiment analysis. They leverage billions of parameters and complex models, leading to more nuanced and accurate interpretations of language. The ability to use API calls for LLMs also allows for large-scale, automated processing, which is crucial for modern data science applications.

Q10: What are the key components of an API request to an LLM?

A10: When making an API request to an LLM, especially for POST requests, you’ll need:

  1. URL: The endpoint for the specific LLM function you want to use (e.g., for chat completions, embeddings).
  2. Headers: This includes authorization (your API key) and other metadata.
  3. JSON Body: This is where you send the actual data or instructions to the LLM. For chat completions, this typically involves a “model” key (specifying the LLM to use) and a “messages” key (containing a list of dictionaries with “role” and “content” for your prompts).

Q11: Can you explain the “role” and “content” in the JSON body for chat completions?

A11: In LLM interactions, there are generally two roles:

  • System: Represents the LLM itself, providing instructions or context.
  • User: Represents you, the one asking the question or providing input.

The “content” is the actual text message associated with each role. For example, the “system” content might be a general instruction like “You are a sentiment analysis bot,” and the “user” content would be your specific sentence like “I like football.”

Q12: What is “training” an LLM, and how does it relate to fine-tuning its responses?

A12: Training an LLM involves customizing its behavior or responses by providing it with specific examples or instructions. There are different types of “prompting”:

  • Zero-shot training: No prior examples are given; the LLM responds based on its general knowledge.
  • One-shot training: A single example or instruction is provided to guide the LLM’s response style.
  • Multi-shot training: Multiple examples are given to fine-tune the LLM’s behavior more extensively.

By providing examples of desired output, you can “tweak” the LLM to respond in a specific way, like breaking down sentiment analysis into “acting,” “storyline,” and “direction” for a movie review. This is called Aspect-Based Sentiment Analysis.

Q13: What is a JSON schema, and why is it important when interacting with LLMs?

A13: A JSON schema is like a blueprint for structuring data. It defines the expected format, types, and properties of JSON data. It’s crucial because LLMs, being language-agnostic, need explicit instructions on how to structure the output data.

A JSON schema typically comprises:

  • Type: Specifies the data type (e.g., object, string, number, array).
  • Properties: Defines the fields within an object, each with its own type and optional description.
  • Required Fields: Lists which properties must be present.

By providing a JSON schema in your API request, you can instruct the LLM to return data in a predictable and usable format, which is essential for automated processing and integration with other systems.

Q14: I noticed a “context_window” parameter in the schema. What is its purpose?

A14: (Self-correction by the presenter) Sorry, that’s not a parameter in this specific schema. That refers to a different context.

Q15: How can LLMs help with unstructured data, like addresses?

A15: LLMs, guided by JSON schemas, can effectively extract structured information from unstructured or semi-structured data. For example, if you have a list of addresses that aren’t consistently formatted, you can provide an LLM with a JSON schema defining fields like street, city, ``state,zip`, etc. The LLM can then parse the unstructured addresses and return them in the specified structured format. This is much more efficient than traditional programming methods for complex data extraction.

Q16: I had an issue where the provided link to the GitHub repository didn’t work. Is it just me?

A16: I’ve noted that the second link doesn’t work for some reason. The general solution is to copy the link and paste it into a new tab, and it should work. I will confirm this with Anand.

Q17: Could you summarize what we covered regarding LLMs and APIs today?

A17: We covered quite a bit today! We started by understanding what LLMs are, their basic mechanics, and their applications. We then looked at sentiment analysis, comparing TextBlob (a faster, cheaper alternative) with LLMs. We delved into the structure of LLM API calls, specifically for chat completions, emphasizing the JSON body with “role” and “content.” We also explored how to “train” or fine-tune LLMs using different prompting techniques (zero-shot, one-shot, multi-shot) and the concept of Aspect-Based Sentiment Analysis. Finally, we learned about JSON schemas – how they act as blueprints to get structured data back from LLMs, which is incredibly powerful for processing unstructured information like addresses. The API approach is super cheap compared to browser-based interactions, and we’ve just begun to unlock its potential.