2025 05 21 Week 2 Session 1 TDS May 2025#

2025 05 21 Week 2 Session 1 TDS May 2025

Duration: 2h 31m

Here’s a summary of the live tutorial in an FAQ format:

Tools in Data Science (TDS) Live Tutorial FAQ#

Q1: How is the TDS course progressing (GA1, GA2)? Are there any common challenges students are facing?

A1: GA1 (General Assignment 1) is complete, and GA2 is currently in progress. I’ve noticed some of you are finding the coding aspects challenging, especially those without a prior coding background. This course will be particularly important for you. Regarding specific assignments, some feel that certain video content isn’t always directly relevant to the assignment tasks.

Q2: I’m having trouble with the GA2 markdown assignment, specifically how to create the markdown file and understand the assignment requirements.

A2: Markdown is essentially a way to write formatted text, very useful for documentation and even platforms like Discourse posts. The core of markdown is just plain text with specific syntax (e.g., # for headings, **bold** for bold text, triple backticks for code blocks). The assignment requires you to create a single markdown file (README.md) that demonstrates the use of all the key markdown features covered in the course, such as top-level headings, sub-headings, bold text, italic text, lists, and code blocks. You can create this file easily in an editor like VS Code. While VS Code might offer a live preview (due to an extension I use), markdown itself doesn’t require any special installation; it’s just text. The purpose of this assignment is primarily to get you familiar with markdown formatting, which will also be useful for clear communication in platforms like Discourse.

Q3: My Week 1 LLM assignment isn’t working; I tried to replicate the steps, but the output I’m getting is incorrect.

A3: This particular assignment (Week 1 LLM) is straightforward. The most common issue students face is that the output isn’t in the expected format. The output needs to be a list of numbers, specifically in the format [num1, num2, num3, ...]. As long as your output adheres to this list format and contains the correct numbers, it should be accepted. I’ll address this more fully at the end of the session.

Q4: My Week 1 assignments show “not submitted” with a zero score. When will scores be visible?

A4: Don’t worry about the “not submitted” or zero score you see on the Seek portal right now. We will process all scores tonight. We’ll take your latest submission before the deadline. You’ll receive an email with your score, and that email will also include a link to a discrepancy form. If you believe there’s an error in your score, you can fill out that form. Your final scores will be published on your dashboard a couple of days after the email is sent out.

Q5: Are past session recordings available? Where can I find them, especially since I received an email saying recordings wouldn’t be available?

A5: Yes, all past sessions are uploaded to the official TDS YouTube channel, except for the very last one, which will be uploaded today. The email you received about recordings was a generic message from the BS team to encourage live attendance, as many students were just watching recordings later. However, for these specific live sessions, we are recording them and making them available. You can find the YouTube playlist by visiting the main TDS website and looking for the link at the very bottom of the page. I recommend subscribing to the YouTube channel; that way, you’ll get a notification as soon as a new session’s recording is uploaded, so you don’t have to keep checking.

Q6: What are the recommended prerequisites for TDS, especially for those without a coding background, to succeed in modules like deployment and LLM? How can I approach these new topics effectively?

A6: To truly excel, especially in advanced topics like deployment and LLM (Large Language Models), you should have a solid understanding of Python programming and be comfortable with the Bash terminal. Linux (or Mac OS, which is Unix-based) is also highly recommended, as it provides a robust environment for application development. While you can get Bash for Windows, it’s not ideal because the underlying OS remains Windows, which can lead to inconsistencies. WSL (Windows Subsystem for Linux) is a better alternative if you’re on Windows. The key to approaching new modules and tools is practice. Go through the documentation, watch the videos, and then actively try to replicate the steps yourself. If you encounter errors, use resources like TDS GPT (our virtual TA on the main webpage) to help debug and understand where things went wrong. The information in the documentation and videos is sufficient, but hands-on experience is crucial.

Q7: Can you explain GitHub and APIs in more detail, as these are new concepts for me?

A7: Absolutely. We’ll be using GitHub throughout this session for version control, and you’ll see how APIs (Application Programming Interfaces) work. An API essentially provides a way for different software applications to communicate with each other. It defines a set of rules and protocols for building and interacting with software. For example, when we created the FastAPI endpoints, each endpoint was a function that our web application could “hit” to perform a specific task (like adding a job). GitHub is a platform for version control using Git. It allows developers to collaborate on projects, track changes in code, and manage different versions of their work. You saw how we created a local Git repository, committed changes, and then pushed it to a remote GitHub repository. We also used the GitHub CLI (Command Line Interface) for speed. Both APIs and GitHub are fundamental tools in data science and software development.

Q8: Why should I use Vercel for deployment, and what exactly is a “serverless” application?

A8: Vercel is highly beneficial for deploying web services, especially lightweight ones. Traditionally, you’d spin up a virtual machine (VM) to host your application, which means the server is constantly running 24/7, incurring costs. For applications that don’t need continuous operation – like our TDS discrepancy form, which only a limited number of users access intermittently – a full VM is overkill. Vercel operates on a “serverless” model. In this model, you don’t run a full, continuously active server. Instead, each API endpoint is essentially a function that only executes when it’s specifically called or “hit.” Once the function completes its task, it scales down to zero, meaning no resources are consumed until the next call. This makes Vercel extremely cost-effective and scalable for limited-use cases, temporary campaigns, or small, modular applications that don’t require the overhead of a dedicated server.

Q9: How do I create a GitHub repository from the command line interface (CLI) quickly?

A9: Using the GitHub CLI (gh) is much faster than going through the browser. After installing gh, you can simply use the command gh repo create [your_username]/[repo_name] --public --source .. This command will:

  1. Create a new repository on GitHub under your username with the specified name.
  2. Make the repository public (you can choose private too).
  3. Push all files from your current local directory (.) to this new remote repository. This process takes mere seconds, significantly speeding up your workflow compared to manual steps in the browser.

Q10: How do I deploy a FastAPI application to Vercel?

A10: To deploy your FastAPI app on Vercel, you need three main components:

  1. api/index.py: Your FastAPI application file (e.g., app = FastAPI(), defining your endpoints).
  2. requirements.txt: Lists Python dependencies (e.g., fastapi, uvicorn).
  3. vercel.json: A configuration file in your project’s root directory that tells Vercel how to build and deploy your app. It defines builds (specifying the source file and builder, like @vercel/python) and routes (mapping URL paths to your application). Once these are set up and pushed to your connected GitHub repository, you can go to the Vercel dashboard, import your Git repo, and click “Deploy.” Vercel automatically detects FastAPI and handles the deployment, providing you with a temporary URL.

Q11: How do I handle CORS (Cross-Origin Resource Sharing) in my FastAPI application on Vercel?

A11: CORS is a security mechanism that prevents web browsers from blocking requests to a server from a different origin (domain). To enable CORS in your FastAPI app, you need to add CORSMiddleware to your main.py file. This middleware allows you to configure which origins, methods, and headers are permitted to access your API. For example, you can allow all origins (allow_origins=["*"]) or specify a list of trusted domains. While allowing all ("*") is easy for development, for production, you should restrict it for security.

Q12: How do I retrieve URL parameters (like ?name=John) in FastAPI, especially for multiple values?

A12: In your FastAPI endpoint, you can access URL query parameters through the request object. If you have a parameter like ?name=John, you can get its value using request.query_params.get('name'). For multiple values associated with the same key (e.g., ?item=apple&item=banana), use request.query_params.getlist('item'), which will return a list of all values for that key. This allows your API to flexibly handle different types of query inputs.

Q13: How do I send data in the request body (JSON) to my FastAPI application, and how do I define the structure of that data?

A13: To send data in the request body, you’ll typically use a POST request with JSON data. FastAPI integrates well with Pydantic for this.

  1. Define a Pydantic Model: Create a Python class that inherits from BaseModel (from pydantic). This class defines the expected structure and data types of your incoming JSON data (e.g., class Job(BaseModel): name: str; cost: int; start_date: date).
  2. Use the Model in your Endpoint: In your FastAPI endpoint, define the request body by type-hinting a parameter with your Pydantic model (e.g., @app.post("/create") async def create_job(job: Job):). FastAPI will automatically validate the incoming JSON against your model. You can then send POST requests using tools like curl, specifying the JSON data in the request body.

Q14: How do I delete specific items using an API endpoint, and how do I test these CRUD (Create, Read, Update, Delete) operations?

A14: To delete a specific item (like a job) via an API:

  1. Define a DELETE Endpoint: Create an endpoint that accepts an identifier (e.g., job_id) as a path parameter (e.g., @app.delete("/delete/{job_id}")).
  2. Implement Deletion Logic: Inside the endpoint function, use the job_id to locate and remove the corresponding item from your data store (e.g., a list). To test CRUD operations, you can use curl commands. For a DELETE request, use curl -X DELETE [your_vercel_url]/delete/[job_id]. You can verify the deletion by then making a GET request to your /jobs endpoint. We implemented “create job” (POST), “get jobs” (GET), and “delete job” (DELETE) operations.

Q15: How can I make my API more intelligent by integrating with an LLM, and what is the plan for the next session?

A15: To make the API intelligent, we’ll integrate it with a Large Language Model (LLM). The plan for the next session is to:

  1. Install Ollama: We’ll set up Ollama, which allows you to run LLMs locally on your machine.
  2. Run the Ollama Service: We’ll ensure the Ollama LLM service is running and accessible.
  3. Integrate with FastAPI: We’ll modify our FastAPI application to send prompts to the local LLM via Ollama and receive responses. For example, we could prompt the LLM to generate steps for a specific job.
  4. Dockerize the Application: Finally, we’ll dockerize the entire application (FastAPI + Ollama) into a Docker container. This makes it self-contained and easily deployable.

Q16: Where can I find TDS GPT, the virtual TA mentioned for debugging?

A16: You can find TDS GPT, our virtual TA, on the main TDS webpage. It’s usually located at the bottom right corner of the page. It’s a great resource for debugging errors and getting clues on how to fix issues you encounter.